Flutter Deep Dive: Gestures
Flutter provides some really amazing widgets out of the box which comes pre-built for handling touch events such as in InkWell and InkResponse. These widgets wrap your widgets so that they are able to respond to touch events. In addition to doing this, it also adds the Material Ink splash to your widget. InkResponse, for example, has options to control the shape and clipping of the splash as it extends out of the widget's boundary. An interesting thing to note is InkWell and InkResponse don't do any rendering, instead they update the parent Material widget. A common example of this is an image. If you wrap an image in an inkWell, you would notice that the ripple is not visible. This is because it is drawn behind the image on the Material. To make the Ink splash visible, wrap your image using Ink.Image. While useful for most tasks, if you want to capture more events, such as when a user drags across the screen, one should use GestureDetector.
So what is the Gesture Detector? How does it work?
The basic overview of gesture detector is a stateless widget which has parameters in its constructor for different touch events. It is worth noting that you cannot use Pan and Scale together since Scale is a superset of Pan. GestureDetector is used purely for detecting gestures and thus does not give any visual response (the Material Ink propagation is absent).
Here is a table to show the different callbacks GestureDetectorprovides and a short description of what they do:
GestureDetector decides which gestures to attempt to recognize based on which of its callbacks are non-null. This is useful since if you need to disable a gesture, you would pass null.
Let’s use the onTap gesture as an example and determine how this is processed by the GestureDetector.
First of all, we create a GestureDetector with an onTap callback, since this is non-null the GestureDetector will use our callback when tap events occur. Inside of GestureDetector, a Gesture Factory is created. Gesture Recognizer does the hard work of determining what gesture is being handled. This process is the same for all of the different callbacks GestureDetector provides. The GestureFactories are then passed on to the RawGestureDetector.
RawGestureDetector does the hard work of detecting the gestures. It is a stateful widget which syncs all gestures when the state changes, disposes of the recognizers, takes all the pointer events that occur and sends it to the recognizers registered. They then battle it out in Gesture Arena.
RawGestureDetector build method consists of a Listener which is the base class for listening to pointer events. If you would like to use raw inputs like up, down or cancel events which come from the platform, this is your go-to class. Listener does not give you any gestures, only the basic onPointerDown, onPointerUp, onPointerMove and onPointerCancel events. Everything must be handled manually, including reporting yourself to the Gesture Arena. If you don't, then you don't get cancelled automatically and would not be able to partake in the interactions which occur there. This is the lowest level on the widget side.
Listener is a SingleChildRenderObjectWidget which consist of the class RenderPointerListener which extends RenderProxyBoxWithHitTestBehavior, meaning it mimics the properties of its children while allowing HitTestBehavior to be customized. If you would like to learn more about Render Boxes and how they operate, you should check out this article by Norbert Kozsir.
HitTestBehaviour has three options, deferToChild, opaque and translucent. These come from and are configured in GestureDetector. DeferToChild passes the event down the widget tree and is the default behaviour. Opaque prevents widgets that are in the background from receiving the events and Translucent allows for the background widget to receive the event.
So what if you wanted both the parent and the child to receive the pointer events?
Let’s for a minute imagine a situation where you had a nested list and you wanted to scroll both at the same time. For this, you would need the pointer to be received by both the parent and the child. You configure the hit test behaviour so that it is translucent, ensuring that both widgets are receiving the events but things don’t go according to plan…why is that?
Well, the answer to the above question would be GestureArena.
GestureArena is used in gesture disambiguation. All recognizer are sent here where they battle it out. At any given point on the screen, there can be multiple gesture recognizers. Arena takes into account the length of time the user touches the screen, the slop as well as the direction a user drags to determine a winner.
Both the parent list and the child list would have their recognizers sent to the arena but (at the time of writing this) only one will win and it always happens to be the child.
The fix would be to use a RawGestureDetector with your own GestureFactory's which change the behaviour of how the arena performs.
As an example, let’s create a simple app which consists of two containers. The goal would be to have both the child and parent receive the gesture.
Both will be wrapped in a RawGestureDetector. Next, we are going to create a custom gesture recognizer, AllowMultipleGestureRecognizer. GestureRecognizer is the base class on which all other recognizers inherits from. It provides the basic API for classes so that they are able to work/interact with gesture recognizers. It is worth noting that GestureRecognizer doesn’t care about the specific details of the recognizers themselves.
In the above code, we are creating a custom class AllowMultipleGestureRecognizer which extends TapGestureRecognizer. This means it is able to inherit the class of TapGestureRecognizer. In this example, we are overriding rejectGesture such that instead of disposing of the recognizers, it is being manually accepted.
Now we pass our custom gesture-recognizer in a GestureRecognizerFactoryWithHandlers to the RawGestureDetector.
Now we pass our custom gesture-recognizer in a GestureRecognizerFactoryWithHandlers to the RawGestureDetector. The factory requires two properties, a constructor and an initializer for constructing and initializing the gesture recognizer. We use a lambda for passing these parameters. As described in the above code, the constructor returns a new instance of AllowMultipleGestureRecognizer while the initializer takes the property instance which is used to listen for a tap and print some text to the console. This is going to be repeated for both containers, with the only difference being the text that is printed.
Here is the full source code to the sample app:
So what is the result of running the above code?
As you tap the yellow container, both widgets receive the tap and thus there are two statements printed to the console.
The app:
Console output:
What Happens When You Win?
After a gesture wins, the arena is closed and swept. This disposes of the unused recognizers and resets the arena. The winning gesture then performs an action.
Bringing it back to our Tap example, after this occurs, the function mapped to onTap would now be executed.
Summation
Today we looked at how the Flutter framework handles gestures. We started off by looking at the fantastic pre-built widgets Flutter provides for handling taps and other touch events. Next, we moved on to GestureDetector and examined the way in which it works internally. Through the use of an example, we followed how a Tap gesture is processed by Flutter. We journeyed through the land of RawGestureDetector, took in the sounds of Listener and admired the secret Flutter fight club known as GestureArena.
In closing, we covered the majority of the gesture system in Flutter from the perspective of an application. With this knowledge, you should now have a better understanding of a touch on the screen is picked up and worked on behind the scenes. If you have any questions or concerns, please feel free to leave a comment or reach out to me on Twitterverse.
Also huge thanks to Simon Lightfoot(aka the “Flutter Whisperer”) for contributing to this article ❤
