Prototyping for TV screens with Framer

This article introduces a module I have developed to create prototypes for TV screens (or 10-feet UI) using Framer.

The Problem

The most commonly used prototype tools available right now cater mostly to two interactions methods: touch and mouse. Both these interactions methods allow random access to elements i.e. the users can freely tap or click on any element anywhere on the screen to interact with the product.

However, when designing experiences for TV screens we face a unique challenge. Most TV platforms (Apple TV, Amazon Fire TV, Roku, Xbox, Playstation, etc.) do not allow free or random access to the interface elements. Instead, the user has to move a “selection” around the interface using a controller and press buttons (select, back etc.) to execute an action on the currently selected item. This type of an interaction paradigm is not very well supported by most rapid prototyping tools currently available.

I wanted to solve for this problem not only for myself but also for other fellow designers working on 10 feet UI and facing the same problem.

Why Framer?

Once I narrowed down on the problem, I had to choose a tool. I zeroed in on Framer because I find that Framer is the only tool that offers the flexibility of code. This flexibility that empowers you to create literally anything. All it asks in return is the motivation to learn a little bit of programming and the willingness stay with a problem long enough to solve it.

My second choice for this was Origami by Facebook. Origami is a very powerful tool for prototyping, and for many, it might be the right choice. However, there were a couple of reasons for me to choose Framer over Origami.

1. Some folks are very comfortable with patches and noodle paradigm offered by Origami, but I found that without extensive documentation effort it is nearly impossible for me to go back to a project and understand and recollect what is going on. It’s worse if youre looking at someone else’s file. Instead, I find that the simplicity of the CoffeScript syntax (the language used in Framer) works a lot better for me.
2. The design tab in Framer allows designing and coding in the same program thereby infinitely simplifying the workflow.
3. Framer allows you to create resuable modules, that abstract away all the complex code and only surfaces the important details you need to know. This makes collaborating and sharing modules a lot easier.

I also feel that having a little bit of coding knowledge helps designers be better at their jobs. Learning to code exposes you to the nitty gritties involved in making the design “work”. Even if prototyping in Framer exposes you to 1% of the actual complexity faced by the engineers, I believe it helps build empathy towards our engineer counterparts and allow us to have a better dialogue with them. These skills are essential for any UX designer. Now I can go on and on, waxing eloquent about why I think designers should learn to code, but hopefully you get the point.

Principles and Guidelines

I started out with code that I hacked together over a weekend to solve this problem. My initial attempts worked (sort of) for one project, but the code I wrote was very specific to the task at hand and as such wasn’t reusable at all.

Over time as I reused the code for more projects, I made it more modular so that it could be applied to a variety of use cases. Over time I encountered more scenarios that I had not considered initially that helped me to keep refining the module.

Throughout this process, there were a couple of pointers that I always kept in mind to help guide the development:

1. Approachability: I wanted all designers to be able to use this solution, regardless of whether they are just starting with Framer or are already pros.
2. Compatibility: I wanted to ensure that the solution works seamlessly with layers imported from Sketch or created in the design tab.
3. Last but not the least: Simplicity. I am aware that not all designers are comfortable writing lots of code. Which is why I wanted to build a solution that can be easily implemented without the need to write or learn any more code than required to start using Framer.

Demo

The solution I came up with is a Focus Manager module that can be easily imported into Framer and set up with a few lines of code. Users can plug in their PS4 or Xbox One controllers to navigate the prototype. If the game controllers are not available, the module also allows you to use keyboard controls.

Here is an animation of a very simple demo I put together using this module.It features a grid of thumbnails, a scroll component and a fullscreen preview.

Download the demo file from the github repo, or directly from here.

Usage and Instructions

Ok, enough talk. Let’s get down to business and see how to use this module. But before we start, let’s make sure we are up to speed with some basics.

What You Need to Know.

Before we start talking about the module and how to use it, make sure you are comfortable with Events and Event Listeners.

The module uses a lot of custom events to create interactions. Which is why a general understanding of how events work in coffescript and how to write event listeners will be useful.

Downloading and Importing the Module

Start by creating a new project and downloading the module files (focusManager.coffee and Gamepad.coffee) from the github repo. Copy the files in the modules folder of your Framer project.

Copy the files in the modules folder

Next, import the module in your framer project as such:

{focusManager} = require ‘focusManager’

Initializing the Module

The next step is to create a Focus Manager object.

The Focus Manager object keeps track of the currently selected layer, the last selected layer and other settings such as the type of controller, selection appearance etc.

Ideally you should create the Focus Manager object at the start of your project to avoid any unintended behavior as you convert standard layers to selectable layers.

focusManager = new focusManager
     leftStickDpad: boolean
     controller: "string"
     defaultOnState: object
     defaultOffState: object
     defaultSelectionBorder: boolean
     defaultSelectionBorderWidth: integer
     defaultSelectionBorderColor: color

Here are the properties you can use to initialize the Focus Manager object:

1. .leftStickDpad (boolean) specifies whether you want to use the left joystick on a gamepad as a directional controller (same as d-pad). If set to true, moving the left joystick will also move the selection in addition to the directional keys. Default value is false.
2. .controller (string) specifies what kind of controller you are using with your prototype. The acceptable string values are PS4 and XB1. Both the controllers output different key codes for button presses, and therefore it is important to specify what kind of controller you are using so that the module knows what key codes to listen for.
3. defaultOnState / defaultOffState(object) optional properties that specify the states that are applied to selected (defaultOnState) and deselected (defaultOffState) layers. For example:

focusManager = new focusManager
    defaultOnState:
      scale: 1
      options:
         time: 0.15
    defaultOffState:
      scale: 0.85
      options:
         time: 0.15

1. .defaultSelectionBorder (boolean) optional property that specifies whether layers will have a border in selected state. Default value is true.
2. .defaultSelectionBorderWidth (integer) optional property to change the border width for a layer in selected state. Default value is 5
3. .defaultSelectionBorderColor (color) optional property to change the border color for selected layers. Default value is “fff” (white)

Once initialized, you can use the following properties on the focusManager object anywhere in the project:

1. .selectedItem (layer) specifies which layer to select (make sure this layer exists, is visible, and is not off-screen!). For example: focusManager.selectedItem = anotherLayer.
2. .lastSelectedItem (layer) a read only property that returns the last selected layer. It returns null if no layer was selected previously. For Example: print focusManager.lastSelectedItem.name will output the name of the last selected layer.

Connecting the Game Controllers

Thanks to the gamepad module created by Emil Widlund, we can now use game controllers directly with Framer prototypes. The gamepad module is included in the Focus Manager module and does not need to be imported separately. However, it does require you to have mac OS 10.12.6 (Sierra) or higher for it to work.

Game controllers can be connected via bluetooth or a USB cable. However, you have to press a button first to activate or wake up the controller in order to start using it. For e.g. the cross button on activates the PS4 controller.

I have found the PS4 controllers are plug and play on Mac OS X, but Xbox One controllers require drivers to work. Thankfully, these drivers are easily available and can be downloaded from here.

Whether or not you use gamepad controllers, you always have the option to use the keyboard to control the prototype. However, the keyboard controls only offer basic controls: arrow keys to navigate the selection, enter to select / confirm, and backspace or delete key to go back.

Configuring the Layer Properties

Once you have imported the module and created a focus manager, you can now start importing or creating layers. Every layer that you create in your project from now on will have the following additional properties:

1. .isSelectable (boolean) this is a required property that indicates whether a layer accepts selection or not. For example, layer.isSelectable = true
2. .up .down .left .right (layer) these properties are optional and are used to specify the target layer to move the selection to when any of the directional buttons (also see nearest neighbor logic). For example: selectableLayer.down = anotherSelectableLayer

In addition, you can also override some of the default properties set by the focus manager object for an individual layer.

selectableLayer.selectionBorder = false
selectableLayer.selectionBorderWidth = 5
selectableLayer.selectionBorderColor = "rgba(255,255,255,0.5)"
selectableLayer.states.on =
     scale: 1.1
 selectableLayer.states.off =
     scale: 1
selectableLayer.isSelectable = true

Note: Generally, it is a good idea to specify the on / off states before the isSelectable property is set (as shown above). The reason is that whenever the isSelectable property is set, it automatically switches the layer to its off state. However, there is absolutely no harm in setting theisSelectable property before declaring the states. Just make sure you switch the layer to its off state manually by using selectableLayer.stateSwitch("off").

Nearest Neighbor Logic

One of my favorite things about the module is the nearest neighbor logic.

The nearest neighbor logic automatically calculates which layer to move the selection to when a directional button is pressed.

For example, if the left button is pressed, the focusManager will find the nearest visible, selectable layer to the left of the currently selected layer and move the selection to it.

The nearest neighbor logic is applied by default, and can be overriden by manually specifying the .up .down .left .right properties for a layer. For example consider the 4 layers arranged in a pattern as shown below, and the leftLayer selected by default.

Nearest neighbor logic : Caveats & Exceptions.

On pressing the right arrow, one would expect the selection to move to the rightLayer. However, the nearest neighbor algorithm would want to move the selection to the topLayer since it is the closest to leftLayer in terms of distance. Similarly, if you press down on the topLayer, the selection might move to either the rightLayer or the leftLayer.

To override this behavior you can manually specify the targets for the four layers:

topLayer.down = bottomLayer
bottomLayer.up = topLayer
LeftLayer.right = rightLayer
rightLayer.left = leftLayer

You can also specify null as target if you don’t want the selection to move in a direction at all.

Events and Event Listeners

Alright, so far we have learnt how to create a focusManager object, set up our layers, and have the selection move around between the layers.

Now, we need to add more interactivity to our prototype and this is where events come in.

The module emits events on the selected layer when a button on the keyboard or the gamepad is pressed. You can listen for these events on each selectable layer and specify event handlers to perform an action.

The following diagrams depict all the button press events that are generated for the PS4 and XB1 controllers. I have tried to keep the event names as close to the controller button names as possible so that they are easy to remember.

Event map for XB1 and PS4 controllers

For example, the following code generates an event when the up button is pressed on thebuttonLayer

buttonLayer.isSelectable = true
buttonLayer.on "up", ->
     print "up button pressed"

Note: do not confuse the layer properties .up, .down, .left, .right (used to specify navigation targets) with the .on “up”, .on “down”, .on “left” and .on “right” events (fired when a key is pressed).

The keyboard controls are available even when a controller is not attached. The arrow keys emit up, down, left, right events; the enter key emitscross, a, andenterevents; and the backspace/delete key emits circle, b, and back events.

Blur & Focus events

In addition to the button press events, layers also emit blur and focus events when they are selected or deselected. You can listen for these events as shown below:

selectableLayer.isSelectable = true
selectableLayer.name = "buttonLayer"
selectableLayer.on "focus", ->
     print this.name," is selected"
selectableLayer.on "blur", ->
     print this.name," is deselected"

Output when selected: buttonLayer is selected.
Output when deselected: buttonLayer is deselected.

Selection change event

A selection change event is emitted every time the selection changes. Unlike other events this event is generated on the focusManager object.

selectableLayer.isSelectable = true
selectableLayer.name = "buttonLayer"
focusManager.selectedItem = selectableLayer
focusManager.on "change:selection", (layer) ->
     print "selection changed to ", layer

Output: selection changed to buttonLayer.

Event Propagation

Listening to events on individual layers is fine, but you can also listen for an event on a group of layers.

Consider a scenario where pressing a button anywhere on the current screen takes you to a previous screen, and the current screen has 40 selectable layers on it as shown below.

Layer hierarchy for 40 selectable layers

While you could create an event listeners for each of the 40 selectable layers, it would be way more efficient to create an event listener on a single parent layer in the hierarchy.

Every time a button is pressed and an event is generated on a selectable layer, the focus manager also generates the same event on each of its parent layers. This continues all the way to the top until the window.document level. This process is called event propagation.

In our example above, If a circle event is generated on selectableLayer40, it can also be captured on the content, scrollComponent and the flowComponent parent layers, as well as the window.document object. For example:

selectableLayer40.name = "buttonLayer40"
flowComponent.on "circle", (selection) ->
     print "circle event generated on ", selection.name

Output: circle event generated on buttonLayer40

For the document object, the event handler format is slightly different:

selectableLayer40.name = "buttonLayer40"
document.addEventListener "circle", (event) ->
     print "circle event generated on ", event.detail

Output: circle event generated on buttonLayer

Note: if you want a layer to NOT propagate events to its parent layers, you can set the .propagateEvents property for that layer. For example:

selectableLayer40.propagateEvents = false

buttonPress Event

Consider another scenario where pressing ANY button on a layer performs an action. From what we have learnt so far, the only way to do this would be to create an event listener for ALL possible buttons, and then have each event listener carry out the same function. That doesn’t sound right!

Thankfully, in addition to the individual button events, we also have a combined buttonPress event. This event is fired every time an input is received from the controller (or the keyboard). Just like every other event this event starts from the originating layer and propagates all the way to the top.

To capture this event on the layer:

selectableLayer20.name = "buttonLayer"
selectableLayer20.on "buttonPress", (button) ->
     print button, " was pressed on ", this.name

Output: circle was pressed on buttonLayer

To capture the event on a parent layer:

flow.name = "myFlowComponent"
flow.on "buttonPress", (button, layer) ->
     print button, " pressed on ", layer.name

Output: circle was pressed on buttonLayer

To capture the buttonPress event on the document object:

selectableLayer20.name = "buttonLayer"
document.addEventListener "buttonPress", (event) ->
     print event.detail.layer

Output: buttonLayer

Conclusion

I hope that you find this module useful and enjoy using it as much as I did building it. That being said, the module is far from perfect. I hope to keep refining it while adding more features, and making the code more efficient as I continue learning.

If you have any feedback, I would love to hear about it in the comments. If you liked this module, tell your friends about it and share your awesome work in the amazing Framer community on Facebook.

Thank you for reading!

Credits and Shout Outs

Credit where credit is due!

Thanks to Emil Widlund for also tackling this problem before me and sharing his approach in form of the Framer Joystick module. I would highly recommend checking it out if you want to see a different take on the same problem albeit with a slightly more technical and what may also be a more robust solution to the same problem.

Lastly, A big shout out to Jordan Robert Dobson for inspiring me to build this module, and being an amazing sounding board, and an all round awesome prototyper.

Resources and Useful Links

• Download link for the demo project
• Github repo for the module