tvOS 10: Getting Started with SpriteKit and Focus Engine

Almost a year ago Apple released the AppleTV 4 and for the first time opened up the platform for developers with the release of tvOS 9. Probably the most significant difference between tvOS and iOS is how the user interacts with the system. On iOS they mostly use their finger, but on tvOS the interaction is one step removed via the supplied remote, using a game controller or the iPhone ‘Remote’ app.

To manage this new input method Apple created something called the focus engine. Within the focus engine any one element on screen has the current focus, and focus moves between one element and another as the user navigates. If the user presses a button the input is directed to the currently focused element.

The spritekit-focusengine app is the currently focussed item

In tvOS 9 the focus engine was only supported in UIKit, thus leaving SpriteKit developers to roll their own systems. In tvOS 10 the focus engine has been extended to support SpriteKit, with the intention of making interactivity in SpriteKit games feel more in line with UIKit and TVML based applications.

I am currently porting my iOS game Mazy to tvOS and as part of that have adopted the new focus engine to power the menus for my game. I thought I would share my experience here, I hope you find it useful if you are embarking on a similar piece of work.

There are 2 WWDC videos that introduce the new focus engine integration in SpriteKit. Focus Interaction on tvOS (SpriteKit bit starts at 15:20, but if you are new to the focus engine it’s worth watching the whole video). The engine is also covered in What’s New in SpriteKit (focus bit starts at 39:15).

I have a github repo you can clone to follow along through the examples below. You can find it here. It requires the Xcode 8 beta due to requiring tvOS 10, it is also written using Swift 3. There’s a scene for each section below. Just update this line in GameViewController.swift to select the relevant scene:

let scene = SimpleMenuScene(size: view.frame.size)

SimpleMenuScene

Firstly lets get a simple menu working. SimpleMenuButton is a subclass of SKNode that will display a string and will change colour when focussed. We’ll start with 3 SimpleMenuButton’s in a vertical list. To make an item focusable it must implement UIFocusItem. SKNode already implements this so we override it in our SimpeMenuButton subclass like so:

override public var canBecomeFocused: Bool {
get {
return true
}
}

(Note that in Swift 2.3 canBecomeFocused is a function and not a property).

Now we need to handle the focus changing, we do that by implementing didUpdateFocus from UIFocusEnvironment. You can implement this on the item being focused or on one of its parents. Here I have implemented it on SimpleMenuScene

override func didUpdateFocus(in context: UIFocusUpdateContext, with coordinator: UIFocusAnimationCoordinator) {
let prevItem = context.previouslyFocusedItem
let nextItem = context.nextFocusedItem

if let prevButton = prevItem as? SimpleMenuButton {
prevButton.buttonDidLoseFocus()
}
if let nextButton = nextItem as? SimpleMenuButton {
nextButton.buttonDidGetFocus()
}
}

Run the project and you can see 3 buttons, the currently focused one in red. If you’re on the simulator you can use the arrow keys to move up/down between the items.

An important part of understanding the focus engine is to appreciate how it is making focus decisions. Fortunately there is a visual representation of the decisions being made available in Xcode. Start the app then set a breakpoint on didUpdateFocus. Make a focus change then perform a QuickLook (press space) on the ‘context’ variable.

You should see this:

Focus Engine searching for next focusable item

The currently selected focus item is highlighted in red. The two other buttons below are shown in purple. The important thing to note here is that we have not specifically set the size of these focusable areas, they are just the sizes of the SKNodes. Specifically they are the size returned by calculateAccumulatedFrame().

DiagonalMenuScene1

This scene is the same as SimpleMenuScene except the buttons have been placed on a diagonal like so:

Buttons on a diagonal

If you update GameViewController to point to DiagonalMenuScene1 and launch the app you will see that you cannot navigate between the items. If you set a breakpoint on didUpdateFocus you’ll see that it is never called apart from when the app first launches to set the initial focus.

This gives us a bit of a debugging headache as there doesn’t appear to be a visual way to debug this. However we do know what is happening thanks to what we saw in the visual debug in SimpleMenu scene.

When the focus engine receives a request to move ‘down’ from Button1 it looks for any nodes that intersect a rectangle extending down from the frame of Button1. This does not intersect with Button 2 or 3 and thus neither of them can be focussed.

DiagonalMenuScene2

To work around this problem a couple of options spring to mind. Either set a clear color background node behind each button wide enough to overlap, or override calculateAccumulatedFrame so that the frame widths overlap. When using a UIView setting the alpha to zero will exclude if from being selected by the focus engine, however in SpriteKit the node can still be selected (not sure if this is intentional or a bug…)

Let’s override calculateAccumulatedFrame like so in DiagonalMenuButton:

override func calculateAccumulatedFrame() -> CGRect {
let frame = super.calculateAccumulatedFrame()
return CGRect(origin: CGPoint(x: 0, y: frame.origin.y), size: CGSize(width: UIScreen.main.bounds.width, height: frame.size.height))
}

Switch the scene to DiagonalMenuScene2, which uses DiagonalMenuButton and then execute and set the breakpoint again and you can see the focus now works as expected.

Focus Engine with overridden calculateAccumulatedFrame

PositionedMenuScene1

I’m now going to warn you about something that I scratched my head about for a long time! Change the scene to PositionedMenuScene1.

Here the only difference is that the 3 buttons have been added to a ‘menuStack’ SKNode, and the position of menuStack has been set to be (100,100). Run the project and pull up the context QuickLook and you’ll see this:

Focus engine not aligning with SKNodes

Here you can see the alignment between where the focus engine thinks the nodes are and where they actually are is offset. What appears to be happening here is that the focus engine is not converting the position of the node into the coordinate space of the root view.

PositionedMenuScene2

Just to demonstrate the problems caused by this, change the scene to PositionedMenuScene2, this is the same as Scene1 but has a second ‘stack’ of buttons. If you run the project you’ll see that it’s not even possible to select items in the first stack, because the focus engine sees both stacks at the same location.

Focus engine alignment issues

PositionedMenuScene3

The workaround I have been using for this situation is to use ‘shadow’ nodes that are used by the focus engine, but are invisible to the user.

In PositionedMenuScene3 there is a new PositionedMenuButton. This has a function to add a ShadowMenuButton to the same location as itself, but directly attached to the top parent.

The ShadowMenuButton is then set to be interactive, and just passes the buttonDid*Focus functions back to PositionedMenuButton.

func addShadowNodeToTopParent() {

let shadowButton = ShadowMenuButton(size: self.children.first!.frame.size, positionedMenuButton: self)
  var topParent: SKNode = self
  while topParent.parent != nil {
topParent = topParent.parent!
}

shadowButton.position = topParent.convert(self.position, from: self.parent!)

shadowButton.isUserInteractionEnabled = true

topParent.addChild(shadowButton)
}

The ShadowMenuButton’s have been set to be blue here, but works just the same when clear.

Focus engine working with ‘Shadow’ SKNodes attached to SKScene directly

Preferred Focus Environments

Now we can successfully move focus in our menu there are just a couple of other topics I’d like to cover off in this post. Firstly ‘Preferred Focus Environments’.

You may have noticed in PositionedMenuScene3 that Button6 was the default highlighted button, this is achieved by letting the focus engine know what the PreferredFocusEnvironments are. When given a list of environments the focus engine will try each one in turn until it finds one that can be successfully focussed.

preferredFocusEnvionments is a property on the UIFocusEnvironment protocol. For the scene to be asked for its environments the main GameViewController needs to respond to preferredFocusEnvironment and give the scene as it’s preferred environment like so:

override var preferredFocusEnvironments: [UIFocusEnvironment] {
if let scene = currentScene {
return [scene]
} else {
return []
}
}

Then PositionedMenuScene3 can respond with a list of environments, here just the list of ShadowMenuButtons in reverse order

override var preferredFocusEnvironments: [UIFocusEnvironment] {
return buttons.map { $0.shadowButton! as UIFocusEnvironment }.reversed()
}

Handling Selection — SelectableMenuScene

The last thing I’m covering here is handling the user selection. What we want to detect is the user pressing down on the touchpad. To do this we need to detect a UIPress event.

The easiest way to to this is to leverage UITapGestureRecgonizer. By default it detects the ‘select’ button but you can adjust it to detect whichever buttons you are interested in.

Switch the GameViewController to use SelectableMenuScene. You can see the recognizer is added like so:

func addTapGestureRecognizer() {
let tapRecognizer = UITapGestureRecognizer(target: self, action: #selector(self.tapped(sender:)))
self.view?.addGestureRecognizer(tapRecognizer)
}

And when processing the tap we take advantage of UIScreen.main.focusedItem to get the currently focused item, and then pass the tap through the ShadowMenuButton to the PositionedMenuButton.

func tapped(sender:AnyObject) {
if let focussedItem = UIScreen.main.focusedItem as? ShadowMenuButton {
focussedItem.positionedMenuButton?.tapped()
}
}

The PositionedMenuButton simply prints out the text value of the SKLabelNode.

Thanks for reading this far!

I hope you’ve found this post useful, the focus engine in SpriteKit is brand new, and in a few places a bit weird! It took me a couple of days to get going with all of this, I hope the above has helped you on your way.

Feedback appreciated as always!

Show your support

Clapping shows how much you appreciated Gordon Johnston’s story.