Accessible SwiftUI for easy UI testing

Rob Sturgeon
Dec 17, 2019 · 12 min read
Image for post
Image for post

Apple’s VoiceOver feature was said to be the most popular mobile screen reader for blind people in 2018. The company marked the occasion by telling the story of Scott Leason, a blind surfer. At their developer conference WWDC in 2019, Apple emphasised the importance of accessibility again when they featured blind architect Chris Downey as one of their guest speakers.

In the United States, the Department of Veteran Affairs gives iPhones to 70% of their blind veterans as a result of how easy the VoiceOver feature is to use.

Developers already have a huge incentive to add strings called accessibility identifiers to their UI elements, as they reduce the flakiness and verbosity of UI tests. Even if you don’t set out to make the world’s most accessible app for disabled people, there is no end to the stability improvements you can gain from thoroughly testing your app, and there’s no reason why these two goals can’t be achieved simultaneously.

The more accessible your app is to disabled people, the more stable you can make it with UI testing.

Unfortunately accessibility identifiers used for UI tests are NOT read by VoiceOver. There are other attributes, such as accessibility labels, values and traits that will help us in that regard. In the absence of an accessibility identifier, UI tests can find an element by the label just as easily. By default in SwiftUI, this is set to the string that you pass as a parameter when constructing a UI element.

This could be the content of a Text element, or the filename of an Image element.

Storyboards allow you to add an accessibility identifier in the Class Inspector when a UI element is selected. SwiftUI can make this goal a little bit more complicated. The clean simplicity of this new framework has astounded the developer community, so it can be tempting to preserve this by reducing the number of modifiers you add to each individual view. However, accessibility can be improved with a single modifier, the aptly named .accessibility(label: Text("Your label here")) modifier.

When a view has too many modifiers, the design of SwiftUI already encourages the practice of tidying them away into separate files.

Modifiers can also be combined into a single View extension, which is simply a function that returns ‘some View’ with whatever modifiers you add to it. Instead of adding .modifier(Title()) to your View, you can then simply add .titleStyle(), as in this example by Hacking With Swift. This syntax is much more consistent with the provided modifiers, and it lets you pass parameters to the function while a custom ViewModifier struct does not.

Here’s my example of how you can make a better modifier for accessibility.

Now that I mention it, I might as well recommend that you check out Hacking With Swift’s vast library of free SwiftUI resources. It has been my favourite source of SwiftUI tutorials and examples, and I have no doubt that it could help you too.

It’s time to start the tutorial!

Text views and buttons with titles are easily accessible in SwiftUI and UIKit, because their accessibility label automatically matches the text they display. I’m not going to waste time adding these to my SwiftUI example, because there isn’t a lot you need to do to improve their accessibility. Sure, you could set an additional accessibility label that makes it clear what the text is or what the button does, leaving the label to describe the content actually visible on the screen, but let’s make it hard for ourselves.

The UI for the app

  • A coloured Button with no text that changes colour every time it is pressed
  • An Image showing “large logo” that changes to “small logo” when the coloured button is first pressed
  • A Slider with a Text that displays its current value
  • A Stepper with a Text that displays its current value
  • A Toggle that shows and hides its own Text label
ContentView has all the UI elements we need for our UI tests

Let’s go through what I’ve added here.

State properties

Marking my properties with State tells SwiftUI that we want it to pay attention to changes made to these properties. After all, they are used as the values that our controls change and display, so we want to know that the UI will be redrawn whenever these properties are changed. You actually can’t modify var properties in SwiftUI without this, as you will get an error saying ‘self is immutable’ when you try to modify the variable in a button action.

First we have a colour property that stores what colour the button will be, an index for the current colour, and an array of colours to loop through. Then we have largeLogo, which is true when the logo is initially large but changes to false when the button is first pressed and the logo becomes small.

Finally we have sliderValue, stepperValue and toggleIsOn, the values that the other controls will hold.

Form

Embedding our views in a Form makes all of the controls look nice, as well as adding the much needed ability to scroll when the content is too large for the screen. Making scrollable content in SwiftUI couldn’t be easier. There’s no need to waste time setting constraints only to be told ‘scrollable content size is ambiguous’ like you would with a UIScrollView in UIKit.

It just works!

Identifying our controls

In his post SwiftUI Accessibility: Named Controls, iOS engineer Rob Whitaker gives an example of what happens when controls are labelled poorly. Since VoiceOver uses reads elements in a natural reading direction, two toggles with labels above them would have their labels read first, and then the toggles would both be referred to as ‘toggle’, making it difficult to know which action you were taking by toggling them.

Note that accessibility identifiers are NOT read by VoiceOver, and only have value for UI testing.

Coloured Button

All controls can be named in the closures when they are created. It would be easy to give the button a Text name which would be easily found by our UI tests. If we made it an image, it could be found by searching for the image filename. Images can also be given names, much in the same way as a button can. But a custom button might look like anything, and this example is designed to make it difficult to select by UI tests. I have a label that describes what pressing the button does, and a value that describes the current colour.

VoiceOver identifies the coloured Rectangle as ‘Next colour, red’, because the label is read before the value. If I was to add an accessibility hint, this would be read at the end. Note that I have added an identifier that is different from the VoiceOver label, but I could use either for my UI tests. “Next colour” or “colour” can both be used by UI tests, although “colour” would arguably not describe the action of the button sufficiently for VoiceOver users.

The identifier is also something static that shouldn’t change, whereas the action of the button (and therefore the label) may change.

This would mean our UI test would have to know the correct label for the button at all times.

The Image changes from ‘large logo’ to ‘small logo’ when the coloured button is pressed. You may not think that the size of the logo would be important to a VoiceOver user, in which case you could add an accessibility label that just says “logo”. This would would be read instead of the filename, and would therefore not inform VoiceOver user about the specific content of the Image. You could use the label to describe the general category of the image and the hint to describe it more specifically.

For instance if your Image was a dog that changes to a cat, you could use a label of ‘Animal’ with a hint that changes from ‘Dog’ to ‘Cat’.

Slider and Text in an HStack

However, it seems that Slider is the only control that doesn’t display its name by default on iOS. As of Xcode version 11.3, naming a slider will not give your UI tests access to the name, nor will it display it to the user. Don’t trust the official Apple documentation either, which currently says:

“The appearance and interaction of Slider is determined at runtime and can be customized. Slider uses the SliderStyle provided by its environment to define its appearance and interaction. Each platform provides a default style that reflects the platform style, but providing a new style will redefine all Slider instances within that environment.”

It seems that the only part of this that is true in Xcode 11.3 is that Slider uses the SliderStyle provided by its environment, because it is visible on MacOS but not iOS. In order to make the name visible, we would need to create a custom SliderStyle. Creating custom styles for SwiftUI controls is detailed in the post SwiftUI Custom Styling by The SwiftUI Lab. If you go to that post, you might see the comment I wrote asking if there was a way that we could customise a Slider, as the documentation seems to indicate this.

The author of the post replied saying:

There is no such thing as SliderStyle. I have seen other references to non-existing types, so I am not surprised. For the time being, afaik, there is no way of customizing the Slider. The only customization is the one that SwiftUI does according to the running platform.

I am leaving the Slider’s name in the closure, as it still names the control for VoiceOver. However, since Stepper always displays the name to the left of the control, I have placed the Slider in an HStack with a Text that mirrors the content of the Slider’s name. Hopefully this will change when SliderStyle becomes accessible to us, as we would then be able to decide where the name goes as The SwiftUI Lab was able to with a Toggle.

Stepper

The Stepper displays its name to the left just as our Slider in an HStack does. Note how I’m prefacing every control name with the type of control it is. In an actual app, this would be a good place to say what the control actually does. For instance, in a ticket app your stepper might be prefaced by “tickets:” to indicate that you can use the Stepper to buy multiple tickets of the same type.

Toggle

Like with the Button, I have intentionally made it difficult for our UI test to find the name for the Toggle. The Text of the name has an empty string until the toggle is switched, at which point the name changes to “Toggle label”. I’ve added an accessibility label with the same text, as VoiceOver does not currently read the name even when the string isn’t empty. Note that names given within the closure of a control cannot have their own accessibility identifier.

UI tests will still not be able to find it and check the content.

Image for post
Image for post
See more of what our Digital team does by visiting the Twinkl Reality page

Writing a UI test

If you’ve never written a UI test before, this paragraph is for you! Go to the Test navigator in the left side panel. The tab icon is a diamond with a horizontal line inside it, and you’ll find right at the top. Now in the bottom left corner of the Test navigator (and therefore of Xcode itself), click the + button and click New UI Test Target. This will be the automation runner that launches first and then interacts with your app like a person would. You can call it anything, but the convention is YourProjectNameUITests. Unfortuantely my project name was AccessibleUITesting, so I ended up calling mine AccessibleUITestingUITests.

You don’t need to repeat my mistake if you don’t want to.

Then create a New UI Test Class with the same + button in the Test navigator, and you can call this class the same as your test target if you want to.

This is how I usually start a test class

When I write UI tests, I tend to create a global variable for the representation of the current app called app, which stores the value of XCUIApplication. This makes it possible to easily query the app for elements on the screen by using the property name app instead of this far longer name. It’s also useful to launch the app in the setUp function, as this removes the need to launch the app manually for each individual test. I have deleted the default comments, as well as the tearDown function that won’t be necessary for this test.

Testing the coloured button

UI tests don’t have the same direct access to our source code that unit tests have. This presents a challenge for visual changes like colours, which are difficult to detect with conventional UI testing techniques. The easiest way to check what colour is being displayed on the button is to set the acccessibility label to the colour name. Unlike in UIKit, SwiftUI Images are automatically given an accessibility label that corresponds to the name of the image they are displaying. This makes it very easy to access the current image being displayed, as can be seen here.

This test checks that the button changes both its own colour and the image displayed below it

Testing the slider

Since the Slider is representing the values between 0 and 1, we expect to be able to move the Slider to these points and see the changes reflected in the Slider’s label. You may notice that I have commented out a line every time I move the Slider. This line ought to check the label that we passed to the Slider when we declared it, but when the test will fail on these lines if they are uncommented. I expect that this bug will be fixed in a future version, as it appears that the accessibility label of the slider is always an empty string despite the fact that we have given it one.

This test checks that moving the slider to 50% causes the label to change accordingly

Testing the stepper

This test checks that incrementing and decrementing the stepper causes the label to change accordingly

In UIKit, testing a stepper is relatively easy. Every UIStepper contains two buttons in its hierarchy: one called “decrement” which decreases the value, and another called “increment” which increases the value. As of Xcode version 11.3, these buttons do not exist and therefore cannot be found for a tap interaction. The best workaround I could figure out for this is using a coordinate with a normalised offset. With these coordinates, the origin at the top left of a view is considered to have an offset of (0, 0) while the bottom right corner has an offset of (1, 1). The decimal offset essentially describes the width and height as 1, so 0.5 in either axis would be centred.

A number greater than 1 in any direction would be outside of the view.

Unfortunately the offset for this test seems to include the entire row of our form as being part of the stepper, which is why I have the decrement as (0.8, 0.5) and the increment as (0.9, 0.5). This may be different on devices and orientations that have different widths, but it’s the best way I can figure out until Apple add the decrement and increment buttons to the view heirarchy.

Testing the toggle

This test checks that switching the toggle causes the label to be displayed correctly

The name that we gave the Toggle when we created it should not be displayed at first, because its default value is an empty string. Although it is visually shown to the user, it is inaccessible to VoiceOver and UI tests. That is why I put the same string into an accessibility label, which is why we can access it through the toggle’s label property.

Next steps

Don’t rely on UI tests to tell you what is going to be read by VoiceOver. According to UI tests, an identifier is a perfectly accessible way to find an element on the screen. But it is in fact accessibility labels that will help your tests and disabled people alike. If you write good labels, you may not need identifiers at all. This really only scratches the surface of what you can do with accessibility in SwiftUI.

For more information on what you can do, visit Rob Whitaker’s series on SwiftUI Accessibility.

The best way to find out test your app’s accessibility is to activate VoiceOver on a physical device (it is unavailable in the iOS Simulator). On versions older than iOS 13, go to Settings > General > Accessibility > VoiceOver. On iOS 13, you no longer need to go to General to find the Accessibility settings menu. Tap on your UI elements to have VoiceOver read them, and double-tap to press a button. Does the information you are hearing describe what you are seeing clearly enough? Swipe to go to the next UI element. Tap to hear it again.

SwiftUI is still very new and continually evolving.

The accessibility and testability is only going to get better!

Let me know what you think in the responses below.

Image for post
Image for post

About Rob Sturgeon

Rob is a Placement App Developer at Twinkl. He’s currently working on their iOS and Android apps, enabling teachers to view, organise and download resources on the go.

READ MORE:

How SwiftUI helps kids create their first iOS apps

Make a SwiftUI synthesiser with AudioKit

Twinkl Educational Publishers

Updates and insights from the team at Twinkl

Rob Sturgeon

Written by

An iOS developer who writes about gadgets, startups and cybersecurity. Swift programming tutorials and SwiftUI documentation too. robsturgeon.com

Twinkl Educational Publishers

Updates and insights from the team at Twinkl

Rob Sturgeon

Written by

An iOS developer who writes about gadgets, startups and cybersecurity. Swift programming tutorials and SwiftUI documentation too. robsturgeon.com

Twinkl Educational Publishers

Updates and insights from the team at Twinkl

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store