Grouping Semantics in Jetpack Compose UI

The semantics modifiers let us change aspects of the semantics tree in Jetpack Compose UI — a representation of the UI that’s helpful for accessibility services and the testing framework.

We can use these modifiers to group a number of widgets into one logical element, making it faster to navigate a list of similar elements using an accessibility service like TalkBack.

For example, in YouTube Music, the current playlist is represented as a list of tracks, each with:

• a title
• artist
• track length
• play/pause action
• see more/menu action
• (and state — is playing)

Screenreaders like TalkBack will focus on each element with an intrinsic description (text) or action, which means that navigating from one track to the next track will cost users ~5 gestures.

Re-writing the semantics

We can re-write the semantics information for the track so that it’s represented as a single logical element:

Using the clearAndSetSemantics() modifier is necessary; it’s not enough to use the semantics() modifier in this case.

Modifier.clearAndSetSemantics()

Modifier.clearAndSetSemantics() will clear the semantics information for all descendant nodes and update the current node with the given properties.

The documentation for this function says:

this can be used to provide a more polished screen-reader experience: for example, clearing the semantics of a group of tiny buttons, and setting equivalent actions on the card containing them.

That’s exactly what we were after! However, there are implications to clearing descendant nodes’ semantics:

• any text-based descendants will no longer be focusable nor included in the content description. Our content description has to be a faithful description of the whole row.
• any clickable descendants will no longer be focusable. We have to expose these actions (customActions) on the row instead.

Modifier.semantics()

Modifier.semantics() lets us add semantic information to the current node. Our aim was to simplify the representation of each track to a single node, so this wouldn’t have helped here.

In fact, we’d have made it worse by making the entire row focusable (in addition to what we already had).

Modifier.semantics(mergeDescendants = true)

Using Modifier.semantics(mergeDescendants = true) is slightly more useful because it will reduce the number of focusable elements.

Setting mergeDescendants to true will:

• remove all descendant nodes (unless they’re also using mergeDescendants = true)
• try to merge properties together where possible

Elements with a clickable modifier won’t be removed because they use mergeDescendants = true. In our case, this means the play/pause and the menu actions will still be focused.

The title, artist and track length nodes aren’t clickable. Here, there’ll be an attempt to merge the semantic information from these nodes, which will result in duplicated information because we set the content description manually.

Recap

Modifier.clearAndSetSemantics() gives us a lot of control over the semantic representation of a node and its descendants in Compose UI. While it’s more powerful than Modifier.semantics(), we also need to be more careful with it.

As a rule of thumb, consider reserving its use for elements in a collection (lists or grids) and always test the behavior.

If you liked this post, have any questions, comments or corrections, please reach out on Twitter.