Launching BR Radio on CarPlay Audio

How we brought the rewind feature into our users’ cars

BR Radio is the new mobile radio application of BR. This post will outline how we approached adding iOS 14 CarPlay support, e.g. the feature to rewind the live program. Here’s how our developers made it work.

BR Radio Home Screen (station selection)

Update 21–11–30: We made some changes to handling remote images for CPListItems (See relevant section)

Table of contents

1. Embracing our rewind feature with the new CarPlay
2. The bumpy road of entitlements
3. Show progress and current event interval in station selection
4. Adding carPlayImage for correct remote image scaling
5. Handle regional stream selection
6. Initiate Playback
7. Adding custom now playing buttons
8. Using the “AlbumArtistButton”
9. Using the Up Next Button
10. Handle tapping the image row title
11. Handle tapping the list image row item
12. Fixing non square images
13. Alert helper

In our app users can listen to all their favorite radio stations in one place, access the program schedule, the broadcasts, the title information and rewind the live program. In the following article you can read why we made certain design decisions and how we implemented some of the features while highlighting the things we have learned along the way.

When we were watching the WWDC session “Accelerate your app with CarPlay” we got pretty excited about the new features for CarPlay Audio. We have already implemented basic CarPlay support using the MPPlayableContentDelegate/MPPlayableContentDataSource from the MediaPlayer framework but the dedicated CarPlay framework is a vast improvement. Previously, you were required to declare the entire app structure upfront with nested MPContentItems which lead to a lot of index path juggling and made any non-trivial navigation a pain to develop and maintain.

Now, CarPlay provides a view-controller-like navigation api with functions like pushTemplate and presentTemplate. This way, every template (aka view controller) is self contained and only responsible for its own content.

Even more appealing is the fact that there are now way more customization options in the now playing screen (called CPNowPlayingTemplate). While pre iOS 14 CarPlay basically just mirrored what iOS was showing in the now playing notification center, you can now have custom playback controls and present new templates when the "album/artist title" or the "Up Next" buttons are tapped.

Different components in now playing screen

CarPlay also gives us some new templates like alerts and grids and a new CPListImageRowItem to show a collection of images within a single row. Unfortunately, audio apps don't have access (i.e. crash at runtime) to action sheets or the information template which might be available in a later release.

Templates overview (see documentation page 16)

Embracing our rewind feature with the new CarPlay

The one thing that makes our radio app special is the possibility to rewind within the last 5 hours. While implementing this feature we added MPSkipIntervalCommands to rewind/fast-forward a certain amount of seconds from the notification center.

Since the now playing screen uses the same shared MPRemoteCommandCenter to populate its playback controls, we got this functionality for free. But while the control is nice for a quick "what did she/he say?" it's definitely not a very pleasant user experience having to tap the rewind button multiple times to go back to a certain point in time (especially considering that the driver should focus on the traffic).

Now playing screen on iOS 13

So for our new implementation we set the following goals:

• Use the “UpNextButton” to show all programs¹ within the past 5h and the selectable audio elements² for the current program
• Use the “AlbumArtistButton” to show selectable audio elements for the current program
• Use a custom news button in the now playing screen to seek to the latest news
• Use a custom “live” button to jump back to live (if rewound)
• Show progress and current event interval in station selection
• Handle regional stream selection in now playing screen

[1]: programs are what you might call an episode on tv. In our context they contain the metadata (like title and image) and are associated with an event which knows when the corresponding program was broadcasted (start/end).

[2]: audio elements are the more granular part within programs and provide, for example, information about music title/artist or when a news segment starts

The bumpy road of entitlements

Since we already had the CarPlay entitlement (note: the com.apple.developer.playable-content) we thought we could just jump right into the code skipping the often tedious process of requesting another one (surely it would be recycled or something right? 😅). Assuming you have already enabled scene support in your app, the changes to your Info.plist are pretty much the same as stated in the documentation (see Build your CarPlay app):

Note: We initially thought that we had to set UIApplicationSupportsMultipleScenes to true since the CarPlay scene is like a new window, but since it still behaves likes an external screen this doesn't seem to be the case.

Now we have to link the CarPlay framework. If your deployment target is lower than iOS 14, make sure to link weakly:

Choose Optional in the Link Binary With Libraries phase

In the entitlements file you add the new key for carplay-audio (alongside com.apple.developer.playable-content if you also support CarPlay on pre iOS 14).

Now create a CarPlaySceneDelegate NSObject subclass that conforms to CPTemplateApplicationSceneDelegate and set a random template as the root template on the interface controller provided by templateApplicationScene(_:didConnect:to:) and run the app. The CarPlay simulator can be found here

Show CarPlay in a second window

We do not recommend using the simulator for anything but playing around with the framework since it does not correctly simulate the behavior on the real device (especially the now playing screen). At this point fellow developers jokingly like to ask: “So do we get a car as a testing device?” The answer is “No” and that makes sense for several reasons. For one, I wouldn’t want to leave my comfy desk whenever a CarPlay bug report comes in 😄.

So instead, we advise getting a car radio (we have a Sony XAV-AX1005DB) with CarPlay support and hooking it up with a small microphone (“Hey Siri”) and some output device to check whether the audio is actually playing when the player indicates it.

CarPlay compatible test device

Following our own advice, we wanted to run the code on the physical device right after the minimal scaffolding was done so we plugged in an iPhone, hit Run and… Boom💥

Entitlement missing in provisioning profile

#HelloDarknessMyOldFriend

So there is a new entitlement for carplay-audio and we did have to issue a new request for it. Your team agent can do that here. After a few days we also contacted them via the old email address that we initially received a message from (pre iOS 14 CarPlay).

In the end, we don’t know whether the additional message accelerated the process but we received an answer within 10 days 🥳. If you don’t have CarPlay support yet and want to support CarPlay + Media Player frameworks we suggest making that clear in your request. We can’t tell if you are automatically assigned 3 entitlements (CarPlay, Media Player, CarPlay+MediaPlayer) or if we just got that because we already had the old one.

Supercharged with the new entitlements hop over to the developer console, (re)create your provisioning profiles and make sure to include the entitlement from the dropdown before hitting save.

Select the correct entitlement

Now that signing is working, let’s see how we went about implementing the goals we have set in the beginning.

Show progress and current event interval in station selection

Our root template should look pretty similar to the station selection we had in the iOS application. The obvious choice is the list template with a single section

Whenever an iPhone is connected with a CarPlay compatible device, the scene delegate’s templateApplicationScene(_:didConnect:) is called which in turn calls the connect function on the TemplateManager. This is the same approach that Apple took in their sample project CarPlay Music. At the end of the connect function (#2) we immediately transition to the now playing screen when playback on the iPhone is already in progress. This is undocumented behavior but it seems to be widely adopted in CarPlay audio applications.

The updateStationSelectionItemsFromCache (#1) is called from several points in the app and it's responsible for creating CPListItems for each station. Let's take a look at the implementation:

Let’s break it down:

1. We wanted to have the same order of stations as defined by the user in the iOS app.
2. Since Apple just loves exposing images via a synchronous image api when it comes to anything related to the media player we have created a convenience initializer for CPListItem to take care of the image fetching and also exposing an identifier which is the station id in our case.

3. Unlike on the iPhone we only have two lines per item so we combined the program’s kicker and the event interval.

4. Since we get the station logo’s urls via our backend we have to select the correct light/dark version at runtime by checking the interface controller’s carTraitCollection. Even though the documentation states "Don’t use other parameters [than the displayScale] in the carTraitCollection" we went the easy way³ of just using the correct version depending on the current interface style. Note that you cannot use UITraitCollection.current since it will use the iPhone's trait collection.

5. We set the playback progress depending on how much of the program has already been played. Notice that a progress of 0.0 doesn't render anything and a value of 1.0 will just render a ☑️ instead of a completely filled bar.

6. In order to show the isPlaying indicator, the isPlaying property needs to be set correctly. This will render an animated or still image of an equalizer when the player is playing or paused, respectively.

7. Every list item has a handler closure that is called with the corresponding item and a completion handler whenever the user selects a row from the list. This completion handler must be called as soon as possible on the main thread. If you forget to call it, an NSInternalInconsistencyException will remind you 🙃.

[3]: If you want your light/dark image behave like one coming from an asset catalog (switch when interface style changes) you have to create a UIImage() and register both versions with the underlying imageAsset. In that case you have to load both images before setting the combined image on the list item.

Here is the result

Station selection screen

We currently call updateStationSelectionItemsFromCache every 30s or when any of the stations' current event changes. Unfortunately, we cannot update the progress values every second because this results in an ugly reload animation.

UPDATE (21–11–30): Adding `carPlayImage` for correct remote image scaling

While working on another project we noticed that the images in list items should actually be bigger and some investigation (including contacting DTS) pointed us in the right direction. Here is what we use to get correct image scaling:

The basic idea is to put the image in a UIImageAsset and take it out again which magically makes things work. Other than that we also crop it to a square and resize it to the maximum allowed image size.

Handle regional stream selection

At BR, we highly value regional and local content which is also reflected in our radio stations. Some of them have multiple regional streams so you can choose the content most relevant (close) to you. If you tap on a station without having previously selected a preferred region on the iPhone, we show a list of available regional streams to pick from. The action sheet would have been the perfect candidate for this but as noted before it’s unavailable for CarPlay audio apps. A list template it is then:

1. Pop the list template and initiate playback for the station.
2. You might think “this code is pretty obvious, what’s there to explain?”. Well, we went down the rabbit hole caused by undocumented behavior which is why we wanted to share our findings. First, why on earth is there no .checkmark on CPListItemAccessoryType 🧐? Fine, let's use an SF Symbol then. What? It renders nothing? Maybe we have to use the carTraitCollection for the UIImage(systemName:compatibleWith:) initializer? Uh-uh. It turns out SF Symbols only seem to work as tab images. Alright, but we're on iOS 14, can't we at least use SVG images in the asset catalog? Nope. We tried every combination of Appearances, Devices and Renders as in the asset catalog but the tinting just didn't work out. While reading the documentation again we interpreted "provide image resources that are display-ready" as "tinting doesn't work the way you want it to" so we went back to plain PNGs which now looks like this:

Sample CarPlay image asset

The documentation doesn’t provide guidance on the size of disclosure indicator images so we went for 20pt x 20pt single-color square PNGs. Only with this setup we could get tinting during row highlight working (previously it would have been rendered white here).

Tinted disclosure indicator

If you look back at the code of #1 you see that we pass the interface controller’s car trait collection to receive the correct image. This will include the “Car Play” device trait and the correct display scale.

Apple’s sample code is actually incorrect since they just use the same assets for iPhone and CarPlay (fine) and don’t pass any trait collection. This results in the iPhone’s main screen’s display scale for the asset lookup and whenever this value differs from the car radio’s display scale the wrong image is resolved.

Initiate Playback

Previous code samples have already shown two invocations of initiatePlayback(for:) so let's take a quick look at the implementation:

1. If we don’t have a preferred stream for this station we present the stream selection shown before.
2. If the user picked the same station and livestream we just show the now playing template. This occurs when navigating back from the now playing screen and selecting the same station again.
3. If a new station or different regional livestream for the same station was selected we tell the player to initiate playback.

We don’t immediately show the now playing screen in #3 because this would only account for playback initiated from CarPlay. Since the user could also pick a station from the iPhone our use Siri to start playback our CarPlay UI would be out of sync. Therefore, we use a delegate function from our player for this:

1. We store the last selected station in a currentStation property. If the station didn't change we return early.
2. If the new station is different from the old one, we toggle the old item’s isPlaying property.
3. We set the new item’s isPlaying property to true.
4. Present the now playing template. This will now also work if a radio station was launched from Siri.

Showing the now playing screen was actually not as trivial as it may sound, so we built a helper function for that:

1. Don’t do anything if the now playing template is already visible.
2. This is a common workaround when the now playing screen doesn’t appear on the simulator. The right combination of setting the player rate often and handling remote events in our player module made this call superfluous but we left it here for fellow developers.
3. If there are other templates pushed on top, we just pop to the now playing template.
4. Now playing screen is still not visible so we just push it on the stack.

Adding custom now playing buttons

One of the coolest features of the new CarPlay framework is that you can use custom buttons in addition to the MPRemoteCommands. We wanted to show a news button to jump to the latest news (we broadcast news ~ every hour) alongside a stream selection button and a button to jump back to the live position (if rewound). For the news we have two different buttons with ⏪ or ⏩ symbols depending on whether the time to seek to is in the past or the future relative to the current playhead. For the other cases we just have single buttons:

There is a unit test that would fail if any image isn’t available so force unwrapping is ok here 😉. For the additional buttons we created a helper struct:

The current config is stored in a property and updates the now playing buttons whenever it changes:

Whenever we need to update the config we call the updateNowPlayingButtonConfig. It evaluates the current state of the player to determine the correct values of the properties. Additionally, we update the isStreamSelectionEnabled whenever the currentStation property observer is called:

This function is called every time the player’s current time changes and just before the now playing template is shown. If the config hasn’t changed since the last call, the buttons just stay the same. We decided to only call updateNowPlayingButtons(with:) when necessary because even setting the same buttons would lead to ugly re-rendering and unpleasant animations. The implementation is just a switch over the different cases and collecting the corresponding buttons:

With this in place our now playing screen looks like this:

Now playing screen (left: Live, right: Rewound)

The first screenshot shows a livestream at the “live” position. The news button points to the past and the button for regional stream selection is enabled (it triggers the same list as shown before). The second screenshot shows the player seeked back 5 minutes so the “live” button is visible and the news button points to the future (current time is 06:59 PM and latest news are at 07:00 PM). Both screenshots already give a sneak peak at the final two features. Let’s check them out.

Using the “AlbumArtistButton”

When the isAlbumArtistButtonEnabled property on the shared now playing template is set to true, the last line of the now playing screen turns into a button. According to the documentation, this can be used "to present more information about the current track". Naturally, most traditional music apps show all titles within the current album or recent albums/titles from the current artist. In BR Radio, the last line is the kicker of the currently playing program. Therefore, tapping the button should present a list of audio elements within the program:

1. This shouldn’t happen in practice but if we have no current event we show an alert (see Alert helper).
2. The items are all the audio elements in reversed order (newer items appear first).

Audio elements that cannot be seeked to are filtered out. Upon tapping an item we seek to the element’s start date. The implementation of seekToDateIfPossible(_:completion:) makes sure to pop back to the now playing screen.

Audio elements within the current program

Using the Up Next Button

Similar to the isAlbumArtistButtonEnabled there is an isUpNextButtonEnabled flag. When enabled, you can further customize the title to anything you like. Without this, our rewind feature wouldn't make any sense. The default English title of the button is Playing Next which couldn't be more wrong when trying to display the list of previous elements. So the first thing we do after enabling the button is to change its title to indicate that tapping it will reveal a rewind screen:

CPNowPlayingTemplate.shared.isUpNextButtonEnabled = true
CPNowPlayingTemplate.shared.upNextTitle = Localized.rewind()

So what do we do now? One idea was to list all audio elements in reversed order grouped by their associated program (using the event interval and program kicker as section title). This approach would have had several drawbacks. It’s very hard to reach elements that aren’t at the very top of the list. Also, we cannot scroll to a certain item. So if we are playing a previous program, the user would have to scroll through all future titles to reach the current and previous elements. That is, if the items are even presented. The list template has a limit (CPListTemplate.maximumItemCount) that can be as little as 12. Always take into account that some items might not be displayed. If you have an order of preference for certain items you should query the maximumItemCount value and use subsets of items if necessary.

It’s obvious that we cannot replicate the same flat list of elements like on iOS. Another idea would be to first present a list of programs on one screen (images, titles, event intervals) and then show the corresponding audio element upon tapping them. This way, the user would have to select a program before she/he sees any tappable audio elements.

In the end, we decided for a hybrid approach leveraging the new image row list item. It will appear in the first section and show the first programs that the user can seek into. The title is tappable and will present the full list of programs. In the next section we show the currently playing program to enable quick access to its audio elements.

Rewind screen (overview)

The second section is the same that we created for the album artist button. For the image row list item we created another convenience initializer since the api felt quite unintuitive. Actually, you can have multiple CPListImageRowItems in a single section but we’ve never seen this used in any app. To us, the UI looks like a section with multiple items aligned horizontally instead of vertically. The api of our initializer aligns mirrors this visual appearance:

1. We create a helper struct that holds the information for one item. The image is non optional because there is no text so the image is the only representation of the item.
2. For remote images you can provide a remoteUrl in combination with a placeholder that is used until the image has been fetched successfully (or if the fetch failed).
3. The imageRowItems parameter resembles the items parameter of the default CPListSection initializer.
4. The number of rendered images is not the same as in the images array of the CPListImageRowItem. Rather, only CPMaximumNumberOfGridImages are used (and are exposed via the item’s gridImages property) so we just slice the images array accordingly.
5. The listImageRowHandler of a CPListImageRowItem is called with the index of the tapped image which we’ll use to access the correct handler of our ImageItems.
6. The handler of the image row itself is invoked when the image row title text is tapped.
7. This is basically the same dance we did for the CPListItems. Note that we first access the gridImages of the row, reassign the downloaded image and then update the list item’s images (CPListImageRowItem.update(_:)).

Handle tapping the image row title

Tapping the Vergangene Sendungen inside the image row list item will present a list of all events that a user can seek within. The handler of the program rows is the same as described in the next paragraph:

A list of all programs within the last 5h

There is one thing to take into account here. Somewhere in the documentation you can find the following warning

There is a limit to the number of templates that you can push onto the screen. All apps are limited to pushing up to 5 templates in depth, including the root template.

When navigating to the audio elements of a program via the path mentioned here, we have root/now playing/rewind overview/all past programs/audio elements of program. So there may be no further nesting.

If you have a complex navigation flow in mind you might want to reconsider and focus on key features of your app instead since it doesn’t make sense to bury features deep within a navigation stack.

Handle tapping the list image row item

When the user taps on either an image in the image row list item or on one of the rows shown in the last screenshot, we show the list of audio elements for that particular program. We also added a shortcut to seek to the beginning of the show as the first item:

A list of audio elements within a program

Fixing non square images

One problem you might have already noticed is that all images look distorted. The reason for this is that CarPlay expects square images but we only have 16/9 images (for now). Until our backend provides us with square images we have two options:

• manually crop the image or
• render the non-square image in a square image context

For the first option we would have to decide for either a center or a top-left crop (some images contain text that’s top-left aligned). Both of those variants might remove important information from some images.

We went for the second option and use the following extension in all the CarPlay items’ convenience initializers that we have shown before:

Use 16/9 images for image row list item

Alert helper

For some errors we use a helper function to present alert templates. Due to limited space, an alert template doesn’t have a title and message like a UIAlertController. Rather, there is a titleVariants array to provide a list of titles sorted from most to least preferred. Since we have never used multiple titles, our final helper on the template manager only has a simple title parameter.

1. We only used .default actions so we could encode the actions as simple title+completion handler tuples.
2. After executing the handler of the caller, dismiss the alert (this is not automatic, not even with the .cancel action).
3. Since at least one action is required we always create an ok action with a .cancel style here.
4. This one is especially interesting: We want the ok/cancel action to appear last so we append it to the other actions. At one point we show an alert with one additional action but only one action is shown. The odd thing: it’s the ok/cancel action 🤪 What is happening? CarPlay checks the CPAlertTemplate.maximumActionCount value and if only one action is allowed it will use the first .cancel action in the array. Only if there is room for more, additional actions will be added. We have never seen a maximumActionCount greater than 1 so be prepared that your actions won't be visible to most users. If an action triggers a critical activity and you want to be sure it can be selected, you might want to consider pushing a list template instead (see stream selection).

Wow, that was quite a lot for a simple CarPlay application but we hope it fills some of the gaps missing in the documentation and that it helps other developers to learn from the mistakes we have made along the way.

CarPlay Audio was really fun to work with. Since the UI is so limited, we could focus on the features and how to package them within the components Apple provided for us. Although CarPlay on iOS 14 is way more powerful than the Media Player framework, there is definitely some room for improvement. We’ll just leave our wish list here in case someone from the CarPlay team reads this 🤓:

• Enable tint colors for templates (this doesn’t even work on the CPWindow)
• Only re-render necessary subviews of list items (progress change re-renders entire row)
• Allow higher resolution images (CPListItem.maximumImageSize seems too small most of the time)
• Allow text under CPListImageRowItems
• Run CarPlay on the iPhone for cars without modern radios (Android can do this)
• Make SF Symbols available everywhere UIImages are allowed (or document missing support)

BR Radio is currently in Beta and the new CarPlay implementation will soon be available on the App Store. We’d love to hear your feedback! If you have any questions feel free to write us a message 😉.

BR Radio is available for iOS and Android.

Contact

alexander.tuerk@br.de