Learning AVFoundation — Part 3 (Playback -Playing Assets)

Continued: Part 2: https://medium.com/@divya.nayak/learning-avfoundation-part2-understanding-and-using-avassets-ef70e64d7ade

Playing an AVAsset needs a Player. Player manages to play a media, pause a media and seek to a particular time. So AVFoundation has AVPlayer to play a single asset. We have so many classes that are coming on our way that might confuse. Let's make this easier.

Players :

Players know the exact state of your playback. We have,

AVPlayer: Player object is used to play a single asset.

AVQueuePlayer: This is a subclass of AVPlayer itself. As the name says this is meant to play a number of items in a sequence. And hence we have options to add /remove the items that has to play.

So a player needs something to know what it has to play right?.

AVAsset!!!???? NAAAaaah! 😰

Player Items:

Apple has abstracted AVAsset and AVPlayerItem for its purposes. AVPlayerItem manages about playing a media on AVPlayer.

AVPlayerItem: It is meant for AVPlayer to manage the playback. It is created using AVAsset object, like an asset has tracks; player item also have AVPlayerItemTrack corresponding to the AVAssetTrack.

Player Layers:

Presentation of the output of player is done on specialized Core Animation Layer AVPlayerLayer or AVSynchronizedLayer

AVPlayerLayer: CALayer used to present the visual content of the media

AVSynchronizedLayer: CALayer used to present the visual content of media which is in sync with the timing of the media.

To make more clear we have a picture of the relation between all the above ones.

Playing an Asset

Why this abstraction for asset and player item? Because we can use a single AVAsset object to create many player items and play it in different players. We can modify the player item properties and get it rendered in different player objects. These abstractions will be helpful when we have customized features supporting the app.

Ex: Disable soundtrack of player 1 and enable the soundtrack of player 2 for the single asset with two-player items created from it, exhibiting different properties.

Creating and Using AVPLayerItem

• We have, init(url URL: URL) and init(asset: AVAsset) to initialize the player item.
• An asset can be media from the server/live streaming/device itself.
• As we know when we initialize an asset it need not have all the necessary properties ready(duration, tracks, status, etc). It's applicable to the player item as well.
• We must observe the “status” whether the item is ready to play from the live stream. Similarly, we must observe the “tracks” key if we have the tracks to play for an item.
• To support playback we have player.play(), player.pause(), player.seek(to: CMTime), player.rate apt method/var names.

Let's understand the above concepts more precisely with the code.

1. Play an item from Http live stream
2. Play a sequence of items

1. Play an item from Http live stream

A simple code to set up a player and player layer. Also, add observers to observe if the item is ready to play. After that use, the properties observed in the observer method. There are a lot many things to observe (like if video player item is ReadyForDisplay, what if the status of the player item fails at a later point for Http live streaming, etc). Based on the requirement these properties status being a success/fail should be handled.

We need to have an observer for notifying us when the player reaches the end.

For that,

player.seek(to: CMTime(seconds: 10, preferredTimescale: 600)) used to seek to specific time. This has performance overhead, so the seeked time through this method need not be very precise.

To have more precise seek to time, one must use:

player.seek(to: time, toleranceBefore: CMTime.zero, toleranceAfter: CMTime.zero, completionHandler: nil) -> this is used when the precision is more important over the performance (like editing apps)

2. Play a sequence of items

As I have already mentioned about the AVQueuePlayer, which can be used to play a sequence of items.

player = AVQueuePlayer(items: [playerItem1, playerItem2]);

Supports insertItem:afterItem:, remove(_:), removeAllItems()

Before inserting a player item check for “canInsertItem:afterItem:” Adding the same player items isn't supported.

It is more of theory now, we shall learn more of practical stuff in the coming blog where we use a playback, seek to time, image generator, etc in the iOS app.

Continued: Part 4 https://medium.com/@divya.nayak/learning-avfoundation-part-4-playback-demo-437390ec4f20

Thank you!