Implementing Background play & Picture-in-Picture in SwiftUI: Building a Video Streaming App with AVPlayer and SwiftUI

Welcome to the comprehensive guide on mastering Picture-in-Picture (PiP) and background play in SwiftUI. Where I will walk you through the process of implementing background play/services and picture in picture video playback in an SwiftUI app using AVPlayer and SwiftUI.

Overview:

1. Setting Up the Project
2. Integrating AVPlayer
3. Implementing Picture-in-Picture
4. Enabling Background Play
5. Advanced Customization

Prerequisites

Before diving into the implementation, make sure you have a basic understanding of SwiftUI, AVPlayer, and the overall structure of a SwiftUI app. Additionally, ensure that your project has the necessary permissions in the Info.plist file to access media files and network resources.

To get started, create a new SwiftUI project in Xcode.

Project setup and enabling PiP mode.

• Add background services/ permissions — Audio, AirPlay and Picture in Picture.

• Increase deployment target to at least iOS-14.2, PiP mode is only supported on or after iOS-14.2.

Integrating AVPlayer

• Lets start by creating the PlayerController.
  The PlayerController class is designed to handle video playback using the AVKit framework in a Swift application. It includes functionalities to set up an AVPlayer, configure metadata such as title, artist, and artwork, and manage player controls.

import Foundation
import AVKit

class PlayerController: ObservableObject {
    // MARK: - Published Properties

    // Published properties to trigger UI updates when changed
    @Published var playbackVideoLink: String = ""
    @Published var playbackTitle: String = ""
    @Published var playbackArtist: String = ""
    @Published var playbackArtwork: String = ""

    // MARK: - AVPlayer and AVPlayerViewController Properties

    var player: AVPlayer?
    var avPlayerViewController: AVPlayerViewController = AVPlayerViewController()

    // MARK: - Initialization and Setup

    func initPlayer(
        title: String,
        link: String,
        artist: String,
        artwork: String
    ) {
        // Initialize playback properties
        self.playbackTitle = title
        self.playbackArtist = artist
        self.playbackArtwork = artwork
        self.playbackVideoLink = link

        // Setup AVPlayer and AVPlayerViewController
        setupPlayer()
        setupAVPlayerViewController()
    }

    // MARK: - AVPlayer Setup

    private func setupPlayer() {
        // Initialize AVPlayer with the provided video link
        player = AVPlayer(url: URL(string: playbackVideoLink)!)

        // Set up metadata for the current item
        setupMetadata()
    }

    // Set up metadata for the current AVPlayerItem
    private func setupMetadata() {
        let title = AVMutableMetadataItem()
        title.identifier = .commonIdentifierTitle
        title.value = playbackTitle as NSString
        title.extendedLanguageTag = "und"

        let artist = AVMutableMetadataItem()
        artist.identifier = .commonIdentifierArtist
        artist.value = playbackArtist as NSString
        artist.extendedLanguageTag = "und"

        let artwork = AVMutableMetadataItem()
        setupArtworkMetadata(artwork)

        // Set external metadata for the current AVPlayerItem
        player?.currentItem?.externalMetadata = [title, artist, artwork]
    }

    // Set up artwork metadata based on UIImage
    private func setupArtworkMetadata(_ artwork: AVMutableMetadataItem) {
        if let image = UIImage(named: "Artist") {
            if let imageData = image.jpegData(compressionQuality: 1.0) {
                artwork.identifier = .commonIdentifierArtwork
                artwork.value = imageData as NSData
                artwork.dataType = kCMMetadataBaseDataType_JPEG as String
                artwork.extendedLanguageTag = "und"
            }
        }
    }

    // MARK: - AVPlayerViewController Setup

    private func setupAVPlayerViewController() {
        // Assign AVPlayer to AVPlayerViewController
        avPlayerViewController.player = player
        avPlayerViewController.allowsPictureInPicturePlayback = true
        avPlayerViewController.canStartPictureInPictureAutomaticallyFromInline = true
    }

    // MARK: - Playback Control

    // Pause the AVPlayer
    func pausePlayer() {
        player?.pause()
    }

    // Play the AVPlayer
    func playPlayer() {
        player?.play()
    }
}

For styling purposed refer to the following code blocks to setup the Notification Centre, Control Centre and Lock screen media player with the proper playback meta data.

    // Set up metadata for the current AVPlayerItem
    private func setupMetadata() {
        let title = AVMutableMetadataItem()
        title.identifier = .commonIdentifierTitle
        title.value = playbackTitle as NSString
        title.extendedLanguageTag = "und"

        let artist = AVMutableMetadataItem()
        artist.identifier = .commonIdentifierArtist
        artist.value = playbackArtist as NSString
        artist.extendedLanguageTag = "und"

        let artwork = AVMutableMetadataItem()
        setupArtworkMetadata(artwork)

        // Set external metadata for the current AVPlayerItem
        player?.currentItem?.externalMetadata = [title, artist, artwork]
    }

    // Set up artwork metadata based on UIImage
    private func setupArtworkMetadata(_ artwork: AVMutableMetadataItem) {
        if let image = UIImage(named: "Artist") {
            if let imageData = image.jpegData(compressionQuality: 1.0) {
                artwork.identifier = .commonIdentifierArtwork
                artwork.value = imageData as NSData
                artwork.dataType = kCMMetadataBaseDataType_JPEG as String
                artwork.extendedLanguageTag = "und"
            }
        }
    }

I’m using the “Artist” local asset for the cover image, from Assets.xcassets.

• Configure the VideoPlayer.
  The VideoPlayer struct is designed to integrate the PlayerController with SwiftUI by conforming to the UIViewControllerRepresentable protocol. It acts as a bridge between the AVPlayerViewController and SwiftUI, allowing the player to be used seamlessly within SwiftUI-based user interfaces.

import Foundation
import SwiftUI
import AVKit

// A SwiftUI view wrapper for an AVPlayerViewController
struct VideoPlayer: UIViewControllerRepresentable {
    // MARK: - Observed Object
    
    // ObservedObject that manages the underlying AVPlayer and its playback state
    @ObservedObject var playerController: PlayerController
    
    // MARK: - UIViewControllerRepresentable Protocol Methods
    
    // Update the AVPlayerViewController when SwiftUI view updates
    func updateUIViewController(_ uiViewController: AVPlayerViewController, context: Context) {
        // Update the view controller if needed
        // (e.g., handle updates when the underlying AVPlayer changes)
    }
    
    // Create and configure the AVPlayerViewController
    func makeUIViewController(context: Context) -> AVPlayerViewController {
        return playerController.avPlayerViewController
    }
}

• Finally the PlayerView-UI
  The PlayerView struct defines a SwiftUI view that incorporates the PlayerController and VideoPlayer for displaying and controlling video playback. It includes buttons for playing and pausing the video and sets up the initial state of the video player.

import Foundation
import SwiftUI

// A SwiftUI view representing the player interface
struct PlayerView: View {
    // StateObject that manages the player controller and its state
    @StateObject var playerController = PlayerController()

    var body: some View {
        VStack(alignment: .center) {
            // Use GeometryReader to adapt to the available space
            GeometryReader { geometry in
                let parentWidth = geometry.size.width

                // Display loading message if the player is not initialized yet
                if playerController.player == nil {
                    Text("Loading")
                } else {
                    // Display VideoPlayer if the player is initialized
                    VideoPlayer(playerController: playerController)
                        .frame(width: parentWidth)
                }
            }
            .frame(height: 200)

            // Play and Pause buttons
            Button {
                playerController.playPlayer()
            } label: {
                Text("Play video")
            }

            Button {
                playerController.pausePlayer()
            } label: {
                Text("Pause video")
            }
        }
        // Initialize the player on view appear
        .onAppear {
            playerController.initPlayer(
                title: "SomeTitle",
                link: "https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4",
                artist: "Khondakar Afridi",
                artwork: "Artist"
            )
        }
    }
}

// Preview for PlayerView
#if DEBUG
struct PlayerView_Previews: PreviewProvider {
    static var previews: some View {
        PlayerView()
    }
}
#endif

Enabling Background Play and Services

• To enable the background services, it is crucial that we modify the AppDelegate and include the appropriate AudioSession settings.

import SwiftUI
import SwiftData
import AVKit

// The main entry point of the app
@main
struct Pip_PlayerApp: App {
    
    // Application delegate adaptor to set up the custom AppDelegate
    @UIApplicationDelegateAdaptor(AppDelegate.self) var appDelegate
    
    var body: some Scene {
        // Main app window content
        WindowGroup {
            ContentView()
        }
    }
}

// Custom AppDelegate to handle application lifecycle events
class AppDelegate: NSObject, UIApplicationDelegate {
    
    // This method is called when the application finishes launching
    func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey : Any]? = nil) -> Bool {
        
        // Configure the audio session for background audio and video services
        
        let audioSession = AVAudioSession.sharedInstance()
        
        do {
            // Set the audio session category to playback
            try audioSession.setCategory(.playback)
            
            // Activate the audio session
            try audioSession.setActive(true, options: [])
        } catch {
            // Handle errors related to audio session setup
            print("Setting category to AVAudioSessionCategoryPlayback failed.")
        }
        
        return true
    }
}

And Tada, just like that we have a streaming video player, with picture in picture mode, background play and an interactive media player in Notification Centre, Control Centre and on the Lock Screen.

Conclusion

In this article, we’ve explored the process of creating a video player app with PiP mode using AVPlayer and SwiftUI. By structuring your code with the PlayerController, VideoPlayer, and PlayerView, you can easily manage the video playback and integrate advanced features like PiP mode.

Feel free to customize the UI, enhance the controls, and incorporate additional features to make your video player app even more user-friendly. SwiftUI provides a powerful and intuitive platform for creating immersive multimedia experiences, and with the right implementation, you can offer users a seamless and enjoyable video playback experience.

Get the full source code at — https://github.com/WorkWithAfridi/PiPPlayer-SWIFTUI