Step-by-Step Tutorial: Configuring Sessions for 60FPS Video Capture on iOS

Image by Freepik

Introduction:

Capturing video at a high frame rate can significantly enhance the user experience in your camera app. In this blog post, we’ll delve into the Swift code required to configure frame rates and achieve smooth 60 frames per second (FPS) video capture.

Step 0: Before we start

Begin your journey by tapping into valuable resources that offer well-structured boilerplate code, addressing input device configurations and error handling effectively. Key resources include:

• Capture setup
• AVCam: Building a Camera App
• AVMultiCamPiP: Capturing from Multiple Cameras
• AVCamBarcode: Detecting barcodes and faces

Those resources are sample projects or codebases that can be referenced for common camera-related tasks. By leveraging these resources, you gain access to well-structured and battle-tested code snippets, streamlining the development process and ensuring effective handling of potential errors in your camera applications.

Post-Setup Code:

Now that the groundwork is laid, let’s examine the essential code snippets needed in your viewDidLoad function:

override func viewDidLoad() {
  super.viewDidLoad()
  
  // Set up PreviewView
  
  // Initialize and configure the video preview view.

  // Check video authorization status. 
  // Video access is mandatory.

  // Setup the capture session.
  sessionQueue.async {
    self.configureSession()
  }
}

This viewDidLoad function sets the stage for your camera app. It initializes the PreviewView, checks video authorization status, and asynchronously configures the capture session using configureSession function. This structure ensures a systematic and organized approach to preparing your camera app for optimal video capture.

Note: Ensure that you include informative text in the Privacy-Camera Usage Description prompt.

Privacy — Camera Usage Description

Step 1: Check Setup Result

guard self.setupResult == .success else {
    os_log("Could not start configuring session. Reason - setup result is: %{public}@", type: .info, "\(setupResult)")
    return
}

Ensure the initial setup result is successful; otherwise, configuration won’t proceed.

Step 2: Begin Configuration

// Begin configuring the AVCapture session.
session.beginConfiguration()

// Ensure that the configuration is committed, whether the configuration succeeds or an error occurs.
defer { session.commitConfiguration() }

Initiate the configuration of the AVCapture session. The defer statement guarantees that session.commitConfiguration() will be executed when the current scope is exited, ensuring proper cleanup and finalization of the configuration, regardless of whether it completes successfully or encounters an error. This is a common pattern when dealing with configurations to maintain consistency and avoid leaving the session in an inconsistent state.

Step 3: Set Session Preset

session.sessionPreset = .hd1280x720

Configure the session preset to hd1280x720 for a balanced compromise between frame quality and size. Additional preset options can be explored in the AVCaptureSession.Preset enumeration(documentation). It's important to note that selecting higher presets will improve frame quality but also increase the frame size. This consideration is especially vital if you plan to store frames on the fly, as it impacts storage requirements. Keep this trade-off in mind when choosing the session preset that best suits your application's needs.

Step 4: Add Video Input

// Attempt to get the default video device, prioritizing the back wide-angle camera.
let defaultVideoDevice: AVCaptureDevice?

if let backCameraDevice = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: .back) {
    defaultVideoDevice = backCameraDevice
} else if let frontCameraDevice = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: .front) {
    // If the back wide-angle camera is unavailable, use the front wide-angle camera.
    defaultVideoDevice = frontCameraDevice
} else {
    defaultVideoDevice = nil
}

// Ensure a valid video device is obtained.
guard let videoDevice = defaultVideoDevice else {
    setupResult = .configurationFailed
    os_log("Could not get video device", type: .error)
    return
}

do {
  // Create a AVCaptureDeviceInput instance with the obtained video device.
  let videoDeviceInput = try AVCaptureDeviceInput(device: videoDevice)

  if session.canAddInput(videoDeviceInput) {
    session.addInput(videoDeviceInput)
    // Successfully obtained video device input; proceed with configuration.
    /*
     Configure video device input
    */
  } else {
    // Handle failure to add video device input
    setupResult = .configurationFailed
    os_log("Could not add video device input to the session", type: .error)
    return
  }
} catch {
  // Handle failure to add video device input
  setupResult = .configurationFailed
  os_log("Could not create video device input: %{public}@", type: .error, error.localizedDescription)
  return
}

Add the video input to the session, ensuring it can be added successfully.

Step 5: Configure Video Device Input

Configure the selected video device input, set the active format, and specify the desired frame rate (60 FPS).

// Desired frame rate - 60FPS
let targetFrameRate = 60
// Disable smooth auto-focus if supported by the video device.
if videoDevice.isSmoothAutoFocusSupported {
    try videoDevice.lockForConfiguration()
    videoDevice.isSmoothAutoFocusEnabled = false
    videoDevice.unlockForConfiguration()
}

// Set the desired frame rate and configure the active format.
try videoDevice.lockForConfiguration()

// Select the initial format as a fallback.
var formatToSet: AVCaptureDevice.Format = videoDeviceInput.device.formats[0]

// Iterate through available formats to find the one matching the desired frame rate and resolution.
for format in videoDeviceInput.device.formats.reversed() {
    let ranges = format.videoSupportedFrameRateRanges
    let frameRates = ranges[0]

    // Check if the format matches the desirable frame rate and resolution (1280x720).
    if frameRates.maxFrameRate == Double(targetFrameRate),
       format.formatDescription.dimensions.width == 1280,
       format.formatDescription.dimensions.height == 720
    {
        // Set the format to the matching one.
        formatToSet = format

        // Log the chosen active format.
        os_log("Video device active format was chosen: %{public}@", type: .info, format.description)

        // Exit the loop as the desired format is found.
        break
    }
}

// Apply the selected format to the video device.
videoDevice.activeFormat = formatToSet

// Set the desired frame rate(60FPS).
let timescale = CMTimeScale(targetFrameRate)
// Ensure activeFormat supports 60 frames per second before setting frame duration.
// This place would crash if you didn't set activeFormat to one that can handle 60 frames per second.
if videoDevice.activeFormat.videoSupportedFrameRateRanges[0].maxFrameRate >= Double(targetFrameRate) {
    videoDevice.activeVideoMinFrameDuration = CMTime(value: 1, timescale: timescale)
    videoDevice.activeVideoMaxFrameDuration = CMTime(value: 1, timescale: timescale)
} else {
    // Log a warning if the selected format doesn't support 60FPS.
    os_log("Selected active format may not support the desired frame rate of %{public}d FPS.", type: .error, targetFrameRate)
}

// Unlock the video device configuration.
videoDevice.unlockForConfiguration()

// Set the configured video device input for later use.
self.videoDeviceInput = videoDeviceInput

Note: At the point where the desired frame rate is set (activeVideoMinFrameDuration and activeVideoMaxFrameDuration), a check is introduced to ensure that the selected active format indeed supports the desired frame rate of 60 FPS. Without this check, setting frame duration on an incompatible format could lead to a runtime crash.

Step 6: Set Video Data Output

Add video data output to the session for handling video data objects.

// Add video data output.
if session.canAddOutput(videoDataOutput) {
    // Add video data output to the session.
    session.addOutput(videoDataOutput)

    // Set this view controller as the delegate for video data objects.
    videoDataOutput.setSampleBufferDelegate(self, queue: videoDataObjectsQueue)

    // Set the pixel format for video frame capture.
    videoDataOutput.videoSettings = [String(kCVPixelBufferPixelFormatTypeKey): kCVPixelFormatType_32BGRA]
} else {
    // Handle failure to add video data output.
    setupResult = .configurationFailed
    os_log("Could not add video data output to the session", type: .info)
    return
}

Conclusion

This step-by-step tutorial guides you through configuring the AVCapture session for 60FPS video capture in your Camera app. Ensure to handle potential errors during each step and adapt the code according to your specific application needs. Happy coding!

Source Code:

https://github.com/oleksiikolomiietssnapp/CaptureVideo