How to Analyze Frames with AVFoundation on a Video Preview
Learn How to Develop a Custom Camera with Live Preview using AVFoundation on iOS 11
Understanding AVFoundation
AVFoundation is a framework developed by Apple and available on iOS . This framework is used to create, edit and playback media content. It gives direct access to camera and microphone to developers.
In this tutorial we will see how to access the camera stream to perform operation on each frame.
Note: AVFoundation is a high customization framework. If you have an application that only require picture taking for example, you should use the UIImagePickerController.
Building a simple implementation of AVFoundation for live capture
To configure AVFoundation in an application, it is necessary to use a AVCaptureSession. This class owns the different configuration needed to access the camera and use the input properly. It will link the input configuration (camera, microphone), the output configuration (photo, video stream, audio stream) and the live preview.
The architecture of each class will look like the following:
The following example is based on the following class :
Create the AVCaptureSession
An AVCaptureSession variable can be created with the default initialize.
Configuring the Input
In our case, we will use the rear camera as input.
First step, we need to get an access to the rear camera device. All devices can be accessed with the DiscoverySession function of AVCaptureDevice.
Then we will be able to configure as we want the camera. Always lock the camera before updating the configuration, otherwise your application will not be considered as the “owner” of the camera. It will result in a Exception.
And finally the camera is ready to be imported in the AVCaptureSession variable previously created.
Configuring the preview
Once you are done configuring the input, it is recommended to provide user with a visual feedback. The easiest way is to add a view displaying the current input. The only object you have to create is a AVCaptureVideoPreviewLayer that will depend on the session you created.
Configuring the output
Depending on what type of information you are interested in, you will have to create a different output object. In this example, we want to get each frame of camera available. The AVCaptureVideoDataOutputSampleBufferDelegate will make it possible with its “didOutput sampleBuffer” function.
Be careful on the resource consumption of your application, this function will only be called at every available frame.
Once the delegate is implemented, it is possible to link the corresponding output to the session.
Starting the session
Once you have configured your input and output, you can start the current session. As it is blocking call and it can take a long time, you should always start your session on a background thread.
You can check at all time the status of the session with the isRunning property of captureSession. It is also recommended to stop the service when it is not necessary anymore.
Bilan
This sample code is a very simple solution to get all available frames of a camera stream. We use this type of implementation on our application Facelytics for iOS, don’t hesitate to test it!
Do you want to know more about WASSA?
WASSA is an innovative digital agency expert in Indoor Location and Computer Vision. Whether you are looking to help your customers to find their way in a building, enhance the user experience of your products, collect data about your customers or analyze the human traffic and behavior in a location, our Innovation Lab brings scientific expertise to design the most adapted solution to your goals.
Find us on:
