ARKit: How to make your own Audio-Video Recorder.
Download an AR camera starter project here.
For TLDR, jump straight to Practice.
Last time I posted, I shared how to save video in ARKit but noted that it did not record audio.
Today, I am happy to say that I have found a way to do both — together.
Problem
As listed in this tutorial, theres a serious issue recording ARKit video with any existing package — especially Apple’s ReplayKit that breaks at any given moment with no solutions as how to fix it.
In that tutorial, I listed a way to capture AR Frames as UIImages and convert them into video but explained that I had yet to crack Audio — as the existing recorder is very taxing on the system and needs a clean solution that will not break the system.
Solution
Things I tried
- AVAudioRecorder — Crashes the system in tandem with recording video.
- AVAudioEngine — Crashes the system in tandem with recording video.
What Worked
Creating a AVCaptureSession with the iPhone microphone as the sole input and its raw data as the sole output.
As the audio CMSampleBuffer comes in with its own ‘time’, the final touch is to correct the time of the recording to match the time of the video frame.
This is then appended onto AVAssetWriter — all of this is covered in Audio Recording Functionality.
Practice
Please refer to this tutorial to explain how video works and once you have covered that one — change the following pieces of code.
New Variables
New didUpdateAtTime
Please note that the system crashes at 30 FPS (currently at 24FPS on line 7).
New Recording Functionality
New Asset Writer
Audio Recording Functionality
Please leave any questions or comments in the section below.
Oscar
