Realtime Video Closed Captioning in SwiftUI

Using SFSpeechRecognizer offline, on device, and uninterrupted

This project is dedicated to my Dad, who overcomes his own hearing loss every day and inspires me to write code that can improve accessibility in tech.

Recently, I got pretty pumped about the changes to SFSpeechRecognizer Apple put this year. The game-changer is that it now has the capability to run locally on the device: no internet connection needed. This means that:

• Users no longer need to worry about using data when mobile
• Privacy FTW — everything is done locally instead of going back and forth with some server out there
• There’s less of a transcription delay, especially in realtime
• No longer restricted to only transcribing one minute of audio for a limited number of times a day. It’s unlimited (or until your voice gives out or your battery dies)

This is big news for speech-to-text, accessibility, and speech technology in general.

Use Case

Having grown up in a household that usually watched TV with the closed captioning (CC) turned on, it made perfect sense to make that my focus to test SFSpeechRecognizer’s latest updates. When thinking about how CC is…