Live Data Scanning on iOS: A Quick Look at Apple’s VisionKit Framework

Prashant Singh is a consultant and iOS Specialist who loves helping clients design, architect and implement mobile apps using the latest technologies.

Apple’s VisionKit framework allows the developer to use the device camera to scan data present in images. The framework is capable of scanning machine-readable codes and can be configured to look for specific content in the text (E.g. email, street address, phone number, URL, shipment tracking number, and date). With the introduction of iOS 16, data scanning is possible with a live camera feed. In this blog, we will look at the implementation and various configurations of the live data scan using the camera video feed.

Scanning the Data:

Data scan is performed with the help of the “DataScannerViewController” class introduced in iOS 16.

An object of DataScannerViewController needs to be created after setting the required parameters containing the scan options.

Note: In SwiftUI, the DataScannerViewController needs to be wrapped using UIViewControllerRepresentable as it is a part of UIKit.

let scannerViewController = DataScannerViewController (
recognizedDataTypes: [.text(), .barcode(symbologies: [.aztec, .qr])],
qualityLevel: .fast,
recognizesMultipleItems: true,
isPinchToZoomEnabled: true,
isGuidanceEnabled: true,
isHighlightingEnabled: true)

Scan Options:

The following scan options should be configured before starting the scan.

• Recognized Data Types: It takes both barcodes and textual types. The types can be further configured depending upon the language and content for text and symbology for the barcode.
• Quality Level: Fast, balanced or accurate.
• Recognises Multiple Items: Recognise all the groups of items in the camera viewfinder or only one.
• Pinch to Zoom: Use the pinch gesture to zoom in and out of the live feed. When zooming, it uses all the cameras (Main, Ultra Wide and Telephoto) present on the device.
• Enable Guidance: Set this to true if the app should provide text cues when scanning the items, e.g., “Slow down.” when moving the phone fast.
• Highlighting enabled: Highlight the recognised items

To start the scan, the “startScanning()” method of the created view controller object is used.

try? scannerViewController.startScanning()

In the below image, the scanned text is printed on the second half of the screen on a custom view. In the first half, the view presented by DataScannerViewController is shown.

Specific text content scan

To specify the text content type to be scanned, the required text content type should be specified in the “recognizedDataTypes” parameter while initialising the DataScannerViewController object.

let scannerViewController = DataScannerViewController (
recognizedDataTypes: [.text (textContentType: .fullStreetAddress),
.barcode (symbologies: [.aztec, .qr])],
qualityLevel: .fast,
recognizesMultipleItems: true,
isHighlightingEnabled: true)

Single item scan

Single-item scanning mode can be configured using the “recognizesMultipleItems” property in the DataScannerViewController initialiser.

let scannerViewController = DataScannerViewController (
recognizedDataTypes: [.text(), .barcode (symbologies: [.aztec, .qr])],
qualityLevel: .fast,
recognizesMultipleItems: false,
isHighlightingEnabled: true)

In single-item scan mode, the required item can be selected by a tap if multiple items are present.

Parsing the Data

The scanned results are provided in delegate methods (didAdd, didTapOn, didRemove) of the DataScannerViewControllerDelegate protocol. The items are represented inside the RecognizedItem enum containing the recognised data type (barcode or text).

func dataScanner (_ dataScanner: DataScannerViewController, didTapOn item: RecognizedItem) {
tappedItem = item
}
func dataScanner (_ dataScanner: DataScannerViewController, didAdd addedItems: [RecognizedItem], allItems: [RecognizedItem]) {
recognisedItems.append(contentsOf: addedItems)
}
func dataScanner (_ dataScanner: DataScannerViewController, didRemove removedItems: [RecognizedItem], allItems: [RecognizedItem]) {
recognisedItems = recognisedItems.filter { item in
!removedItems.contains (where: {$0.id == item.id })
}
}

To get the value of the scanned text from the RecognizedItem enum, the “transcript” property needs to be accessed. For barcodes, “payloadStringValue” contains the scanned value in the String form.

switch item {
case .barcode (let barcode):
print (barcode.payloadStringValue ?? "Unknown barcode")
case .text (let text):
print (text.transcript)
@unknown default:
print ("Unknown item")
}

Limitations

While VisionKit is a powerful and versatile framework, it does have some limitations that developers should be aware of when designing their apps. Some of these limitations are given below.

1. Scanning using the live camera feed is only available in iOS 16 and above.
2. The device should have a minimum of an A12 Bionic chipset.
3. The framework struggles when scanning for the text that is not present in a straight line.
4. When scanning for multiple complex items such as Version 40 QR codes, the device needs to be held very still.
5. Not all languages are supported by text scanning. As of iOS 16, the following languages are supported: English, French, Italian, German, Spanish, Portuguese, Chinese, Japanese, and Korean.