24/7 Accelerometer Tracking with Apple Watch

In general, Watch apps are considered foreground apps; they run only while the user interacts with one of their interfaces. — App Programming Guide for watchOS

If that was the real state of things, we could close Xcode, sign retirement and go fishing. Here is the story of how back in 2017 we’ve succeeded to build 24/7 accelerometer recording app for watchOS3.1, based on my side notes written during work on the project.

Photo by Fabian Albert on Unsplash

• Human Interface Guidelines for watchOS
• WatchKit Programming Guide
• WWDC 2016 — Designing Great Apple Watch Experiences
• WWDC 2016 — What’s New in watchOS

1. Apple Frameworks: the building blocks

1.1 Background tasks

• WWDC 2016 — Keeping Your Watch App Up to Date
• WWDC 2016 — Architecting for Performance on watchOS
• App Lifecycle on watchOS

The default state of an app on watchOS is not running. Apps running in foreground with screen on is running. Apps from Dock and apps with complication on active watchface are in suspended state, loaded into memory and ready for quick resume. Apps running without being presented on watch screen (either if screen is off or another app or watchface is in foreground) are in background state, which is strictly budgeted and actually is our goal.

Apple provides four background task options, which watchOS can pass to your extension delegate’s handle(_ backgroundTasks: Set<WKRefreshBackgroundTask>) method:

• WKApplicationRefreshBackgroundTask is a general background task type that is not specialized for a particular refresh type. The most common use case for this type is for scheduling other tasks. In our app it is used to trigger pretty much all of the work: request accelerometer data from CoreMotion, write it to file and start file transfer to phone side. In order to receive such task from watchOS, you should implicitly request it.
• WKSnapshotRefreshBackgroundTask is a task type specifically for when your app needs to update its snapshot (which is used as a launch image, is shown in the Dock, etc). WatchOS gives this tasks from time to time to every app in Dock, but also can be requested.
• WKWatchConnectivityRefreshBackgroundTask is a task type for when you have transferred data from your iPhone to your watch via the WatchConnectivity framework. We are not using it currently.
• WKURLSessionRefreshBackgroundTask is a task type given to your extension delegate when a background networking task has completed. We are not using it currently as well.

In order to request WKApplicationRefreshBackgroundTask:

Altho documentation claims scheduledCompletion should be called after the background app refresh task has completed, it is called after watchOS schedules or fails to schedule the task (confirmed by Apple people on developer forums and told at WWDC 2016 Session 218). I've never seen scheduling to fail, but beware: watchOS will call (launch your app if needed or wake from suspended state) you extension's delegate handle(_ backgroundTasks: Set<WKRefreshBackgroundTask>) method any moment after requested date, depending on system state — battery level, CPU usage of other running tasks, etc. It can call it one second later, it can call it ten hours later, it can call it in the year 2525.

Important: watchOS has limited number of tasks to be given to processes, so after work of a task is done, you have to call it’s setTaskCompleted() method. Soon after that your app will be suspended again.

Number of WKApplicationRefreshBackgroundTasks given to your app is budgeted; you can expect:

• up to 1 task per hour if your app is in the Dock
• up to 2 tasks per hour if your app has complication on active watchface
• no tasks ever in other cases

Seems pretty cool, until you’ll see the last restriction: timing of WKApplicationRefreshBackgroundTask execution is also limited. If the limit will be exceeded, system daemon called watchdog will terminate your app with 0xc51bad01, 0xc51bad02 or 0xc51bad03 crash report. Altho precise time limit is not stated in the documentation, I've witnessed 9 seconds on Apple Watch 1st Generation and somehow less than 4 seconds on 2nd Generation. Moreover, the limit is being changed as watchOS learns behavior of your app and user's patterns of interacting with it.

1.2 Workouts and start from phone

• WWDC 2016 — Building Great Workout Apps

Workout is a specific mode of watchOS app that originally is intended to run during user’s workout: sensors are tuned for specific type of workout, gyro and hear rate are measured, etc. In workout mode, app will rise foreground every time user switches on the screen.

Workout is managed by HealthKit via following objects:

• HKWorkoutConfiguration - indoors, outdors and kind of sport (which was suddenly deprecated in watchOS4).
• HKWorkoutSession represents the workout - start and end date, flow (paused, running, ended, etc), configuration and object's delegate.
• HKWorkoutSessionDelegate - protocol defines an interface for receiving notifications about errors and changes in the workout session’s state. HealthKit calls it's methods on an anonymous serial background queue.

HealthKit requires user permissions on the phone: either to read or to write data from HealthKit protected store, which means you have to provide NSHealthShareUsageDescription or NSHealthUpdateUsageDescription in Info.plist, respectively. Even if we will not write or read anything, we have to conform to these conditions in order to run workout session, as described in HealthKit Documentation.

Once user granted permissions, workout on watch can be started from foreground iOS counterpart:

1. iOS side: Create HKWorkoutConfiguration
2. iOS side: Call HKHealthStore's startWatchApp(with:completion:) method with this configuration and (Bool, Error?) -> Void completion handler which will be informed whether start was successful or not. Phone will inform watch that user is going to work out, watch will launch the app if needed or wake it in background and call it's extension delegate's handle(_ workoutConfiguration: HKWorkoutConfiguration) method.
3. watchOS side: accept workout configuration in extension delegate’s handle(_ workoutConfiguration: HKWorkoutConfiguration) method. App will be background active till return from this method. You are not obliged to actually start workout at this point, but if you will, app will immediately rise foreground.

1.3 WatchConnectivity

• WWDC 2015 — Introducing Watch Connectivity
• WatchConnectivity Documentation

Data exchange between iPhone and Apple Watch is managed via WatchConnectivity framework. Despite stackoverflow gossips, AppGroups are not an option to share data between watchapp and iOS counterpart, but only between watch app and WatchKit Extension.

There is a number of methods allowing to send Dictionaries, Files, Data, etc with a different level of instancy and requirements. The table in Choosing the Right Communication Option and Guidelines for Communicating Between Apps sections of Sharing Data Programming Guide describes all the aspects pretty well.

Two unobvious things:

1. message sent via sendMessage(_:replyHandler:errorHandler:) with nil reply handler will be accepted only by counterpart WCSessionDelegate's session(_:didReceiveMessage:) method. Message sent with non-nil reply handler will be accepted by session(_:didReceiveMessage:replyHandler:) method. Same with sending data.
2. delivery of an instant message from watch to phone always succeeds (if counterpart’s WCSessionDelegate is not busy processing previous message reply handler for too long), but delivery from phone to watch requires watchapp to be running with active or background states.

1.4 CoreMotion

• WWDC 2016 — Health and Fitness with Core Motion
• Core Motion Documentation

Access to CoreMotion sensors data requires user permission with NSMotionUsageDescription key in Info.plist file. Even though we will use watch accelerometer and gyroscope from watch app, authorization alert will be shown on phone, and key should be in iOS counterpart's Info.plist. Another inconvenient aspect (as of iOS10 and watchOS3) is that CoreMotion do not have dedicated method for authorization request, and it is triggered automatically on the first attempt to get motion data from CMMotionManager.

1.4.1 Accelerometer: CMSensorRecorder

In order to get accelerometer data, you have to do following things.

Request motion sensor to write accelerometer data into it’s own storage. You can request to write up to next 24 hours, and data will be kept and accessible for next 3 days. This should be done on non-main thread:

Request data from CMMotionManager, on non-main thread. There may be a delay of up to three minutes before new samples are available. As of iOS10 and watchOS3, app will crash if start date is older than 3 days or newer than current time. The difference between the start date and end date must be 12 hours or less. For correct dates, optional CMSensorDataList is returned:

CMSensorDataList can be iterated as a collection with a little class extension, as a Sequence of CMRecordedAccelerometerData objects:

1.4.2 Gyroscope: CMMotionManager

Gyroscope data is accessible only during workout, which have two APIs: Handling Motion Updates at Specified Intervals and Periodic Sampling of Motion Data. Both of them are described in documentation in details, so I will not repeat it here.

I’ve found startGyroUpdates(to:withHandler:) method more convenient to use for getting all the samples in one file:

1. Open or create file at desired location with FileManager
2. Instantiate file handler via FileHandle(forWritingTo:)
3. In gyro updates handler closure put the file pointer to the end of file using fileHandler.seekToEndOfFile() method and write serialized to data sample there
4. Finally, close file calling fileHandler.closeFile(). Make sure that you will not attempt to write any more samples to file by calling stopGyroUpdates() before closing file. I did not want any samples to be lost, so I've found convenient to place file closing closure in the end of same serial queue that I've passed to startGyroUpdates(to:withHandler:).

1.5 Notifications

• Notifications on watchOS Programming Guide
• UserNotifications framework
• WWDC 2016 — Introduction to Notifications
• Quick Interaction Techniques for watchOS

Since watchOS3 was released together with iOS10, it’s apps do not support deprecated LocalNotifications in favor of UserNotifications framework. Old-styled notifications are correctly shown, but watchapp’s extension delegate will not receive any callbacks from user interaction with the notification or notification’s actions.

Unobvious places in documentation:

• There is an opportunity to schedule local user notification from watch that will fire only on watch. On the other hand, silent push notification can be accepted only by iOS counterpart. All other cases have complex logic defining which device will deliver notification to user, but it is managed by the system and developer have no influence there.
• If user will tap on watch one of notification’s UNNotificationAction that have .foreground option, watchapp will be opened foreground. Make sure to catch notification in watchapp's UNUserNotificationCenterDelegate and present something reasonable in UI.

2. Our approach: Apple Watch side

2.1 Data Delivery Infrastructure

General idea of how to deliver accelerometer data to phone side is:

1. break timeline into 10-minute chunks
2. get raw accelerometer data from the motion sensor for a chunk (see 1.4 CoreMotion section)
3. write this data chunks to files
4. transfer files to phone side (see 1.3 WatchConnectivity)

In our solution, this flow is incapsulated into NSOperation's subclass SendSamplesOperation. NSOperationsQueue's subclass called SendSamplesOperationQueue can run operations serially or concurrently, track failed and succeeded operations, place callbacks at different moments of operation execution and refill itself according to it's internal state. Manager called OperationsCoordinator changes the state of it's SendSamplesOperationQueue, causing different types of stream.

All this should be done in background at every opportunity (see 1.1 Background tasks and 1.2 Workout), and we’ve tried a number of different strategies discussed in 2.5 OperationsCoordinator strategies.

Since every watchOS app will be terminated in case of resource violation (either CPU time, wall-time or memory), most of stateful objects should be persisted in UserDefaults.

Objects used in transfer of data to the phone should not know any difference between cases when motion sensor gave us some motion data for a chunk and cases when it told us that it do not have any. Altho these cases utilize different WatchConnectivity methods of watch-to-phone transfer, they are incapsulated into DataFile object and Operations do not know that they are any different.

2.1.1 OperationsCoordinator

Responsibility of OperationsCoordinator is to provide interface to the operations queue: whether new sync cycle should be started, callback with transfer status from WatchConnectivity was received, etc.

It has a number of accelerometer data sync entry points which basically just configure operationsQueue refillment rules, give it closures to be executed between operations and manage background time gaining strategy.

It has callbacks forSessionManager to be called when it receives status of file transfer from WatchConnectivity framework: successfullyTransferred(chunkOf:) and failedToTransfer(chunkOf:file:).

In cases when user logged out from the app on phone or gap in transfers was too long to try to transfer all missed data, we want to restart transfers flow in one of two ways:

• discardCurrentOperations() will finish currently running operations and discard all the operatins waiting in queue.
• discardOperationsHistory() will also remove info about which time chunks have already been processed and which ones should follow.

2.1.2 SendSamplesOperation

Responsibility of SendSamplesOperation is to get accelerometer data for one time chunk and pass it to SessionManager for transfer.

SendSamplesOperation has two possible purposes:

1. we need to request data from motion sensors, pack it into file and transfer to phone.
2. we already have data packed into file and operations should only transfer it. This can happen if delivery failed or if watchapp was terminated before delivery confirmation from phone arrived.

Possible algorithm of SendSamplesOperation

2.1.3 SendSamplesOperationQueue

Responsibility of SendSamplesOperationQueue is creation and queuing of SendSamplesOperations. It tracks succeeded and fallen operations, refills itself with new operations for time chunks it will choose, has it's own state to allow preventing refillment with precise operation kinds from outside (for example, in ShortTasks we want to allow requesting-data-from-sensor operations only for first 4 seconds, but create sending-previously-prepared-files is ok as long as we have them without any limits).

Possible algorithm of SendSamplesOperationQueue refillment

Time chunk for new operation will be chosen in such priority:

1. Timestamp previously used, with file previously failed to transfer, if there were less than numberOfRetriesIfFailedToSend attempts fallen in a row.
2. If queue’s stage allows, timestamp previously used, without file ready to be sent (which means that app was terminated while preparing file).
3. If queue’s stage allows, new timestamp calculated as newest previous timestamp + chunk duration, if it is older than chunk duration In other cases, operation will not be created and self-refillment will stop, finally executing performWhenEverythingTransferred closure.

2.1.4 DataFile

DataFile object incapsulates accelerometer/gyroscope data file options that can be sent from watch to phone. It is enum with three options:

• notRequestedYet is a default case, used by Operations for flow stages before motion data was requested from motion sensors manager.
• noData stands for situations where motion data was requested, but motion sensor do not have any data recorded.
• url(URL) is case with associated URL value stands for situations when motion data was requested, received, processed and written into file placed at the url.

In order to be saved to UserDefaults, DataFile conformed to our homemade StringCodable protocol in tough times of Swift 3.2 😒

2.3 Motion Sensor Manager

MotionSensorManager provides accelerometer, gyroscope and other services of CoreMotion framework and writes their raw data to files.

It has a number of public methods:

• requestAuthorization() triggers privacy authorization request (see 1.4 CoreMotion) on the phone.
• askSensorToContinueRecording() just tells CMSensorRecorder to continue writing accelerometer data for next 12 hours (maximum allowed by CoreMotion API)
• getAccelerometerData(startTimestemp:endTimestamp:completionHandler:) should be called to get accelerometer data written to file. It is the most important method in this class, which covers all the async sword dance between CoreMotion and FileManager.

Algorithm behind getAccelerometerData(startTimestemp:endTimestamp:completionHandler:) can be roughly described such way:

1. Call requestSensorData(from:to:completion:) on a high priority dispatchQueue , which will give us CMSensorDataList or error into completion closure.
2. In case of error push it up to initial completionHandler. In case of data list call writeToFile(marker:sensorData:).
3. In writeToFile(marker:sensorData:) iterate over samples of type CMRecordedAccelerometerData and write them to file. Iteration should be done inside autoreleasepool { } in order to prevent excessive memory usage (watchOS kills apps that use more than 30mb of RAM). If data was successfully written to the file, call completionHandler with the URL.

We serialize CMRecordedAccelerometerData samples into strings and then to data in serialize(vector:at:) method. ThreeAxisVector protocol helps us to generalize implementation of this method for both gyroscope's CMRotationRate and accelerometer's CMAcceleration objects.

This can be optimized: instead of printing decimal numbers into string we can print hex numbers, which will be shorter -> less characters in string -> less data in file to transfer to phone -> less battery usage, less workout time, faster LongTask, faster work.

While Accelerometer data is kind of historical, gyroscope data comes to us almost in real time thanks to CMMotionManager, which places closure to our serial gyroscopeQueuefor every recorded sample. Inconvenience of this way is that CMDeviceMotion do not give us any normal epoch timestamp, but the amount of time in seconds since the device booted. In order to convert it to UTC, we're saving deviceBootTime timestamp in MotionSensorManager's constructor.

2.4 Data Transfer

SessionManager is WatchConnectivity's WCSessionDelegate responsible for all the communication with phone.

Method sendFile(_:via:) is intended to transfer mostly accelerometer data files to phone. Since files are delivered to iOS app only when it is awake, this method first sends instant message (which launches or wakes the counterpart app when being sent from watch) and after delivery confirmation sends the file by means of fastsendFile(_:via:). This is done to transfer files while both apps are running in background.

Method fastsendFile(_:via:) is also used for sending watch logs files or gyroscope data files - both happens when iOS app is foreground and no wakeup-message is needed. Depending on DataFile case, fastsendFile(_:via:) can:

• transfer UserInfo dictionary with chunk's timestamp in order to let iOS app know that watch motion sensor do not have any data for the time interval. This is done by means of WatchConnectivity transferUserInfo(_:) method.
• transfer the file by means of WatchConnectivity transferFile(_:metadata:) method.

Whether delivery was successful or not, WCSessionDelegate will receive one of the following callbacks:

• session(_ session: WCSession, didFinish userInfoTransfer: WCSessionUserInfoTransfer, error: Error?) in case of dictionary transfer
• session(_ session: WCSession, didFinish fileTransfer: WCSessionFileTransfer, error: Error?) in case of file transfer

Both of them will map file/dictionary back to it’s contents type (accelerometer chunk timestamp, gyroscope or logs) and report success or failure to OperationsCoordinator, so it could manage following actions - workout status change, refillment of queue, etc. Pay attention that OperationsCoordinator will not know if file or dictionary was actually sent, it only knows if delivery was successful or not.

2.5 OperationsCoordinator strategies

2.5.1 Short task

Short task is efficient in terms of battery usage, but risky in terms of app termination. It's idea is to run 4 concurrent SendSamplesOperations such way that their CPU-expensive part (while MotionSensorManager prepares file with data) would be done in first 4-9 seconds of background task, and then workout mode will be switched on to keep app active while SessionManager will wait for file delivery confirmation. Refillment will continue while queue state (switched by countdown) allows.

Efficiency: battery-expensive workout is running only while waiting for confirmation of 4 concurrent deliveries.

Risk: if CPU-expensive MotionSensorManager work will not be accomplished before workout will start, there is a risk of exceeding CPU usage limitation and 0xc51bad0X termination. Also, if by some reason watchdog will decide that our hardcoded (and discovered heuristically) timeout of 4-9 sec is too much for WKApplicationRefreshBackgroundTask, the app will be terminated.

This flow also correctly works in cases then MotionSensorManager do not have any data and processing do not take any sensitive time, so by the end of canSendFor number of seconds workout should not be started because dictionary have already been delivered to phone.

2.5.2 Long task

Less efficient in terms of battery usage, Long task almost never gets terminated by the system. It's idea is in starting workout as soon as possible, running SendSamplesOperations serially and waiting for average CPU usage will not exceed cap of 15% by sleep()ing between operations. Refillment will continue until the newest data will be transferred to phone.

Efficiency: battery-expensive workout is running during whole task — including idle sleep() intervals. That's A LOT of time: about 10 minutes of workout to transfer only 30 minutes of raw data recordings.

Risk: the only possible reason for termination is exceeding of 15% CPU usage cap. We've run the app in Instruments to find out correct timing and chunk size so this will never happen.

2.5.3 Entry points

• WatchExtensionDelegate's method handle(_ backgroundTasks: Set<WKRefreshBackgroundTask>) when system gives us WKApplicationRefreshBackgroundTask
• app by some reason goes foreground (see applicationWillEnterForeground() method of WatchExtensionDelegate)
• triggered from iOS by sending workout configuration (see 1.2 Workouts and start from phone). This can happen if iOS app will notice that it did not receive anything from watch for too long

3. Our codebase: iPhone side

3.1 Accepting Accelerometer Files

WCSessionDelegate accepts file or dictionary (see 2.4 Data Transfer for more details on sending side decisions) and callbacks session(_:didReceive file:) or session(_:didReceiveUserInfo:) are called. Both of them are binding to received(file:with:) which will check what kind of data arrived and in case it's accelerometer data (or dictionary with timestamp if sensor had no data) will call sensorController's handlers one by one.

Handler will:

1. In case of file arrived, rewrite file to document directory, since copy provided by WatchConnectivity will be removed by iOS as soon as session(_:didReceive file:) returns. In case of dictionary proceed to the following step.
2. Timestamp and file URL (or blank URL in case of dictionary) will be placed into pendingFiles array. This array works as a queue, storing all unprocessed chunks that we've received from watch some data about. It’s custom getter is smart enough to return nil when handler should interrupt cycle of processing accepted files: in case app runs out of execution time, in case of gaps between chunks or in case of empty queue.
3. Starts iOS finite background task by means of UIApplication.shared.beginBackgroundTask(withName:)
4. Call processPending(for:), which will try to get following data file from pendingFiles, and if it is non-nil, pass it to further processing into the deep of iOS app. It will also update lastChunkBeforeInterrupt timestamp — an events horizon of processed chunks. All chunks with timestamp earlier than lastChunkBeforeInterrupt are considered somehow processed and will be removed from pendingFiles array soon after.
5. Repeat Step 4 until pendingFiles will return nil.

Data files are being deleted after data was accepted by internal mechanisms via updateLatestDataArrived(with:) callback. Rollback can be done during initialization of delegate, which protects against crashes occurring while iOS app processes data samples in background.

What about watchOS4?

Long story short, if the project was started on watchOS4, it should be done in a totally different way: via CoreBluetooth framework as described in a very cool Watch your Bluetooth! post .

All above was originally written in 2017, the age of Swift 3.2, Xcode 8, iOS 10 and watchOS 3. Application was accepted by AppStore, and according to performance tests it did not decrease watch battery life as dramatically as we originally expected. Following the userdata breach scandals of last weeks: app was used for a medical research, all the users were participants aware of tracking.