Syncing with CloudKit in a clean architecture (swift)

Hi, i’m the creator of the macOS Jirassic app, an app that will track your worked time mostly automatically https://github.com/ralcr/Jirassic Recently I had to implement syncing and backup for few reasons: iOS app is waiting to be developed; when working from the office and from home on two different computers logged time will be scattered and hard to manage; future plans of syncing with other services like Jira tempo. I never used it before but i chose to use CloudKit because users are already logged in and because it is not doing any magic of caching stuffs for me like Firebase. I also believe it will not die suddenly like Parse, although it is free, we are actually paying for it through the developer account and iPhones, like our macs pay for that spaceship Apple is building.

Finding info about a sync mechanism with CloudKit was hard, mostly scattered articles, hope this working example will be useful for other developers.

So, what i mean by clean architecture is that the repository is designed to be swappable to any other repository whenever you wish without modifying code everywhere in your application. This is true even for an over the network repository, so it is wise to design it with completion blocks in mind even if you don’t really need (for the local db on a Mac app you’ll most probably don’t need). I wrote about this in my previous article when I moved away from Parse: https://medium.com/@cristi.baluta/how-to-move-away-from-parse-9f8b82edf6ca

Syncing is not easy, and it might be overwelming trying to get it right from the beginning, better to take it easy and advance when you feel comfortable understanding the current step you’re at, why something works and something doesn’t. It took me few weeks part time.

Components

1. a local repository (sqlite in my case)
2. a remote repository (CloudKit)
3. structs to carry data outside the repositories
4. a sync manager that will keep them in sync

This article assumes you already know what CloudKit is and you know how to configure it. Because my developer email is different than my iCloud email i’m not even able to show you screenshots with real data from the db, i guess Apple didn’t thought to that when recommended to keep separate accounts for development and for personal use.

The interface

The interface for the two repositories is the same, their implementation is different and some methods might not be possible to implement so it’s okay to leave blanks. For example queryUnsyncedTasks synchronous method is not possible to do with CloudKit.

Something to note: deleting a task permanently from CloudKit is not possible, it will only be marked as deleted. I implemented the same thing in my local repository, logic that I didn’t had previously as I didn’t needed, when something was deleted was gone for good but now I want to tell CloudKit what was deleted. The permanent deletion will be done only when CloudKit tells you something was deleted on server. When working with the network you have to always assume that an error can occur and you need to re-do the operation later, and this is why we keep locally deleted items till the server tells is ok to delete.

The struct used in the app

Sync steps

Let’s look at the sync steps first, then we’ll look into the CloudKit repository implementation. I understand things better if I draw them

And here’s the code

This sync works by uploading local changes first, then getting the server changes later. What is nice is that if you delete something locally, you can wait for the CloudKit to tell you that it was deleted, so you do the permanent deletion from local db from a single place. The drawback of this logic is you have to fix the conflicts on the server, which is not possible with CloudKit.

1. get the unsynced + deleted local tasks and store them in arrays. The local repository should know what is unsynced based on the lastModifiedDate and markedForDeletion properties. An idea might be to store the deleted tasks in a separate table.
2. send the changes (save + delete) to CloudKit. I chose to send them one by one recursively with the syncNextTask method and the reason for this is that my app has little info to sync and it will not exceed the quota or something. Another reason is the difficulty of saving, we’ll see that later. You can save in batches too with CKModifyRecordsOperation
3. get the latest changes (updates, insertions, deletions) from CloudKit and save them to or delete from the local repository. ColudKit returns changed and deleted records in the same call so I designed my repositories to return in the same manner.

After sync finishes, if there is any change coming from server we’ll return hasIncomingChanges=true in the block, it is useful to refresh the UI after sync.

The CloudKitRepository

CloudKit has one public database and multiple private databases that are differentiated by zones. There’s a private db that has no zone too.

For a fully working sync we need to use a private database with a custom zone, for two resons:

1. getting changes from db requires custom zones.
2. custom zones are possible only in private db.

Another reason would be if you want push notifications which requires custom zones too. In this example we don’t use notifications because being in sync instantly is not crucial.

This is how you init a custom zone. The case of failure is not handled, but you should.

Repository protocol implementation

Now let’s implement the repository methods that are used in the sync process.

1. firsts are save and delete. Note that we receive from outside a Task struct but to work with CloudKit we need CKRecord objects. This is what cktaskOfTask is doing, is fetching from CloudKit the right CKRecord. If no CKRecord is found we create one now when saving. When deleting is no reason to create one. This might not be the most efficient approach but when you have little data is fine. Apple recommends to cache the CKRecord but seems complicated to me, where are you gonna cache it? If saving in sqlite, we’ll break the clean architecture and pollute the struct too, i think is a matter of what you want to sacrifice. After you have the CKRecord you need to update it and save it back to server.

IMPORTANT: The recordID of the CKRecord must be the same with objectId of the Task. recordID is not a string but it has a property recordName that must be unique. You need to specify the zone too.

CKRecordID(recordName: task.objectId, zoneID: self.customZone.zoneID)

If you don’t create the id manually it will be created by the server and returned to you after saving, but this will be much more complicated to handle because you need to save it back to Task, and if that Task is in use you’ll lose it and eventually get duplicates in db. task.objectId is created by the local repository.

2. next and last public method is for getting the server changes since our last sync. We use CKFetchRecordChangesOperation to get them and we call it recursively till moreComing property of the operation is false. Next thing is CKServerChangeToken which comes with the operation and must be stored locally and used next time changes are requested, this ensures that you get only the changes from the last sync and saves everybody time. First time you do the call you pass nil, but later you have to use the token given by CloudKit. We’ll store it in the UserDefaults

Here some helpers for:

• fetch CKRecords
• create a Task from CKRecord
• extract string ids from CKRecordID
• save to UserDefaults the CKServerChangeToken

What do you think, did it help, will I run into troubles?