Stored data in iOS: save, update, use

Hi there! My name is Alex, and I’m an iOS developer at hh.ru.

Countries, cities, professional fields, languages, currencies are the names of directories within our mobile app. The names are changed very rarely but used everywhere, therefore they must be up-to-date and users and developers should not waste time updating them. This article will show you how to work with directories simple and convenient.

Where we started

We have been applying directories in our application for a very long time. Until recently, they were just ordinary text files where we saved responses from the server in JSON-format. There were about 20 MB of such data which is really a lot for a mobile application. Loading this data into the app was very slow, and the wrapper was written in Objective-C. In short, the horror and nightmare of legacy.

A year ago, we started working on a new application for employers. One of the challenges for us was to create a new feature for working with directories. In its current form we didn’t want to drag it into a fresh and brand-new application, so we decided to change the directories.

I contacted various developers, talked to them about preferences for the new feature, and made a list of requirements:

• data should be available as quickly as possible, right away from the application launch;
• it is necessary to provide updates of directories from the server;
• take into account that the structure of information can change. For example, new directories can be added, old links can be changed, so we need to provide for the possibility of migration;
• provide a quick search for information in these directories;
• the whole thing should work in a separate feature, which will be shared between our applications.

After looking at these requirements, I realised that the question “how to store directory information” simply does not exist — we need a database. It will provide us with the storage of directory information, quick search, the ability to migrate and update.

So most of the requirements we meet are practically “out of the box”. One thing remained to understand — what kind of base we wanted to use and how we would work with it.

Eventually, we ended up with several tasks:

• First, we need to learn how to deliver the data along with the application;
• Second, we need to make it easy to work with the data inside our application;
• Third, we need to make it so that the data is updated only when needed.

In a perfect world, the developer doesn’t need to update data in the bundle manually. In a perfect world, all the necessary updates happen either on schedule on the CI side, or during the creation of a release candidate for regression. Some scripts are launched and a fresh version of the database is pushed to the repository. The validity of that database is ensured by integration tests. In general, we needed to automate the delivery of data with the application as much as possible and combine the generation of directories between iOS and Android.

Databases and their precious fellows

Taking the main requirements as a basis, I have collected the key options for working with the database in iOS. There were three options: CoreData, Realm, SQLite. In addition, I looked at less popular options like YAP, but quickly gave them up. Now I will briefly talk about the pros and cons of each of the main options.

CoreData.

Pros: it is as native as possible, adding a framework does not affect the size of the application, and most of our developers have already worked with it. Plus, it’s easy enough to write migrations.

Cons: The database can be only generated on Mac or iOS. You can’t just go ahead and generate a SQLite file and swap it in. You have to be sure to use the model version from your project. CoreData is only suitable for iOS — no cross-platform. It also has a rather bulky API. Apple has improved it, but CoreData still loses a lot to other solutions in terms of usability.

Realm.

Pros: cross-platform, fast data search, simple and clear code in the project.

Cons: To generate, you need a project with Realm connected — you can’t do without a script here. This is a third-party library and an additional dependency, which also increases the size of the application from 5 to 14 MB.

In addition, our Android developers did not want to bring Realm into our application. Accordingly, we lose the advantage of the cross-platform.

SQLite.

Pros: The main advantage is that the database is easy to generate. Just write scripts that create tables and insert data. It’s the ultimate cross-platform: the database engine in Android also uses SQLite.

Cons: The minuses and other pluses depend on which library you choose to work with the database. I considered three options.

One of them is native, it has a big advantage in the form of no dependencies. But there is a big disadvantage: the API is inconvenient. Apple suggests using the OpaquePointer to work with the database.

GRDB.swift.

Pros: easy to work with data, fast speed, excellent support of the author and the community. Cons: it has no support for Carthage, and it was a principled position of the library’s author. For us this was a stop factor, because we actively work with Carthage and all our main libraries are connected with it.

And finally, SQLite.swift.

Pros: easy and convenient, there is community support. We already had this library in our project as a dependency on another library.

Cons. At the time of research, the last commit was in the repository about two years ago. But before recording the video version of this article, I visited the library’s page, and it was updated quite recently. Another thing to note is that a separate manager is used for migrations, but that’s a common problem with working with SQLite.

For each of the five database options, I created a small test project that demonstrated the possibilities of interaction: how to work with models, how the database itself can be generated and updated. And then my team and I held a small vote to single out the absolute favorite. It is worth noting that we were solving a specific task of working with directories, rather than choosing a single database for our application.

We evaluated the work with the database according to 4 indicators: convenience of generation, convenience of work in the code, data migration and quick search in the database. But there were only two really important criteria — the convenience of generation and the convenience of working in code, taking into account the goals we set. In the end, the 1st place in generation was deservedly taken by SQLite, the 2nd by CoreData, the 3rd by Realm. In our conditions, it was very difficult to generate a database on Realm.

But in terms of work convenience Realm took the first place, of course, taking into account the work scenarios that we developed according to the voting results. In second place was SQLite, not the native one, but with the use of some libraries. And in third place was CoreData. As for the rest items — any variant gives a good search speed, and we refused migration at all. But we will come back to it later.

After all, we chose the easiest way to generate SQLite and the SQLite.swift library, which was already in the project. SQLite database generation itself is quite simple. We wrote a script in Python that downloads data from the server and puts them into tables, with each individual directory putting them into its own table. Or as, for example, in the subway, in two tables: separately for stations and separately for lines. Our final database weighs about seven megabytes, but before adding it to the bundle, I put the whole thing in a zip-archive and compressed it to two and a half.

Directories, tests, data

I won’t go too deep into the work with the directories, because there’s nothing interesting here. In general terms, for each table a different provider was written. And it had two types of get-methods: to work synchronously and asynchronously through Combine.

Synchronous methods were needed to simplify the work with legacy code, because everything worked synchronously there. And in order to switch to directories gradually, I added an additional legacy wrapper for them. Synchronous calls are sometimes much easier to build, and for simple queries it works very quickly.

And perhaps the most interesting part is the tests. Tests have been written on the entire database and on each table to make sure that the model in the code matches the tables in the database, which is in the application’s bundle. This is how we make sure that our Python script has generated exactly what we need and that the code is up and running.

And now we have only one task left to solve: updating the data while the application is running. In the course of discussions and the choice of the database, to make things easier, we decided to eliminate the option of migrating from one version to another.

Now everything works as follows:

• First, if the user downloads a new version of our application, at launch, the database from the bundle will replace the one available now. This simple mechanism allows us to completely forget about migrations and not to be afraid that the data models in the code and table will not match.
• Second, we decided to add a special table with metadata to the database. We store the latest update date for each directory, as well as a special ETAG number in it. Seven days after the latest update date, the application will try to download new data, taking ETAG into account, and will change the date in the table to the new one. Using ETAG means that if the data has not changed on the server, nothing will change in the database either, and there will be no unnecessary traffic or load.

Summary

By switching to SQLite for directories, we got rid of JSON files and reduced the size of our application by 15 mb. Updating the version of the directories in the bundle has become much easier, and working with them in code has become much more comfortable. And my main message for today is: “Do not be afraid to use the SQLite database in your projects. In some scenarios, it can give a tangible head start to other solutions.”

That’s it for now. Ask any questions in the comments to the article or on our Telegram. See you soon!