Dealing with legacy code: moving from ORMLite to Room

There’s a lot of great posts explaining how to move from ORMLite to Room in an Android app, but most of them focus on how to do it in perfect conditions. In our case, parts of our main Android app are old, very old, and follow many different patterns, some of them making it hard to update a core element like the database without breaking anything.

We’ve recently released an update that migrated the old ORMLite database to Room, this article will focus on how we dealt with our legacy code during this change.

Why switch to Room?

The main issue with our use of ORMLite was that since it made it easy to query the database from anywhere and get results seemingly instantly, we did call it from anywhere, and often on main thread.

Calls made from the main thread without realizing were becoming a big issue. ORMLite is really good at what it does, making you forget you’re calling a database. This is an example from our actual code, that we used to determine whether a category was a leaf (no subcategories):

getCategoryById calls the database, it’s pretty clear, but getSubcategories also does. Twice. Now imagine a category with 20 subcategories for which we want to check whether they are leaves or not, using this leads to 60 database calls just to find out which of these are leaf categories, and this on the main thread (easily changed to 40 by changing call to getSubcategories, but that’s still 40 too many).

Making sure we don’t write this again was of course a priority. Fixing existing calls is another story though, some of this code is pretty complex and would require major rewrite to make it asynchronous.

Given the amount of refactoring required, this presented itself as a great opportunity to consider more modern tools. After considering a few options we decided that Google’s Room was the best option, it supports RxJava and thread validation.

We had a few requirements for this update:

• All new calls should be forced out of the main thread.
• All calls to the database should be proxied, no more direct calls to get a list of ForeignCollection for example.
• Solid data migration from old database to the new one.
• Some data is “static” and very infrequently updated, we wanted that data to already exist on app install.

Sudden vs progressive migration

We could either switch from ORMLite to Room in a single release, or slowly migrate while keeping both side by side for some time. A progressive migration makes dealing with legacy code easier, for example keeping the “worst offending tables” on ORMLite while doing new stuff with Room, then slowly refactor those calls, make them async and finally switch the table to Room.

We decided never to have ORMLite and Room side by side. While this seems to make some things potentially easier, it makes maintenance and testing a nightmare. Migration scripts would get more complex, needing to maintain 2 databases at the same time. But worst of all, we must be careful about always being able to retrieve data from previous ORMLite tables when switching to Room, and this for any previous version of the app. Slowly migrating means many different versions of both databases still exist in the wild, as a lot of users don’t update their apps regularly.

Preparation work

While suddenly switching to Room means in one single release we dropped ORMLite dependency and replaced it with Room, this does not mean all our work went into this specific release. We did a lot of preparation work before this switch happened:

• We looked at every table in the database, and asked ourselves “is this really something that should be in a database?”. We identified a few tables for which a database use did not really make sense. Why store complex objects over multiple tables when you never store more than one, and never really query it? Simple serialization to disk is a much easier way to deal with these.
• We also looked at tables that could be now unnecessary. Our API evolved a lot during the past years and some “static” data we stored locally is now always available in API replies when we need it. This also helped us remove a few tables.
• Finally, all new work for a long time before the switch was made using asynchronous calls (with RxJava), that are easier to convert to Room than direct synchronous calls.

Implementing the new database

Many good articles and Google codelabs already explain how to implement Room properly, I’m not going to focus on this. To make our legacy code easier to support, our tables are mapped to objects that reflect almost exactly the ORMLite objects we were using, this helped make changes to existing code minimal.

e.g: we had a Category table in ORMLite, now we have aCategoryEntity in Room, that we map to/from Category objects that are almost identical to the ORMLite objects:

database/
├── model/                     — public model
│   └── Category.kt
└── room/                      — private package
    ├── converters/            — converters for custom types
    ├── dao/
    │   └── CategoryDao.kt     — all SQL queries for category table
    ├── migration/             — migration scripts
    ├── tables/
    │   └── CategoryEntity.kt  — category table
    └── Database.kt

Dealing with main thread

As said before, with ORMLite we often made calls from the main thread. Room prevents this by default, which is a very good thing, but made legacy code support hard for us. Room has an “allowMainThreadQueries” setting, but that’s a global switch that would remove thread requirement for all calls, we don’t want to enable this.

We wrote a small workaround for this, which consists of a method that pushes a call to a background thread, but waits for it on current thread. DO NOT use that code unless you’re 100% sure about what you’re doing. This is a bad practice, and should be avoided as much as possible. But sometimes when dealing with legacy code you have to write things like that:

In this particular case, we decided that it was OK to use this, because it works “as before” with ORMLite, and fixing all calls on main thread immediately was impossible.

One good side effect of using this: now ALL your legacy calls go through the same method, which makes it really easy to profile. We used this to find out which calls were the most common and were taking the most time on main thread. This gave us some really good insight on which calls we should focus on fixing first.

Using the above code slightly modified to store calls with counts in a tree instead of direct logs, we were able to pinpoint the worst offenders very quickly. Our search screen was doing over a hundred calls to the database when being simply opened (isLeaf example above). This tiny piece of code made us find out about it immediately, and that was one of the first fix we made right after migration was complete.

Migrating data from ORMLite

Migrating data was pretty straightforward, registering a callback on Room database onCreate event, we check for a previously existing database, if it exists, we call our MigrateToRoom migration code:

We can then open the previous database using the using lower level SQLiteDatabase objects:

Once we’ve opened it, we can read its content using cursors and insert into Room using standard DAO operations:

We had to deal with multiple versions of those tables, sometimes querying sqlite_master table to get info about current version of the table (checking if table exists, or if a column exists), and decide how to migrate depending on this.

For example, checking if table “biddetail” exists:

Or checking multiple versions of “recentsearch” table, in some very rare cases (a few users who haven’t updated for years), we drop table content, for other, we check for existing columns to decide how to update. In this example, we drop migration if the table does not contain “title” column, and if it does not contain “dateModified”, we update our migration queries accordingly:

Another pain point was that we used Serializable data in some of our ORMLite tables. This is something that ideally should not be done, we can easily read them from BLOB database fields, and convert them to our new Room entities:

Database prepopulation

We also used this opportunity to update the way we provide some of the static data we keep in database. Before, we’d ship the app with a json file coupled with an ETag indicating the version, that file would be parsed on first launch to populate objects that we’d then insert in database. This was easy but very inefficient, the switch to Room was a good time to make things better.

We now ship the app with a prepopulated database that we create during build, which made our first launch almost as fast as the following ones, often close to one second on modern devices. More information about this in the following post:

Android: Prepopulating database during build