Offline search
A week ago I published a small Android library with big ambitions.
Offline-locations-lib is a tool that grants any Android app access to text search of the OS OpenNames data set, fully offline.
OS OpenNames is a data set released by Ordnance Survey that contains every place, street name and postcode in the UK.
The use cases for fully offline search are a little unusual. Here are a few potential reasons you might choose to use the library:
- Operating on geographical data under flakey (or zero) data conditions (perhaps underground, inflight, offshore, or in other regions with poor signal).
- Wishing to avoid the impact of high bandwidth use or API fees when communicating with an online locations search service.
- Avoiding the risk of communicating with unvetted online services for your geographical analysis work.
- Working offline or inside a private network that is not connected to the internet.
Granted, these use cases are a little out of the ordinary, but privacy is valuable to many people for many different reasons — and a primary benefit to users of offline-locations-lib is that it does not rely on online services to perform lookups.
Can I try it out?
Yes! Alongside the client library, that you can build into your own apps, offline-locations-lib provides:
- The Ordnance Survey Offline Data Source app, which makes offline location data available to other apps on your Android phone.
- An Offline Places Sample Client app, which illustrates how you can quickly code a map to use offline-locations-lib for its search capability.
Here’s how to get started…
- Install the Ordnance Survey Offline Data Source app from Google Play.
- Open the app, and choose to extract the data.
- Once this is done, download the Offline Places Sample Client app from Google Play.
- Open it, and use the floating action button to search for a location, eg. Bristol.
The search takes place entirely offline, by communication with the Content Provider contained in the Offline Data Source app.
NB. The sample app is illustrative and uses Google Maps as a base map. The map itself does require internet access to render.
How do I use it?
When you’re ready to build offline-locations-lib into your app, follow the instructions in the github repository README to get started.
So how does it work?
The offline-locations-lib provider uses data downloaded during installation of the app. This data can be either delivered as an APK expansion file from the Google Play Store, or bundled inside the app as an internal resource.
In either case, the app uses the raw OpenNames dataset zip file as provided by Ordnance Survey (from the open data downloads section of their site) as its source.
This source file is reasonably small when zipped (a little over 100Mb). When unzipped it expands rapidly — coming in at several Gb! In order to keep its download size to an absolute minimum, offline-locations-lib uses the zipped form of the data, and indexes it (storing its retrieved place data in a Room database) without ever fully unzipping it.
This process isn’t quick! Indexing of all places in the data set can take up to half an hour (tested on a Samsung Galaxy S6, 2017). This is a one-time hit however, and once done the user won’t need to do it again until the data set updates.
Once the Room database is filled, the app makes a Content Provider available, which other apps can access. Currently, you’ll need to know the authority for this provider (com.flt.liblookup.opennames.provider), and use the appropriate permission in your manifest (com.flt.liblookup.permission). That permission is automatically granted.
Whilst it is possible to communicate with the Content Provider directly, I’d recommending that you use the client library provided to keep things simple!
Questions and answers…
Can I search by location coordinates or location type? Not yet I’m afraid! The search library currently provides just a simple text-search returning exact matching results and results containing your search term. Search by location and search by location type are top of my personal wanted-features list. I’ll keep you posted!
Why do I have to provide the space in the middle of the postcode? This is a temporary issue. Postcodes are stored with the space, and a simple text search can’t currently match the postcode if you don’t include the space. A specialised search for postcodes is planned, too!
How can I update my locations data regularly? Right now, you can’t. However, by the time you actually ask this question in earnest, I anticipate having built an update to the app that can use Google Play’s APK expansion features to update itself; use an alternative download location you (or the user) provide; or use a connected storage device to update from.
Are searches quick? The provider may take several seconds to return a search result. The client library performs this as an asynchronous task. During that time you’ll need to provide your user with an indication that a search is taking place. The client library’s search dialog takes care of this for you. As a rough guide, searches seem to take as long as you might epxect them to take when using a online web service.
Are there any rate limits? No, however you should be aware that searches consume a lot of resources, and so you may decide you should rate limit your usage of the ContentProvider to help keep the phone responsive (and preserve battery life!)
Why am I restricted to UK locations? Ordnance Survey are a UK data provider, and authoritative source for UK locations data. The OpenNames data set does not contain locations outside of the UK. If you are interested in helping to adapt offline-locations-lib for another country or state, and you have access to a freely available and authoritative data source for that place, do please get in touch.
