How to make your Flutter app offline-first with Couchbase Lite
Why I developed Couchbase Lite for Dart + Flutter, how to get started with it and how to implement full-text search locally
Oh wow, it has almost been a year, already. Time flies. In January, I was working on a Flutter app as a side project and wanted to make it offline-first. As many features should work as well as possible, with bad or no connectivity. With mobile network availability always improving, one might think that the offline-first approach has lost relevancy, but doing things locally is also beneficial for responsiveness and user privacy, among other things.
I started out using Firebase Firestore as a storage solution, which is usable offline but has limited and coarse controls for related features. The only setting to determine what is stored locally is the cache’s size in bytes. You can still write to the store when offline, but there is no conflict resolution mechanism. As far as I can tell, the last write to the server wins.
And then there is full-text search (FTS), or rather the lack thereof. The Firestore documentation basically tells you to check out Elastic, Algolia or whatever… just figure it out. 🤪 Hm, ok, what about SQLite? It comes preinstalled on many systems and has multiple FTS extensions. Looking into it, I found that while possible, it’s not exactly simple to make it work. Not all preinstalled versions of SQLite have the same FTS extension enabled or any at all. Of course, the versions of SQLite itself differ between platforms and platform versions, so you have to take that into account. As with any relational database, you have to manage a schema and setting up a FTS index in SQLite is not super simple, either.
Searching for a mobile storage solution that gives me full control over what is stored locally, supports conflict resolution and FTS, I eventually found Couchbase Lite, an embedded NoSQL database which fulfilled all my requirements and some more.
At the time, there were multiple packages available on pub.dev, that were implementing Coubhase Lite for Flutter, with various levels of maturity, platform support and feature completeness. None of those were a perfect fit, so naturally I abandoned my side project and started shaving a Couchbase Lite sized yak. 🙊
I decided to start a new project because I felt a specif architecture was necessary to support both standalone Dart and Flutter, but thank you to the creators of those packages for giving me points of reference.
About a year later, cbl-dart is in a pretty stable and usable beta, with full support for all platforms that are supported by standalone Dart and Flutter (except for web). The same Dart API can be used to persist data in a Flutter app, inspect databases in a CLI tool or customize prebuilt databases in a server app.
If you’re interested in the architecture, how to get started with Couchbase Lite and how to implement full-text search with it, continue reading.
Architecture
This section walks you through the project’s components and explains how they fit together. If you're not interested in the architecture of the project, you can skip to the next section.
Both cblite and cblitedart are native dynamic libraries, that expose a C API. cblite is shipped as part of the Couchbase Lite C SDK. This is where the core functionality of Couchbase Lite is implemented. dart:ffi allows Dart code to call native C APIs, but has some limitations, which make a support layer necessary. That layer is provided by cblitedart.
The internal Dart package cbl_ffi encapsulates all the code necessary to call into cblite and cblitedart from Dart.
cbl is implemented on top of cbl_ffi and provides a Dart API for Couchbase Lite. This is the package that user code interacts with mostly.
cbl needs to be initialized, involving things like loading and initializing the native libraries. That's the purpose of cbl_dart for pure Dart apps, and cbl_flutter for Flutter apps.
Getting started
Getting started is simple. I’ll show how to prepare a Flutter app for usage of Couchbase Lite, including in unit tests. We are going to
- add the required dependencies,
- initialize Couchbase Lite in the app and
- initialize Couchbase Lite in unit tests.
Add the following dependencies to your pubspec.yaml:
Initialize Couchbase Lite early in the app lifecycle:
Similarly, initialize Couchbase Lite in a setupAll hook in your Flutter unit tests:
Being able to use Couchbase Lite in Flutter unit tests is great because they launch much quicker than integration tests on devices or simulators. The short feedback loop enables exploration and test-driven development, which can become tedious with full-scale integration tests. Be aware though that unit test run in a headless version of Flutter on the development host. They cannot replace testing of database code in integration tests on real devices or even simulators.
Implementing full-text search
To give you a taste of what it’s like to work with Couchbase Lite and to demonstrate how easy it is to use the FTS feature, I’ll show you how to implement FTS for a simple note model. We are going to:
- open a database,
- create new notes,
- create an FTS index for the notes,
- use that FTS index in a query and
- extract data from query results.
First, let’s look at how to open a database:
Note that we are opening an AsyncDatabase through Database.openAsync. For now, all you need to know is that the whole Dart API has a synchronous and an asynchronous version, and you should use the asynchronous API if you are unsure.
When opening a database without specifying a directory, a default directory is used. For Flutter apps, that is the directory returned from path_provider's getApplicationSupportDirectory. If the database already exists it will be opened, otherwise a new one will be created first.
With the database open, we can create notes. In this example, notes have an ID, a title and a body:
Before we can run queries that make use of an FTS index, we need to set one up and define which fields of the documents to index:
We’re going to encapsulate each search result in a simple data class. The query will only need to fetch those fields that we need for this class:
Finally, we can use the FTS index in a query:
The query is written in a query language that is called N1QL (pronounced nickel, like the metal). It’s like SQL for JSON. There is also a type safe query builder API, if you prefer that.
If you are familiar with SQL, you will recognize the typical structure of a SELECT statement. Something that will be new is the META() function. Every document has metadata that is managed by the database and can be accessed through that function. The unique id of a document is part of that metadata. Regarding the FROM clause just know that you need it and need to select from _ .
The part of the query that relates to FTS are the match and rank functions. Both take the name of the FTS index that should be used as the first argument. match additionally takes the FTS query as the second argument.
Note that we appended * toqueryString, before initializing the query parameters, to perform a prefix match on the last word. When you run the query, each time the user types a character, they will get search results before they have finished a word. You can learn more about the supported FTS query expressions here.
The rank function returns the rank of the search result relative to all results.
Now you know all that is necessary to build a fully local, always available full-text search.
I’d love to hear your feedback and answer any questions.
Learn more
If you want to learn more about how to use Couchbase Lite with Dart and Flutter, have a look at these resources:
- The
cblpackage README: For more on key concepts of the API and usage examples. - The
cblAPI reference - The
cbl_dartpackage README - The
cbl_flutterpackage README - The
cbl_flutterexample - Couchbase Lite Swift Docs: For more information on Couchbase Lite concepts. The Swift API is very similar to the Dart API.
- N1QL Language Reference
- Sync Gateway Docs
