Tech Stack Choices: Data Store

This is the first entry in my series on tech stack choices when you’re working on side project with a small team (or even by yourself). You can find the introduction here. Today we’re going to be discussing data!

Some applications work quite well with only local storage. Most, though, will need some form of centralized storage for long-term persistence and recovery, social or sharing features, data insights, the list goes on and on. As a small team, you don’t have the luxury of managing a database, dealing with authentication systems and credentials, networking code, data syncing, offline-first (or at least offline-capable) principles, optimistic updates, network recovery… the list goes on and on.

You need a data store that can handle all of the above, scales when you need it to, and, most importantly is easy to set up and maintain.

Enter Cloud Firestore.

A few alternatives

Before I touch on how Firestore came out on top, it’s important to note that there are others in this space. Quite possibly too many, with more popping up all the time. To be completely transparent, I have not actually used any of the services list below, simply factored them into my research when making this choice. Please leave me a note if I’m off base with anything presented here!

Realm

Realm was one of the first providers I looked into. It checked off a huge amount of the requirements (offline first, automatic data syncing, etc.), and was available for any language you would be likely to use for a mobile application.

After evaluation, there were three things that stood out to me and were enough to knock them out of the running:

Pricing

You get 30 days to evaluate Realm before you have to enter into a paid tier (at the time of writing, the ‘Standard’ tier was $30/mo). That doesn’t give you long to evaluate the product, and even worse, you’ll have to spend money every month before you even have a viable product. Even a severely limited starter plan would go a long way here.

From there, the next (and final non ‘contact us’) tier is ‘Pro’, entering in at a whopping $250/mo. Most of that increase must be to cover the premium support availability, because you’re not getting five times more storage and bandwidth as the ‘Standard’ tier.

To be clear, I’m not making an argument against services that cost money. There are very real and valuable products that are worth every penny you give them. As a small team, especially a small team that’s just starting to build a product (or even side project), the beginning of the process is not the time to start laying out cash.

Confusing product lineup

There’s Realm Platform (both Cloud and Self-Hosted), Realm Database, and Realm Studio.

Simply by the information provided on the pricing page, it sounds like one can use Realm Database as local storage in an application for free without tying to their cloud service. It’s then directly compared to other local storage technologies it can replace: SQLite, Core Data, other ORMs and… Firebase? I’m no longer sure if we’re talking about a local database technology or cloud services.

Data syncing

There are two types of ‘Realms’ you can open… ‘query-based sync’, and ‘full-sync’, each with their own use-cases.

From what I can gather any ‘full-sync’ Realms (which are indicated to be for data that needs to be accessible by many clients) live entirely on each client that has requested them. I’m sure there is likely a good reason for this (and part of me hopes my understanding is incorrect), but if true, would indicate potential scaling issues if your application were to become wildly popular after adoption.

Couchbase

For starters, there is no published pricing for Couchbase. That’s a huge unknown for a small team, and implies that Couchbase is intended for larger-scale endeavors and enterprise teams. The feature-set they provide further supports that hypothesis.

Other than the above, Couchbase looks really solid. Additionally, they offer full-text search as a first-class citizen which is something that Cloud Firestore lacks… more on that in the future.

Firebase (Realtime Database)

While Firebase and Cloud Firestore have many similarities, there are a few sticking points… likely best explained by their own documentation.

There are some applications where it still makes sense to go with Firebase over Cloud Firestore, the clearest to me being anything that has a requirement for real-time user editing or collaboration. Think Google Docs, a mobile game, chat services (especially those with public rooms), etc. The charge-by-data-transfer model will fit your needs far better than the charge-by-data-access model of Cloud Firestore.

Separate from the pricing structures, I also prefer the way Cloud Firestore manages data querying and structure.

Your own database solution

I’m not going to spend long on this entry, as there’s really a wide-open field of choices. Running any sort of server or technology as a small team should usually be your last resort. Even with the ease of AWS (or similar), there will always be work you have to take on to keep things running.

On top of that, if you choose to run a more traditional database server (instead of one of the fancy hosted NoSQL options), you’re likely committing to running (or creating) your own API middleware.

Benefits for a small team

Cloud Firestore runs on the Google Cloud Platform. As such, it’s heavily integrated with their other cloud products, and using all of them in concert is relatively straightforward when you consider how many separate providers you would have to weave together on your own to get the same functionality. The less time you spend context switching between providers and services, writing integration layers, or otherwise fighting with systems you’re trying to tie together, the more your team is going to be able to accomplish.

Core Firebase services

As of this writing, the core connected offerings are:

  • Authentication (spread across several available providers)
  • Database
  • File storage
  • Hosting (CDN)
  • Cloud Functions (more on this in a later article)
  • Machine Learning

Not to mention the huge list of other services you can take advantage of while building your application.

It’s also pretty clear that the Flutter team has paid close attention to their Firebase and Cloud Firestore integrations, making it trivial to use data streams when rendering content (more on Flutter coming soon).

The concepts and terminology used by Cloud Firestore aren’t coming out of left field, which makes ramp-up time for new folks minimal. The dashboard has everything you need to manage and explore your data, as well as connecting your applications. In addition to not having to write any networking code (which is not really limited to just Cloud Firestore and is true of most of the NoSQL offerings), not having to worry about managing users and their credentials is a huge plus! One less thing to worry about.

On top of how easy it is to get started with Cloud Firestore on your platform of choice, their starter tier is absolutely perfect for your initial development phase. You’re likely not going to have to pay for anything until your application starts seeing normal and consistent usage.

Are there any cons?

Not specific to Cloud Firestore, but if you’re used to traditional database structures it will time to get used to the NoSQL approach. Specifically: ‘What do you mean it’s ok to duplicate data sometimes?’. I’ll be writing more about structuring data in Cloud Firestore at a later date.

Cost could be a potential concern with Cloud Firestore. Their pricing seems insanely good on the outset, but it’s near impossible to calculate what your actual cost is going to be without data and usage patterns (that you won’t have until you’re very much further along). Additionally, Cloud Firestore (and Firebase for that matter) can be susceptible to inflated usage due to bad code or programming errors.

A lot of the benefits listed above could also be considered cons because they’re all tied into the same provider. If you start using cloud functions, storage, hosting, analytics, etc. and ever decide to leave Cloud Firestore for any reason, you have to try to decouple somehow, or have a fragmented toolset.

Final thoughts

The major players in the NoSQL space offer very similar features across the board. However, for a small team, it comes down to flexible initial pricing and the available integrations and features. As of right now, Cloud Firestore strikes the perfect balance of services offered, cost for initial integration, and more importantly cost savings for time you no longer have to waste futzing with maintaining and running other services.

Larger teams have the luxury of fighting and managing vendor lock-in, but I would argue that smaller teams should embrace the lock-in to increase productivity.

Always remember your time has immense value. Having more time means you can spend it enhancing your application and engaging your users. If it comes down to spending $100/mo on an amazing service with clear benefit that will regain even an hour or two of your time (especially once your application has normal usage), that’s likely well worth it.