Browser Data Persistence

The options available to us:

Browser data persistence is NoSQL

SQL (Structured Query Language) and NoSQL (Not-only SQL) are not databases themselves but rather ways of interacting with data. And while the boundary between them is becoming slightly blurred, with some SQL databases taking on NoSQL-like functionality and vice versa, it’s still worth quickly reminding ourselves of the difference between the two.

SQL

• SQL is the universal interface for Relational Databases, of which there are many.
• A Relational Database is a collection of tables that hold rows of data categorised in columns, and that can have functional relationships to other rows of data in other tables.
• The types of functional relationships can be one to one, one to many, many to one, or many to many.
• Requires a Schema, an up-front blueprint/configuration for your entire database, thus being a lot more rigid and requiring a lot more planning.
• Scales vertically. more power comes from increasing the computational output of your server via CPU, RAM and physical storage speeds.
• Supports transactions, a collection of actions that either all pass or all fail, allowing for greater data integrity and reliability.
• Examples include MySQL, PostgreSQL and Oracle server.

When to use SQL

• Data requirements can and must be identified up front.
• Enterprise level architecture that must maintain the highest level of integrity and reliability.
• Cast-iron established standards (ACID) and wide developer support.

NoSQL

• As the name implies, some NoSQL databases may in fact use SQL-like behaviour, but they are not bound by them and typically will not use them.
• Instead of storing data in tables, data is stored in key-value pairs, graph databases or columnar stores. Most often analogised as storing “Documents” as opposed to tabular relationships.
• Does not require schemas or much up-front planning, can be dynamic. Making them much easier to set up or change. Much easier to use in the short term.
• Scales horizontally, more power comes from adding more servers to the database infrastructure.
• Typically will not use transactions, but they are sometimes available.
• Examples include MongoDB, Redis or IndexedDB for the browser.

When to use NoSQL

• Unknown or fast-evolving data requirements.
• Faster to production, prototyping.
• Speed and scalability of data infrastructure.

WebSQL and SQLite

There once was WebSQL which was a SQL database based on SQLite for the browser but due to differences between Mozilla and Microsoft and the rest of the major browser producers it was deprecated several years ago in favour of a new standard, IndexedDB.

SQLite touts itself as being the most used database in the World, and it’s pretty sweet, but finding a good way to use it in all major browsers is proving tricky.

So, having said that…

Let’s now look at what we have available for our client-side data storage needs.

Cookies

These are small collections of data (key-value string stores) that can be stored in the browser and get sent back and forth between client and server. They typically manage:

• Tracking.
• Personalisation.
• Session state.

Cookies are the most limiting and least performant of the data persistence methods, clocking in at a maximum of 4kb each and a recommended maximum of no more than 50 cookies per domain, and being the most transfer-inefficient due to being shipped back and forth with every HTTP request whether or not they’re required.

However, for the above use cases they are very useful and probably best practice. Once upon a time they were used widely for client side storage but we have far better options now.

Note that there is legislation governing cookies.

Session Storage and Local Storage

Both are types of native HTML5 Web Storage. Akin to cookies, they are NoSQL key-value stores but they can store any JavaScript value, including string, ints, bools, functions, etc.

The other big difference between cookies is the size. For all intents and purposes we can consider web storage limits to be 5mb per object, 10 mb per domain and 50mb (max) per computer.

• + Supported on all major browsers.
• + Simple API.
• + Unlike cookies, they do not get transmitted with every HTTP request.
• - Poor performance for large data.
• - Unstructured data can lead to poor integrity.

Local storage will persist data for a domain until cleared or altered by the user or JavaScript, whereas session storage will only persist data while the browser tab is open.

Overall this is probably the best way to store simple data that is not personal to the user and not too large or complex, and it’s super easy to interact with as a developer.

File System

Definitely not SQL and you’d be hard pressed to describe it as NoSQL, but it is a way to persist data for the browser so I’ll quickly mention it.

• + Good for storing very large files and binary data, e.g. Images and PDFs.
• + Great performance and asynchronous.
• - No transactions.
• - Poor (native) browser support, only Chrome and Opera out of the box.

IndexedDB

This is where we solidly re-enter the world of browser databases. Whilst it’s not a NoSQL database in the same sense as MongoDB or CouchDB, it only stores JavaScript objects, it does however give us access to complex database like behaviours such as transactions, and decent performance to handle large datasets.

Therefore it’s considered by most to be a NoSQL database, and the child of necessity that was born out of the demise of the WebSQL standard.

It’s got a lot more capacity compared to all previous entries, weighing in at a total of 50% of your free hard drive space, with no one origin being able to take up more than 20% of that allocated space.

While on it’s own it isn’t bad, it’s really designed to shine with Service Workers. The main point to call out here is that IndexedDB will block the DOM while it performs a transaction, unless it’s used with a Service Worker. That might mean a few milliseconds or much, much more.

• + Good performance.
• + Good documentation and easy learning curve.
• + Supports versioning.
• + Works well with Progressive Web Apps and Service Workers.
• + Supports transactions.
• + Potentially large capacity for storing data.
• - Can have an unwieldy API with multiple nested callbacks.
• - Will block the DOM unless programmed via a Service Worker.
• - Not as performant as a proper SQL database.

MDN’s introduction to IndexedDB is a great starting point, and then move on to Google’s own indepth coverage.

In conclusion

For anything essential to personalisation or that contains secure, private information I would suggest a Cookie.

For data that can help build and serve content, or data that is not private and not too large I would suggest Web Storage.

If you happen to be working with Service Workers or creating a Progressive Web App then IndexedDB is an excellent choice because of the middle ground it takes between all the browser based NoSQL solutions and more traditional SQL solutions.

Also great for caching your website offline for users.

You can indeed use it for large data storage needs client-side but you’ll have to be careful how you use it because you don’t want to block the DOM and destroy the UX, or clog up your users’ computers and devices with loads of guff.

If you want to see how various websites are persisting data in your browser then just open up the Chrome DevTools and head over to the Application tab.

The feeling I’m getting from reading around about it is that while it fills a gap in the market it falls somewhat short of being an amazing technology that could still be useful decades from now.

It just feels like it doesn’t deliver enough to please everyone and that something newer on the horizon might be the actual answer we’re looking for. Something called NewSQL. A new breed of database that seeks to combine the scalability and ease of NoSQL with the consistency and performance of SQL, like Google’s Spanner for instance.

But, at this point, those technologies are still immature so we’ll just have to make best use of what we have and keep our ear to the ground.