Welcome to the Real-Time World

What is hidden beneath the «real-time» keyword? Can you make a real-time application using the traditional technology stack? In this post, I will review the real-time technology stack and try to answer these questions.

Real-time Application

An application is considered to be «real-time» when it has the next features:

• Data consistency. A guarantee that all clients have valid data on their devices.
• Non-blocking user interface. No lag for user actions during data operations.
• Self-updating data. No further action (pull to refresh or screen reopen) is required to update the data.
• Offline support. The possibility to work with data independently of internet connection status.

Let’s examine how developers try to achieve these features with a traditional technology stack.

Traditional Server

A traditional technology stack includes a DB, a server and one or multiple clients that are connected with REST and WebSocket protocols.

Clients are accessing data from the DB through the server. Each exposed DB operation is wrapped by the server-client protocol: if you want to add, edit or remove entity in the DB you need to request the server for that. A traditional server is a proxy with a custom protocol between the clients and the DB. Anything you do with the DB goes through the server.

Traditional Client

In the initial stages of mobile development application data was a replication of the server data. Applications were simple views of the server data and had to request the server for each data operation. A characteristic feature of such applications is a blocked user interface during data update from the server:

These applications are reliable, but not very useful. It’s a good thing, then, that such applications are rarely seen today.

The next improvement was somewhat of a trick: non-blocking data update operations without a server approval. Update data first and request the server for an update after. In case something goes wrong, revert the update on the client:

With a non-blocking user interface a lot of new problems are introduced, as well as specific use-cases the developer has to think about and handle with care, including:

• handling each server request and reverting data changes
• situations when the user quits the application before the request is processed by the server
• implementing a queue of server requests
• more complex offline support mechanisms

Besides, there is no way to achieve the requirements for the self-updating data and offline capabilities without server support. These features have to be implemented on the protocol level. The server should notify the clients about data changes and handle data synchronization between them.

A summary: real-time requirements are a pain for both client and server developers who have to spend a lot of time working on implementing applications with an «almost» real-time look and feel. Developers regularly make choices between a real-time feel and complexity of the application.

What if in reality we are using a wrong approach and not the right tools?

Real-time Databases

A real-time technology stack includes a DB, a server and clients that are connected to both DB and server at the same time. The server doesn’t duplicate DB data operations and is using the database the same way as clients do.

Real-time databases and their SDK’s support all real-time features from the box. Developers work directly with self-updating data with offline capabilities. An example with a Realm database:

Real-time database enables automatic synchronization of data on the server and across all the clients. All protocol code (including networking) is hidden in the SDK’s: no need for HTTP requests or WebSocket commands.

The server, simply put, is one of the clients, but with admin database permissions. Clients use the server for massive operations that can’t be performed on mobile devices, for example report generation or data analysis.

The database persists data locally and the user can continue using the app and modifying data without Internet connection. Once the connection is reestablished, the client receives all changes it missed on, synchronizing with the current server state.

In addition to basic real-time functions the database has some interesting features:

Authentication

Every real-time database has an authentication system that is required for client identification. The developer can use default authentication modules or integrate a custom authentication system. The set of default modules depends on the database you chose for the project. Some of them only support the OAuth protocol, while others provide implementation for authenticating with Google, Twitter, FB, Github and iCloud.

Conflicts Resolution

A conflict usually occurs when two clients are offline and make changes to the same part of data. Some databases provide interfaces for conflict resolution. But most real-time databases automatically resolve such conflicts as inserting items in the same position or editing the same property. For example, the Realm conflict resolution strategy:

Deletes always win. If one side deletes an object it will always stay deleted, even if the other side has made changes to it later on.

If two items are inserted at the same position, the item that was inserted first will end up before the other item. This means that if both sides append items to the end of a list they will end up in order of insertion time.

Migrations

Data will change over time for any real-world application. Real-time databases possess embedded mechanisms for schema updates and migrations definitions. In Realm:

Observations

As was mentioned, a «real-time» application must always keep the data updated to the latest available version. Real-time databases support observing data properties to receive data updates, for example Firebase:

In the same way, you could observe data queries, here’s Realm’s version:

Application developers must keep in mind that realtime-capable databases are not magic. They are complex products that use advanced data structures and algorithms to enable real-time features for client-side convenience, resolving most technical issues you previously had to resolve yourself. As such, all of them have their advantages and issues. It means you have to do some research before choosing the right real-time database for your needs. The good news is that in 2017 there are multiple projects you can choose from, some of them commercial. Here’s a small list of possible candidates:

• Couchbase: open source, ObjC+Swift | Java | JavaScript| .NET
• Realm.io: ObjC+Swift | Java | JavaScript | .NET
• Firebase: ObjC+Swift | Java | JavaScript
• RethinkDB: open-source, Java | JavaScript
• Horizon: JavaScript

Conclusion

Developing a real-time application with a traditional technology stack as hard and unrewarding. Thankfully, there is a stack that can make your life as application developer a lot easier, at the core of which are real-time databases. Some of them are open sources, while others are proprietary. A good idea would be experimenting with a free PAAS database, and later upgrading to a paid product which supports your business needs or deploying your own local environment database.

Welcome to the real-time world!