Why you should consider it, the pain points and how to write your own lightweight reactive store

Hugo Dolan
Aug 22, 2017 · 8 min read
Keeping multiple simultaneous clients updated can be a nightmare

I’ve written this article at the end of my latest Angular4 project: Moveseats.com , which was architected to work in real time across many clients (See demo). Now its time share what I learnt while its still fresh!

Git repo link (For walkthrough below)

If there are issues or helpful improvements you wish to make, leave a comment or fork my repo, Thanks.

An Introduction

A reactive application can help to create:

  • Fluid user experience
  • Maintain consistent application state across many simultaneous clients
  • Save time in testing when every component and service is using the latest app state

A reactive applications can be the right choice if you have any of the following issues:

  • Need to keep multiple users viewing the same version of content
  • State changes by a client which directly impact or potentially break other users experience. (e.g. editing an already deleted item)
  • Have complex app state mutated by many components

Technologies Available

There are several technologies available out there for building reactive applications, with different benefits. I have only used Firebase and Socket.io, but other technologies include Redux, Ngrx and RethinkDB.

Firebase :

  • Provides a full api (API / Angularfire) compatible with RxJs
  • Constantly being improved with more functionality (recently : including hosting solutions and cloud functions)
  • I have only ever used their real time database, it works pretty well and can only be enhanced by the available integrations
  • Have to build your Db structure from the ground up for Firebase as it will only access data through a url. (Querying does not work like traditional databases)
  • Everyone should be careful about directly embedding the API directly into your application in case of pricing changes or the service dying (This article will demonstrate one way to avoid this)

Socket.Io:

  • Can be used in combination with your existing state management and databases
  • Passes data by firing events bi-directionally between backend and client.
  • Can be used to create ‘rooms’ to selectively broadcast data.
  • It takes about 30 mins to setup by following the docs.

Read on to learn:

  • Write a lightweight front-end realtime store
  • Abstract firebase from your code
  • Integrate to your existing DB using a Socket.io and Express.js

Building a Lightweight Front-End Store

Side note: I am not a security expert, please make sure to implement the appropriate security (to prevent injection attacks etc.) if you are dealing with sensitive date.

Service setup

Since firebase is a popular realtime db, I chose to base the design of this store roughly from the firebase api.

First off, In your Angular app create and provide a new service called db.service and implement the following:

/* Rest of db.service.ts */// Todo: (Next)import { DbObject } from ‘./db-object’;private userDb = {};private pathError = (search, path) =>{ return {msg: ‘Error access value at path’, accessing: JSON.stringify(search), ‘path’: path}};// Accepts a JSON Object, e.g. userDb snd searches for value at url// Eg. ‘users/test/nickname’ ,// Returns error if invalid url, else the value at route.private Value(route, search): any {   if(route.length < 1) return search;
let next = route.shift(); let data = search[next];
let dataUndefined = data !== 0 && !data;
if(dataUndefined) throw this.pathError(search, next);
return this.Value(route, data);
}// Access Db implicitly without passing userDb variableDbValue(route): any { return this.Value(route, this.userDb);}

The code above essentially declares an empty ‘database’ which we will pull from the backend later. It also declares a function to recursively access a value which may be nested within many layers of JSON in the database object.

Introducing Rxjs Subjects

Next we will create a reactive DbObject which the Db.Service will return when a client calls Ref(url). This will ensure that the latest data available is always pushed to the application through the Update() Method.

import { BehaviorSubject } from ‘rxjs’;export class DbObject < T > extends BehaviorSubject < T > {private dbPath : string[];private dbService: any;
constructor(state, path, dbService) {
super(state); this.dbPath = path; this.dbService = dbService;}// Called to update state of db && fire event to all subscribers Update(value) { this.next(value); }

The DbObject uses RxJs subjects to achieve a realtime database reference.

A Subject in RxJs, is both an observer and observable. Meaning that values can be pushed to it when the database is updated using .next(value). Those values can them be remitted to subscribers to the DbObject.

A BehaviourSubject will always emit the last value pushed to it when a new observer is subscribes. This has the added advantage of components immediately receiving the latest version of data without having to wait for changes.

Working prototype

https://stocksnap.io/photo/0A92VJXHAL

Finally to get a basic implementation, we need to be able to store a distribute DbObjects to observers through the Ref() function.

/* Rest of db.service.ts */private userDb = {};private pathError = (search, path) => {msg: ‘Error access value at path’, accessing: JSON.stringify(search), ‘path’: path};/* Accessing DbObjects*/private userDbSubjects = {};
Ref(url: string): DbObject< any > {
let subject: DbObject< any > = this.userDbSubjects[url]; if(!subject) { let route = url.trim().split(‘/’); let routeValue = this.Value(route.slice(), this.userDb); subject = new DbObject(routeValue, route.slice(), this); this.userDbSubjects[url] = subject; } return subject;}private Value(route, search): any { /* */ }

If we provide and import db.service into any component (given some populated userDb), we can now access and update data:

/* Some-Component */import { DbService } from ‘db.service’;import { DbObject } from ‘db-object’;obj: DbObject < any >constructor(db: DbService) {   this.obj = db.Ref(‘pathA/pathB/object’);     this.obj.subscribe(changes => console.log(“Obj: ” + changes));}// Assigned to some button (click)MakeChange() {obj.Update(“Test”, this.obj);}

From here it is relatively straight forward to add more Operators to the DbObject such as Add(), Push(), Replace(), Remove(), Increment etc, for convenience when dealing with arrays etc.

This realtime setup will simultaneously update all components which are subscribe to the DbObject. This is really powerful as we can utilise all the operators of RxJs, map, filter, switchMap etc to create dynamic lists etc.

Connecting store to a database

We’ll use web sockets to connect our front-end to our backend. Once the data reaches your backend you can integrate it with whatever database solution you like, mongoDb, firebase, rethink etc.

Basic setup

First install the necessary modules:npm install express socket.io — -savenpm install youFavouriteDatabase// Build ng app to dist folderng build

Create a server.js in your project root and with the following content:

// This script should encompass any db logic you haveconst db = require(‘./db’);const express = require(‘express’);
var app = express();
app.use(express.static(path.join( ____dirname , ‘dist’ ) ) ); // Return Ng4 app app.get(‘/*’, function(req, res){ res.sendFile(__dirname + ‘/dist/index.html’);});
// Listen on port / production port from Herokuvar server = app.listen(process.env.PORT || 3000);// Socket listenvar io = require(‘socket.io’).listen(server);

This script declares a standard express app, listening on a predetermined / default port. A web socket is then instantiated, listening on the server. We have imported a db.js which will contain your own custom database logic. (MongoDb implementation can be found in the repo)

Database Connection and Data Setup

The following code uses the db.js script to connect to the db and a callback to save the current db state to a store object.

/* server.js continued… */var collection;var store = {};db.connect(url, function(err) {   if(err) {      console.log(“Error: ” + err);      return;   }   / * Implementation by database * /   collection = db.get().collection(‘COLLECTION NAME’);    db.open(collection).then(data => store = data);});

Sending and receiving Socket data

Sockets work by sending and receiving events.

The socket parameter returned refers to the socket client which fired the event, .on() is used to listen for events from clients.

Broadcast excludes the socket client which fired the event and sends data to all other clients.

Hence broadcast.emit allows notifications of new data be sent to clients when the application state is altered by another user.

/* server.js continued… */io.on(‘connection’, function(socket) {     io.emit(‘db’, store);     socket.on(‘dbUpdate’, function (dbData) {         SetValue(dbData.path.slice(), dbData.data);         socket.broadcast.emit(‘clientUpdate’, dbData);    });});
var SetValue = function(route, data) {
UpdateValue(route, store, value);
/ * Custom per database implementation * /
db.update(collection, store);}
var UpdateValue = function(route, store, value) { var next = null; //Return condition (Assign value to store[next]) if(route.length <= 1) { next = route.shift(); store[next] = value; return; }; next = route.shift(); var subStore = store[next]; UpdateValue(route, subStore, value);}

SetValue() and UpdateValue() purely exist to update the custom database implementation and the realtime store that is distributed to user at io.on(’connection).

Making the front-end socket ready

To connect the backend socket-driven-database to our front-end we need to import to socket clients and setup events to receive:

  • Initial state from the backend
  • Any subsequent updates from other clients.
/* db.service */// Socket client importimport * as io from ‘socket.io-client’;
private userDb = {};
/ * Accessing DbObjects * /private userDbSubjects = {};
// Socket
private socket: io.Socket;
constructor() {
// Declare socket and establish connection this.socket = io();
//Receive initial data payload
this.socket.on(‘db’, (data)=>{ this.userDb = data; this.socketLoaded = true; this.socketMonitor.next(this.__ socketLoaded); });
// Any updates broadcast from other clients
this.socket.on(‘clientUpdate’, (data: any)=>{ if(this.__ socketLoaded) this.UpdateSubjects(data.path, this.userDb, data.data); });}

We created a ‘socket monitor’ which ensures that we do not prematurely send an empty database to any part of the application.

The below code modifies the Ref() method to incorporate a ‘socket still loading’ temp data to observers.

// Updated implementation for socketRef(url: string): DbObject {   let subject: DbObject < any > = this.userDbSubjects[url];   if(!subject) {      let route = url.trim().split(‘/’);      if(this.__ socketLoaded) {      / * Code as previous * /      } else {      // New object awaiting socket data      subject = new DbObject({’socketLoading’: true}, route.slice(),   this);      }
}
return subject;}

This final section of the db service ensures updates from other clients are applied:

  • To the front end database representation.
  • To any active db-objects currently subscribed to by the application.
// Update any active DbObjects + Write socket changes to userDbprivate UpdateSubjects(route, store, newValue) {   this.socketMonitor.next(this._socketLoaded);   let subject: DbObject = this.userDbSubjects[route.join(‘/’)];   this.UpdateDb(route, store, newValue);   if(subject) subject.QuietUpdate(newValue);}
// Same implementation as for backendprivate UpdateDb(route, store, value) { var next = null; //Return condition (Assign value to store[next]) if(route.length <= 1) { next = route.shift(); store[next] = value; return; }; next = route.shift(); var subStore = store[next];
this.UpdateDb(route, subStore, value);}

Finally a few changes need to be made to the DbObject class.

The constructor is modified to ensure that data loaded from the backend is pushed to the app.

A quiet update method ensures that any updates from other clients are not propagated in an infinite loop.

/ * db.object.ts * /constructor(state, path, dbService) {/ * … * /if(state.socketLoading){   this.dbService.ListenForSocketLoaded().subscribe((loaded) => {   if(loaded) {       let newValue = this.NewValue;      this.QuietUpdate(newValue);  }}); }}//Update all other clientsUpdate(value: any) {   this.QuietUpdate(value);   this.dbService.SocketUpdate(val, this.dbPath);}// Receive updates from other clientsQuietUpdate(value: any) { this.next(value); }   private get NewValue(): any {   return this.dbService.DbValue(this.dbPath.slice());}

Wrapping Up!

This light-weight store enables us to create a reactive application to sync across all clients in real time.

It is also abstracted from the database solution, which means it can simply be plugged in or replaced.

All the code for this application is available on the repo. There may still be some bugs or improvements to be made in the code, so feel free to leave a comment or fork the repo.

Thanks for reading!

Hugo

OneHourCode

A blog for hobbyist coders and graphic designers

Hugo Dolan

Written by

UCD Statistics & ACM, Learning Data Science, Winning Team @ Citadel Dublin Data Open. www.hugodolan.com/linkedin | Mailing List: http://eepurl.com/gkV7ov

OneHourCode

A blog for hobbyist coders and graphic designers

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade