Building a real-time, multi-user collaborative whiteboard using Fabric.js — Part I

11 min readAug 24, 2020

We all miss those office days. Want to brainstorm or communicate ideas to the team? All you needed was a whiteboard and a marker. However, as the pandemic turned us away from the classic office into remote work, there was a need for a virtual whiteboard to collaborate rather than just staring at each other’s faces while video conferencing.

We were looking for a good whiteboard solution for our team to collaborate and visualize ideas and one that can be integrated with our internal website. All we needed was some essential features, like an old Paintbrush app, to draw using a brush, draw shapes, add text notes, change colors, add images, etc. but collaboratively

In this multi-part series, we will describe how we made one such multi-user whiteboard with real-time collaboration for our team. In this first part, we will begin by describing how to make a single-user whiteboard and then will describe how to make it collaborative so that multiple people can contribute in real-time. We will cover more advanced topics like multi-party undo, auto-scaling, etc. in the next part of this article series.

Let’s get started from the basics to build a whiteboard for free drawings and shapes.

Interactive Graphics with HTML5 Canvas

The Canvas element is one of the most powerful features of HTML5 that provides API for drawing shapes, strokes, text, animation, data visualization, photo manipulation, and real-time video processing.

Canvas is supported by almost all the latest browsers and we will be using it to make our whiteboard. Canvas APIs are simple but powerful. For example, if you want to draw a red rectangle using canvas, here’s what you do.

Add a `<canvas>` element in your HTML file:

and use the canvas API in JavaScript

The result:

As you can see, the Canvas APIs are simple to use, yet it offers the raw power and lowest level of graphics rendering capabilities on browsers. However, the downside is that you have to code everything yourself for complex tasks, for example, selecting and moving an object, etc.

However, there are a couple of good libraries developed on top of Canvas APIs which simplify some of these tasks. Fabric.js is one of the leading libraries which comes loaded with many essential features needed for 2D graphics editing.

Fabric.js — Powerful Canvas APIs

Fabric.js is a powerful JavaScript library that greatly simplifies things by providing an object model on top of vanilla Canvas API [1]. You just to include fabric.js in your code:

<script src=”https://cdnjs.cloudflare.com/ajax/libs/fabric.js/4.0.0/fabric.min.js"></script>

Let’s write the same code using Fabric.js, to add a red rectangle [1]:

You can add different shapes by creating objects of different shape classes like Circle, Rect, Line, Triangle, etc. or even images! You can also enable freehand drawing mode (pencil, marker, or brush).

The result:

As we can see, you can create a rich whiteboard experience using Fabric.js easily. There are plenty of examples or tutorials available on the Fabric.js website to quickly master Fabric.js.

While that is good enough if only one person is using the board. What we wanted was a whiteboard where multiple users can contribute and collaborate in real-time, something like a shareable Google doc which allows everyone to type and edit each other’s text. Unfortunately, neither Fabric.js nor any other canvas library supports this option. We were on our own for building this feature on the top of Fabric.js.

Building a collaborative experience

One of the simplest ways to synchronize a whiteboard across multiple users is to publish the entire state of the board including all the objects in the whiteboard and their properties like position, color, stroke width, etc. At the remote end, the whiteboard is (re)initialized with these objects using the properties so that everyone can see the same instance of the board.

Although this approach is simple, it has disadvantages. Firstly, it is slow since the entire board state needs to be published even for a small change resulting in a large latency. Secondly, it is also prone to race conditions, for example, two users made changes at the same time, even though two different objects, one of them may lose their edits, as illustrated below.

Race condition: Changes made by A are lost over B

A better approach would be to replicate the state of individual objects rather than the entire board. Every time someone adds, modifies, or removes an object, say, a shape or a freehand drawing, publish the state of that particular only to all the users. At the remote end, the object is created, modified, or removed based on the state of the object just published to all.

Since Fabric.js treats everything as individual objects, it makes it easy to synchronize individual objects. Additionally, Fabric.js has a well-defined event architecture and provides events that we can observe in real-time. For example, whenever a new object is added, removed, or modified, events will be fired. All we need to do is to capture these events and send them to all the users so that they can synchronize their whiteboard in real-time.

Capturing Fabric.js Events

Whenever an object is added or modified to the canvas, fabric’s event listeners are called [3]. We need to capture these events and send it to other users so that they can replicate the same changes on their boards.

For example, let’s say we create a rectangle and add it to the canvas by calling canvas.add(). Then the object:added event will be fired with the target object that was added. We can now inform all the users that an object is added along with object properties (coordinates, colors, etc.) so that they can also add the same object with the same properties on their canvas.

Similarly, when an object is removed we will send a signal that this particular object was removed and everyone can remove that object from their canvas.

Transmitting events in real-time

Once we capture these events, we need to publish these events to the group of users working on the board, in real-time. We can convert the object for which the event was received to the string before sending it. At the remote end, it can be converted back to an object using JSON.

There are a few ways to send this object to all the users. Simplest would be to use REST APIs to store it on the server. Each user will continuously send any changes made by the user to a server. Then, each user needs to poll the server at regular intervals for changes made to the board. But this approach has problems. For one, you will not be notified of changes to the board immediately without delay as you are polling at fixed intervals. Secondly, it will put unnecessary load on the server as users will have to poll even if there were no changes.

A better approach would be to use WebSockets which always stays connected to a server. Whenever the server receives any new information, it can publish it to all the users who are already connected (live) using WebSockets. Although real-time and effective, this approach is complex and requires us to do a lot many things ourselves, server setup, connection management, packetization, groups and permission management, etc.

There are a couple of good real-time platforms including Google Firebase available on top of WebSocket which simplifies some of these tasks. Among these platforms, mesibo is the leading platform that offers real-time APIs for messaging, group messaging, and we decided to use it.

mesibo — real-time group communication APIs

mesibo is a powerful real-time platform offering APIs for Android, iOS, and Web. We have been using mesibo messaging and call APIs for some of the apps we have built. In this project, we will be using mesibo group messaging APIs.

mesibo allows you to create groups of users (members). Once you create a group, you can send messages to the group, and all the group members will receive the messages. In this project, each user working on the board will be a group member. If we send a message to the group containing the changes on the board, all members of the group will receive it immediately without any delay. Each user can then read the synchronization message and sync the state of the board.

Before we proceed with code examples on how we did this, you need to have a basic familiarity with mesibo APIs. You may refer to some of the mesibo tutorials to know about how to create mesibo groups and using mesibo API. However, we will briefly cover the mesibo API relevant to this tutorial. You can read about mesibo API [4].

Using Mesibo APIs

First install mesibo by including the script as follows in your project

<script src=”https://api.mesibo.com/mesibo.js"></script>

mesibo is easy to use.

You first need to sign-up with mesibo to get the API keys.
Create an end-point for each user. mesibo will generate an access token for each user.
Create a group for each whiteboard and add each user as a member
Every user can send a message to the group which will be received by all other members.

Refer to mesibo Getting started guide and first app tutorial to learn more.

For each user, you can initialize mesibo JavaScript API as follows:

You also need to implement MesiboListener to receive messages and status in real-time.

That’s it. Now you are ready to send and receive messages with mesibo.

Follow the steps below to set up basic group messaging with mesibo:

Create a group with a set of members. Each group will have a groupid
To send a group message, set the groupid in the message parameters.

The Mesibo_OnMessage listener will be called at each member of the group when a message is received.

By using mesibo APIs, we have made real-time synchronization of Fabric JS objects simple, without messing with complex APIs or setup. Once you have mesibo API up and running, it’s time to integrate Fabric JS and mesibo.

Integrating Fabric JS & Mesibo

In previous sections, we have learned how to use Fabric.js and mesibo APIs separately. In this section, we will be integrating both the APIs for synchronizing the Fabrics JS objects between multiple users in real-time.

In the following code, we will create a canvas and then create a rectangle on the click of a button.

Add following script:

Now let’s capture the object:added event when the rectangle is added to the canvas. Once we have the object, we need to Serialize the object(JSON stringify it) and send it to everyone in the group. We are also setting the type of the message to indicate a canvas sync message so that we can distinguish it from other messages (since we also use mesibo for normal group chat).

On the remote end, Mesibo_OnMessage will be called on receiving this message. We can now convert back from the message to JSON object by using JSON.parse() and then add it to the canvas

That’s all. You have synchronized your first object in just a few lines of code.

Object Identification

In the previous section, we learned that creating an object and syncing across multiple users was simple. What if instead of adding an object, we are modifying or removing an existing object. As you can imagine, we also need to tell everyone about which object we are modifying or removing. Hence to achieve that, we need a mechanism to uniquely identify each object.

Additionally, we also need to solve a recursive issue in our code above. In the above code, when an object:added event is called on one end, the user will send a sync message to the group containing the object. On the remote end when a user receives the object, it will be added to the canvas. This will in turn fire a new object:added event on the remote end which will again send the object for syncing to the group! Now you have the recursive sync even for the same object, added the same object to canvas, object:added is called again, and so on. And… You are lost in limbo forever with never-ending object inceptions.

To solve this recursion issue, we need to tag each object by the user who created the object. You sync a new object only when YOU have created an object by drawing on the canvas and NOT when you have added because of the sync.

To address both the issues, we will now add a user-ID and an object-ID every time a new object is created. You can do it immediately after creating each object. However, a better way to achieve this in our object:added event handler so that we don’t need to do it everywhere. In this simple example, we are creating only one ID by combining a user id with a random number. However, you may use a different scheme suiting your app.

Since now we have an ID field, we need to ensure that it is included when we stringify this object. By default, the JSON.stringify on Fabrics object will not include the ID parameter. To solve this, we need to extend Fabric Objects using fabric.util.object.extend.

Now on the remote end first we will check if the object already exists by looking up the object id in the list of objects present in the canvas. If it does, we simply update the object by calling set. Otherwise, we will create a new object and add it to the canvas.

Modifying or removing an object

Now that we have implemented synchronization and object identification, synchronizing the modification or the removal of objects is trivial and not much different from adding an object. Let’s add a button to remove a selected object

<button onclick=”removeSelected()” >Remove Selected</button>

To remove an object, click on it to select and click on the Remove Selected button which will then call removeSelected().

When an object is removed, the object:removed event will be fired. We capture this and send a message to the group containing the object that was removed.

On the remote end, when we receive the object we check if the removed property is set and then remove the object from the canvas.

Here is the complete code sample for integrating fabric and mesibo.

script.js

In this article, we have explained and implemented some of the basic building blocks of the whiteboard using fabric.js and mesibo.

We will soon publish the next part of the article, implementing some more complex features like undo, redo, scaling, modifying and editing existing objects on the board, and more.