illustration by Adrien Griveau

How to make Meteor web apps communicate together

Vianney Lecroart
Mar 18, 2015 · 6 min read

a comparison with the REST API method

As a Hacker in Residence at eFounders (which is a startup studio, a startup that builds startups), I develop internal tools to help our startups grow faster and better. One of them is an app that enables our marketing team to build curation newsletters very easily: users can curate web pages using a bookmarklet or a Slack command line, format the newsletter — or create templates — and send it (we use Mailjet to send the emails). Each curation list comes with its own pre-built website, where people can browse all the curated content.

You can see an example here: SaaS Club (newsletter about SaaS) or here: BuildTogether (newsletter about Startup Studios).

It was important for us to create a clear separation between our backend tool servers (only accessible by us, which means a low traffic) and the public curation frontend servers (accessible by all user, which means potentially a lot of traffic).

That way, we can have two different server architectures at the same time. One for our internal tools and one for curation frontend. These 2 clusters can evolve differently and even if we have too much traffic on the curation frontend. If the frontend is down because of a peak in traffic, our internal tools are still working.

Why the “Classic” (non-meteor way), a REST API, couldn’t work

In a “classic” stack (ror, mean …), we would have created a REST API on our backend server, so that the frontend server could access this API to get the content and store it. Then we would have created another REST API on the frontend server so that the clients can access this content.

Since it’s a REST API, the frontend server would have had to regularly call the backend server to know if there’s new content, modified content, removed content.

Why and how we chose the Meteor way (DDP)

Now, let see how we did that with Meteor.

Since we have a real-time platform, Meteor cannot rely on REST API.

It mainly uses WebSockets (or long polling if WebSockets are not available) to create a real-time communication channel and then uses DDP (Distributed Data Protocol) to transfer data in both ways. DDP is like “REST for WebSockets”. Like REST, it is a simple, pragmatic approach to providing an API. Yet, it is WebSocket-based, unlike REST, allowing it to be used to deliver live updates as data changes.

How is it done?

On the internal curation tool (backend server)

Curation contents are stored on a MongoDB collection called CurContents.

CurContents = new Mongo.Collection('curContents');

We have to expose a subset of data contained inside this collection so that the frontend server can get them. In Meteor, we use the publish / subscribe system to do that:

Meteor.publish('curContents', function (curListId) {
check(curListId, String);
return CurContents.find({ curListId: curListId, curCampaignId: { $exists: true } });
});

It creates a new publication called curContents that will check the curListId parameter sent by the subscriber and then return all documents related to this list to the subscriber.

curCampaignId: { $exists: true } tells Meteor to only give the contents already sent by email in a campaign. We don’t want to expose the contents that is already in our tool but not released in a newsletter.

That’s all we have to do on the backend.

On the curation website (frontend server)

In a classic Meteor app, it’s the client who subscribes to the server publications.

But in our case, we don’t want all clients to connect to our backend so it’ll be the dedicated curation website (frontend server) who will subscribe to the backend.

So on the server part of the frontend server:

  • We create a connection to backend server:
var cnx = DDP.connect('ip_of_our_backend_server');
  • Now we have a connection, we create a collection that will contains all the documents sent by the backend:
CnxCurContents = new Mongo.Collection('curContents', { connection: cnx });
  • Since Collections are persistent (stored on mongodb), we clear all the data at the frontend server launch time:
CnxCurContents.remove({});
  • Let subscribe to the backend publication to get the data:
cnx.subscribe('curContents', Meteor.settings.curListId);

With this 4 lines of code, the frontend now has a fully synced collection with the backend server. Each time a content is added (and published in a campaign), modified or removed, the backend will send it instantly to this frontend.

Sync between frontend server and clients

The final part is to send the data to a client connected to the frontend. We just have to create a new “publish”, this time on the frontend server, and a “subscribe” on the client.

Imagine we send all the data to the client (don’t do that for real, it’s only for the example!): in the real application, you only publish what the client really needs (i.e.: 20 last contents).

  • on the frontend, we create a publication:
Meteor.publish('curContents', function () {
return CnxCurContents.find(),
});
  • on the client, we create a subscribe:
ClientCurContents = new Mongo.Collection('curContents');
Meteor.subscribe('curContents');

And voila, the client now has all the data and can access, filter, sort them with mongo queries, directly on the client. For example, to get the 5 most recent contents:

ClientCurContents.find({}, { createdAt: -1, limit: 5 });

Conclusion

As we’ve seen, Meteor doesn’t use REST API (even if it’s really easy to create your own REST API) because it doesn’t support the real-time nor the websockets needed by modern applications. It defines a new simple open protocol called DDP. Today, more and more wrappers exist in different languages to handle DDP.

That said, the fact that Meteor uses websockets as the way to communication between client and server can be problematic. Websocket-based applications are still not easy to scale and it can be a big challenge if your app has to handle a lot of simultaneous users.

Hope it’s useful. If you’re a Meteor-friendly curious developer, I might have a job for you (in Paris !) — ping me on Twitter.

Unexpected Token

Bi-monthly stories about code, tech and shared experiences.

Vianney Lecroart

Written by

Founder at http://talkus.io. Your help desk. In Slack

Unexpected Token

Bi-monthly stories about code, tech and shared experiences. For developers. UT is brought to your by eFounders.co — startup studio building great SaaS startups.

Vianney Lecroart

Written by

Founder at http://talkus.io. Your help desk. In Slack

Unexpected Token

Bi-monthly stories about code, tech and shared experiences. For developers. UT is brought to your by eFounders.co — startup studio building great SaaS startups.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store