Realtime Web Apps with Nginx Nchan and Python

If you have been doing any kind of web development, the name Nginx should immediately resonate with you. But what you may not know is that Nginx is built on top of an extensible and customizable module chain. It is these modules which actually do most of the heavy lifting when it comes to serving requests, sending files or communicating with Redis , Memcache or FastCGI servers. One such (perhaps not so well known but extremely powerful) module is Nchan . Nchan makes writing realtime web based pub/sub applications a breeze. In this article, we will build a simple systems monitoring dashboard which displays process information in realtime similar to what you would see when you run the unix top or htop commands. We will use python as the backend server publishing process information and react as the front end javascript client consuming and displaying that information on our dashboard.

Setting Up Nginx Nchan

To use Nchan with Nginx, you would need to build Nginx from source and configure it with the Nchan module. It is a straight forward process and instructions could be found on Nchan’s webpage. Once you have built and installed Nginx and Nchan, the next step is to configure Nginx to act as a pub/sub broker by adding the nchan_publisher and nchan_subscriber directives to the nginx.conf config file.

If you are familiar with Nginx, you know that the server and location directives are the core building blocks of the Nginx routing configuration (This article wont go into details regarding Nginx configuration but if you are new to Nginx, a good introduction to Nginx configuration could be found here). Nchan directives (nchan_subscriber, nchan_publisher) work by mapping the server and location sections to publishing and subscribing end points on a given channel ( topic or subject in other pub/sub messaging par lance). With Nchan, publishers publish messages on a channel by making simple post requests to an end point configured with the nchan_publisher directive. The nchan_channel_id variable holds the name of the publishing channel and could be set to any Nginx variable . In the config example above, it has been set to the query string param id. With respect to the above configuration, publishers wanting to publish messages on a channel say channel_1 will submit post requests of the form http://localhost:8080/pub?id=channel_1.

Subscribers subscribe to messages on an end point / channel combination via the several mechanisms (i.e Websockets, Server Sent Events) supported by Nchan. For example, a subscriber interested in receiving messages published on channel_1 via Server Sent Events (SSE) can simply initiate an EventSource connection and start receiving updates as follows:

Setting up the Python Publisher

Our publisher is a simple python script which uses the excellent psutil module to collect process and system information. It then publishes this information every two seconds via a simple http post request to the Nginx- Nchan /pub end point on the system_stats channel . You do not require any web frame works or special modules to publish to Nginx-Nchan. All you need is the ability to make post requests. It does not get any simpler than that!

Setting up the Subscriber

For the subscriber client, we create a react application which subscribes for updates on the system_stats channel via Server Sent Events (SSE) and passes these updates to a table for display. We could alternatively have used Websockets but for pure pub/sub where there is no bidirectional communication involved between the publisher and the subscriber, SSE is much simpler and cleaner to use than Websockets .

If you are not familiar with react, don’t despair. Key is to note that you can simply subscribe for updates being published to an Nginx-Nchan server from any javascript application by simply instantiating an EventSource (or Websocket) object and providing a callback hook and that’s it!

Conclusion

The intent of this short post was to show how easy it is to begin writing scalable web pub/sub applications using Nginx-Nchan. There is of course a lot you can do with Nchan and I encourage you to further explore the documentation of the module.

You can find the full source code of the pub/sub application discussed in this post here.