Weekend Project (Part 2): Turning Flask into a real-time websocket server using Flask-SocketIO
If you haven’t already, you can read Part 1 of this series here.
Since you’re reading this blog post, I’ll assume you’re no stranger to Python and modern web technologies. In the Python community, there’s a plethora of frameworks for building web applications, APIs, and websites. To name a few… Flask, Django, Pyramid, Bottle — these are all very popular for building applications of any scale and any variety. Personally, I only have experience with Flask, and limited experience with Django. Flask is a very versatile and lightweight framework that can be molded and twisted to fit many use cases. This includes building basic websites, web APIs, CRUD applications, or in the case of this blog post, a websocket server.
In my experience, Flask makes a great server for rendering JSON responses and templated HTML using Jinja2. It’s a very lightweight and manageable framework for these use cases, making it very popular in the Python community. However, when it comes to event-based websocket servers, the community tends to lean more on Node.js. Node’s major draw is the event loop — a continuously running loop that executes “events” or actions as they’re queued. This allows for a very flexible and powerful concurrency model when compared to Python’s typical multi-threading. This “event loop” is something that is required to convert Flask into an “evented” websocket server.
The fundamentals behind this requirement are due to the Python interpreter’s “blocking” nature. Typically, when Flask receives an HTTP request, it executes a function, and the server has to wait for this function to finish before accepting another request. Transforming Flask into an asynchronous evented beast allows it to process requests faster, independent of one another. An evented Flask server can accept a request and process it while accepting another request asynchronously. This is a behavior we’ll need for websockets, since there will be the possibility of Flask being hammered with websocket messages.
Getting Started
If we look back at the diagram from Part 1, take note of the few things Flask is responsible for. Flask’s primary purpose is to manage our game lobbies and to send/receive data via websockets. In addition to acting as our “API,” we’ll also be using Flask to serve out our compiled front-end code. This is a bit of a hack, but I chose to do it this way so the app can be run using only Flask. The alternative is to leave the compiled front-end code in a static directory and serve it using a standard web server such as Apache or nginx, but I wanted to leave that as an optional requirement.
Flask Refresher
If you’re new to Flask or haven’t used it in a while, here’s a quick run-down of how it works.
Before jumping into anything, we have to install our dependencies. Let’s go ahead and install all of the dependencies for a websocket Flask server.
pip install flask flask-socketio eventletNow, let’s dive into creating our Flask app. Flask requires a few things — the main app script — I’m calling this one flask-simple.py, and a templates folder for our HTML Jinja templates. Optionally, you can add a static directory to store your Javascript, CSS, and other static files loaded by the application. For the sake of simplicity, we’ll omit that folder for this project.
Now that our application structure has been established, we need a script to run it. Let’s edit our flask-simple.py file.
As you can see, there’s really not a lot of work to do to get Flask up and running! After importing the required modules, there are three main things that we do. First, we initialize a Flask object. Then, we set up our index route, which will route us to localhost:5000/ in the browser. The decorator @app.route() wraps our index function with some Flask code that essentially tells it to use the output of this function as the response. In this case, we’re rendering a template called index.html. The final piece of this code is our main function, which runs the Flask server in debug mode, allowing us to receive more verbose console output and enables hot-reloading.
Our index.html file needs a bit of work so we can show something in the browser to prove that Flask is doing its job. Let’s just edit it and enter some basic Hello World HTML.
Now we’re ready to run our app! Type python flask-simple.py and load up http://localhost:5000 in a browser window. You should see “Hello World!” in large bold letters. If your app renders a blank page or returns an HTTP error, check the terminal for any error output. Make sure all files are named properly and free of syntax errors, and make sure you installed the dependencies!
Introducing Websockets
If you’re a stranger to websockets, it can be a bit overwhelming to understand at first. Websockets work similarly to polling using AJAX. Think of the notification icons on many social media websites — as you’re sitting on the page and a new notification rolls in, the page instantly knows about it and displays an icon notifying you. The page has to get this info from somewhere without you refreshing, right? One way to do it is to use standard AJAX requests and ping the server every 15 seconds for new notifications. This can be a bit clunky… there must be a better way to do it, right?
This is where websockets can come in handy. A websocket is a persistent connection between your browser and the server that messages can be sent across. Establishing this connection requires a simple handshake between the server and the client, after that the server can essentially push messages to the client. Using a websocket connection will allow our server to “push” notifications to our client, rather than our client polling for updates. This results in a more seamless system for receiving new data from the server.
Similar to click handlers, we have callback functions that run when a new message comes in that we care about. These handlers will process messages on a specific channel that are delivered via the websocket.
socket.on('notification', function(msg) {
console.log('New notification! ', msg);
});As messages come across the websocket on the connect channel, our callback will be called and will have access to the full JSON message.
Namespaces, the typical way of setting up a channel using websockets, have no way of separating users. A message from the server on a namespace will send to all connected clients with listeners for that namespace. To separate our users, we will use “rooms.” Rooms in Socket.IO are just like chat rooms — users can join/leave and send messages. Rooms have a unique identifier, so users can join a unique room and receive messages intended only for users connected to that room.
In the context of our Codenames game, users will join unique rooms to receive game updates. Using a namespace doesn’t fit well here, because our game updates will be sent to all connected clients listening on that namespace, which would cause their games to become corrupted and display the wrong set of words!
Getting Flask working with Websockets
To convert Flask to use websockets, we first need to import a few things from Flask-SocketIO. We’re going to leave our /index as a standard Flask route since we’ll be using it to actually render the index template. In our app, there are a few actions we’ll want to do using websockets — create a room, join a room, and take a turn. Instead of creating standard Flask routes, we’ll need to create some websocket event handlers. Before doing so, our Flask app needs to be wrapped by Socket.IO. Instead of starting the app with app.run(), we’ll instead use our Socket.IO instance and call socketio.run().
EDIT: This example uses pieces from my codenames repo. To replicate this example for your own project, remove the import and any references to
gameand replace with your own data structures (dictionary, list, string, etc).
Notice how our decorator for on_create() differs from the decorator for index(). This is because we’re leveraging the Socket.IO decorator to create a websocket event handler instead of a standard Flask route. This works in a similar fashion to the standard Flask routes, except instead of speaking with GETs and POSTs, we’re speaking using rooms and channels over a websocket connection. When the client sends a message on the create channel, our on_create handler will process it.
Our handler does a couple of things — it initializes a game object, joins a “room”, and emits a message on the join_room channel with the room identifier. This allows the client to subsequently send a join message with that ID to join the room. Utilizing unique rooms, we will be able to keep our Codenames clients all up to date with a single send call.
The final step is on line 28, where we call socketio.run() instead of app.run(). Again, this is due to us wrapping our app with the SocketIO object.
Making sure everything is working…
To demonstrate our working websocket server, in our index.html we’ll write a little bit of vanilla Javascript to send a websocket message to our Flask server to solicit a response. We’ll create a button with a simple onclick action that simply emits a message on the create channel telling the server to create a game using some default parameters. In addition to that button, we’ll want to verify our communication channel by subscribing to the join_room channel and logging the output. This will confirm that our client and server are communicating properly.
Now, as with our basic Flask example, start the app using python flask-socketio.py and open up http://localhost:5000 in a browser window. We should see a plain webpage with a “Create Game” button, with the words “Websocket connected!” in the Javascript console window.
And voila! Our Flask server and client are now speaking in a new language! Well, the same old language, just over a new medium… websockets! As you can see in the image below, our logged output shows our expected outcome — establish the connection, click the button, send the create message, and receive the join_room response.
Now that we’ve got our Codenames game lobbies created, there’s only two more actions to add to our server — join_room and flip_card. It’s as simple as it was to write our create endpoint — define the function, add the decorator, and send our message.
The lines to take note of here are lines 10 and 20. In our on_create handler, we used the emit function to send a message back to the client on the join_room channel. This was because we hadn’t joined a room just yet. In this example, we use the send function with the room parameter. Instead of sending a message on a specific channel, this will send the message to all connected members of our room. In line 9, the server adds us to the designated room — we will now receive all broadcasts using send for the room with the ID we sent to the server.
Thanks for reading along and sticking with me through this journey! We’re about halfway to a fully functioning Codenames game — in the next installment, we’ll ditch that vanilla Javascript code and work towards integrating websockets into a larger-scale Vue.js app using Vuex.
If you have any questions or just want to chat, join our Slack channel!
Update 2018/01/23 — Read about scaffolding Vue with vue-cli and integrating Vuex in Part 3 of this series!
