EuroPython 2016
In the beautiful Bask city of Bilbao (a city where you only can eat pintxos) a group of six developers from both the Elements Almere and Barcelona office attended EuroPython, the biggest European Python conference. EuroPython is a one-week fully immersive conference with talks, workshops and sprints about everything related to Python. The venue was the Euskalduna Conference Center, the auditorium of the University of the Bask Country, and the conference was organized in time slots with many parallel talks. Obviously we cannot cover everything from the conference in this blog post, but we will try to give a condensed overview by summarizing some of the talks that really stood out.
Exploring Python byte code
This talk by Anjana Vakil was about the .pyc files that always pop up in your project directory. By exploring these files with the dis library it provided some examples on how to analyse Python byte code. The presenter even showed how to use this to debug your code and find out why some code is faster than other. Understanding the execution steps in your script can give you an insight what the machine is actually executing. By making efficient use of the dis library, you can also find out exactly where your code is failing, giving an interesting alternative to standard stack trace and debugging.
By using the mentioned library you will be able to see exactly what instructions are executed by the virtual machine. When printing these instructions you will see something like the following:
Which can look a bit obscure at first, but it is actually very easy to read once you know what each column represent:
Just analyzing the important bits of it we can say that the first column is the line in your source code where that action is triggered. The next important part is the operation name. You can find a list of what each operation means well documented in the Python dis documentation. While probably there won’t be too many cases where you will be needing to read this information, it still helps a lot in the process of understanding what your code actually does. And you can dis basically everything, from variables to functions to classes and even entire modules. Really cool for who is really interested in understanding what is going on under the hood of Python programs. Let’s dis everything!
Building Service interfaces with OpenAPI / Swagger
This talk by Yelp developer Stephan Jaensch zoomed in on the possibilities of auto-documenting your API in a manner that complies with the OpenAPI specifications. If you are not familiar with these, basically they are a set of specifications aimed at constructing better APIs, ones that are easy to develop, to maintain but on top of everything, easy to use for the client.
“The goal of the OAI specification is to define a standard, language-agnostic interface to REST APIs which allows both humans and computers to discover and understand the capabilities of the service without access to source code, documentation, or through network traffic inspection.”
— The OpenAPI specification
On top of that the talked focused on another interesting point: automatic API documentation. This is often skipped (coding is done, right…?) and automation seems like a huge benefit. Stephan pushed the talk even further in the sense of what can be done to automate more steps in the process of building an API. Apart from not having to manually write content for each and every endpoint you will build, it would be nice to be able to consume the endpoints automatically. I mean having some sort of connection between the process of building them on the back-end and making the client aware that they exist without having to write a single line of code. That is the deeply lazy motivation that led the guys at Yelp to develop the Bravado library without having to write a client. Bravado is an open source Python project hosted on the Yelp’s GitHub page to help them develop and maintain their APIs according to these principles. Among the features that were shown to us, one of the most interesting was undoubtedly the fact that it provides a system to automatically build endpoints based on pre-formatted YAML file where you are free to define exactly what you want the said endpoint to show. Of course there are limitations in the possibilities you have regarding what an endpoint can or cannot do, but this functionality should be good enough to cover all the basic needs of an API. And, being compliant with the OpenAPI principles, you will be sure that its core structure is solid. If on top of that you decide to use the recently released version 3.4 of the Django REST Framework as your framework of choice, you get even more perks.
Protect you users with Circuit Breakers
Assessing the status of your whole infrastructure is something really important. There are many ways to tackle this challenge but the perspective offered by Scott Triglia from Yelp was surely interesting. He talked about Circuit Breakers. To begin with let’s explain what are these special constructs. Basically a Circuit Breaker is a piece of code that has the responsibility of monitoring the status of some other part of your code (or your infrastructure) and take actions in case something fails. It basically checks the status and can return three different results:
- Healthy (or closed)
- Recovering (or half-open)
- Unhealthy (or open)
It is the developer responsibility to decide what to do in each case.
To ease the understanding of this matter, Triglia used a fitting analogy. Think about a restaurant: how does the whole experience work? You sit at a table, wait for a waiter to come collect your order, the waiter brings it to the kitchen and stacks it on a pile of other orders and then the kitchen processes it (and of course you are served with the chosen dish). But what happens if that pile of orders in the kitchen starts to grow too much — let’s say above the ability of the chefs to serve all the tables in a reasonable amount of time? That is when the maître should stop new customers from flooding the kitchen with new requests. And he should also take actions in order to avoid losing those clients by, maybe, offering them to drink something at the bar, for example. This example works very well if we think of the customers as users of an application, the waiters as the network infrastructure (servers, client libraries and load balancers); the kitchen would our back-end (or any worker or set of workers) and the maître is the Circuit Breaker.
You can also think of this as a setup for a task queue/job queue system or any other system where a lot of incoming requests need to be handled efficiently. The idea is that, no matter which setup you have, at some point things will fail (because they will) you will need to be prepared and do something to avoid making things worse. This is exactly the purpose of Circuit Breakers. To keep the analogy: if too many requests (orders) are incoming we might want to detect that and make new requests wait — maybe display some message, redirect them to some other page/task or show them some cool animation (get them at the bar) — in order to allow the workers (the kitchen) to carry out the already received tasks. This way you will be able to serve efficiently to the maximum of your capacity avoiding bottlenecks in the best way possible, because you have control over failure.
And this doesn’t even mean that you should really do nothing on the back end when an unhealthy status pops up. You might still be doing things (maybe to test your application under stress conditions, or for other purposes) while the user thinks you are doing nothing. Based on the examples we heard during the talk, you can decide to:
- Tell the users that their request has been blocked, but you will try to process it anyway (playing on the surprise factor when the back-end manages to complete it);
- Block the users requests, but process fake ones to test the system under stress conditions (which has the disadvantage of not being representative of a real scenario but can give you interesting insights on your actual maximal capacity);
- Block the request completely.
Or many more things. The point is that you are in charge.
But back on the most efficient way to monitor the status of the system. More questions arise in particularly complicated architectures where multiple sets of servers carry the need for more Circuit Breakers. How to make them communicate? How to decide if the overall status of the system is unhealthy? We could determine that if at least two Circuit Breakers are returning an unhealthy status then the system should react in some way, otherwise we can ignore the issue. In this case a solution would be adding a centralized “Statuses Database” that operates as a bridge and collector of information from all the Circuit Breakers in the system. This solution works quite well, but it adds another node — making things inevitably a bit more complicated to manage. Anyway, the gain in terms of control that derives from it could compensate for the added complexity.
At this point it is easy to build more features to this monitoring system. It would be nice, for example, if the back-end (or our kitchen) could send messages independently to signal that they are starting to be unhealthy without waiting for an explicit check. This is done by making the Circuit Breakers listen to signals coming from all the other nodes. Again, it adds complexity to the system. It is really a matter of deciding how much are you willing to pay in order to have a certain degree of control over your application and the solution we exposed here could be something to keep in mind for the next big project you will be working on.
Originally published at www.elements.nl on July 26, 2016.
