Microservice with Nameko

Why and how we implemented Nameko as our microservice framework

We have been interested in microservice since the early beginning of ForePaaS. As you may already know, our platform has a microservice software architecture based on containers, to ensure the scale and growth of our clients’ projects while delivering a high-quality experience. For those not familiar with these concepts, applications with microservices are broken down into their smallest components, instead of a monolithic approach where everything is built into one single piece.

As you can imagine, deploying and synchronizing more than 20 services in less than 3 minutes is a huge challenge for us. Kubernetes, the orchestration service we’ve been one of the first to use in Europe, has been of great help for accomplishing it.

As part of our 2019 roadmap, we have decided to create a third version of our ETL now called the DPE (Data Processing Engine) which is the component where data extraction, transformation, and loading happen. Everything is automated, from the management, deployment to the scalability.

Key elements of our DPE include:

• Actions: processing, aggregating and integrating custom python programs
• Workflows: structuring actions chains and orchestrating them

We decided to go for a simple microservice architecture while revamping this component:

After defining our architecture, we had to make the decision to go with a framework or our own implementation. Making our way could have been a good solution to give us something that would fit completely and be specific to our product, but still, this would have taken us a long time to develop. So we decided to go look at some microservices framework that would still fit our platform while ensuring a high development speed.

This is when we discovered Nameko.

Hands-on Nameko

Nameko is an easy-to-use solution for microservices in Python. It helps to develop scalable and distributed services as soon as you get to use it. This framework will manage all the communication, transport and concurrency, you only have to focus on your project specifications.

Before even getting into Nameko, you will first need to install it. A simple pip install will do the trick (you need to have the version 3 of Python)

pip install nameko

After this, you need to launch an AMQP server (it will be your main backend for communication between services). We will use RabbitMQ for this example.

docker run -d -p 5672:5672 --name rabbitmq rabbitmq:3

Before getting your hands on the code, I would advise you to take a good look at the CLI documentation. This way you will be able to run your services and test them efficiently.

Now let’s get our hands into our first service. It is easy to use and also to test. Here is a small implementation of a service with a single RPC call:

You only need to add the decorator rpc to the desired function (here hello) and then you will be able to use this function from another service.

Run this service in a terminal:

$ nameko run hello
starting services: hello

And then test it from another one:

$ nameko shell
>>> n.rpc.hello.say(name="World")
'Hello, World'

To use it from another service you can use the RpcProxy built in dependency provider.

Make it yours

Nameko is incredibly easy to use and is a very “modular” framework. As you begin to understand the different “entry-points” that Nameko gives you, you will discover how easy it is to develop your own. For instance, we found out that we needed a way to initiate our service. This is when we decided to develop an “init” entry-point.

The content below describes the init entry-point

MongoDB dependency provider

Another extension that you might want to discover is called Dependency Providers. You might want to integrate your database connection and usage with this helper class. Here is an example of a MongoDB dependency provider:

Now define the Nameko service

We need a configuration to run this example since we added a MongoDb connection variable:

Now you just need to run the service and test your service:

nameko run --config config.yaml service_a
# In another terminal
nameko shell
Nameko Python 3.8.0 (default, Oct 30 2019, 11:05:01)
Broker: pyamqp://guest:guest@localhost
>>> n.rpc.service_a.find_one()
>>> n.rpc.service_a.find_one(filter={"name": "Hello"})

Wrapping up

Nameko is a very extensible framework and those are just a few sets of examples of how you could use it. Overall our experience with it was spot on:

• Scalability,
• Extensible to our various backends and factory,
• Lightning speed development time,
• Fitted to our product.

Nevertheless, we still have a lot to learn about this framework and I’m sure we will discover more about it while migrating some of our components using Nameko. Don’t hesitate to share your thoughts and feedback on your usage of Nameko in the comments below.

If you want to integrate a microservice architecture yourself, head to their website https://www.nameko.io/ and learn more about their amazing product and how to use it.