Nothing is micro about microservices

What is a microservice?

There is nothing micro about eMAG but as we like to do things as simple and as efficient as we can, I’d like to talk to you a little about how we use microservices and how they improve our work and make our projects shine.

So what exactly is a microservice? You might find lots of articles, each having a different approach on how they explain it but, in a nutshell, it means using lots of small applications that communicate with each other and get the job done fast, and not having just a few big ones that need entire teams to handle them. Not to mention that releasing a new version of a big application implies a lot of work, a lot of developers and a lot of things can go bad, so it is self-explanatory why micro is better than monolith designs.

You might be using microservices already, but if you want to know whether they are your best choice, you should ask yourself the following questions:

• Are the services easy to replace?
• Can they be handled by small development teams? Small means a team of five to nine developers.
• Are the services clearly organized around different functionalities?
• Can the services be implemented using different programming languages or databases, hardware and software environments?
• Is the architecture symmetrical rather than hierarchical (producer-consumer)?

If you answered yes to all of the above, then it’s safe to say you are microservicing your way to success.

So how do we use it in eMAG?

Our first request that involved microservices was a project for the user details in the account section of each customer. More specifically, the section where you have your personal data, delivery address, company section (if applicable). We had heard a lot about microservices prior to this project, but we did not really know how to apply the knowledge, so we brainstormed it in a few meetings with the interested departments and together with the Network Operations Center team, we came up with a plan.

We decided to use Silex framework, a light version of Symfony, simply because we were accustomed to it and it was the fastest method to integrate a new light framework. You can read more about it here: http://silex.sensiolabs.org/.

We also decided to use MySQL for database operations, and for communication with the exterior we used HTTP at first, then we tested Gearman and Google Protocol Buffers. We arranged the framework to support more than one type of request by building a custom response/request system.

Technical stuff ahead

Let’s take a brief look on how we made microservices work in eMAG:

1. First we developed a microservice file structure using the Silex Framework mentioned before.
2. We created a tight relation between requests, response and service through data objects
3. We did the refactoring of the code by separating the business logic from the presentation code, moving it into a separate Silex app using the main dependencies employed by the main app and we removed the dependencies for each and every action and created a new database structure for microservices.
4. We set clear standards to follow such as having services named clearly so you can easily identify them

Let’s see some detailed description about building the user details microservice at eMAG:

The base section

When we planned the structure, we thought that if every application from our “kitchen” will be a microservice, these will have in common some (a lot of) controllers/services/libraries and this will generate some duplicated code. That is why we decided to collect all the “general” available information in a base section. This section is also used to override some Silex default sections, like Application for example. We did this because we thought that maybe we will need to do some custom stuff later like registering global services.

The converters section

Because we started this project with the possibility of multiple communication types in mind, we agreed that we need a converters section from where we could shape a DataObject in all the needed forms. So, we created the DataObject that each request type should extend.

The encoders section

We decided how the request looked like and we had to establish the response format too, so we created an encoders section in our Silex structure from where we could encode a message using JSON, XML or Plain Text for now.

The generators section

We defined above some standards for the request, response and service, so each time we needed one of these, there was a protocol to be followed. Well, this was not fun at all and that is why we replaced the manual configuration with some commands that help us generate the desired element.

These commands look like this:

[snippet id=”61"]

The “utilities” section

After we defined the skeleton, the structure that could be a starting point for any of our future microservice, we knew that we have to define the current microservice utilities. These were that external “needs” required by the user details context, like the list of cities, regions, education levels and so on. This is a true Symfony bundle with requests and responses created through our DataObject model, services, resources and routes.

The validators section

Because of the context of the microservices, we got to this section too, having to validate different data forms from the user account section. First, we thought that this would be an easy step, but we underestimated the challenge to gather in one place all the information below: the validation for all countries (Romania, Bulgaria, Hungary, Poland) regarding user company registration number, user identity card serial number, user national identification number, country VAT identification number, user IBAN and many other custom validations.

The first question was: who should do the data validation?

The possible answers were: the application that uses the MS; the MS; both

If validation is handled by the application, it means that the microservice should take “as good” all the data that arrives at it. Well, this means to trust or not to trust… We thought not to trust is the safest method, so we took into consideration to put the microservice to validate the data, even if they were already validated into the application. This would mean duplicated code in app and microservice. So, the final solution was to have a package of validation rules created by one entity, the microservice. This way, each application that needs validation rules would request them to the microservice, and eventually store them locally/cache them. When the microservice modifies the rules, it should notify the application to remove its local copy and get the new ones.

The validation rules were kept into a database. To be able to process custom expressions we built a CustomExpression handler. This structure is actually a Symfony bundle with all the specific sections, from the routing system to the controller, through the service.

The “core” section

Think about all the above sections as additional steps to the main goal: the microservice itself. The “core” of the user details microservices is a collection of requests-responses pairs for each request type created after the DataObject model discussed above, models, controllers, services and views and, of course, the routing system.

These being said, we have a last issue to discuss: how do we access it? Well, there are two options:

• a basic accessing method: just call it
• a SDK client method

To avoid overhead and duplicated code over the microservice and the applications that use it, we went with the second one.

Why did we insist on using microservices?

Let’s just take a look at a few advantages:

• Microservices are small and presume an easy to understand code. Each little service does its own thing and it is much easier to use the micro management approach here. Less code means you can understand it quick and there is less risk involved when changes occur.
• They are easy to scale and that’s a big thing because when you use a monolithic application, when you change something you change it all at once. In our case, for the user details project, if you have a problem with one of the features you can’t just change one thing, you have to change everything. When we use microservices we just change what we need and that is it.
• Since everything in eMAG changes fast you want to be able to change features fast, so using microservices makes that very easy. You don’t like a feature and you think it’s outdated? No problem — throw it away, out with the old, in with the new. It is just that easy!
• You can easily deploy them, since you can just deploy one service instead of deploying an entire application.
• You can use different technologies since every microservice does its own thing, and you can make it work however you like and in whatever way works best for your project.
• Strong essences come in small bottles — you use less manpower since you just manage a service at a time, and you do not need the entire development team to change the entire application — you just change what you need.
• And last but not least, reliability. When you use a monolithic type design, if an app stops working, the entire application may crash. If a microservice crashes, just a functionality will crash and not the entire application. That by itself should make you think twice before working with monolithic designs instead of a microservice type architecture.

Taking all these into account, I for one can safely say that using microservices clearly made our jobs a lot easier and I definitely recommend it in opposition to monolithic type architectures. It took us some planning and thinking outside the box, but the end result was worth it and I think this will be the way to go from now.