How a microservice (and a new role) is born

Deciding on the right architecture is never an easy task. There are usually pros and cons to every decision you make. Today’s domains of responsibility will not be the same as tomorrow’s…

“Prediction is very difficult, especially if it’s about the future.” Niels Bohr

Among the biggest sins of software design, you’ll find YAGNI, premature optimization, and others of the category of premature design.

So you might be thinking something like — well that sucks… you’re saying there’s nothing I can do to get things right from the start?

Well yeah. Truth is — there isn’t. But that’s ok!

If you spend forever trying to perfect your product, architecture and code, you’re not only ignoring the business importance of time-to-market; You are also ignoring the importance of a fast feedback loop. Getting your product out there so you can better understand the market fit, and what your users actually want.

So what should I take from all of this?

1. When starting a new project from scratch, you should mainly design for what you need* at that moment.
2. When taking on a new task in an existing project, you should… well again, mainly design for what you need* at that moment.

* Non-functional requirements are a necessity as well. So make sure you always consider them and never mistakenly treat them as YAGNI.

OK, when do I create a new microservice already??

When a requirement comes that justifies it. We’ll get there soon.

Some context and then the example you’re waiting for:

5 years ago… Natural Intelligence has over a 100 comparison websites in many different fields.

Users looking for the best dating site / antivirus / mortgage, or anything else we cover, can use our sites to find the best option for them.

The sites were all served by the same monolithic codebase at the time, and we wanted to migrate all of them to our (then) new in-house built CMS platform.

It would allow us to manage the content, as well as the look & feel of every page in every site.

But these sites looked very different from one another, since they target very different needs. So we’ve had lots of different look&feels, as well as other features, to implement in our new CMS, for every site we wanted to migrate to it.

My director knew I wanted to become a team leader, and gave me the opportunity to lead an amazing new team with this task at hand.
Yay! an exciting change for yours truly :)

The VP R&D already had a plan of execution(including a tight schedule), and he laid it out to me.

It looked something like this:

foreach site in sites_to_migrate:

implementMissingFeatures(site)

implementUI(site)

Every site had an aggressive deadline in that plan. It was a challenge, and we were looking forward to it.

Identifying the need:

After moving just 1 site, it was plain to see that the implementUI(site) part of our plan, was rigorous work, that took way too long, and added way too much complexity to our code.

Let’s look at a super simple example of a component that looks different from site to site.

Consider you’d like to promote some products, and build a promotion component.

It might look something like this for Dating sites:

And totally different for Online Gaming sites:

So as you can see, we have some questions that will be answered differently per site:

• Which products do we emphasize?
• Maybe the top choices because they’re the best fit for the user
• Maybe a specific product that has a special deal to offer
• Which features, and how many do we want to use?
• What look & feel should we use?

To support all of these options, we needed to code and deploy in 3 services(!):

• Add the content & style options to the backoffice editors — Backoffice UI Service (1)
• Save the user choices for these options — Content Service (2)
• Add the html/css/js partials of the new options — Rendering Service (3)

This is a very simplified version for the sake of the example, but it pretty much fairly represents what we had to do, to add a new look and feel, with some basic user inputs and ui logic.

It shouldn’t be this complicated though, should it?!

The solution:

Create a new microservice to manage that for us — finally we’re getting there!!

Its responsibility: handle templates

Main features:

• Hold all templates available and their tags, for example — promotion / chart / article / review / etc’
• Generic handling of these templates as data in 1 place (instead of as code in 3 different services as it was before)
• Serve them to the Backoffice UI Service so the user can choose from them. Usually getTemplatesByTag(tag)
• Serve them to the Rendering Service so it can render them. Usually getTemplateByID(id)

Bottom line — now when adding a new template, or editing an existing one:

• No deployment is required*
• There’s no need to write backend code in Ruby & PHP. Only html, scss, and js

* Instead of deploying 3 different services, that at the time were only released once every 2 weeks, and at a very high cost (full stacks, manual qas, regression bugs, etc’).

I know — I promised you a new role was born too:

We said there’s no longer a need to write backend code, and that there’s no need to handle any deployment pipelines, or complex regression issues.

So there’s really no need for an experienced full stack developer, and qa anymore.

Now, all we needed was someone who specializes in html, css, and js.

And that was super easy to scale because the templates were now data, and not tangled in complex backend code, so a lot more people could work on it at the same time.

So let’s introduce a new role to the company — the Web (FED) Developer!

And while it could have been fun to pat ourselves on the back for being geniuses, coming up with this great idea, that the new microservice means we have a new role in the company… it’s actually a very basic and intuitive derivative of Conway’s law:

“Organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations.”

So our case is actually Conway’s law in reverse. A system design forced a structural change on the organization!

If we are to be true about it, it is a business need (TTM) that forced the system structural change, and then the organizational change.

I encourage you to find your microservices when they’re required. Hopefully (and usually), you won’t have to create new roles within your company to accommodate them :)
I promise you it’s really fun and productive.

Hope you enjoyed reading this one.

Feel free to give a clap or two, or leave a comment.