Dynamic vs. static ui composition

I want to tell you guys a story. In my company(e-commerce shop) we took over a monolith from an external agency. Someone decided that we want to develop in-house, so we were forced to take the monolith and go further steps. At some point someone decided that we want to make micro-services. (SOA). So we started to develop “micoservices” on a green meadow. But there was no migration back to the old shop monolith, so we needed to call services from the monolith. We were very unhappy with development at the monolith (long running release circle, long running e2e test suites, technology zoe, front-end was just hell (jquery, knockout, require.js, jquery.ui and more and zero tests) many teams were blocking each other and so on.

History

At some point someone decided that we want to move away from that. (understandable) Strategie goals were: Independent teams, short release circles (continues deployment), minimal coordination efforts, consistency on UI/UX (living style guide)

We already had a prototype process which was basically a grunt task which was using include and replace for html composition to create static html files an less for compiling less to css. That’s it.

We aimed to reduce waste, so we decided instead of using include-replace to use a real (more or less) stupid template engine with default escaping enabled. We ended up with handlebars. It was plug & play ready. For example if you have a more complex data structure ( recursion for menus etc.) you could just use some helpers. Templates are ready to be used at client side as well as server side (which was very important to us)

Migration step 1

As someone decided we want to rewrite the header, we started to think, how we can migrate it to the “new world (microservices)” We also had the requirement to make A/B testing for the new header. You need to know that 80% of the shop functionality were displayed in the header(above the fold). Everyone of us knew it will be pain to refactor stuff. Write new stuff. We introduced a standalone “assets project / repository ” which was using handlebars and mock data to assemble the header, javascript with require.js and sass for css paired with galen framework for layout cross browser testing. Afterwards, we precompiled the templates to javascript functions, which will be used by our ui-serivce (more details later on) Now, we started to write unit and component tests for javascript as well (and it saved as a lot of times our jobs). The price was high. The deadline was moved several times (typical character of dead lines). Very stressful time. Two teams were involved to push this header out of the door. Many of the frontend developers have never being faced with testing (galen, javascript). It took us a lot of time to coach people to do that. They did, but mostly test quality was not that good. The end result was good. We could deploy assets and markup independently from backend in a few seconds on a continues basis. The ui-service just gets precompiled templates (javascript functions), requests the old shop-monolith and replaces the header tag with the new rendered header. The ui-service was written in Java using the google v8-engine instead of rhino, because of performance and memory issues.

After s short period of time, we recognised that this will not scale on a long-term goal. We decided to make this more scalable, by splicing it into useful parts. Every team is not only the owner of parts of an page, but even for ui-components like slider etc. This team will take care and claim main responsibility to maintain and develop this as an internal open source repository and will share it to make it useable by everyone, maybe outsiders as well. Every team is the owner of at least one page. There is no way teams share responsibility because of everyone can manipulate my page. We introduced the atomic-design and went DRY + static, and ITCSS + BEM to reach this goal. For the javascript parts, we decided to use require.js, but precompile this AMD-modules with r.js to have one bundle at the frontend and only that resources which are needed by an specific page, not just everything. This makes things more cacheable and more fast by utilising bandwidth correctly.
At the end this resulted in a simple dependency driven architecture which was assembled on multiple levels to simple bundles which will land on the CDN again. We have used a package manager (bower paired with npm) to achieve a very flat dependency graph and to find bottlenecks at “compile time”. The new more general architecture looks more like this:

Whats the sense of testing

Many people think, that testing is there to prove, that my software is without bugs or works as specified. But this is not the whole truth. We do testing to ensure, that the customer will not be affected in any way, because of technical dept or similar problems. We want any customer at any time on every supported platform and browser to be able to use our shop. From a technical point of view, we want to be able to refactor stuff, checkout if my changes broke something (fast feedback circle) and maybe I want to make a cross browser and cross platform test for a specific css and javascript feature (Chrome on windows does not act the same like on mac for an css point of view). Maybe you need to check if a specific breakpoint will take the right rules. We also aim for a high level of automation and confidence(sleep at night).

Data driven

It’s pretty important to have good and clear KPIs even for the client side. We need to distinguish between business (conversion rate, visitors per day etc) and technical measurements like errors and performance KPIs on browser and platforms. To find correlations between business data and performance data, you need a good dashboard and reporting mechanism.

Thoughts about consistent ui

Using DRY to prevent stuff from being written twice is a good idea. Adding separation of concerns to it might be another good idea. Preventing teams from solving the same problem in different ways is another good idea. That means: Components. That means, you have a clear vision from UI/UX perspective of what is going on. Having a living style guide and a common understanding of what is already there(naming things), still in development or deprecated might be a good idea . One great benefit of an living style guide is, that you can reduce cost and complexity, because you develop, test and integrate small parts of the UI, which can be highly parallelised. You can even get feedback from UX/Design to correct minor issues directly. Overall changes can be made with low efforts. For example, changing the color of a button on every page. Another great benefit is, that estimating complexity for a huge UI feature might be more easy. Having flexible components means, you can use them in many context and they will still work (might even be a cms). If it comes to page level, you can delivery only what’s necessary to run the UI smooth.

Consistent and functional ui is important

Never ever, save money and time on the cost of customers and UX. This is very important to understand, because saving money and time instead of delivering a good, functional and useful software to the customer could kill you. You cannot measure the damage a sad customer could cause you.
Consistency even represented on the markup is highly recommended. For example for SEO or accessibility reasons.

Do not run in circles (expect your continuous integration)

You may have noticed, that I wrote a lot of times “someone decided”. Our company was pretty bad in documenting things and have clear understanding of who decided what (caused a lot of pain) We discussed same topics over and over again, because there was no decision backlog you could have taken to answer raised questions.

What ist architecture all about?

I will answer this question with the following image:

What do you think from a business perspective is important for an architecture to be accepted by teams and technical leaders?

I want to answer this question with some image and graphs. I do not use any numbers, because it’s quite hard to measure the exact impact of an architecture.

From my point of view, with an period of time, costs and efforts should reduce on a long-term goal.
This “decrease” of costs and the agility of the architecture is in my opinion the main goal to stay as a company competitive. It should scale on high performance, on a technical perceptive as well on the company perceptive by having growing teams and feature numbers.

The following image should make clear, what impact an architecture decision really has. Be clear with yourself, never choose an architecture because of personal preferences, this could cause a lot of damage.

As you can see on the image, you need a period of time to measure how good you performance actually is. What I’ve drawn on the image is in my opinion one ideal way of how a project should perform. To achieve a goal like this, the decision for the right (frontend) architecture is very essential.

You also need several other things:

• A good and experienced team
• A strong product management
• Skilled UX/UI guys
• Continuous integration
• Continuous deployment
• Good logging even on client side
• High test coverage
• Some flexible development process maybe Kanban or Scrum

How to rate my architecture and how to decide which way to go?

First we need to figure out what’s your goals are. Do you aim for a very WET driven development process or do you try to solve problems once by practicing DRY as you development process.

What does WET (write everything twice) + dynamic really mean?

It means, that you will not share code via internal libs or package repositories. Every code you want to share needs to be open sourced maybe on github or so. YOU DON’T SHARE CODE!

Dynamic means, your page will be assembled while runtime. You have maybe an entry template which resolves some kind of links to a full page. Look here for an example.

Your page will be created and parts of it will be delivered as HTML from multiple services. This will end up in a full HTML page and delivered to the client.

Pros:

• You can deploy global changes at any time and any point to change UI
• You can save your contract tests if your service is not consumed by anyone else maybe via JSON.
• You can develop from scratch and have the full end-to-end responsibility for your UI parts
• Global UI changes like color schemes can take place very fast
• You do have the freedom of choice how to test, when to test, what to test

Cons:

• You need to have a full-stack team which has experts on different levels QA, backend, frontend and so on. Maybe super dupa fool-stack developers.
• You need to have a server instance to render templates. This means you have to start maybe an web server, an vagrant or docker machine to see your development results.
• End-to-end integration tests are very complicated and only possible on integration level.
• Bugs may be hard to be reproduced because of an continuous, unstable system. Keep in mind: Every deployment on the system can change everything. Keep also in mind: Client-side means, that you are acting in a global scope. Yes, it is true you can work with name spacing and different scopes, but never put your trust on that, global errors can effect everyone. On CSS level it may happen that specificity wars start to burn.
• It also can happen, that while you are fixing or reproducing a bug, someone changes the system and your work starts over. (reproduction of bugs could start to be a mess)
• Management of asset deployment could come very heavy. Keep also in mind that cache invalidation is still a problem sometimes.
• Frontend assets can get very heavy when using strictly none shared code base.

Now comes a list of some very good lies I have heard in the past about this approach:

• “I can reproduce bugs in a few minutes and deploy an fix for that”
• “High test coverage is not that important for decision of an architecture”
• “Our customers don’t care about bugs, they can live with minor issues”
• “Developers are t-shape more or less. Just everyone can do client side an server side development as well.” (and we are sure every developer wants to)

Don’t ever play with your companies brand and your user experience, because of technical misunderstanding of client side / browser side development and architecture. Act like a lawyer of the customer in order to achieve a good understanding in your company for the UI topics.

What does DRY (Don’t repeat yourself) + static really mean?

DRY means you will share code to ensure same problems are not solved multiple times. It also means you have only on central point for technical changes. You need to sit down with people to define “components”, look, design, behaviour, UX. You also need to name things by their name, to enable people to talk about the same things at the same time. This results in a product catalog now days called “style guide”. To slice the right portions a practise separation of concerns, you can also build, test and deploy flexible parts of an UI.

Static means, all most everything happens at compile time. Early and fast feedback circles allow to intervene very early.

Pros:

• It can help to reduce waste, instead of just building a prototype, stuff could be reused very easy.
• You only delivery what’s really necessary to the client side.
• Decapsulated development of components makes clear what needs to be where.
• Testing gets a great benefit, because you don’t depend on a backend service to be up an running to run your integration tests. You can do integration testing in an very early stage of development, which is by the way very important for product quality on client side, if we think in perspective of cross browser and cross device tests.
• Doing data integration tests could be done easily by executing (consumer driven) contract tests.
• High and independent development of several components in parallel
• Very consistent UI
• Global UI changes like color schemes can take place very fast
• Versioning of UI components enables everyone to deploy stuff (non-blocking release circle)

Cons:

• Setting up the build process and coaching of developers for task automation could be time consuming and pain full.
• Developers do not look into the catalog to use what’s already done. Things might be written twice.
• Third parties can block my feature development, maybe of bad quality.
• Versioning of UI components can cause damage if one component will be needed in two versions.
• Components can use different technologies, which can cause additional efforts for development and bad performance, because of heavy frontend assets.

How to figure out which way to go?

I have created an image which should help you to understand how to look on an architecture. For sure I can not cover every point which is important to you. It is more or less an point you can start with.

1. This draws right trough quality, long-term projects and pulls down to consistency. This could only mean one exact thing: This ideal for product development, that means, if you have to maintain a classical platform or you want to build up an application like this with a team with some great size. On long-term, quality and consistency are very important to you? You should go DRY and static. But keep in mind, you need a very disciplined group of developers to get this rolling. Also communication overhead can become a bottleneck.
2. It is also possible to have a mixture of both concepts, but everything comes with a price. It could happen, that your development speed will suffer because of unclear quality management. It is not very clear if the project, you should work on, is setup for long-term or short-term , so you need to keep at least the door open.
3. You need to setup a UI with multiple teams at once? In very short time?Just some kind of UI? Quality does not matter, people need to get their stuff out of the way right trough to production? Rewrite stuff, deploy independently, but keep in mind, quality will suffer a lot of times. It also can happen that you gain speed a development, but lose it when it comes to debugging on production. Also performance of being dynamic can cause a lot of problems as well, because same problems are maybe solved twice.
4. This number is the “I really don’t care decision”. That means: You need to setup something, which is done fast to show something very soon. This is not to be meant to be used on a daily basis, because of not existing or bad quality. It´s also not meant to be used for product development. Just to setup something up very fast without talking to others.