Frontend Architecture at Scale for Large Organizations

Hi! I want to discuss with you how to manage Frontend architecture in large organizations. It feels to me that there are not many articles about this topic and it is not explained well.

By large organization in this article, I mean companies, in which the number of front-end engineers starts to be bigger than 15 and the company has at least multiple frontend projects.

I don't want to discuss management problems or business domain issues, the ones which are commonly seen in such big companies, but let's focus only on Frontend architecture instead.

Frontend architecture is involved in too many areas nowadays, so to begin with I would propose to divide it into the following sections:

Frontend Architecture scheme

1. Visual Code

Let’s start from the easiest topic, in my opinion, it’s the visual code of our applications.

Most likely because we develop multiple frontend applications at the same company we want them to have:

• brand recognition
• the same UI/UX

To achieve that we need to have a design system. The design team has to come up with the design system and follow those design guidelines in all future product designs. Even this already is a very complex task and needs a lot of discussions and alignments between the design team, engineers, and product.

From the front-end standpoint, we need to provide a tool to implement and to share this design system across engineers. A good solution for that can be the creation of the npm package, which will be represented visually by a Storybook or some similar tool. In my opinion, it's very important to have a dedicated web resource (with URL) with documentation about how to use this package. This web resource will be used by front-end engineers and by designers and can be a great reference point in conversations between them.

Visual Code

Summary:

At this point in time, we have a design system and its representation in a package and interactive documentation for it. We do share and adopt this package across all our front-end applications.

2. Code structure

Let's talk a bit about engineering daily basis. We do implement new features, fix bugs, refactoring code if needed. We do care about our codebase, trying to make it nice and easily understandable. But what happens when we start to have not 1, not 2, but dozens of small or big projects in the company? Usually, people are grouped around some of the projects and start to work only with this group of projects. By our human nature and the limited time, usually, we can't take care of more than 2–3–5 projects at one period of time. So naturally, we start to have those groups of influences. Btw, thanks to NASA for such an amazing photo for my analogy 😃

Groups of influences

But, we start to have more and more cases, when people have cross-teams collaborations, need to check each other's code and solutions, fix some bugs even in other applications or add something that they need inside some external app (external for their group of influence). The bad news — that this process is happening in big companies almost uncontrolled. The good news — we can help to improve it.

How? Correct. By providing the same code structure, coding, and structuring guidelines for all the projects in the company.

Let's dig deeper into what we mean by code structure:

• The folder structure in the project
  When engineers jump into a new project the first time — having the same folder structure as in their project, where they know everything helps a lot to navigate and search.
• Configuration or supportive files
  Files like package.json , .gitignore , .editorconfig , webpack.config, etc. They should be always in the same place, in every project. The same is connected to tests configuration files or CI files if they are needed.
• Fixed location of the file types
  If the location of the same file types is following always the same structure it also works great. For example, if the component folder always has a style.css file:

/Component
--/Component.tsx
--/style.css
--/index.ts

• Components internal structure
  The structure inside files should be the same: order of imports, exports, the position of public function, types, etc. In each type of file, you should know what to expect.
• Naming conventions
  This includes names of the folders, files, variables, functions, classes, types, etc.
• Coding conventions
  In general, I'd say, coding conventions is a very broad section, but here I just want to include things, that didn't fit into the rest of the sections, such as ; or not ; 😁 and similar.

The same code structure and the project toolset in practice are very tight together and help each other to co-exist. By toolset I mean CLI tools (project bootstrapping, linting, testing, etc.), IDE extensions, etc. We will discuss the toolset topic in the next sections.

Code structure

Summary:

After applying this section and adopting it we should have all projects across the organization with the same folders structure, naming guidelines, files structure, etc. Every developer ideally can jump into any other project and not being completely lost there. This 'completely lost' is also very connected to the technology stack which is used inside the project, so let's talk about it.

3. Tech Stack

Similar to the previous section, we do want to have a unified tech stack across the projects of the organization. The reasoning for this is very similar — we want our engineers to be comfortable with all the projects across the company.

In Frontend projects, the components of tech stack can be: framework, based on that project is built, the main language, styles processor, data layer (ex. Apollo), state management, tests, code linting, build system, and others.

Of course, in all rules there are exceptions. And in this topic, I would also want to make a remark, that sometimes some technologies extremely fit some specific projects, even if those technologies are not part of the global company-wide tech stack. But still each time this idea comes to move away from the company tech stack you should think twice because the cost of supporting such things is very high, so benefits have to be dramatic.

Let's mention here some generic tech stack which can fit now most of the projects (of course it covers only part of the real tech stack, but can be a good starting point for someone 🤷):

• React
• Typescript
• Apollo
• Styled Components
• React Router
• Jest
• Cypress
• Create React App

Stack

After we defined tech stack in our company and agreed on it, there are also other very important pillars of success.

First — we need to write down the tech stack to the document. The document, which should be well and easily shared in-between engineers, so they can always give a link to each other as a prove.

Second — we should again write down and share the document with the way how the new projects should be started and bootstrapped, by using the defined tech stack.

Tech Stack

Summary:

After we implemented and adopted all that we've mentioned above you should have all your projects at the organization to share the same tech stack. Ideally, without any differences. Not ideally — with insignificant differences. If we add also the previous section, the code, folders, file structure should also be almost identical in your projects. So navigating across any project should be a very easy task. Good 😊

4. Tooling

This topic is very important. We use some additional tools now almost everywhere now — for linting, building our apps, CI, components generators, and much more. So that’s why I see if we can choose the correct tooling for our projects — it’s crucial. Good tooling vs bad tooling (or no tooling at all), it’s the same as a comparison between automation vs manual testing.

We just talked before in previous sections about tech stack and code structure and mentioned that we need to write a lot of documentation to make people following them. But the correct toolset can give you the opportunity to automate following the guidelines in your company.

For example, if you have a specified coding style — you can give people the linting toolset, which by default is following those rules. If you have the defined tech stack, then a good CLI tool will give you an opportunity to bootstrap a new project with those specific technologies from your tech stack in place.

Let's try to discuss which parts of your frontend architecture can be covered by tools:

• Code style and structure
  As we discussed before this can be easily automated by tools
• Project bootstrapping
  You don't need to come up with a new project structure, installing manually all packages, which are needed, etc.
• Components generation
  Most of the time some component in your application contains not single file even, so files creation, linking/importing them can take time, so this can be automated.
• Start and Build
  Of course, the most obvious thing to automate — how you build or start your application.
• Tests
  The process of building your app for tests and actually running all types of tests — unit, integration, etc.
• Dependencies management
  As we know, around 80% of our code now is dependencies. So we need to keep them updated and managing that in a huge company is not an easy thing to do.
• Cross-projects dependencies
  Most likely our projects are not working in isolation and might depend on other projects or to be a dependency for some other project, so we might need some tools to make easier the process of linking them, developing in a combination of multiple projects (like Bit, for example), etc.
• CI
  CI is an essential part of our daily-basis toolset and automation and unification of it can be a very beneficial work for your organization.

If you don't want to develop a new own toolset, I can recommend the NX toolset, as one of the most interesting candidates. Also, it looks like the creators of Babel do a similar solution, which you may want to check out — Rome. Those tools are not covering everything, but a big part of what we talked about, so can be a good starting point.

Tooling

Summary:

Just imagine how great our architecture can become after we have all sections done and adopted 🧙

Every project is the same and maintained and managed by the unified toolset. Each project can start and build in the same way. New components are generated at the same place and with the same naming guidelines.

But is it that "sweet" or has downsides? As with any solution is has downsides. One of them — it needs some time for onboarding new engineers to your company. But if everything is done in a very natural way (as people got used to already in other existing tools) and documented, then this becomes not a huge issue when you compare it with the benefits of development speed, an opportunity for engineers to work with any codebase easily, etc.

5. Production

About this part of the frontend architecture usually, engineers take care of the least. Maybe because it's not connected with the coding itself most of the time🤷 and probably not that exciting, but not least important ☝️

In production we usually need to take care of the next things:

• Analytics
  All kinds of different tracking events, etc. Things like Google Analytics, Segment, HotJar, etc.
• Status monitoring
  This includes things, like health-checks, can be even tests run on production, error reporting (like Sentry), etc.
• Performance
  This is kind of similar to the previous item, but more oriented to performance. This includes measuring response time, first meaningful paint, etc. (probably tool #1 — Lighthouse)
• A/B testing
  All kinds of A/B testing solutions or feature flags.
• Caching
  Such tools like Varnish, Cloudflare.

All of them can be unified across your front-end applications in the company, which will simplify the life of your engineers. For example, by adding packages with the initial configurations predefined, and adding those packages to your bootstrapping project.

Another piece of the production is the code preparation, such as:

• Minification
  Just code minification 😄
• Source mapping
  Source maps might be needed for some other tools also, like Sentry.
• Assets preparation
  Preparation for screens with different resolutions, cropping images, etc.

Those are great candidates to be added to the toolset we have for managing front-end applications, which we were talking about in the Tooling section.

Production

Summary:

Ideally, all those should be added to each frontend project by default on the bootstrapping phase. And engineers would take care only about adding correct API keys in the correct places for the tools.

6. Development

CLI Tool

Partially we already discussed development experience in the Tooling section, when we touched the frontend CLI tool. Unified tooling is a big part of engineers' daily-basis, but not everything.

API

Comfortable API to work with locally is the second awesome thing we can do to improve the developer experience and the development speed. Usually serving API locally for front-end engineers is not a trivial process: this might include installing tools or frameworks, which they can be not familiar with. Even if the setup dockerized this can take a huge amount of computing resources from the developer's machines (if it's not — this can be considered as a setup). What can be a solution in this case — external served API. This can be a single server for all engineers or some mechanism to bootstrap easily your own dedicated server for development.

CI

CI is the third big part. I might assume, that your company already has chosen some CI tool for frontend and use this single tool (ex. Circle CI, Concourse CI, or any other). If not — you should unify that.

CI configuration of the particular project, in my opinion, should be part of the job of the team, who is working on the project. This gives chances to CI to be stable because there are people who are interested in CI to be working, using it every day, and have the power and skills to fix, configure, and improve it.

However, not all work should be done by the team. For front-end applications exists a pretty specific bunch of jobs, which potentially can be needed. Especially, if we could manage to unify folder/code structure, tech stack, etc. So after some time in your company, it might come point when you would be able to detect those patterns for stages of your CI tool, implement those stages as some kind of "building blocks" and give an opportunity for your engineers just construct their CI pipelines from those unified "building blocks".

Demo Environments

And finally the last — verification of the implemented feature. After engineers have everything done and implemented — they almost always need somehow to check how it looks like and how it works and share this with other engineers, designers, or other stakeholders. For such needs, it works super well the temporary deployed version of your application for the specific PR with the provided URL.

App temporarily deployed

This solution speeds up the communication between different teams and people a lot and I think this is just a must-have. However, this temporarily deployed version should be as close to production as possible, because can also be a great tool for checking some surface errors or bugs.

This can be easily added to your projects and automated if your frontend applications build and deployment process pipelines are unified. Also, such or similar tools like Kubernetes and Helm can help a lot in the implementation.

Development

Summary:

Comfortable and efficient development flows with a single CI tool with building blocks to make any CI pipeline, with unified CLI frontend toolings and the demo environments. All these should make our development process as smooth as possible. Multiply this by the number of the engineers you have at the company and you will understand how beneficial it is.

7. Modularity

This topic is very big and might take a separate article to talk about, but I would try briefly to go through it.

In large organizations, huge codebases are not a rare thing. With all the known issues which come together with them, like slow CI pipelines, issues with collaborative work, slow tests, etc. So the big piece of the frontend architecture is a decision on how granular we want to see independent frontend applications/modules.

There are 3 main models we have now:

• Monolith
  One big repository with a single project and all the code there, all the teams are working at this repository together at the same time.
• Monorepo
  Many projects, but still one big repository (monorepo in the wiki). All the teams are still working at the same repository, but with different projects. There are already opportunities to fix some issues, which we have with a monolith approach, by running pipelines only for specific projects, projects have smaller test suites, etc. Such tools like Lerna can make your life way much easier in case you choose this approach.
• Repo per project
  Each project has its own repository and all the supportive things, like CI pipelines, deployments, etc.

In all those models the project can mean an independent frontend application, page, independent frontend module, etc. This depends on how granular you want to decide to split your front-end applications. In most cases, this split should be in sync with the desired organization structure and people management.

The second big topic after you decided how to split your application will be how to connect those pieces together (if you decided to split your application).

And here we have the next approaches:

• Build-time composition
  Your projects can be just npm packages, installed, and composed during the build time.
• Server-side composition
  This approach usually including Server Side Rendering and the composition happening on a server. Such tools like Hypernova can help to better organize it.
• Client-side composition
  Composition of the projects inside the browser. In Martin Fowler's blog, there is a pretty good explanation of some approaches — here. Also, very important to mention Module Federation, a new approach introduced in Webpack 5.
• Route composition
  Super simple — just each project has its own URL and on an "Nginx level" we decide where to redirect users. (sorry, for such an explanation 😄 but I thought this will be the easiest to understand)

As I said before, it's super difficult to cover this topic in such a small format, but I hope you at least found some interesting ideas for yourself and will dig into the topics by yourself 🙏🏻.

Modularity

Summary:

We found a way to make our teams happy and productive by organizing the repositories, splitting our projects into smaller pieces (most likely), by making teams more independent from each other.

Welcome to paradise 😁

8. Testing

There are so many resources available about testing for front-end applications, so I would try to not get into details, which you can easily find, but focus more on large organizations’ issues and how we can solve them.

The first of them — each engineer understands testing technics differently, also which technic to apply in which situation, how to write “good” tests, etc. So it makes a lot of sense to document pretty precisely all the nuances and guidelines of the testing levels used at the company and guidelines for each of them.

Possible testing levels you want to have in your testing strategy:

• Unit testing
• Integration testing
• E2E testing
• … and others

Additionally, as a second step, we need to unify them across different frontend applications at the company, so people would not have questions about how and what to test when moved to a different project.

If you managed to unify the testing levels and approaches you automatically can help to solve the second issue — testing infrastructure setup. Each project on its own need to set up and configure it locally and on CI some testing infrastructure. For example, we decided that we use Cypress and it needs to be run inside a docker image. This needs some time to be set up locally and on CI. If we multiply that by the number of projects we have — it's a tremendous amount of time. So, the solution — to unify that again and provide some tooling for the projects. Sounds easy, but takes a huge amount of time to implement.

Non-development time testing

I also wanted to touch another way of testing your applications after features already implemented and deployed. And yes, monitoring of course it part of it 😁.

We have already mentioned in previous sections error and performance monitoring for frontend applications, uptime monitoring, the response from different locations.

It’s great also to have Lighthouse tests run on your website (can be included in the CI pipelines). To find easier performance bottlenecks, accessibility issues, and improve web pages quality in general.

And the last — production tests on the most important business flows. Most likely they will be working best if you run them directly on the production environment, but also can be possible a modification when we can run such tests on a staging/test environment, which is very close to the production one or even mirroring it. There is a very nice article regarding this topic — here.

Testing

Summary:

We have clear testing guidelines with defined necessary testing levels for each frontend application. We have a single CI pipeline for every application and we have a single CLI tool, which helps to run tests locally.

Final word

I really hope you guys found something interesting in this article. I'm very sorry, that in most of the topics I've just scratched the surface of the issues. And will be happy to discuss them more, so please leave your questions in the comments or ping me directly on twitter — @denieler.