Front End Resource Sharing in Fiverr’s Decentralized System
How Fiverr balances resource sharing and component independency in a decentralised Front End system.
Our front end architecture at Fiverr dictates multiple gateway applications, serving “vertical” experiences. Each vertical creates web pages out of multiple components, from diverse sources, who share one runtime environment — the window, or global scope.
In our organisation, many of these components share extremely similar technology stacks (React, Redux, Lodash…) but we still want to keep them independent, and not reliant on resources outside of their control. Some components may remain as is with no maintenance for a long time, while others are changed frequently — new components and features are built all the time. We want to accommodate them all.
There will always be conflict between an environment supporting legacy code, and the desire to innovate with new technologies.
Our platform architecture has evolved into a structure which I personally feel is maintainable, performant and allows for easy development.
Throughout the system we’ve recognised the most commonly used resources and have categorised them into 3 different groups:
- Vendors — Large libraries and frameworks, usually well known open source projects, which have reasonably low major version release cycles.
- High Recurrence— Modules that have been tailored to our development needs so well they have become a second nature to developers and are expected to be available in all environments, among them our i18n solution, and statistics reporters like technical and business monitoring.
- Miscellaneous Utilities — These utilities may have various levels of re-use. They include helper functions, useful classes, repeatable business logic or technical functional operations, which are not decisively related to any feature or domain.
1. Vendors
Our approach to vendors resources is simple: It is extremely likely a single page will contain multiple programs requiring the same dependency, often a large one. We want to make sure they are able to use the same resource, and to do so we need to make them available globally. This approach is meant to create optimal browser caching and minimal (JIT) compiling.
Components declare the dependencies they intend to use from a pre defined list (the list itself is maintained as a common dependency). It’s a type of contract between components and services. For example; component A will declare: react, react-dom, lodash and component B will declare react, react-dom, redux. They also make sure to include those dependencies as Webpack externals for their respective browser targeted builds, so they are omitted from the bundled solutions of the code. Lastly, they have to assume future upgrades within major version are out of their control, so experimental features are usually off the table.
vendors.json
{
"lodash": "_",
"react": "React",
"react-dom": "ReactDOM",
"react-redux": "ReactRedux",
"redux": "Redux"
}externals.js
module.exports = Object.entries(require('./vendors.json'))
.reduce(
(collection, [route, name]) => Object.assign(
collection,
{
[route]: {
'commonjs': route,
'commonjs2': route,
'amd': route,
'root': name,
'var': name,
},
}
),
{}
);Gateway applications who consume these components are the owners of the runtime; they forge HTML documents. They know all the vendors by consuming the same “vendors” package. They can also upgrade patches and minor versions freely, assuming no serious breakage will occur. They will account for the requirements of all components, will load only required vendor scripts, and either expose them on the window or within the global.
What we found to be especially useful in this approach is, because we pipe the representation of all vendor scripts from one location, we can superimpose the global names. Originally we exposed React as React, but we’ve figured we’d be better off exposing which major version we’re using, so React will be represented as window.React15.
Seeing that all components use Webpack externals, the global name is ambiguous to the code itself — they simply import React from 'react’, and their configurations will point to the desired version. This approach allows us to migrate gradually and have the same page serve both versions of React without collision. We have, however, decided to deadline migrations in the company, to avoid vendor scripts bloat over time.
2. High Recurrence
Ultimately, we decided to break this group up and divide the tools between the two existing solutions — “Vendors” and “Miscellaneous Utilities” — based on size, maturity, and likelihood of breaking changes. At the moment of writing, “Vendors” includes only two of our own dependencies which we previously considered of the group “High Recurrence”, our i18n solution and a decorator method for the browser’s fetch API.
The rest of our utilities, however frequently used, have been merged into the utilities library, to be consumed by, and bundled into individual components (see below).
3. Miscellaneous Utilities
As we wanted to let each package consume exactly what it needs, with no bloatware, the challenge lay in creating a single utility library which could be shaken to the bare necessities and contain no repetitions whilst serving as a single point of lookup for common helper operations.
We assume common modules will be required from methods within the utility library itself, but if we were to export each module individually as a bundled solution we would cause code duplication within components. Small, individual packages create duplication but are also likely to introduce version multiplication. We definitely wanted to avoid this.
Our solution offers a utility library in the form of raw Harmony modules. Each consumer bundles in all they need from those modules, and the internal linkage between modules remains intact as well. This brings an optimal bundling solution to each individual component.
This solution allows us to introduce breaking changes in our utility programs in a non destructive manner. Consumers update at their own pace. The fact that all utilities are maintained in a single library creates a comfortable environment where consumers and other utilities will always use the same version for each utility program, within a bundle (closure). Finally, this approach allows for the evolution of bundling methods to keep consistency within consumed modules: polyfill deprecation, different ES version exports, and chunked bundling are in the control of the consumer. For example, a decision to serve ES6 code to supporting browsers now includes all utilities as well.
There is a limitation that is imposed on the utility library — it has to use only widely supported Javascript features, experimental features may not be covered by the consumers’ “transpile” process (e.g. babel plugins).
A utility
export const resolve = (string = '', context = global) =>
string
.split('.')
.reduce((prev, current) =>
typeof prev === 'object' ? prev[current] : prev, context);Utilities root
export * from './deepAssign';
export * from './env';
export * from './inTimeRange';
export * from './multiEventListener';
export * from './pluck';
export * from './resolve';
export * from './select';
export * from './sendEvents';All modules are exported as named modules. Keeping method names is helpful for cross-project maintenance and crucial for developers’ ability to jump in and out of projects. It also prevents naming collisions.
For this reason, and to simplify consumption, whether they are namespaced or nested inside directories, modules are exposed on the root of the utility library.
import { inTimeRange, env, resolve } from '@fiverr/util';The utilities library is optimised for creating functional units, and it cleans up feature code from technical operations. This repository is also covered by a testing environment with real browsers (including edge!), creates automated documentation, and is generally made to be optimal for developing functional units of code. Developers are encouraged to use it even for modules with no re-use potential.
We feel we’ve found a good balance, where large dependencies enjoy optimal caching and sharing whilst a maintainable utility library provides a rapid development environment with the ability to introduce breaking changes safely.
The world of complex web applications is growing fast, I’m certain more organisations encounter similar challenges. We hope this illustration of Fiverr’s solution will help others tailor their own.
Bonus Gotcha — Webpack exclude / include
It is common to exclude dependencies’ output from the “transpile” process (e.g. babel). Consumers of the raw Harmony modules must “un exclude” the utilities from their process. This point also applies to Jest settings, and any other transpiler.
const MODULES_TO_INCLUDE = ['@fiverr/util'];
const exclude = new RegExp(`node_modules/(?!(${MODULES_TO_INCLUDE.join('|')})/).*`);...
module: {
rules: [{
loader: 'babel-loader',
exclude,
