Bluecore’s Frontend North Star — Building Frontend at a Newly Minted Unicorn

At Bluecore, our mission is to help retailers find their best customers and keep them for life. We do that by providing data science as a service so our customers can personalize their content for their users.

Over the years, we’ve invested heavily in our data science and infrastructure teams, while building a relatively light frontend. This got us pretty far, we recently announced our Series E, raising $125 million at a $1 billion valuation.

As we approach our next chapter, we need to enable more complex workflows for our customers. In service of this, we wrote a frontend north star for our web application. This post contains our internal frontend north star document that we’re using to guide frontend development going forward. It covers several things:

• Terminology for engineers from other domains
• Component types and code organization
• An interface for an API Client
• State Management
• Testing Guidelines

We think it’s pretty good, so we decided to share it with the world. If this type of work gets you excited, we’re hiring people to build it.

Introduction

This document details the architecture for a modern web application built on top of React. It is opinionated with regard to system design, but not with implementation details or open source libraries.

It recommends a series of primitives and abstractions to make it easy to build new pages and flows. Some layers of abstraction build on top of others, however it’s designed such that we can ship individual pieces as soon as they’re ready. This allows us to provide incremental value to ongoing feature development while marching towards our goal.

Some notes:

• We will determine an execution strategy after we have aligned on a design strategy.
• This document details our in-browser architecture. We will document portions that live on the server in a future RFC.

Terminology

Pure Functions

This document frequently mention pure functions or impure functions. Pure functions are functions whose inputs always have a 1:1 mapping to their output and have no side effects. Impure functions maintain some kind of internal state, trigger a side effect, or depend on state not passed in as an argument.

Components

Components are the primary mechanism provided by React to power applications. Components are (sometimes) stateful nodes in a tree that power an application. Many types of components are detailed in this document.

React provides two types of components, function components and class components. Most of our components will be built on function components. This provides a clean, declarative, programming experience.

React applications are trees of components rendering each other. Function components are stateful (impure) functions that handle business logic, render layout, and display UI. State is typically passed up and down the React tree through props and callbacks.

Props

Props are the arguments passed when calling the function that creates the Component. React will automatically rerender or update a component when its props change. Passing a callback as a prop allows a child component to send data back up to its parent.

React has an internal render loop that separates DOM mutations from the code that describes them. When this was first released it was a remarkable development in fronted engineering.

Hooks

Hooks are a primitive provided by React to share business logic by using stateful functions. These functions occasionally have system-wide side effects. React maintains internal hook state by remembering the order they were called in.

As a convention, all hooks begin with the word use like useEffect or useState. Developer tools will assume that any function that starts with use is a hook.

Context

React provides a primitive called context that allows users to pass data up and down the tree without threading callbacks through every component. Internally, React will update the relevant components when a context changes that it depends on.

Think of them like Non-Fungible Tokens 😝.

More seriously, context has two parts, a provider and a consumer. The provider is a component that takes an initial state prop from its parent and provides a bit of state anyone can go looking for. The consumer will often be a hook that returns a getter and a setter for the state.

Design System

This is a component library filled with any UI elements that are displayed on a page. This library contains components such as: buttons, modals, icons, loading spinners — you name it.

The design system should be a tool that developers and designers use as leverage with each other. Sketch, Invision, and Figma all have a notion of components (sometimes called symbols), and we can provide a 1:1 mapping of those components to a real React component. This creates an environment where designers and developers are speaking the same language and ensures our product is consistent across products/teams.

The design system can also feature layout components and importable styles (although most styling should be done in the design system itself).

ls
- src/
- src/components/
- src/layout/
- src/styles/

A good rule of thumb is anything that renders to the DOM, belongs in the design system. To avoid being overly prescriptive, we should follow this rule 80% of the time.

Core Design System

This is the layer where we convert the components in Sketch/InVision into actual code. These will likely be written manually (as opposed to exporting them from the design application).

These components will map 1:1 with a component/symbol in our actual designs and will be focused on rendering to the DOM. The prop types for this component should allow a user to pass anything to the component’s root node. We can strictly type this.

Magic Components

These are components that we will build that will be tightly coupled to one or more of our APIs. They’ll allow us to reuse business logic for things like typeaheads, search bars, and data tables.

These will require a strongly typed API and a robust API library. Another name for these would be widgets, but I like Magic Components so much better.

Product (Application) Components

Product code will be composed of magic components, layout components, and design components. For custom components written for one feature, we should optimize for simple interfaces like the ones above so that they can be promoted to more generic magic components.

Testing and Mock Data

Our design system will use Storybook, a popular tool for demoing and manually testing individual components. We will write a set of strongly typed mock input data that we use for both our manual tests and automated tests (below).

It is easier to reproduce and test errors manually since we’ve maintained the same data set for both automated and manual testing

Magic components will leverage the API Mock Data described below.

API Library

We will have a base API class that supports auto-generated routes. This client will be able to take a path to a route, a typed object of options.

bluecoreApi Example Call

bluecoreApi(
 ‘/api/v1/create_campaign’,
 {
   request_arguments: {
     campaign_name: ‘Frontend’
   },
   headers: {
     auth: “TOKEN”
   }
 } 
)

Implementation Detail

Typescript supports keying a type on a string, which allows a strictly typed auto-generated API. Below is sample code of how TypeScript supports function overloading based on a string argument.

type PathOneResponse = {one: true};
type PathTwoResponse = {two: true};
function bluecoreApi(path: '/path_one'): PathOneResponse;
function bluecoreApi(path: '/path_two'): PathTwoResponse;
function bluecoreApi(path: string) {
  if (path === '/path_one') {
    return {one: true};
  }
  if (path === '/path_two') {
    return {two: true};
  }
return undefined;
}
const responseOne: PathOneResponse = bluecoreApi('/path_one');
// ts(2741): Property 'two' is missing in type 'PathOneResponse' but required in type 'PathTwoResponse'
const responseTwo: PathTwoResponse = bluecoreApi('/path_one');
// error: The call would have succeeded against this implementation, but implementation signatures of overloads are not externally visible.
const responseThree: PathTwoResponse = bluecoreApi('invalid_path');

API Hooks

We will provide access to the API directly via react hooks to power magic components. These hooks will be strongly typed against the API client described above.

The hook will take the request arguments as function arguments. The example below shows how types can propagate up to the React layer.

type APIPaths = '/path_one' | '/path_two';
type APIResponseMap = {
  '/path_one': PathOneResponse;
  '/path_two': PathTwoResponse;
};
type APIResponse<T extends APIPaths> = APIResponseMap[T];
function useApi<Path extends APIPaths, Response = APIResponse<Path>>(
  path: Path,
  // TODO: Implement APIArgs
  apiArgs: APIArgs<Path> = {}
): Response {
  // Authentication will be an implementation detail of the API.
  return bluecoreApi(path, apiArgs);
}

Promise Wrapper

Each API hook will be wrapped in a function that returns an easy to consume object as shown below.

function MagicComponent() {
  // useSpecificApi (or useApi) will be as simple as possible on the
  // outside, and as powerful as necessary on the inside to create a
  // state machine that abstracts a promise into a simple response.
  const {response, isLoading, error} = useSpecificApi();
  
  if (isLoading) {
   return <LoadingComponent />
  }
  
  if (error) {
   return <ErrorComponent />
  }
if (response) {
   return <UIComponent />
  }
}

Data Fetching

Application data will be fetched with the bluecoreApi module. The interface will provide EntityTypes rather than ResponseTypes.

We will provide an interface for product components to consume Entities. These interfaces will employ strictly pure functions that have no side effects. They will usually (but not always) take a single ResponseType and return an EntityType.

function transformCampaign(response: CampaignResponse): CampaignEntity;
function useCampaign(campainId): Campaign {
 const {response} = useSpecificAPI(campaignId);
 if (!response) {
  return;
 }
 
 return transformCampaign(response);
}

Client-Side Caching

We will include some form of client-side caching for APIs. This will only include API state and previous response data. Caching is kept as an abstract and only available to the API Library itself. We may decide to implement tools around polling or refetching of data. It is also possible we find a library that does this for us.

Mutations

Mutations are writes to the server that often have a corresponding UI change. These are supported inside of the API Library.

async function writeCampaign(
  campain: Campain
) {
  // Optionally supports optimistic rendering   
  writeCampaignToCache(campain);
  return writeCampaignToServer(campain);
}

Optimistic Rendering

Optimistic rendering is when we update the client before the server has had a chance to respond, allowing interactions to appear faster to the user. The potential tradeoff is that the server might provide a different response than we expect, in which case we now have to undo it and display an error.

Users want most things to process instantly, but they expect some things like a bulk import to take a moment. For toggles and typeaheads, we’ll usually do things as fast and optimistically as possible. For anything that requires trust, we’ll want to display some form of animation to show the customer what we’re doing.

Aside: This was a fun article about what to do when your writes are too fast. https://www.fastcompany.com/3061519/the-ux-secret-that-will-ruin-apps-for-you

Testing and Mock Data

Every API should have at least one checked in sample request/response. We should use these sample requests/responses both in our integration tests against the API and as inputs to tests for Magic Components and Product Code.

Product Code

Product code leverages the primitives and interfaces detailed above to build new features, workflows, and functionality quickly. These features will be maintained by the product teams working on them. These are the teams that will work directly on shipping products and interfaces to the users.

Application State

Forms

Forms are historically a nightmare to implement in React. Fortunately, there are a wide variety of form libraries we can employ to do this job.

Transition State

Legacy code will continue to use Redux. We will likely move away from Redux towards libraries with better ergonomics. Another option is to invest in the modern Redux Toolkit.

For the unfamiliar, Redux is a pub-sub event dispatcher that maintains an internal data store. However, it often looks remarkably like trying to implement your own database in the client. The nice thing with Redux is it puts all your application state in one place for better debugging.

Application Shell

At the root of the (in-browser) application will live an application shell that handles page routing, metadata, and context providers. It will take an initial payload from the server, usually in the form of a token used for API authentication.

Miscellaneous

Routing

We will continue to use React router for client-side routing. It is the standard tool for the job.

Logging

We will work with our data team to determine requirements for client-side metrics. We may decide to use bluecoreApi against a single logging route, or decide to use a lighter weight implementation.

Feature Gating

We will continue to use our existing feature gating system, however we may build new hook and component based interfaces for simpler React integration.

Types

We will rely heavily on TypeScript for maintaining contracts between parts of the system.

Unit Tests

We will use Jest, a popular test-running for headless unit testing. This will mostly be on a per component basis.

Integration Tests

We will use mock API responses and write integration tests in the same system we use for unit tests (Jest).

End-to-End Tests

We will continue to use Cypress for our end-to-end tests.

Mock Data Strategy

Every layer of our application should have robust mock data to quickly generate testing states. For example, we may want to auto-generate a state that logs a user in and skips them to a specific step of a flow.

In-Page Debug Tools

We will provide a debug panel in a corner of the page (likely lower-right). This will only be available in the development/Qa environment. This debug panel will be used as a surface for triggering state changes or automatically dropping the developer into a specific workflow for testing.

Bug Reporter

We will ship a bug reporter available in production to Bluecore employees. This bug will be an icon in a corner of a page that allows anyone to submit a bug about the page they’re on. This tool will create a JIRA ticket and include as much context about the current state of the application as possible.

We will only display this icon for Bluecore employees and we can hide it on demo surfaces.

We’re Hiring!

Wanna help build this system? We’re hiring talented frontend engineers. Come join us!