Creating your own conventional framework

Or — “Why should you order your code (and enforce it)”

In the modern world of front-end development, there’s an influx of coding patterns and approaches, from rendering “libraries” like Vue and React, to full-fledged frameworks like Angular, Next.js, or Nuxt.js, But even with these tools, there’s a large degree of freedom left, A degree of freedom, that might be costly.

Hopefully, by the end of this article you’ll have some practical tools to organize your code, and some theoretical tools to make design decisions about the way you organize your code, so buckle up!

This article can be split into two parts — one part theoretical/introductional and one part practical. I highly recommend reading both in order, but if you want you can dive directly to the code.

A brief history of code-splitting

Before I’m diving in into design patterns, open-source projects, and actual coding, it’s important to talk about what brought us to this (amazing and comfortable) point — code splitting.

Not so long ago, when we wanted to break our code down into smaller parts or use a third-party library. we would have added a <script> tag in our HTML, pointing to our own or third-party script, and assumed that script added some way to interact with it (via global variables).

This approach had many drawbacks, but it pretty much let you do whatever you wanted to do! but it did not impose, or encourage code order.

With the growing complexity of web applications and teams working in collaboration on complex projects, came various solutions both for code splitting, and website/application development.

For example, Require.js allows to split code locally or configure external sources. Using the AMD (Asynchronous Module Definition) code style.

Require.js splitting modules with the AMD syntax

Another example might be AngularJS, which introduced their API to share code between pieces of an application And offer both a set of premade features, with an opinionated approach.

These two examples show an implementation of two coding patterns we’re going to be using today —

• DI — Dependency injection — “In software engineering, dependency injection is a technique in which an object receives other objects that it depends on, called dependencies.”
• IOC (Inversion of control) — “In IoC, custom-written portions of a computer program receive the flow of control from a generic framework”.

Both implement dependency injection, but AngularJS provides more than just that, it provides an opinionated generic framework — In the sense it “forced” you to write your code in a specific way, With the benefit of writing less code to get more things done (like DOM updates).

As the need for an official way to split code in Javascript grew, module support is natively provided by more and more browsers, and an official API starts to take form. Already widespread today using bundling tools like Webpack, Rollup, ESBuild, and others — we can now break our applications into simple shareable pieces of code.

What’s the word on the street today?

So now that we have a way to split our code and write very complex applications and not just simple (or not so simple) scripts in a single file… how should I split my code? 🤔

I’ll avoid the manual ways to write applications and render HTML using Node.js, and jump right into the common ways you’d create applications using popular open-source libraries. To simplify things I’m going to separate by (my own) criteria to three groups—

• UI Rendering libraries — Allowing us to create reactive and modern single-page applications, although very robust today, these libraries still mainly handle UI rendering and state management. Some notable open source libraries in this group are Vue, React, Svelte, Solid.js, and more.

React offers a simple way to create reactive applications

Application Frameworks — Either for front-end or micro-services development, these frameworks try to cover all the needs of an application developer. From a pre-baked IOC to various tools to handle requests and routing. Some notable open source libraries in this group are Angular, Nest.js.

Angular provides a way to share code between “Services” and offers built-in services

• Conventional Frameworks — In the last few years these types of frameworks grew a lot of traction in the community, beyond supplying most of the tools you need for a full application, they expect you to manage your application’s filesystem in a very specific way, and reward you with automation, ordered readable code, and reduced boilerplate. Notable open-source libraries in this group are Next.js, Nuxt.js, and Gatsby.js

In Next.js, a page is a React Component exported from a .js, .jsx, .ts, or .tsx file in the pages directory. Each page is associated with a route based on its file name.

These frameworks implement the coding by convention approach.

Convention over configuration — “Convention over configuration (also known as coding by convention) is a software design paradigm used by software frameworks that attempts to decrease the number of decisions that a developer using the framework is required to make without necessarily losing flexibility.”

It’s important to note I split them a bit arbitrarily and there are many other approaches, just as an example in Wix we use Repluggable, which provides IOC coupled with React and Redux, but doesn’t serve as an entire framework.

Should you build your own framework?

As you saw the open-source world today is rich, stable, and has a lot to offer, so I wouldn’t say yes so quickly, but I’ll try to go over some of the concerns in choosing whether to even use (or develop your own) framework.

Application complexity

I like to imagine application complexity as a spectrum, ranging from the simplest landing page websites to complex software such as site builders, graphical tools, and more.

From landing pages to complex software such as site builders & graphical tools

• Landing page— Creating a landing page, maybe with a contact form? the time you’re going to spend creating a framework most likely would be longer than creating the page itself, and using a site builder such as Wix, Webflow or any other would probably do the trick faster and easier.
• Content website— Creating a blog, store, or similar content/purchasing website, which probably requires an admin view, various pages, and layouts? I would still argue site builders are the right choice here, providing a full back-office system, user management, payments, and more. But in case you build it on your own, existing frameworks such as Next/Nuxt or Gatsby are tailor-made for this use case.
• Native/Complex software — Creating very complex software, such as graphical software, map-based software? graphical tool? Maybe the paradigm of pages, layouts, and static data as offered by Next/Nuxt is not for you, and you need a tailor-made framework for your needs.

Coupling to specific technology

Most open-source frameworks are coupled to various technologies from choosing their code bundles to their UI library, maybe you want a very specific set of dependencies and would like to avoid using a specific bundler or UI rendering library.

For example, coupling your application to experimental decorators might have implications and breaking changes in the future once decorators are supported natively

Maintenance

Open-source frameworks (at least the ones I covered here) are great! they offer a huge set of capabilities, they’re battle-tested and well maintained. There’s great hubris in thinking you (or your team) can create and maintain better tools that’ll supply the full set of capabilities they offer without putting similar effort as open-source contributors put.

When you’re creating a framework, you might just be eliminating one bottleneck (business and product implementation) while creating a new one (Infrastructure maintenance and features) — If you’re creating your infrastructure it should be documented, easy to contribute, simple to use, and tested. No different from creating a good open source project.

The task at hand

So (finally!) let’s get to code, to be clear — it’s not going to be the entire set of features Next or Nuxt provide(I’ve mentioned hubris before), but hopefully I’ll leave you with some practical ways to order your code and get started with framework building.

My (very minimal) definition of done —

• A way to automatically “gather” pieces of code — Implement a way that our framework would find and execute various pieces of code (Implementing coding by convention to some degree).
• A standard way of requesting APIs — Supply a standard way for a piece of code to “request” another piece of code (Implementing Inversion of control).
• Pages feature — Support a way to contribute pages (UI + routing) to the website
• Layouts feature — Support a way to contribute layouts to the website(multiple pages can use the same layout)

I’m going to implement 2 different approaches — one generic (generic doesn’t mean better) and one more conventional (or specific)

For both approaches, I’m going to use React as the UI rendering library and Webpack as the bundler (using create-react-app)

1. Starting with typescript + React boilerplate:

npx create-react-app AppName — template typescript

After creating the initial application, I’ll put all of the “framework” code within a folder named framework in the src folder.

The generic approach (Everything is a module)

To not couple myself to a specific product, I’m going to create a very simple and generic approach for the task at hand.

• Simple — in the sense that there’s only one convention (module).
• Generic — This single convention doesn’t have any applicative/product meaning, it’s just code that you execute.

I’m going to show partial code snippets. I recommend going over the full example to understand the broad context better

2. Modeling the system —Describing what is a module using Typescript

I like to start with the types, so what is a module? It’s a file that has a default export with a contract (interface) that the framework is going to use -

Notice that even these few lines “made” several design choices —

• Using default — Every named export becomes a reserved word, and a magic string in your framework, I prefer using the already reserved word default and describing the rest of the contract with an interface. (it also makes refactoring easier)
• Using a method `execute`— I could export a function, but I chose an object interface (that you can also implement as a class). Objects are easier to extend than just function and allow more flexibility.

3. Gathering modules from the filesystem

To achieve it, I’m going to use Webpack’s require.context. It would do the trick for now, but it might not be the best option, see issues with using require.context

We replace the code generated by create-react-app in the index file — to take control of the app lifecycle

Now every file located under `src/modules` (and sub-folders) which ends with modules.ts or modules.tsx is going to be executed when bootstrap is invoked.

src/index.ts is defined as the application’s entry point (the code that’s going run when we navigate to our application) in create-react-app.

4. Creating the first module

Just by adding this file to the filesystem, it’s going to be executed.

I can move the React rendering code (generated by create-react-app) into this module, but that just means we’re moving the entire application into 1 module… and that doesn’t achieve the goal.

So I’ll implement a way to break our application into pages and layouts and allow the modules to communicate by an API provided via the framework (Inversion of control)

5. Modeling inversion of control

Similar to the convention I’m going to start by defining the API a module declares, and the APIs the module can use, to achieve this I am modifying the framework types -

Again we encounter some design choices:

• 1 module 1 API — It’s a limitation I’m imposing, supposedly to separate concerns that each module doesn’t do more than it needs to.
• createApi doesn’t have access to getApi— Another limitation that reduces the chance of a circular dependency, at the cost of not being able to create API that uses another API (which is a very important and powerful feature)

6. Implementing Inversion of control within the framework

There are many open-source projects the provide methods to provide IOC, I’m going to use @owja/ioc, since it’s very simple, lightweight, and allows me to implement my frameworks without using decorators which I’d like to avoid.

If you do want to use decorators @owja/ioc supports them as well, but you can also consider using Inversify or TypeDI.

By using Container and createResolve I’m going to create the method that resolves a module’s API. It requires a small modification to our bootstrap function

Notice I chose to bind the APIs as singletons for simplicity

7. Creating a modules types and identifier

I expect each module to expose an Identifier, which has 2 purposes:

1. Expose a unique symbol to be used to declare/get the module’s API
2. Define a type for the API provided through this module

Since this identifier will be used in other modules to “request” its API, I’ll put it into a separate file.

8. Declaring our modules, now providing an API

• State — Since my module now needs to maintain state (an array of pages), I chose to decouple the framework from the state management and implement the state within the module. I chose MobX due to the limited amount of boilerplate code needed for this demonstration.
• The ‘createModule’ helper function — I want to provide developers with an easy way to create modules without the hassle of using too many types, so I’ve exported a createModule function to infer the expected API types correctly.

I created a similar module for the layout feature that’ll be used by the application.

Using a module in another module (Wiring the App)

I can now use the getApi method to get another module’s APIs and use it

In this example app.module uses pages and layouts modules to render routes, but I could have also used layouts in the pages module — the generic module approach gives me this degree of freedom.

It’s not the topic of this article but the actual rendering of the code can be seen in the App component itself, or the layout and page components. I used react-router-dom to configure the routes (which would probably require changes to support SSR) and used semantic-ui-react for the base UI elements.

9. The resulting structure

Key takeaways from the generic approach:

• The framework is agnostic to state or UI rendering.
• There’s a single convention (module) and it doesn’t have a built-in applicative meaning (everything is a module, a module can be anything).
• Most of the application’s concerns are implemented within the modules themselves, keeping the framework’s code very small.

The conventional (specific) approach

The generic approach leaves quite a large degree of freedom, and might be suitable for complex applications to implement various features as modules (e.g codeEditor.module, toolPanel.module, etc…), but for our specific use-case there’s a large amount of boilerplate code just to get the pages and layouts working, how can we reduce it? by moving more concerns into the framework itself.

We’ll use the generic approach as a blueprint, and add conventions on top of it.

I’m going to show partial code snippets, I recommend going over the full example to understand the broad context better

1. Modeling the framework to fit multiple conventions

Right off the bat, it’s very clear that we’re moving a lot of concerns from the modules into the framework itself:

• Pages/Layouts —Now the expected type of a page or a layout is maintained within the framework.
• Module — Modules are still supported in the same way to share code within the application (although in this context I would consider renaming them into services).

I also chose not to treat layouts or pages as a module, as they can’t declare an API or execute code in the same manners module could — to keep them to a single purpose, UI rendering only.

Notice that in the first line we’ve coupled our framework to React this is a clear trade-off between velocity (by providing ready-made utils and tools by the framework) and flexibility (the ability to replace the UI rendering library in the future).

There are probably several ways to support specific conventions without coupling to the UI rendering library, but that’s a topic for another day

2. Adapting the convention gathering

I’ve separated the logic to “gather” various conventions into a dedicated file and adapted it to support pages, layouts, and modules.

By separating the conventions, I can now manage the pages and layouts in their dedicated folders.

It’s pretty apparent that require.context, as functional as it is for this educational purpose, might not be suitable for multiple conventions due to its drawbacks.

3. Bootstrapping the application

Similar to the models, to render our application, we’re going to move the React render flow from the app.module into our bootstrap function, which (leaving out some code related to handling HMR) looks like —

Now that I control the app’s rendering, I can choose to create various types of lifecycle events, for now, I’ve left only execute, but if I want I could add more (For example before the app renders).

The creation of the runtimeApi and the way we execute the modules remained identical, I just extracted it into a dedicated file.

4. Adding a page or a layout to our application

It now becomes very simple to add new pages and layouts to our application

By just exporting (using our typed helper functions) the very simple interfaces, and providing a React component, the framework adds them to our application.

Notice that unlike Next.js, which creates the page route based on the filesystem, I explicitly state it in my Page model — This is by design, I like using conventions to reduce boilerplate code, but prefer to keep the applicative logic within strict interfaces and within the code.

Wiring the pages to work based on the filesystem should be rather simple since you have access to their path in the filesystem.

5. Using modules within pages and layouts

Since pages and layouts are no longer “modules” in the sense they don’t have an execute method and have no access to getApi directly, I can pass them the runtimeApi in several ways, I chose to go with exposing a hook from the framework —

the getApi method itself hasn’t changed, and we can pass it a module’s identifier.

9. The resulting structure

The resulting filesystem shows clearly — by moving more concerns to the framework we ease the product developer’s life, but we move concerns (and maintenance) to the infrastructure.

Key takeaways from the conventional approach:

• Moving more concerns into the framework couples it with certain technologies (such as state management, and UI rendering library).
• There are meaningful, applicative conventions (pages, layouts) that serve specific product purposes.
• A lot of the application’s concerns are encapsulated within the framework, keeping the actual applicative code base very lean and simple.

Taking it to the next level

This is just the tip of the iceberg, and I’ll just leave some ideas here for potential features:

Code Generation

Now that all we need is to add parts of our application into a folder, we can use code generation to make developers lives even better, I added a generator to my examples repo using hygen

Server Code

Next/Nuxt also supplies server endpoints based on convention, as I wrote — require.context doesn’t work with Node.JS without Webpack, you can either bundle your server code with Webpack, or use a native approach to gather modules from the filesystem using globby.

Enforcement and validations

To reduce the room for error, and to make your approach stricter you can add file system linting (ls-lint for example) to make sure all your conventions are made just right. or/and create a script to gather the conventions and validate they export the expected structure.

Require.context drawbacks

The choice of using require.context was mainly because it’s fast to set up, but it has many drawbacks —

• Webpack only — It doesn’t work outside of Webpack’s build context (e.g jest, or Node.js)
• It’s not configurable — If you want to share your conventions (configuration of convention over configuration 😆) between applications, consider alternatives like creating a Webpack plugin or using prebuild scripts that use Node.js to create conventions.
• It has an issue with HMR — I worked around it (see generic/conventional example), but it’s far from perfect and may have grave implications on the application’s state
• Bundle size — If used without a specific configuration, it will bundle all of the code together into the main bundle.

To summarize

Creating and maintaining your own framework takes a great deal of effort, and in case you’re not creating a project on a very large scale, with many contributors it’s probably not worth it. If you’re creating a large-scale project, with many developers actively maintaining it — enforcing coding style and standards makes onboarding and navigating the code a lot easier.

I wanted to introduce another spectrum — similar to application complexity we see here a relation between the features of a framework (and its’ inherent complexity) to the effort required by consumers to develop their application.

What’s the right approach? well it’s pretty clear there’s no absolute answer here, it depends on the character of the developers you work with (are they more product or infrastructure oriented?), the nature of the software you’re developing (does it have a clear end date? should it last “forever” and trade owners?) and so much more….

Hopefully, I leave you with either practical tools, or at least some new approaches to test out for your next project.