The Layperson’s Guide to Code Scaffolding
In today’s software development, many organisations are still seeking ways to deliver valuable work to their customers as quickly as possible without having to go through the headache of building prerequisites from scratch. At the same time, organisations are struggling to maintain sensible defaults of engineering practices. In this article, we will see how code scaffolding (code generation) that has emerged over the years solves these problems, related technologies and techniques, and key principles of building successful code scaffolding.
Code scaffolding, what and why?
When creating a new project, service or application, it is not only about writing a few lines of code, but also sensible defaults, and engineering practices that must be taken into account and agreed upon by the team. Some modern library/framework has already even inserted them out of the box. This means that with every new project, engineering teams have to make in-depth decisions to address these concerns and go into setting these things up from scratch:
- The format and level of application loggings.
- Automated testing tools, and libraries.
- File, directory and package structure.
- Integration with third-party that shared with multiple services.
- Coding standards, static code analysis, linting tools.
- Setting up and connecting to the CI/CD resources.
- Cross-cutting concerns, such as security, monitoring and distributed tracing.
Although these things do not directly contribute to generating business value or revenue, wouldn’t it be nice if we shortened these steps?
With that said, many organisations have created platforms to reduce pain points that are common within teams. Yet, they are getting more complicated in larger, and more diverse organisations because teams have different opinions on software engineering biased toward themselves. So, it is the liability of the organisation to incorporate sensible defaults through various methods.
One of them is code scaffolding, which is the technique to generate the source code from a pre-defined “starter pack” of code templates. What users need to do is execute the code scaffolding tool with specific inputs, such as the project name, and required third-party services. As a result, they will get the source code which aligns with conventions on project structure, and requirements. The rest is just to develop according to the business needs.
Technologies and Techniques
Technology choice depends on maintenance costs, extensibility, and organisational contexts. Note that not every organisation has to always have code scaffolding (we will cover this in the last topic).
Template repository
Maintainers prepare a source code template and store it in a repository where users can access, and clone (or fork) the repository out. When maintainers want to update the template, they edit the code and push it to the repository.
Some remote repository providers like GitHub or GitLab have a one-click button to create a source code from the template. There is also the ability to sync the generated repository with the template repository when there is an update available.
This is the simplest starting path of code scaffolding. But it lacks the ability for users to customise based on pre-defined options. To do so, there will be an extra cost on maintaining multiple template repositories.
Sub-project in mono repo
Monorepo is a multi-project repository. It is ideal for development teams to maintain multiple projects where some code is shared or some code is run across projects, at the cost of project size and additional complexity on building packaging, testing and deploying. We can apply code scaffolding by keeping the template as one of the projects. When maintainers want to update the template, just modify the code and push it to the repository.
As the name suggests, this technique suits teams maintaining monorepo. The drawback exists where the maintenance cost is as equal to the template repository technique. As well as users will have to copy a template project into a new project.
Generator
A generator is an application built specifically for scaffolding, often in the form of a web application or a command-line interface (CLI). Nowadays, we can use generators from platforms in communities around the world. Many frameworks already provide generators by default so that users can start a new project quickly without having to build them over again. For example:
- Create React App is a CLI for creating applications developed with the React library.
- .NET default templates is a CLI for creating applications (console, web API, web MVC, unit testing module) developed with the .NET framework.
- Spring Initializr is a web application for generating applications developed with the Spring Boot framework. Users can choose the framework version, package management tool (Maven or Gradle), programming language, dependencies, and metadata.
- Fresh App Factory is a web application which generates popular application frameworks, mostly a single-page application (SPA), but it also can generate code used for implementing a Visual Studio Code extension. The factory runs TypeScript CLI to update templates daily, and keeping the up-to-date template repository in GitHub allows users to leverage the template features through GitHub.
- Backstage is a developer portal platform that integrates solutions to solve common frictions across organisations, including lack of discoverability, DevOps workflow, and especially launching new services. Once the code is generated, it can be automatically registered to the Software Catalog, and CI/CD resources to address those issues. Read the article below to learn more about the developer portal:
These generators above have common limitations where they can support only a few combinations. Some of them are one-off; once it is generated then it takes additional effort to keep them up-to-date with templates.
To address these issues, we can build our generators from the platform, depending on our needs and aptitudes in a programming language. For example:
- Yeoman is a CLI to build generators. Users can either download generators from an existing open-source community or create their customised generators using the JavaScript language. The most popular one in the market is JHipster which is a generator for web applications available on both the client and server sides.
- Cruft is a CLI that can be used to generate code with pre-defined options. Developed in Python via Cookiecutter templating library, it can keep the repository up-to-date with the template repository regardless of remote repository choices. Users can also choose to accept, view, or even skip updates as desired.
Principles
“What is more important than choosing a technology is how code scaffolding will develop so it is useful and grows with the organisation for the long run.”
Do you even need code scaffolding?
As mentioned earlier, not every organisation has to have code scaffolding. The right organisations should have the following characteristics:
- The number of products and teams has increased considerably, and not correlating with the current organisational structure and culture.
- Teams start to do repetitive tasks that are not directly related to the business value generation.
- Teams are slower to deliver value to the business by building new projects because they are dependent on other teams.
- Teams begin to adopt new tools and technologies that were more difficult to use but better than the old ones.
- Organisations need to address cross-functional requirements such as security, privacy, and reliability.
On the other hand, organisations with the following characteristics should not implement code scaffolding:
- Organisations with a small number of products and small teams.
- Teams use a wide variety of technologies.
- Organisations are still in the experimenting or exploring stage.
Not only does it devalue the benefits of code scaffolding, but it also wastes time and energy building it.
Consumer-first
The code scaffolding should be done with the users in mind to make it the best and easiest option for them to start a new project without coercion.
- Users can take code from code scaffolding to further develop with little to no extra configuration.
- Users do not have to ask for a lot of approvals before using code scaffolding.
- Users can patch the code in case there is an update on the template with ease.
Don’t forget to treat code scaffolding as one of the products (and other types of platforms, too). Organisations should apply product thinking by continuously assessing user satisfaction. Then, continuously improve and
iterate on code scaffolding tools, and techniques.
Convention over configuration
Some code scaffolding tools give users a lot of combinations to configure. Even though it will benefit users in choosing what they use, the maintenance costs will increase and it might not be worth the investment considering some combinations can only be used on a single project. The problem that code scaffolding should solve is how to reduce the cognitive load on engineers to create a new project from scratch, while organisations can still incorporate good engineering practices to maintain software quality as much as possible.
Provide “paved road”
Good code scaffolding should pave the easiest road to practise sensible defaults, but it should not prevent users from modifying their source code to suit their use cases if needed. Users must keep in mind that if those use cases overlap with what the code scaffolding has already provided, they have to bear the risks and put in the effort to make it work on their own.
Final thoughts
With the right characteristics of organisations, code scaffolding can lower the cognitive load in the technical landscape during startup across multiple projects.
Focusing on the right balance of each design principle, with product thinking will sustain the code scaffolding for the long run.
Enjoy the content? Find the rest of our articles available in English and Thai below. Cheers!
