Have you ever encountered such a framework, it is very simple and lightweight, easy to use, but when your project becomes complex, it can evolve into a powerful heavyweight framework, without the need to rewrite the entire project? I have never seen it before.
Let us first look at the life cycle of a project. Usually, when a new project starts, we don’t know how long it will last, so we want it to be as simple as possible. Most projects die in a short period, so they don’t need a complicated framework. However, some of them have hit the pain points of users and become successful, and we will continue to improve them and make them more and more complex. The result is that the original simple framework and design are far from meeting the needs, and the only way left is to introduce a powerful heavyweight framework and rewrite the entire project. If the project continues to be popular, we may need to rewrite the entire project multiple times.
Thus, a framework that can evolve itself shows its advantages. We can use this lightweight framework at the beginning of a project and only evolve it into a heavyweight framework when we need it. In this process, we do not need to rewrite the entire project or change any business logic code. Of course, you need to make some small modifications to the code (the part of code that creates the structs). But this modification is many times easier than modifying the business logic or rewriting the entire project.
This sounds great, but is there such a thing? For a long time, I thought it was impossible until recently I found one.
Last year, I created a framework based on Clean Architecture and wrote a series of articles about it. Please check out “Go Microservice with Clean Architecture “. It uses the factory method design pattern to create structs (objects), which is very powerful but a bit heavy. I hope to make it lighter so even a simple project can use it. However, no framework is lightweight and powerful at the same time. I spent a lot of time on this and finally found to make a framework self-evolved is the way to go.
Solution
We can divide the code of a project into two parts, one part is business logic, in which all calls are based on interfaces instead of concrete implementations(structs). The other part is to create concrete implementations(structs) for these interfaces, which we can call Application Containers (for details, see “Go Microservice with Clean Architecture: Application Container). In this way, we can make the program container evolve itself while keeping the business logic unchanged. Most application containers use dependency injection to inject structs(objects) into business logic. “Spring” is a good example. However, for a framework to evolve itself, the key is not to directly use dependency injection as the interface between the two parts. Instead, you must use a very simple interface. Of course, you can still use dependency injection, but only inside the application container, so it is only the implementation details of the application container.
The following is the project folder structure.
The interface between application container and business logic
The interface between the application container and the business logic should be very simple. The main function is to allow the business logic code to retrieve the concrete structs(objects). Most of the time, you only need to retrieve use cases.
The following is the interface:
How to evolve the application container
Currently, I have three models of the application container, from the simplest to the most complex. You can replace the application container to a different model at any time without changing the business logic code. You can also create your models of application container as long as you following the above interface.
The basic model:
This is the simplest model, with no design patterns involved. The biggest advantage of it is simple. I’d argue that you should start most projects from this. With it, you can create your whole project in one day or event in one hour if the project is simple. You can through it away without any regret if you don’t need it anymore. The drawback is that the functionality it provides is really simple, all configuration information is hard-coded in your project. It is the best fit for the POC (Prove of Concept) type of project. You can take a look at “Order Service” for example. It is an event-driven microservice project created to provide order service.
The following is the folder structure:
The enhanced model:
This model is similar to the basic model, with the major improvement being the addition of configuration management. Now, the configuration data are not hard-coded in the project, they are defined in “structs”. You can also validate them. Making changes to the configuration is much easier and you can go to one place to get the whole picture of the project configuration. The framework is still very simple and no design patterns involved. You can switch to this model when the project is stable and need some structure. You can take a look at “Payment Service” for example. “Payment Service” is an event-driven microservice project created to provide payment service.
The following is the folder structure, and the highlighted part is the application container.
The advanced model:
Some project becomes successful and lasts for a long time. In that case, the project can get more and more complex and you also need a complex framework to keep up with it. It eventually will need powerful features such as changing the underline database or dynamically changing configuration without redeploying the code. At that time, you can upgrade it to the advanced model, which is powerful and complex. It will use dependency injection in the application container. You can take a look at “Service template 1” for example. It is a Clean Architecture microservice framework.
The following is the folder structure. You can see the folder structure looks quite different now.
How to upgrade:
Assuming you have a new project. The easiest way to start is to copy the entire “Order Service” project and then change the “structs” in it to be yours. That should be straight forward and easy to do. During that process, you can keep the original project structure. Later on, if you need to upgrade to the advanced model, then the easiest way is to copy the “app” folder from the “servicetmp1” project over and replace the “app” folder in your project and then modify the application container to generate your “structs”. You don’t need to change anything in the business logic.
Key components of the solution
The project has to be designed in a certain way to make it self-evolved. The followings are the four main components of the framework.
- Project structure
- Application container
- Interface-based business Logic
- Pluggable third-party library
Interface-based business logic
I have already talked about the project structure and the application container, here I will focus on business logic. Interface-based business logic is the key to the framework’s self-evolution. In the business logic part of the application, you may have different types of elements, such as “use case”, “domain model”, “repository” and “domain service”. Except for the “domain model” and “domain event”, almost all elements in business logic should be interfaces (not structs). For detailed information on application design and project structure, please read “Go Microservice with Clean Architecture: Application Design”
Internal interfaces:
There are two different types of interfaces. One is the internal interface, which is the interfaces used internally by the application, such as “use case”, which is an important element in Clean Architecture. The following is the code example for the interface of the “RegisterUser” use case.
Outside interface or third party interface:
Usually, a domain model needs to interact with the outside world, for example, logging service, messaging service, and so on. In Domain-Driven Design, these services are called “application service”. There are many libraries that can provide such services and you don’t want to tie your application to anyone of them. It is best to be able to replace any of the services with a different implementation without impacting your application.
The problem is that each service has its interface. Ideally, we already have a standard interface, and all different service providers follow the same interface. This will be the developer’s dream come true. Java has a “JDBC” interface, which hides the implementation details of each database, allowing us to handle different SQL databases in a unified way. Unfortunately, this success did not extend to other areas.
The key to making the framework lightweight is to turn external services into standard interfaces and move them outside the framework to make them a third-party library, which not only contains standard interfaces, but also the supporting code for the libraries that implemented this standard interface. Thus, this third-party library becomes a standard pluggable component. Every time you need to add a new implementation, you change the pluggable component instead of the framework, thus you have a small framework and a big library. Of course, you also need to change the application container in your project to switch to a different implementation, but that should be one line of code change.
To make the application interface-based, I created three general interfaces, one for logging, one for messaging, and one for transaction management. Creating a good standard interface is very hard and need expertise in an area, which is beyond my capability. So, the interfaces I created are not standard ones but are good enough for my application to run.
The following is the generic interface for logging:
Framework or Lib?
The fight between framework and lib has been going on for a long time. Most people prefer a library over a framework because it is lightweight and more flexible. But why should I create a framework instead of a lib? Because I still need a framework to organize all the different libraries together (whether it is homegrown or third-party). The key is to make the framework as small as possible and make other features pluggable components as I described above.
Because the elements in business logic are all based on interfaces, we can regard the framework as a bus (interface bus) and insert any interface-based services into it. This is the so-called pluggable framework, which realizes the perfect combination of framework and library.
Under this framework, the ecosystem of an application consists of three parts, one is an evolvable framework; the other is pluggable standard interfaces (these interfaces can be used independently of any framework), for example, the log interface mentioned above; the last part is the specific implementation library that supports the standard interface, for example, for the log function, it is “zap” or “Logrus”. The evolvable framework is the key to put everything together.
The following is the ecosystem diagram:
Comparison with other frameworks
The framework is based on [“ The Clean Architecture” ]. You can find similar elements in many other frameworks, such as “Spring” in Java, which also has application containers and makes heavy use of dependency injection. The only new thing in this framework is self-evolution.
Generally speaking, most frameworks try to deal with future uncertainties by applying multiple design patterns. And it requires complex logic, which inevitably introduces this complexity into the code. This makes most useful frameworks heavy and difficult to learn and use. But if the future does not match what is expected, then this built-in complexity will be wasted and become a huge burden. “Spring” is a good example. It is very powerful but also heavy, suitable for complex projects, but wasteful for simple projects. To avoid this, the framework has completely changed the mindset during the design and does not make any assumptions about the future, so there is no need to introduce complex design patterns in advance. You can start with the simplest framework, and can later on to evolve into a complex framework only when your program becomes very complex and need a matching framework. The key to make it self-evolved is the interface-based design.
At present, we have entered the era of microservices, when most projects are small services, which makes the need for self-evolving frameworks even stronger.
How does the business logic get the structs from the application container?
In Clean Architecture, the “use case” is a key component. If you want to understand an application, always start from here. The business logic code only needs to obtain an interface of the use case to complete any operation needed, because all other required interfaces are encapsulated in the “use case”.
In the business logic code, “use cases” are defined as interfaces rather than structs. At runtime, you need to obtain the specific implementation(struct) of the use case and inject it into the business logic. Its steps are as follows, first, create a container, then build a specific use case, and finally call the function in the “use case”.
How to call a use case:
The following is the code to build the application container.
The following is the code of “InitApp()” in the application container (In file”app.go”).
The following is a helper function to build the “Registration” use case and it is in “containerhelper.go”.
The following is the code to call the “Registration” use case. It first calls “GetRegistrationUseCase” in “containerhelper”, then calls the “RegisterUser()” function in the use case.
Conclusion:
This article introduced a self-evolved lightweight framework, which can evolve itself from a simple framework to a powerful one as your project becoming complex. During the process, no business code needs to be changed. Currently, it has three models, namely basic model, enhanced model, and advanced model, which is based on dependency injection and is very powerful. I have created three simple applications to illustrate how to use them and each application corresponding to one model.
Source Code:
The complete code is in:
Reference :
1 “Go Microservice with Clean Architecture”
2 Go Microservice with Clean Architecture: Application Container
6 “Go Microservice with Clean Architecture: Application Design”
