Large Scale Laravel Application

Lets face it! Maintaining large PHP application is not easy.

(Click here if you just want to see the solution)

We all know that, Laravel is the most popular PHP framework till today. It’s directory structure is good, well organized, defined and straight-forward. Working with the default project structure provided by Laravel is absolutely fine when we are working in a small or medium sized project. But, when it is a large application with more than 50 models, that is the time we start to choke on our own code base.

Maintaining a large application is no joke. Especially, when it is not organized properly. And the default structure of Laravel is definitely not very helpful for this scenario.

But first, lets take a look at the default structure and what it becomes for large applications.

Laravel’s default application structure is like this-

|- app/
   |- Console/
      |- Commands/
   |- Events/
   |- Exceptions/
   |- Http/
      |- Controllers/
      |- Middleware/
   |- Jobs/
   |- Listeners/
   |- Providers/
   |- User.php
|- database/
   |- factories/
   |- migrations/
   |- seeders
|- config/
|- routes/
|- resources/
   |- assets/
   |- lang/
   |- views/

There is nothing wrong with the structure. But, when we work in a large application; we usually divide our business logic into Repositories, Transformers etc. …. something like the following-

|- app/
   |- Console/
      |- Commands/
   |- Events/
   |- Exceptions/
   |- Http/
      |- Controllers/
      |- Middleware/
   |- Jobs/
   |- Listeners/
   |- Models/
   |- Presenters/
   |- Providers/
   |- Repositories/
   |- Services/
   |- Transformers/
   |- Validators/
|- database/
   |- factories/
   |- migrations/
   |- seeders
|- config/
|- routes/
|- resources/
   |- assets/
   |- lang/
   |- views/

This is a obviously a well structured Laravel project. Now, lets take a look inside the Models folder -

|- app/
  |- Models/
     |- User.php
     |- Role.php
     |- Permission.php
     |- Merchant.php
     |- Store.php
     |- Product.php
     |- Category.php
     |- Tag.php
     |- Client.php
     |- Delivery.php
     |- Invoice.php
     |- Wallet.php
     |- Payment.php
     |- Report.php

Well, it is not that bad at all. We also have Services which handle all the business logics. There are also Repositories, Transformers, Validators folder which has same or more number of classes inside them. But, working on a single Entity/Model requires navigating through different folders and files. Some might find it very tiresome but others are still okey with it.

The problem is not in navigating different folders but in maintaining the code and inter-communication among the services.

Lets analyze the code base and see what we can find -

• It is a Monolith application
• It is hard (for some developers) to Maintain
• Productivity is low (need to think about inter-connection all the time)
• Scaling is an issue

The solution is obvious- Microservice. Even when we use SOA (Service Oriented Architecture), we still have to break our monolith application into smaller independent parts to scale them separately.

Awesome, we have a solution. Then why don’t we just do it? Break the code into smaller parts. It is obviously easier saying than doing it!

Usually separating a service requires two simple steps-

• Move the service’s children (Models, Repositories, Transformers etc.) into a new smaller PHP microservice application.
• Re-factor the service function calls to redirect the target to the microservice (eg. create HTTP request).

But now you need to find all the files related with that service. And surprisingly, you may find out that you have used it’s models or repositories somewhere else in the code by bypassing the service (I did this in the past). But this is not the only problem, there are more. And if we sum them up-

• There are too many files to consider
• Chance of making mistakes is high
• Creates developer frustration
• Need to re-consider the domain logic sometimes
• R.I.P. new developers

The last reason is very important because it is very hard for a new developer to grasp the whole application in a short time. But, the project manager will not give him much time. This leads to monkey patching, wrong code placement and the next new developer will be more confused.

Luckily we already have a solution — HMVC. Dividing the whole application into smaller parts where each part has its own files and folders like the app/ folder, and loaded via composer.json autoload, like the following -

|- auth/
   |- Exceptions/
   |- Http/
   |- Listeners/
   |- Models/
   |- Presenters/
   |- Providers/
   |- Repositories/
   |- Services/
   |- Transformers/
   |- Validators/
|- merchant/
   |- Console/
   |- Events/
   |- Exceptions/
   |- Http/
   |- Jobs/
   |- Listeners/
   |- Models/
   |- Presenters/
   |- Providers/
   |- Repositories/
   |- Services/
   |- Transformers/
   |- Validators/
|- database/
   |- factories/
   |- migrations/
   |- seeders
|- config/
|- routes/
|- resources/
   |- assets/
   |- lang/
   |- views/

But HMVC adds more complexity and when we want to move a particular module into a microservice; we still need to keep the controllers, middlewares etc. in the main codebase. Most of the time, moving to microservice requires to redefine the routes and controllers. Thus, we have to do redundant works. So I am not a big fan of this structure. Because,vI want to separate only what I (must) have to separate not anything more.

Domain Driven Design can be a solution

There is no perfect solution. Everything has a trade-off. But everyone has a preference. I am not going to discuss about Domain Driven Design (DDD) here. Developerul DeLaUnu wrote a great description about DDD in here.

In my perspective, DDD (could) structures your Laravel application into 4 parts (or 3 … see how) -

• Application — usually holds, Controller, Middleware, Route
• Domain — usually holds business logic (Model, Repository, Transformer, Policy etc.)
• Infrastructure —usually holds common services like Logging, Email etc.
• Interface —usually holds views, lang, assets.

If it is this easy then why don’t we structure our application like this and use Namespace ?

|- app/
   |- Http/ (Application)
      |- Controllers/
      |- Middleware/
|- Domain/
   |- Models/
   |- Repositories/
   |- Presenters/
   |- Transformers/
   |- Validators/
   |- Services/
|- Infrastructure/
   |- Console/
   |- Exceptions/
   |- Providers/
   |- Events/
   |- Jobs/
   |- Listeners/
|- resources/ (Interface)
   |- assets/
   |- lang/
   |- views/
|- routes/
   |- api.php
   |- web.php

Because, dividing the project into folders is not going to work. It just means we are adding nothing but a parent Namespace.

Moment of truth

By this time you are probably already annoyed and just want to see the solution. So, here it is -

|- app/
   |- Http/
      |- Controllers/
      |- Middleware/
   |- Providers/
   |- Account/
      |- Console/
      |- Exceptions/
      |- Events/
      |- Jobs/
      |- Listeners/
      |- Models/
         |- User.php
         |- Role.php
         |- Permission.php
      |- Repositories/
      |- Presenters/
      |- Transformers/
      |- Validators/
      |- Auth.php
      |- Acl.php
   |- Merchant/
   |- Payment/
   |- Invoice/
|- resources/
|- routes/

The Auth.phpand Acl.php are the service files inside the app/Account/ folder. Controllers will only access these two classes and call their functions. Other classes (outside of the domain) will never know about the other remaining classes in the app/Account/ folder. The functions inside these services will only receive basic PHP data types like array, string, int, bool and POPO (Plain Old PHP Object) but no class instances. Example -

...
public function register(array $attr) {
    ...
}
public function login(array $credentials) {
    ... 
}
public function logout() {
    ...
}
...

Notice that the register function receives an array of attributes instead of the User object. It is important because the other class who is calling the function is not suppose to know the existence of the User model. This is the base rule of this entire structure.

When we want to separate the code

Our application has become large and we want to separate the Account domain to a separate microservice and turn it into OAuth server.

So, we only move the following parts-

|- Account/
   |- Console/
   |- Exceptions/
   |- Events/
   |- Jobs/
   |- Listeners/
   |- Models/
      |- User.php
      |- Role.php
      |- Permission.php
   |- Repositories/
   |- Presenters/
   |- Transformers/
   |- Validators/
   |- Auth.php
   |- Acl.php

To a brand new Laravel (or maybe Lumen) application -

|- app/
   |- Http/
      |- Controllers/
      | - Middleware/
   |- Account/
      |- Events/
      |- Jobs/
      |- Listeners/
      |- Models/
         |- User.php
         |- Role.php
         |- Permission.php
      |- Repositories/
      |- Presenters/
      |- Transformers/
      |- Validators/
      |- Auth.php
      |- Acl.php
|- routes/
|- resources/

Of course, we have to write code in the controllers and routes as we need to make it an OAuth server.

But what change do we need to make in the main codebase?

We only keep the Service files Auth.php and Acl.php and change the codes inside their functions into HTTP requests (or other methods like messaging) targeting the newly created microservice.

...
public function login(array $credentials) {
 // change the code here
}
...

The entire application will remain the same. And the application structure will look like this -

|- app/
   |- Console/
   |- Exceptions/
   |- Http/
      |- Controllers/
      |- Middleware/
   |- Providers/
   |- Account/
      |- Auth.php
      |- Acl.php
   |- Merchant/
   |- Payment/
   |- Invoice/
|- resources/
|- routes/

And with this less effort, you can move a part of your code into a completely separate microservice. I can not think of any other way to do as little as possible while moving a part of code into a microservice.

Trade-offs

As I said before, everything has a trade-off, this solution is no different. And, here we have a problem regarding the migration! Because in the above folder structure (before separation) all the migration files are places inside the database/migrations/directory. But when we want to separate a domain, we need to identify and move the migrations of that domain too. This could be hard because we do not have any clear indication of which migration belongs to which domain. We some how need to put a identifier in the migration files.

That identifier could be a domain prefix. For example, we can name the migration file xxxxxxxxx_create_account_users_table.php instead of xxxxxxxxx_create_users_table.php . We can also use the account_users table name instead off users if we want. which I prefer to identify which tables to move during separation. Separating migration files could be a bit frustrating but if we use prefix or any kind of marker, the process will become less painful for sure.

I am still experimenting with the structure and planning to build a Laravel package which will provide artisan commands to automate file generation and separation process. I will add the package link here when it is done.

Until then, please share your feedback as I need help to find the best solution.

Have a good day :)