We moved an old Symfony 2 app from PHP 5.6 to PHP 7.1
A Symfony 2.1 app to be more specific. Wait, what? In 2020? Yes. Context.
Context
What’s this app, how has it been built?
The application that we’re talking about is actually the API that has been used for years to serve all TheFork B2C frontends, such as the desktop website, the mobile one, and the iPhone and Android apps (to keep things simple). It’s been created around 2012, with technologies from this era (PHP 5.4, the brand new Symfony 2, and so on). Shinier than ever for 2012. The great feature that Symfony 2 was introducing, compared to Symfony 1, was the notion of splitting code into bundles, and building apps around this concept. But were we supposed to split code into bundles inside our app (by functional domains), or were we supposed to just split the reusable features and put the functional part of the app in a same package? Well, if the last one is what’s actually recommended now, it hasn’t always been crystal clear, even in the official documentation of Symfony, which was specifying that all pieces of code had to belong to bundles. So, at TheFork, we did what was the most obvious thing to do at this time. We split the pieces that could be used by other apps (B2B API, back-office, etc.) into shared bundles, and we also split the code inside our app into functional bundles (restaurant, user, reservation, loyalty, etc.)
All of this resulted, after years of development, in a big monolith holding all the B2C logic of TheFork, and relying on Symfony 2.1-compatible dependencies, either external to TheFork, or internal and shared across all the TheFork PHP projects.
On another note around our infrastructure at this time. This API is hosted at a hosting provider’s and lives alongside other PHP applications. As TheFork is currently transitioning to AWS (but hasn’t totally yet), the cost to scale, if we need it, is big.
Time flies, technology changes
You know how it is. A lot of code has been created since the beginning, relying on interfaces provided by all the bundles our API depends on. These bundles also evolved with their time. Some of them became compatible with PHP 5.5, 5.6, and then 7.0 to 7.4. Some of them disappeared, being replaced by others, some of them became unmaintained, and a lot of interfaces changed. Regarding the internal bundles, as the amount of projects relying on them grew, it became more and more painful to maintain them for all versions of PHP, and all versions of Symfony. Some patches were created, sometimes, for punctual needs, but a lot of factors (lack of time, lack of true ownership, etc.) annihilated all chances of a true maintainability. This API was becoming, month after month, a legacy, with more and more technical debt. Controlled at first. It had been quickly switched to PHP 5.5, and some years later to PHP 5.6. But still Symfony 2.1. The bigger it became, the heavier the cost was to upgrade the version of a framework that, below its 2.4 version, was introducing backward compatibility breaks.
Micro-services
In the early 2017, a big change occurred. The decision to break this monolith to micro-services was taken. All the business logic and data, that was held by our API was going to be transferred to smaller applications, split by functional domains, and the API was going to die silently, once everything had been migrated.
Back in 2020, if some of those micro-services did manage to become the true owners of their scope, other domains are still fully legacy, still handled by our API. As a result, it is still a mandatory path, despite all the efforts that are made to make it die. There is traffic on it, sometimes critical for the business.
The business grows
Finally, TheFork isn’t the same company that it used to be, eight years ago. Today, it is present in 22 countries, has acquired some other companies all over the globe, and is recognized as a true leader in Europe and Southern America in its domain. Which means more users and more traffic but also advertisement, that is followed by peaks on the infrastructure… and Google massive crawling on a daily basis. And that’s precisely this last point that kicked the anthill.
Help! What can we do?
As you can see on this picture, Google was badly hurting our poor application. As we know, it was impossible to scale horizontally. So what could we do? Those are the three options that we considered:
- Accelerate the transition to micro-services? Meh. It had already been three years, and even if a special task force was on it, the situation was too urgent and too critical.
- Accelerate the transition from our hosting provider to AWS? Oh, no. This was way too risky, and too many unknowns were remaining. The project was really big, and relying on a lot of databases, with custom shards strategies and so on.
- Go from PHP 5 to PHP 7? Well, a lot of companies did this move before us and quickly saw impressive improvements. Plus, the PHP community is known to be extremely careful about BC-breaks, so it seemed like a doable move in a short amount of time.
The migration
Where do we start?
Several strategies came to our mind when we started to think about it. Do we go PHP 7, launch the tests, fixing every bug one after another? Do we remain under PHP 5 and upgrade dependencies one after another? Well. Some clues were at our disposal to start.
- PHP 7 support in Symfony starts at Symfony 2.3, so we had to migrate Symfony to at least 2.3.
- We had to support both versions of PHP (5.6 and 7.1) so that we could rollback the PHP version if something went wrong with PHP once in production.
Docker as a work environment
As we had to ensure that our code would be compatible with two versions of PHP, the easier way to proceed was to use Docker, and simply change the version of the PHP image we would be using.
The main differences between the two environments were the names of the PHP extensions packages, that are usually prefixed with “php5-” or “php7.1-”. Fortunately, Docker’s PHP images come with handy commands like “docker-php-ext-install” or “docker-php-ext-enable”, making our Dockerfile easily switchable, by just changing the “FROM” directive at the top of the file.
Composer to help us
And as we had 64 dependencies to work on, we quickly decided that it was impossible to migrate every single one by hand. So we took the decision to let Composer do the job. Composer kills your laptop memory, but finding the accurate dependencies for your context remains its main job, after all.
Again, we had to be compatible with both PHP versions, so we decided to work with the most recent one, PHP 7.1, as it would be the target on our production environment. And we added these lines in Composer to ensure backward compatibility.
“require”: {
“php”: “>=5.6.29”,
…
},
…,
“config”: {
“platform”: {
“php”: “5.6.29”
}
}This way, we were sure that Composer would install 5.6-compatible dependencies on our 7.1 environment.
Then, we set all the versions to “*” in composer.json, under “require” and “require-dev”. Yes. “*”. We kept the versions identifiers that were previously installed (via the composer.lock file), to be sure to rollback a dependency to its previous version if a behavior had changed too much, making our app crash. And we deleted the /vendor directory, the composer.lock file, and ran “composer install”. Do the job, Composer. Find for me the most recent versions for my configuration, both compatible with 5.6 and 7.1.
It took half an hour. Then it crashed because of a lack of memory. Then I restarted my computer. Then I relaunched it. And it took half an hour again. And it crashed because it could not find suitable dependencies.
Fortunately for us, all the non-suitable dependencies were internal, so, one after another, we made them compatible with Symfony 2.8 and PHP 7.1. It took a while, as we had around 20 internal dependencies, and almost all of them needed to be fixed in order to work under newer versions of Symfony and PHP.
But we did it. After having fixed every single problematic internal dependency, composer managed to install everything, and it was compatible for both versions of PHP. Was it over, then? HAHA. You wish.
Tests
At TheFork, this API benefits from 4 suites of tests. Unit tests, PHPUnit functional tests, Behat functional tests, and finally End-to-end tests. The unit tests will test small pieces of codes, such as functions, the functional tests will test the different methods that can be called on the API, by mocking any dependency to another project, and the end-to-end tests hit the frontends on the preprod environment and go to the lowest layers. The tests are mandatory before doing such a migration. We couldn’t have performed it without them.
Unit tests
We started by fixing all the broken unit tests, and of course, this means change the code itself, as the unit tests should just be your method’s contract. As we changed Symfony’s version from 2.1 to 2.8, it’s almost like we changed a major version. The majority of failing tests were related to usages of the framework API. We’ll never repeat it enough, the more your code is independant from your framework, the more the framework will be easy to migrate, or even to change.
Starting by fixing the unit tests provides a good preview of the gain you’ll get by switching your PHP version, especially regarding execution time. We were able to quickly say to all the parties “hey, I don’t know when this will be over, but definitely, this is worth it”.
Functional tests
Fixing the functional tests is one of the most satisfying achievements. When this is done, we know that we’re getting closer and closer to the victory, as our API responds the way we want to the caller, and relies on other APIs that shouldn’t have changed, theoretically. Also, this can be painful. We may have fixed a lot of unit tests, as everything is mocked to focus exactly on what the functions do, it’s during the functional testing session that you’ll discover all the mismatches between your application and all the dependencies you upgraded with Composer, or fixed to make them compliant with new versions of Symfony.
End-to-end tests
Launching the tests before releasing to preprod
It’s very important to be able to have a comparison basis. So, launching the tests before releasing on the environment they hit, will allow us to know if some warnings, or functional errors were there before, and are unrelated to our version upgrade.
Setting up the preprod environment
If you remember what I said earlier, our application was living alongside other PHP applications, that we didn’t move to PHP 7. So we had to prepare our preprod environment to welcome both PHP 5 and PHP 7 applications. I’ve been hiding something from you since the beginning. Our application is not only an API. It also contains a lot of workers, mostly RabbitMQ consumers, that are deployed on other servers. So, to remain simple, we
— installed php7-specific libraries and binaries, and renamed the PHP 7’s php executable to php7
— made the nginx point to the php7.1 fpm socket
— made the workers use the php7 CLI instead of php
Launching the tests for good
I won’t tell you how to launch and verify tests, but monitoring the logs is really important. And as we upgraded a framework and a PHP version, we had to check both the application logs, and the PHP logs. For instance, one of our internal bundles was using a deprecated parameter in an edge, untested, case. As this parameter has been removed in PHP 7, it led to a broken feature (easily fixed, fortunately), but the application logs were not helpful to find it out.
Deploy
Once everything was ok, we decided to setup the prod environment the same way we did for the preprod, and to release and monitor. This is actually the last big test, as our upgraded new app would now be confronted to the real world. And the real users go everywhere, even where your automated tests, and your day-to-day tests don’t.
We started with the release of our code-base on PHP 5.6, to have only one thing to rollback if something was going wrong. And of course, we managed to find some issues that we quickly fixed. After having ensured the warning/errors level was back to “normal”, we decided to switch the PHP version. And that, kids, is how we switched to PHP 7.
Gains
Ok, but… was it worth it? YES.
This was a quick and essential win, to save time before the migration to AWS tools completes, and the full decommission of this project.
If you’re here, then it means that you have been brave enough to read everything, so thank you. I hope it’s been helpful, and if you have questions, please comment, I’d be glad to answer them ;)
Do you want to work with us? Please visit https://careers.thefork.com
