An update on our hosting setup
In October 2020, we published a blogpost on why we decided to use AWS as a hosting provider.
We also promised that we would challenge our setup every three years, with the next time at the end of 2021.
A few months ago, we analyzed our infrastructure needs and risks in terms of different criteria: security, efficiency, reliability, public image…
As we analyzed our tools, we realized that one of our weaknesses was our reliance on Heroku for our deployments.
Since our beginning in 2016, Heroku, a Platform as a Service (PaaS) owned by Salesforce, has allowed us to run Alan’s applications on AWS servers by delegating our servers’ management and allowed us to push to prod many times a day.
Why Heroku was not appropriate anymore
After years of loyal service, Heroku was not appropriate anymore for us:
Heroku is expensive. Although we didn’t have a huge usage, it cost us around €200K per year. We also had to commit to fixed yearly plans in advance that didn’t adapt to our consumption: we paid for everything even if we used half, and it was hard to get more resources.
Heroku is fairly inflexible. There are some baked-in limitations in Heroku (such as the machine and application size) that forced us to find workarounds which were painful every time. Its opinionated stance is great for a small startup, not for us anymore.
Heroku has not improved much in the past years. The majority of updates are technical ones, and there were no new features that were of interest to us.
Our commitment with them was coming to an end in July 2022, so it was an appropriate time for us to challenge its usage.
Where we moved
To move off of Heroku, we first went through the most promising Platform as a service (PaaS) offerings.
As we have a slight bias towards French providers, we went to Qovery, which seemed to fit the bill.
At the time of our testing (beginning of 2022), Qovery was promising in terms of flexibility, pricing and feature set. Unfortunately, they notably missed a solid infra-as-code tooling with Terraform, a very key component for us to manage our infrastructure in a sane way.
Our alternative was to consolidate our tooling towards AWS, which proposed a cheaper and more flexible alternative to Heroku, called ElasticBeanstalk.
In terms of cost reduction, this move enables us to explore opportunities like usage of reserved instances, switch to ARM/Graviton machines, etc. With on-demand instances, you pay a monthly price but there’s no time commitment. With reserved instances, you reserve instances for some amount of time, at a discounted price. By doing so, we estimated we could save at least 5% of our AWS bill.
In terms of flexibility, ElasticBeanstalk provides us with the machine and application size we need, as well as more control on the infrastructure and build. As part of the AWS ecosystem, this solution integrates well with a variety of AWS services, notably with Virtual Private Clouds, which are key to the security of our members’ data.
In terms of migration, the consolidation of all our hosting tools in a single provider (AWS) allows us to be counterintuitively more open to moving everything in the future. Less providers means less boundaries to maintain and to migrate if we decide to change providers.
In the end, its better autoscaling fits the company needs.
How we moved off Heroku
As we aimed for zero downtime, we made both infrastructures live at the same time, to ensure everything worked well before shutting down on Heroku.
On one side, our users had access to Alan’s applications that were running on Heroku. Business as usual for them. On the other side, we were deploying the same applications on ElasticBeanstalk on a test version.
Once the ElasticBeanstalk version was up-and-running, we switched the users to this version without any downtime, and therefore had no impact on the user experience.
In terms of security, remember that behind the curtains, Heroku is using AWS, so we didn’t have to change our strategy.
We had a bit of a rush since our credits with Heroku were ending on 31th of July.
In the end, thanks to the efforts of our amazing Foundation Unit, the transition happened cleanly and we shut down Heroku on the 29th of July (in reality, we had already negotiated an extension with Heroku just in case things didn’t go well). That story would deserve its own blogpost!
Why we didn’t challenge the full hosting
There are a few reasons why we didn’t want to challenge the hosting of both our applications (previously on Heroku) and our data (on AWS) at the same time.
First, we are tight on engineering resources and a full move of our hosting would take a very big toll on what we could deliver in terms of product for our members (Member-First is one of our values!).
We also decided that it would be more efficient to first consolidate towards the same provider, so it would give us more flexibility if we wanted to completely challenge our setup in the future.
By simplifying the process to migrate to another provider, we mitigated the risk of becoming too dependent on AWS. This may seem counterintuitive, yet it’s the interfaces between tools that are the most challenging usually.
And finally, the status of “clouds de confiance” is still in flux in France. We cannot afford to bet on a technology which doesn’t have a solid track record (cf our philosophy of boring technology), and that wouldn’t solve the underlying problem of image. We still believe we are using the best solution in terms of security and privacy given our context as highlighted in our previous blogpost.
Our next challenge for our full hosting will be in 2024. We’ll be happy to review proposals at that time!
