Istio-redirector: the way to go to manage thousands of HTTP redirections
Let’s start with a little bit of context
As the BlaBlaCar business grows, we are migrating our tech stack from a PHP monolith to a Service Oriented Architecture in Java. Along the way we have acquired existing companies to improve our bus operations worldwide and integrate them in our app aside from our carpooling experience.
This operation has sometimes forced us to change some of our backend services, like the PIM (Product Information Management). This PIM stores all of the bus rides we offer and dynamically generates the axis and city pages, which are very useful for users and Search Engine Optimization (SEO).
But the change of technology came with a change of slug generation: the old URLs like https://www.blablacar.fr/bus/destinations/nancy were now https://www.blablacar.fr/bus/destinations/nancy-2, so we needed to redirect them with a 301 HTTP code (Moved Permanently).
Why?
These pages are visible on search engines for users searching for buses, in SEA and SEO.
It was important to redirect these URLs for two main reasons:
- Users who land on the old URL are automatically redirected to the new one with the correct content instead of seeing an error page that is not supported anymore.
- These URLs are well known to Search Engine bots and have acquired some SEO popularity over time, due to their age and the content of their pages. In order to transfer this SEO popularity, we needed to tell the bots that the pages have been migrated instead of showing them a 404 page.
To do so, when the user or the bot reaches the legacy URL, we need to send a 301 HTTP Code along with the new URL.
HTTP: 301 Moved Permanently
How?
Today, our infrastructure is deployed on Google Kubernetes Engine (GKE) with Istio as a Service Mesh. You can read the article from my colleague Thomas to learn more about how we have implemented it.
Our entry point is the Google Cloud Balancer, then the Istio IngressGateway takes care of sending the traffic to the adequate service based on the path of the request.
In order to handle our HTTP redirections, we have evaluated different solutions but not limited to:
- The GCLB level, it is the most performant solution, as this technology has been created to handle network traffic, and is closer to the end-user. However, the amount of redirections we can set is limited, we won’t be cloud agnostic and it would cost more money.
- The IngressGateway level with Istio, this is possible through the creation of VirtualServices deployed on the ingress-gateway pods, the redirection is handle by Envoy in C++.
- The Frontend web application level, we can handle the redirections on the server-side of our React application in Node.js. This solution would make the request go deep down in the mesh, be redirected by Node.js and sent back to the user, not very efficient…
But we also have other requirements related to our StrongDefaults:
- Infrastructure must be defined as Code (Either in Terraform or Flux)
- Ownership must be defined and known
The list of redirections to be created is handled by our SEO Managers. They are used to work with .csv files where they write redirections with a “source, destination, http_status_code» format. Asking them to generate Istio VirtualService and let them deploy the new configuration to our production clusters was not an option, as it could have unexpected impacts on all BlaBlaCar’s traffic.
To ease the work of transforming the .csv file to the Istio VirtualService which is used in production clusters, we have created a dedicated tool named istio-redirector.
Istio-redirector is composed of a Web UI in React.js that lets the user import and preview its .csv files with the redirections detected in the file. Client side parses the file and detects host, paths and status code of the redirection. The kind of parameters that you specify on your Istio VirtualService actually.
Once reviewed and validated, in a single click, the list of redirections is sent to the backend API written in Go.
The API will parse the redirections, generate the VirtualService from a template and validate it against the official CustomRessourceDefinition of the VirtualService.
Then, a Pull Request is opened on GitHub, on the repository that hosts all our Kubernetes manifests. Thanks to the Codeowners feature, the developers of our squad dedicated to SEO will review the Pull Request and deploy it to production.
The manifest will be synced with Flux and the redirections will be available on the website a few minutes later!
By running kubectl get virtualservice -l app=istio-redirector-generated our developers can get all the redirections that have been deployed in the cluster.
But the istio-redirector UI also offers a nice list of all the redirections deployed on the cluster.
Conclusion
istio-redirector has been used to generate more than 25 000 redirections on our production domains! It shows the efficiency of Istio to handle this kind of work as it increases only by a few MB the Memory used by our Istio IngressGateway. It is able to target specific domains, paths, or any request that can be matched on the VirtualService. Istio offers a great way to handle redirections at scale, in a distributed and cloud agnostic way.
This project is a concrete and positive example of collaboration between SEO managers, front-end developers and the infrastructure team!
One more thing, it’s open source!
If you face the same issue in your company, please, feel free to use istio-redirector as we have opened sourced it on GitHub. More features could be added such as handling domain redirections or other proxy technologies.
Big thanks to Marine our SEO Manager, Cyrille the developer of the discovery squad and the Core-Infra team for their support! ❤️
Are you interested in working on such topics at BlaBlaCar? We’re hiring! Let’s get in touch :)
