Why AutoScout24 Changes its technology
TL;DR
written by Simon Hohenadl
European 100 engineer online marketplace AutoScout24 changes its technology from .NET/Windows/on-prem to JVM/Linux/AWS to become more flexible and better participate in the shared ecosystem of the internet industry.
Where we are now AutoScout24 is a pan-European vehicle marketplace used by over 10 million people every month. With around 2000 (virtual) servers in two data centers, we sustain a (human and automated) traffic of about 10 billion page impressions per month. Four internal production teams operate the data centers. One is a standby data center for disaster recovery. The main part of the web and backend platform is developed in nine teams. The corner stones of our technology stack are .NET with C#, ASP.NET MVC, IIS on Windows servers and an Oracle cluster. In the last years we have been splitting up our Monolith into continuously deliverable swim lanes that are cut along the lines of teams’ responsibilities. This platform is doing a great job: it delivers pages and data at good speed, scales to the size we need, accommodates new requirements and pays our salaries.
Project Tatsu On Monday, 3rd of November 2014, a team of ten started reimplementing our watch list from scratch with Scala and running completely on Amazon Web Services. In January, a second team will start on the new stack with the saved searches. Our plan for the next one and a half years is to enable all the teams on the new stack and to move most part of our traffic to the cloud. This means pretty much redeveloping the whole platform. We have given this project the name “Tatsu” internally. Apparently, this means Flying Beast in Japanese (タツ) — our metaphor for this move to the cloud.
Project Tatsu logo
I see five big changes in Tatsu:
from swim lanes to microservices, from .NET on Windows to the JVM on Linux, from our data centers to the cloud, from separate development and operation departments to “you build it you run it” teams and last but not least rebuilding our whole product. Are we out of our mind? Whenever I tell someone this story, we get to the point where they say something along the lines “Well… that’s a really big project!” I agree, but from the look on their face, I know they actually wonder whether we have some loose screws, as we say in German.
So, why are we going on this risky and expensive journey? We are an internet company. As such, we are looking up to the giants of our industry. Companies like Netflix, Amazon, Google and Spotify have pioneered running high traffic websites with high availability, continuously innovating and defining modern ways of developing online products. We want to stand on their shoulders and use their tools and knowledge instead of reinventing the wheel ourselves. Moreover, we want a microservice architecture to leverage flexibility in business and development capacity and to enable end-to-end responsibility. However, with our current technology stack it is difficult to take advantage of that knowledge and to build microservices.
.NET? It’s not that. This is not about AutoScout24 “leaving” .NET. There is no problem with .NET as a technology. Actually, we feel quite comfortable in our Microsoft niche, where open source is becoming more widespread, tool selection is quite manageable and good programming languages are available. Personally I’ve really come to like .NET, its proponents and the innovations they have brought. And C# is still the language I am most proficient in by far.
Big innovation is elsewhere But we often find ourselves left behind by technological innovation in other areas: automation in general is a hassle on Windows compared to Linux. Driver implementation for .NET often lags behind. Other tools are not available at all. Now, you could say: “Why don’t you innovate yourselves and create these tools?” Well, we are not as big as the Amazons or Netflixes out there and have decided to take the easier route:
Standing on the shoulders of giants You want to build resilient and fault-tolerant microservices? The Netflix OSS — with Hystrix, the Simian Army and Co. — or the Twitter Stack will get you a long way. You want to virtualize your hardware resources and automate provisioning? Public cloud providers like Amazon Web Services have solved a lot of problems and run stuff for you. A database that scales and replicates automatically? Amazon has one. A content delivery network and worldwide caches to deliver fast around the globe? Deploy and manage containers instead of whole machines? There is no need to build these things yourself. This list of available services and tools can be continued infinitely. By using all these instead of developing and running them ourselves, we want to focus on our core business and reduce our time to market in the long run.
Technology Ecosystems We don’t think so much in terms of single tools but rather of ecosystems. And we see a technology ecosystem as a flywheel of technology and the excitement, community and talent that evolve around it. These all reinforce each other: the more excitement there is around a technology the bigger a community will form and the more good people will adopt it. The more people work with a technology the more they will innovate and the better the technology will get…
For internet companies, the .NET / Windows ecosystem is simply smaller and spinning at a lower frequency. We want the momentum of the big flywheel.
We want to hire people that we can learn from AutoScout24 is a people company. They are our biggest asset. We have hired good engineers in the past. They have brought know-how in terms of technology we use and even an agile mindset into the company. However, when it comes to building and running a high-traffic website, new employees always learn from us. What we want is to attract people from other internet companies. Hiring relevant internet experience is another way in which we want to benefit from Tatsu.
You Build It, You Run It We have come a long way of tearing down walls between the development and operations departments. At the moment, one or more members of each development team have taken an Ops training and have rights to the production environment. This group of Devs together with Ops takes care of many aspects around releases, monitoring and tools. We are very happy with this collaboration and want to take DevOps to the next step: You-build-it-you-run-it teams. The “You” in there is One Team. That is why we are staffing the Tatsu teams with Devs and Ops. This is as much an Ops project as it is a Dev project, if not more. I am very curious to learn where this will take us and how our final team setup will be.
The Public Cloud is Cheaper? People with public cloud experience say you can save a lot of operational cost: up to 96%. I am not convinced of that. Certainly this number combining all possible savings is too high for our case. But there is a good chance we can reduce some costs by right-sizing our platform on AWS compared to our current two data center setup. Fact is however, we have not a good idea how much that could be. But we want to learn that. And: Tatsu is not a cost-saving project. It is a project to get ready for the future.
All in all, we see this as a know-how transfer and not a migration project. The focus is on learning and not on rewriting our code. And Tatsu is definitely not a big-bang migration where the IT will do nothing else for two years. We will continue improving our services to the consumer.
I plan to post regular updates on our wrestling the Beast here. Stay tuned.
