2021: ManoMano IT Odyssey - Part IV
Beyond the Infinite
After Cyril told us about the Cloud Mission, Laurent about our Architecture Spaceship and Antoine about the Dawn of ManoMano IT, I will continue the odyssey by leading you Beyond the Infinite.
I am Victor and I joined ManoMano first-2020 as a Solution Architect.
Once in the Cloud and with the API-zation project on track, we have put the focus on our next challenge to raise the bar on the management of our ecosystem. We want to have the systems ready to meet our current needs and the future demands: increase of data volume and new business services.
In order to achieve that, we are working on these four pillars:
- Changing Thoughts with Software craftsmanship, we began to invest on quality assurance with tests at every level so that teams may be more autonomous in their development. We recruited software craftsmanship coaches in order to help teams to enlarge their development practices on pair programming, testing strategy, code review…
- Changing Place with Mobility, we put a lot of effort into it because now customers are changing the way they access our websites. Every month the mobile devices gained more and more traffic. Moreover the digital generation is here and this kind of young users prefer to surf and buy inside dedicated apps. The old rules have to be changed.
- Changing Time with a Reactive System: we want a system that reacts to changes, not a system requesting regularly whether something changed. This move is an important one as it allows us to work closer to “real time”, reducing the time to respond, improving user experience and interaction with our partners and customers.
We are looking for a system that continues to carry out its mission in the face of any issue related to reliability. We want a platform with a strong foundation on security and observability but flexible and adaptive as well.
- Changing Future with a Collaborative System: we care to build a system which simulates an ant colony, with individual components working together in a total harmony and coordination.
In fact, we have started to adopt The Reactive Manifesto as a part of our philosophy. But, how to set up all these principles into our growing ecosystem? Revamping everything again and again (no, it’s a joke!). We are defining these five working lines in order to arrive Beyond the Infinite.
- Mobility & Front First
But not everything happens in the Back. We are now focusing on the Front too. We are involved in creating the best possible applications for our customers. We detect that more and more customers come from mobile devices: the “Mobile revolution” has arrived to the marketplaces too. The Manomano websites are fully responsive but in this new context we have to go further. We are building new apps with cool features for B2C customers and PRO / B2B customers, totally adapted for the mobile environment. We are working with ReactNative to build the apps with the same tools as we work on the web, but with the features and performance of the native applications in both platforms (iOS & Android) with the same code.
Our vision is to use the Microfrontend with Web Components and continuous improvement of the Server Side Rendering for the more critical part of the website. We want to totally split the responsibility in small pieces that are more scalable and more flexible, easy to test and deploy as we did in the Back with Microservices. “To break the monolith” consists in breaking it at every level.
2. Flow Domain
The vision of the application is changing. We should view our full Domain in a most vertical way and put the focus on defining the complete flow from the user interface to the database, also passing through Backend-For-Frontend, Backend, etc.
We must build smaller and autonomous applications that can work without dependencies on other applications.
But at the same time we want these applications to work in collaboration and harmony to solve the business problems.
Whether to show a product, do a purchase or show to a seller the evolution of their sales, having smaller components allows us to reduce our time to market. We can have better observability to check with A/B Test or Canary Release if a new feature is working properly without affecting the other features.
3. From Service Hell to Service Collaboration
Every sprint we are releasing several new services, increasing the complexity of our ecosystem. We have to find a solution to improve the management and the communication between services.
Thinking in DDD and applying some patterns such as CQRS, API Composition and Event Sourcing allows us to well define the bounded context of each service. But on the other hand, it brings us new kinds of applicative components such as REST API, Backend For Frontend, Cooker, Workers, Streaming…
Working with small services allows us to introduce new languages (Kotlin, GoLang) and new technologies like Quarkus. We were able to apply those changes in some small pieces without any big impact in the global system.
Today we are deploying them in ECS and in some EC2 instances, but we need more control and some features are missing. We have started to migrate to Kubernetes. It’s a great challenge that will help us “get closer to Jupiter”. Going to Kubernetes and putting more effort in serverless will allow us to reduce cost and have a more flexible system. Probably we will move to OpenFass because we are using some Serverless services yet. This choice opens the capability to have a more elastic system: to reduce or to increase the capacity of the system according to the demand (ex.: for Black Friday).
Deploying is one thing, orchestrating is another one. As a consequence, we want to put a Service Mesh in order to have a circuit breaker, retry mechanism between calls, service discovery or more security with mTLS. Currently we are thinking to use Consul but we don’t close the door to alternatives like Istio or openServiceMesh. Another topic we are working on the reduction of dependencies by using more and more events. This allows us to work more in an asynchronous way and to improve the isolation of the services among them.
4. From Data Centric to Event Centric
We are starting to move from Data Centric paradigm and apply to an Event Driven vision.
Even if we are using RabbitMQ for point to point messaging as mainly waiting queues, we need to raise the bar. We introduced Kafka to create a more reactive system and to sustain the split of our monoliths to services with only one responsibility, following the SOLID principles.
This kind of architecture improves our analysis capacity to understand what is happening in our system. We are starting to work with tools such as Apache Flink, Kafka streams or KSQL that allow us to work with streaming data. So we can have better solutions and less latency.
The goal is to be ready for the “Fast Data” era, handling continuously and in real-time big volumes of data. We want the system to help us, our partners and customers understand what happens in every moment and propose the best solutions.
5. Fast Data Era
The world of data is changing at an incredible speed and we are trying to adapt as quickly as possible. We are facing legal issues (GDPR compliance), new ways to explore and transform data applying streams in near real-time, new datastores to manage (from SQL to NoSQL), events centricity. All these changes transform our IT ecosystem.
Like many companies we have a MySQL monolith database. And sure, we are working to split it in domains to create smaller databases and use the best technology for the correct case (document database, key*value database…). We have introduced PostgreSql, and little NoSQL databases like Redis or DynamoDB. But in the future we are going to explore more technologies.
But we cannot forget about one other big important piece: our DataLake. We moved to Snowflake, to continue to intensively exploit our own data. We use both Change Data Capture and Business Events to populate it in real-time.
To finish what would be of an “IT Odyssey” post without mentioning that we also continue to rely on Machine Learning or Artificial Intelligence so that we may go to explore the infinite data space, to understand what we need and how we can offer a better experience to our users (customers and partners).
As you can see we are not bored and the challenges are enormous. But in the end the idea is simple, technology has to help us solve these challenges ahead in order to boost our business. Our job is to choose the best and simplest way to tackle these challenges and anticipate the future, and who knows… maybe in a few years a HAL computer will replace us. :-p
— THE END —
