The future of e-commerce

What would an e-commerce platform look like if we designed it with today’s technology?

Javier Toledo
Oct 4, 2016 · 8 min read

For the past four years we’ve been developing all kinds of commerce software, including mobile commerce apps, Magento API extensions, in-store applications, funny legacy software integrations, and everything in between.

It’s interesting that, in spite of the current trend of serverless and microservices architectures and the introduction of modern and faster languages like Go and Swift, more than a half of the e-commerce installed base is built over the classic LAMP stack, featuring monolithic architectures.

At the time of this writing, for instance, Magento is the e-commerce platform with the biggest slice of market share, and there are very big players using it. Thanks to its flexibility and extensibility it can pretty much support any imaginable use case in the e-commerce world. That doesn’t come for free, though: Magento internals are really hard to understand, from both the user and development perspectives, so if the default Magento installation is not enough for you (which is the case for most mid-size+ companies), you’ll invariably need the help of specialized Magento developers and have to rely on third-party extensions and services. In addition, achieving good performance and scaling or upgrading Magento can be really challenging quests for your team.

We’ll save a rundown of the strengths and weaknesses of current e-commerce platforms for another post, though. In this article we’d like to share our vision of how a modern commerce system would look based on our experience in this field, discussions we’ve been having with people all around the world, and a great big dose of imagination.

Our dream commerce system

Most people fail in life not because they aim too high and miss but because they aim too low and hit.

— Les Brown

Microservices architecture

We think that breaking those “macroservices” into smaller ones is especially suitable for the commerce domain, because it can be easily broken into smaller subdomains that have entity by themselves. For example, a typical e-commerce application would have an online storefront, an order management system, and a fulfillment system to optimize deliveries among other subsystems. These three subsystems can usually work independently, and, when they need to communicate, they can work as a chain using the data produced in one module as the input for the next one.

We also believe that we need to think about the software systems for a commerce company as a whole — there’s too much clunky proprietary closed software in the market that doesn’t play well with others. We think that the industry needs to switch to a more open approach and we are positive that a microservices approach would be a catalyst for a progressive change in that direction, where businesses would be able to keep using their current systems and progressively replace some of their functionalities as simpler modules are developed.

Splitting the main submodules of commerce applications into microservices could yield many benefits:

  • Smaller, decoupled and domain-focused systems are easier to maintain than a huge, monolithic and generalist codebase.
  • The subsystems can be deployed and scaled independently, so you could have a lot of machines dedicated to the high-traffic storefront and just a few for your warehouse operations. You can also update one service without the risk of affecting the others.
  • The system is easier to extend and customize. Well known APIs between services can be used to create new modules or replace existing ones, without the other modules even noticing. Those APIs make them also easier to integrate with other systems.
  • The system becomes independent of a specific technology, so you can use whichever technology best suits each specific problem.

Event-Driven; Reactive!

One of the hardest challenges we’ve found in commerce systems is keeping everything in sync and consolidated among all the systems in the company. Every company is unique, with ERPs, online catalogs, or point of sale software, all of them customized for the company’s very specific needs and all using a wide range of third-party services and extensions. It’s not surprising that it’s almost impossible to find solutions that work right out of the box.

This is not a problem exclusive to the commerce domain. Actually, it’s a more general problem: how to keep coherence in a complex system that is working in a real-time environment. We’ve learned that Linkedin had exactly the same problem and they found that moving to an event-driven architecture helped them to decouple their systems and make things easier to manage. Lately we’ve also found ourselves using solutions like RabbitMQ or Apache Kafka to solve some specific sync problems and the overall feeling from our development team is that they want more of that.

Front-end development, on the other hand, needs to deal with many of real-time events. We’ve been watching event-driven architectures like ReactiveX, reactive cocoa, and especially Facebook’s React become mainstream, making the industry pay more attention to initiatives like the Reactive Manifesto.

Reactive systems go a step beyond event-driven systems: they see the event stream as something that can be transformed into new event streams, so the system is defined in terms of events and transformations instead of data models. We think that this idea can really make the difference when designing commerce systems, and produce simpler and more decoupled subsystems.

Schema-less data and heterogeneous databases

Most current commerce solutions rely on relational schema-driven databases and, in order to mitigate this problem, they have configurable database structures that are quite complex, and really hard to work with. In recent years there has been an explosion of NoSQL databases, which have influenced the way the SQL databases have evolved in turn. We believe that switching to a more open schema-less approach could significantly simplify critical parts of commerce software while keeping its flexibility.

We don’t think of NoSQL databases as a replacement for relational databases, but they’re definitely tools to consider when designing modern systems, because their different approaches may help to solve some specific problems. Using a microservice event-driven architecture makes it easier to use the best tool for each case, as each subsystem can have its own set of databases and use the event store to communicate with other parts of the system.

Event Sourcing and CQRS

Event sourcing is a technique where a log of committed events is stored and then used to build the system state. This has many implications:

  • It’s easy to replay events to rebuild current or previous states. This makes finding and fixing the origin of problems easier.
  • Subsystems become more robust; if for some reason a subsystem loses its state, it can be easily recreated from the event log.
  • The event store is an append-only database, making it easier to scale and manage.
  • By making a copy of the production event store, you can introduce fake events to predict how the system will behave in certain circumstances.

In any given event sourced system it is important to share a standardized approach to generating events, but every system can read the event source and build its own versions of the data in a different way. This is a technique known as CQRS (Command Query Responsibility Segregation), which can lead to more independence between subsystems, and data models better suited for the specific problem the submodule needs to solve.

Data, data and data

  • Automating recurrent decisions like stock balancing, when to order new goods from suppliers, or when and where to promote a new product.
  • Detecting possible problems before they happen and sending alerts to avert them.
  • Improved recommendation systems that take into account a wider range of parameters.
  • Optimization of operations in general.

Thanks to the event stream and the reactive architecture, data-driven modules can be easily introduced without affecting other subsystems.

User interface independence

We believe that a modern commerce framework must be designed independently from the UI and focus on the domain logic. UI templates are great for starting companies or low-budget projects, but if you have a great API and use modern tools, it’s usually harder to adapt a configurable UI to your needs than to start a new UI from scratch. In the case of customer-facing applications, it’s usually better to avoid templates if you want to achieve a great user experience. As a modern system, we want to avoid the need for re-engineering the backend in the future, even when augmented reality replaces all our devices.

Open source!

Let’s make it real!

The Agile Monkeys’ Journey

We write about what we learn and what we think.

Javier Toledo

Written by

Cofounder and CTO @theagilemonkeys . Tech hipster. Co-invented @boostthecloud . Working remotely from Canary Islands. Husband of @alicia_robaina and 2x dad.

The Agile Monkeys’ Journey

We write about what we learn and what we think.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade