Mar 11
How Wix Accelerated Open Platform Dev with Standardized APIs & Schemas
Over the past few years, Wix has experienced tremendous growth in both scale and complexity.
From a WYSIWYG visual website editor to a complete ecosystem of features that empowers self-creators and design agencies to create professional websites, and allows anyone from individuals to large enterprises to manage their online presence and business.
However, this growth introduced a technical challenge in managing a growing list of disparate API designs and event schemas from hundreds of services.
Wix developers and API experts have implemented various tools and standardized practices to make sure the above challenge is met, dev velocity is kept high, and production issues are kept to a minimum.
Standard APIs & Event Schemas — Standardization effort
In the early days of Wix services, most of them were monoliths, with APIs based on internal use cases only and not exposed to 3rd party developers.
In order to incorporate standardization and cultivate the “open platform” mindset, It was important to move from feature-based APIs (that mostly frontend uses) to domain-based API designs and event schemas.
Designing part of a platform
The idea was to define a service’s main entities, then define the APIs and events and also define how everybody would use them, looking not only at your immediate use cases, but also research the API as a product. Talk to potential 3rd party apps developers, go over similar APIs that exist in other companies.
This allowed for better efficiency throughout the company, as APIs and events were re-used instead of duplicated. At the same time, common infrastructure and terminology were formed.
For more on adopting an API first approach you can read this article.
Exposed externally
Wix now has a comprehensive developer platform that enables the extension of Wix websites and back-office capabilities. The platform consists of two parts:
- Extending websites with Velo — a JavaScript based development environment for both frontend and backend code.
- Wix App Market for third-party applications.
Most of Wix’s microservices internal gRPC APIs are exposed as REST APIs via automatically generated thin wrappers for applications and also Velo APIs.
Wix’s microservices Kafka Events are now automatically open to external developers via Webhooks for applications and Velo Events .
Wix schemas are defined with protobuf
Previously, the standard schema definitions at Wix backend were written as Scala case classes serialized to JSON. It was mainly used for RPC as well as Kafka payloads (At the time, Kafka was less heavily used for microservices). Custom JSON-based de/serializers were created for the JVM and Node.js services (as javascript became a second language for wix backend).
Enter gRPC and protobuf
As Wix scale grew tremendously, both in terms of traffic (500 Billion HTTP requests per day), but also in terms of product areas (many verticals such as hotels, restaurants, bookings, etc…) and developers (>1000 devs), a standard microservice API architecture was introduced based on gRPC communication and protocol buffers (aka protobuf) IDLs.
Why Protobuf
Tools that generate client and server code from protobuf definitions are available for many languages, offer fast parsing, and allow forward compatibility in many cases.
Both RPC and REST
Google provides protobuf http extension that enables the dev to define http mapping per RPC method. It also provides JSON-serializers to/from scala case-classes that were auto-generated by protobuf. This uncovered the ability to use a single schema definition to model both RPC and REST APIs without any additional translation infra.
Event-driven architecture & Domain Events
In the last couple of years, emphasis started to shift to a more event-driven microservices architecture approach with the introduction of domain events.
A common standard was defined for sending events from services related to a specific domain (e.g. product-changed) in order to ensure a streamlined approach for clients to consume events.
This setup serves both Wix consumers (using Protobuf serialized with JSON and streamed via Kafka), and external consumers (via webhooks and Velo events).
Domain events
Domain events are meaningful business events notifying of a change in the state of the domain entity. Examples:
- Entity created
- Entity updated
- Entity deleted
- Email sent to member
- Payment transaction started
The domain events common proto defines an envelope with common metadata in order to make it easier for infrastructure and “horizontal” services to consume them (more on this in the following section).
Discovering APIs & Schemas with Wix Docs
At Wix, we have our own web product for discovering and presenting schemas called Wix Docs. It is used for Wix third-party application development.
Wix Docs utilizes build tools (e.g. Bazel) plugins, which collect API & Schema information from .proto, documentation.yaml and .md files.
The plugin generates OpenApi swagger.json files and sends them to Wix Docs services to be able to store and view the documentation.
Automatic API & Schema discovery & updates
In addition to Wix Docs, Wix also has a service called Business Schema that provides a list of business-entities and a machine-readable “API descriptor” for each Domain Entity with its actions and events. Business-schema API and events allow Wix developers to build client libraries, IDE plugins, and other tools that dynamically interact with Wix APIs.
Schema Evolution & Validation at Wix
Most of the schema validations at Wix are the responsibility of the developer, to make sure that they don’t introduce breaking changes. There are extensive internal API lifecycle guidelines that describe how Schemas should be introduced and changed (Including backwards compatibility, maturity levels, and versioning.)
Because of the API-first “Open Platform” mentality, all developers have built the mindset to make sure they change their public APIs and schemas in a safe manner.
Automatic validation
In order to help developers and avoid compatibility issues, a dedicated tool was developed in order to make sure there are no breaking changes in proto files.
Version Migration
A typical schema version lifecycle at Wix goes through these stages:
- Create the model
- Stabilization
- Add features / abilities / enhances / fixes
- Deprecation
- End-of-life
In case a breaking change must be introduced, the old version is marked as deprecated, and a new version is created.
Summary
In order to reach the “Open Platform” vision, Wix has created detailed guidelines and an extensive array of frameworks and tools to support Schema management and development velocity. These Include automatic discovery, serialization, and validation.
Mindset Change
An organizational mindset change was made to build a truly open platform that treats every single schema very carefully and thoughtfully as it will most probably be consumed by developers external to Wix.
Higher dev velocity
As a testament to this success, the rate of new business microservices creation and overall event consumption has risen dramatically in the last few years.
Despite having over 2000 event-driven microservices in production, Wix is able to onboard a new business domain and integrate it with the surrounding ecosystem, consuming all the necessary APIs and events, in just a few days.
A true open platform
In addition, this new domain is then automatically open to third-party application developers and Wix website developers, creating one seamless experience for both internal & external developers throughout the lifetime of this API & Schema version.
Thank you for reading!
If you’d like to get updates on my future software engineering blog posts, follow me on Twitter, LinkedIn, and Medium.
You can also visit my website, where you will find my previous blog posts, talks I gave at conferences and open-source projects I’m involved with.
If anything is unclear or you want to point out something, please comment down below.
