Schema versioning and upgrade in document store without service downtime
A Document store (or “document-oriented database”) is a type of NoSQL database that manages document-oriented information, also known as semi-structured data. As the term “semi-structured data” suggests, a document store does not enforce a specific document structure or schema. However, document stores used by applications typically derive the structure of the documents from the classes used by the application, hence defininig an implicit schema that is common to all documents within a collection.
During development or maintenance of the application, the classes used by the application change. In turn, this requires changes in the implicit schema of the documents. The fact that the document store does not enforce an explicit document schema facilitates this change; it is possible to simultaneously keep documents having different schema versions in the same collection, either only for a short period or also in the long term. This requires the same application to be able to handle multiple schema versions.
Still, the rollout of a new version of the application is a challenge if no service downtime is permitted. A rolling upgrade means that there will simultaneously be running instances of the current and the previous version of the application. The problem is that the previous software version cannot know the new schema version and therefore cannot handle documents written in the new schema version.
In this article, we show how schema versioning can be done in a documents store and how schema upgrades can be achieved without service downtime.
For this purpose, the following patterns are applied:
- Schema-Versioning-Pattern.
- Schema-Compatibility-Version-Pattern.
The related article https://medium.com/@bernhard.ruch/bbd4062c9819 shows an implementation of these principles using Java, MongoDB and Spring Data.
Document Store
Document stores such as MongoDB manage data in collections containing documents.
A document has two properties:
- An ID that uniquely identifies the document
- A Payload that represents the document content.
The payload can have different formats, e.g. JSON or XML
The following diagram shows two examples of such collections: Train and TrainStation.
Document schema
A collection can contain documents of any type and structure. It is possible to mix different formats (e.g. JSON, YAML, XML) and different data structures representing different entities (e.g. train, train station) in the same collection.
However, an application typically stores one entity type per collection and uses the same data representation for all documents. Therefore, a collection has an implicit schema that typically is defined by the application class (e.g. Java) that is serialized into a string (e.g. JSON). In that way, all documents of a collection have the same structure.
This makes it possible for the document store to define indexes on all documents having the same attributes, allowing an efficient lookup of documents.
The following code example, using annotations from spring-data-mongodb (such as @Document and @CompoundIndex ), spring-data (@Id and @Version ) and lombok (@NonNull) illustrates how the document schema of a MongoDB collection can be defined in a Java class:
@Documentmarks the entity as a mongodb document, also defining the name of the collection.@CompoundIndexdefines the indexes that are generated on the collection.@Iddesignates the attribute used as the document ID.@Versiondesignates the attribute used for optimistic locking.@NonNullmarks an attribute as mandatory.
import lombok.NonNull;
import org.springframework.data.annotation.Id;
import org.springframework.data.annotation.Version;
import org.springframework.data.mongodb.core.index.CompoundIndex;
import org.springframework.data.mongodb.core.index.CompoundIndexes;
import org.springframework.data.mongodb.core.mapping.Document;
@Document(collection="train")
@CompoundIndexes({
@CompoundIndex(
name = "IDX_TRAIN_NUMBER_V2",
def = "{'schemaVersion': 1, 'trainNumber': 1}",
partialFilter = "{'schemaVersion': 'V2'}"
)
})
public class Train {
@NonNull
@Id
String trainId;
@NonNull
SchemaVersion schemaVersion;
@Version
Integer optimisticLockingVersion;
@NonNull
Integer trainNumber;
@NonNull
String from;
@NonNull
String to;
String label;
...
}Schema Versioning
Typically, the schema of an entity evolves over time. Attributes are added, modified or removed, or the structure of the entity changes. In a relational database imposing a strict schema for all records of a table, this requires a data migration of each record. For databases with many large tables and relations, a schema change might require a long running data migration script that makes it difficult to implement a service update without downtime.
Document stores have the advantage that they do not require all documents to have the same schema. Therefore, changing the schema of an entity does not require all documents to be upgraded at the same time.
In order to keep track of the schema of a specific document, a schema version field can be added, allowing the application to correctly interpret the document.
Note: a new schema version is usually only required when the schema change is not backward compatible. For example, if the change consists in the addition of an optional attribute, then the new schema is backward compatible to the old one, and previous documents are still valid after the schema change. In such cases, no new schema version is required.
Indexes on documents with schema version
For collections having documents with different schema versions, it is possible to define different indexes for each schema version, taking into account the differences in the schema.
Usually, this requires adding the field “schemaVersion” into the index. At the same time, the schema version also needs to be added to the queries in order to use the correct index.
If the document store supports partial indexes (as MongoDb does), the value of the schema version can be used as a partial filter, which reduces the size of the indexes.
Service upgrade with new schema version
When a service is upgraded and the new service version requires a new document schema version, the upgrade process depends on the question whether the service is allowed to have a downtime or not.
Service upgrade with downtime
If the service is allowed to have downtime, then all instances of the service that rely on the previous schema version can be shut down before the instances of the new service version are started, resulting in a short downtime. In that case, the upgrade is simple since the new service can always write documents in the new schema version; there is no running service instance that only supports the old schema version.
However, the new service version needs to be able to read documents in the old schema version as long as documents in the old schema version still exist in the collection.
The following figures illustrate the service upgrade with downtime:
Note: In relational databases, if a service downtime is permitted, the service upgrade works in a very similar way. After shutting down the previous version, the schema of the database tables are upgraded by SQL scripts, before the new version of the application is started. This upgrade procedure could theoretically also be used with document stores: running an upgrade script that performs the schema upgrade for each document before the new application version is started. In that way, the new version of the application would not be required to support the previous document schema.
Service upgrade without downtime (rolling upgrade)
If the service is not allowed to have a downtime and a rolling upgrade is required, the upgrade process is more complex since there will be times where instances of both the old and the new service version are running. As long as there are still instances of the old service version, the new service version must still write documents using the old schema version.
For that purpose, the new service version must support a schema compatibility version that indicates in which schema version the documents will be written. At the start of the upgrade process, the instances of the new service version will run with the old schema-compatibility-version and will write documents in the old schema version. As soon as all instances of the old service version have been shut down, the instances of the new service version can gradually be restarted with the new schema-compatibility-version and can thus start writing documents in the new schema version.
Again, the new service version needs to be able to read documents in the old schema version as long as documents in the old schema version still exist in the collection. Optionally, the documents having the old schema version might be migrated to the new schema version after the software upgrade.
In summary, the upgrade process without service downtime involves the following steps:
- Step 1: rolling upgrade from Service V1 to Service V2 (with Schema-Compatibility-Version=V1).
- Step 2: rolling upgrade from Service V2 (with Schema-Compatibility-Version=V1) to Service V2 (with Schema-Compatibility-Version=V2).
- (optional Step 3): schema upgrade of the old documents having Schema-Version=V1 to Schema-Version=V2.
The following figures illustrate these steps with some example deployment states:
Data migration after service upgrade
Once the service upgrade is completed and all service instances support the new schema version (while still being able to read the old schema version), the question is, whether and how the documents in the old schema version are migrated to the new schema version.
For this, several strategies exist:
- Upgrade-on-write:
No explicit migration; old documents are only migrated when they are updated by the application.
When reading documents with old schema versions, they will have to be converted to the new schema-version on-the-fly. - Upgrade-on-read-and-write:
No explicit migration: old documents are only migrated when they are either read or updated by the application.
This means that reading a document by the application has the side effect of upgrading its schema version. - Upgrade-by-script:
An upgrade script upgrades all old documents from the old schema version to the new schema version.
This can be done while the application is running as long as concurrency issues are taken into account.
The choice of an upgrade strategy depends on the requirements of the application:
- Can the application live forever with old documents whose schema version is never upgraded?
In that case, both upgrade-on-write and upgrade-on-read-and-write are viable.
This also means that the application will always have to maintain the source code that converts documents with older schema versions. - Is it the goal to maintain only one schema version both in the code and in the document store?
In that case, upgrade-by-script is the best strategy; after running the upgrade script, the handling of old schema versions in the source code can be removed.
One option is to use a combination of these strategies:
- In general, upgrade-on-write could be used.
- From time to time, when the number of old schema versions to support becomes too large, an update-by-script can be done in order to get rid of the old schema versions and the corresponding migration code.
Schema upgrade scripts
For relational databases, there are tools that explicitely handle schema upgrades, such as Flyway.
For document stores, this is less common. There are some tools that depend on a specific document store (such as “Reschema for MongoDB”), and often, applications use custom upgrade scripts that are executed either manually or by the application.
Reasons for this might be:
- For relational databases, standardized data definition languages (DDL) and data manipulation languages (DML) exist that are supported by many relational database products and on which upgrade tools like Flyway might rely.
For document stores, there is no such equivalent. - An upgrade of all documents having a certain schema version requires reading and writing each of these documents. If this is done by a database upgrade tool, it is not much more efficient that doing the upgrade directly within the application.
