NewSQL: The Global Scale Database Trend And What It Means For You

Data is the new oil, or so The Economist says. As businesses continue to grow and operate in an increasingly digital and complex world, they are forced to collect from many different sources.

The velocity, or how fast the data comes in, volume and variety of this data vary from the source; whether it’s an IoT sensor or a SaaS platform. This has resulted in many ways to store this data: RDBMS, Distributed File Systems like the one used in Hadoop, document storage systems, commonly referred to as NoSQL databases like ElasticSearch and MongoDB, and many other variations.

One new type of database that is starting to gain in popularity is something called NewSQL. There have been many NewSQL databases launched over the past few years like Google Spanner, MemSQL, and VoltDB to name a few. ‍

These databases aim to provide the scalability of NoSQL, meaning fast reads and fast writes, with the guarantees of Relational Databases (RDBMS).
‍

The guarantees, known ACID are critically important for data integrity. ACID stands for:

• Atomicity — This means that every transaction is treated as its own unit of work, no matter how many SQL statements that might be part of that transaction.
• Consistency — Ensures that all transactions on the SQL database leave the database in a valid state no matter what.
• Isolation — Ensures that transactions can be run in parallel while not harming the database
• Durability — This guarantee ensures that once a transaction is committed it is committed forever.

NoSQL, while providing incredible read/write speeds, did not have ACID guarantees built into them, which prevented many systems from full adoption, as they require strong data integrity guarantees. These guarantees, along with lack of distributed computing options and elasticity, cause RDBMS systems to not handle web scale. So how can NewSQL achieve the best of both worlds?

Unlike the RDBMS systems that exist, NewSQL databases achieve this amazing feat of scalability by focusing their efforts in solving issues in one problem domain. RDBMS systems attempt to be a one-size fits all tool, while NewSQL databases tend to solve one thing well. For example, VoltDB aims to do streaming data the best. By focusing on doing one thing well, they can still offer the guarantees of RDBMS platforms while also providing the scalability of NoSQL.

In the future, there will be very few general-purpose databases and they will be replaced by specific purpose data stores that can handle the scale needed for the task at hand. This trend is already occurring with application stacks using external search indexes like Solr and Elasticsearch instead of full-text solutions built into RDBMS systems.

In addition, as analytics become more complex, many businesses are switching to databases tailored towards analytic systems like SAP Hana to handle the workload. While this allows for scale to be more achievable, it creates its own problems, such as distributed data management. These problems are already being tackled by new systems as they are prevalent in certain system architectures, like microservices.

As business continues to evolve, NewSQL, and potentially other types of systems, will be adopted to solve common problems like real-time analytics on gigabytes/terabytes of incoming data, financial planning across complex corporate structures, offer management for businesses to stay competitive in the digital world, and even real time sales quoting programs.

The expectation of users is real time feedback loops powered by intelligent systems.‍

‍Gamers expect their games to have no lag.

Every ad that isn’t rendered is lost revenue and opportunity.

E-commerce sites are expected to have tailored experiences for every user.

These systems may require extreme scale in order to provide the experience expected by these users.‍

‍‍NewSQL can be part of the solution that helps provide these experiences to users.

**This article was originally published on www.skiplist.com