Hello, World!

The authors of Zetabase are proud to announce our v0.1 release. We’ve been hard at work for the past couple years developing what we think is the “perfect database,” and we’re very excited to share it with you today!

In only those couple years that Zetabase was under development, we saw technology progress rapidly around us. In the open source era, things move quickly. (This is one of the many new realities we had to design for.) But for all that rapid progress, one component never seemed to get much attention at all: the database.

This despite the fact that it sits at the center of nearly every architecture of every major system ever built. It’s critical for the performance of the system and it can often be the make-or-break factor for scalability. It’s the database, after all. And yet — or perhaps therefore — its progress seems to move slowest of all. Most people have been stringing together Redis and Postgres and some variant of Lucene and Kafka and so on for years; it remains a clumsy and awkward architecture, and yet no improvements exist.

Until now. Zetabase is the product of several years of re-thinking how a database should work. We put in all the things we cared about and thought were important, and it is our mission to continue augmenting and improving the platform based on feedback from our growing user community. To start, we made a few — really big — modifications to the idea of a database. These are:

1. Always serverless: databases crashing, failing to scale adequately, getting corrupted and so on are some of the worst heartaches an engineering team could experience. And despite the fact that every day we make use of services that seem to have no trouble scaling ad infinitum, our cloud databases are still just stock images of Postgres or MySQL on a virtual machine that we rent from Amazon or Google. There’s nothing built-in to scale automatically or optimize for performance on-the-fly. We wanted to change this, because a serverless database is a database free of heartbreak.
2. No more ORMs and mapping to rows: that’s right. ORMs are a pain, so we got rid of them. And to get rid of them, we had to get rid of rows, so we got rid of that too. You can insert bytes, and if you define a table to know how to introspect objects, you can index those objects to allow for complex queries. In other words, we decouple those two features: storing and indexing data are orthogonal concerns, and they are treated as such.
3. The security model is broken and that is making databases less useful: we didn’t realize this until we started thinking about it, but it’s really true. First off, many folks still rely on a secret password as their method of authentication. This is clearly not great from a security perspective. But it’s also bad from a usability standpoint. If any database client needs to know the password, a Javascript client on a website, for example, clearly can’t use it. We solve that problem with permissions and constraints.

Everything should be simple and fast to get started

Have you ever noticed you start a project with a file-backed data store, then replace it with a real database, then replace that with a sharded version, then replace that with Cassandra, and so on as the application grows in size and scope? You’re not alone: it’s a pain to get a database up and running. That’s why SQLite is a thing. And the high cost of running them is the reason most people upgrade databases as needed, often multiple times during an application’s lifetime, as the application grows.

We wanted to make a database that was so quick and easy to get started with that you may as well just start there.

As a result, it’s very simple to get started. Once you’ve installed the command-line tool, you can use it to create an identity:

$ zb manage -t newuser

At this point, you might try your hand at creating some tables, and you could use any of the client libraries to access your databases. Everything is pretty consistent across the client libraries. For example, to put and get some data using the Javascript client, it looks like this:

// Put: takes (table name, key, data)
await zb.put("tweets", `tweets/1`, "hello!")
// Get: takes (table name, key pattern)
zb.get("tweets", `tweets/%`).all().then((res) => { 
  console.log(`Got ${res.length} tweets!`) 
})

Changing the way we think about the security model

As it stands now, most databases use one of several old models for authentication. Typically, the model is that a backend REST service has all the privileges it might need to act on behalf of any user; the clients then effectively proxy their requests through these REST services.

We’ve devised a way to change this: by allowing for very fine-grained privileges, we can construct tables that restrict actions based on the identity of the requesting user. Thus, the “database user” is no longer the REST service, it’s the actual end user. And instead of the REST service enforcing the application’s constraints, the database does that work. This new architecture, discussed in full in this document, allows for new levels of scalability and an ultra-simple deployment model for a wide variety of applications.

It probably sounds like some kind of configuration nightmare, but it’s actually quite simple. It’s a few extra characters of effort when defining a table. The easiest way to do it is to enforce read and write constraints based on a field that must contain the requesting user’s ID. This can be set up with a simple “user constraint,” for example:

$ create mytable json "PUBLIC READ,USER APPEND special uid @uid" uid lex

This example creates a table that indexes a single field, uid. Any user can read; any user can append, so long as they correctly tag their message's uid field. It's a simple way of doing business, but it works! (For the curious ones: the designation USER instead of PUBLIC establishes that only designated sub-accounts of your account - your users - can write to the table; it gives you full administrative control over user accounts.)

This enables an entirely new category of applications:

Using Zetabase, you could scale an application to the size of Twitter using nothing but GitHub Pages.

Get started!

Hopefully you’re as excited to get started working with Zetabase as we are to present it to you! We look forward to hearing more from you. Please consider joining our community Slack or following us on Twitter — and of course, get started with Zetabase!

Originally published at https://zetabase.io.