The real-time philosophy behind Zeit
Welcome to GET PUT POST, a newsletter all about APIs. Each edition features an interview with a startup about their API and ideas for developers to build on their platform. Want the latest interviews in your inbox?
For this edition, I spoke with Guillermo Rauch, creator of socket.io, about his new company Zeit. This startup aims to make cloud infrastructure as easy and fast as a mobile app. We dove into their first product: a CDN for code that lets you instantly deploy to production. We also discussed cloud vendor lock-in and how to build a developer community.
Disclosure: I’m an investor in Zeit. This article is transcribed from a live interview.
How did Zeit evolve out of Cloudup?
We built that project to help people share their files in streams. You could put anything into that stream — a file, a URL, you could even put a markdown file and it would convert it to HTML. Those user-facing features for quickly sharing had a very interesting technology underneath.
We were thinking about what happens later on to a file beyond just uploading or fetching data. What notifications can you get about it? What if anything changes about the metadata? For example, when you’re uploading something and start consuming it right away, you want to get notifications on the progress of the upload.
The main idea behind Zeit is to build real-time tools for data and code in the cloud. The first one is what we call now and it’s about real-time deployment. We call it real-time because the same principle applies — you start deploying something and you get linked to it right away.
What are some of the advantages of real-time deployment?
The deployment might fail or succeed, but both outcomes are interesting. Any modern codebase includes a build process. Some people in the JavaScript community are not as used to it, but failure is very common when building. We give you a link to your deployment right away and that allows you to communicate with your team. You can send a link and say, “Look, the deployment went wrong. How can we collaborate to fix it?”
Also, now includes immutable deployments. Every deployment is a unique link that represents the state of the project. It’s kind of like git for deployment where even just one little change in one file will initialize a new production server for you automatically. This is common for real-time systems because you’re always concerned about getting the deltas for a topic that you’re subscribed to.
With continuous integration we’ve made a big leap and everyone is deploying several times a day. It’s become a distinguishing feature in technology companies. It’s bragging rights. “We deploy X number of times a day.” We’re trying to help people go down the path of hot code reloading. The technology we’re building is optimized for thinking about data and code as sort of the same thing. The data is always flowing to the user and the code is changing continuously. I think we can propel everyone to the next generation of apps that are changing several times a day, as much as the developer needs or wants.
A big part of that is blurring the lines between development, staging, and production, and understanding how to break down services more effectively. If you work with microservices, you’re a lot less scared of shipping a change to production.
All the real-time power is exposed through a very simple CLI. Go to a terminal, type `now`, press enter, and it’s in the cloud instantly. We think that’s a very powerful proposition.
What are some of the challenges of shipping microservices?
One example: we’re not going to do databases. We tell our customers to take advantage of all the amazing cloud-based databases out there.
Migrations cause problems and I think it helps to treat your data as fluidly as your code. When you have to perform a migration on data before even launching a new service, that’s going to slow you down. If possible, it’s best to rely on a fundamental schema and then add code on top of that to keep the data in a compatible state going forward. This helps a lot with iteration speed.
Another example is naming via DNS. A lot of people have asked, “How do I go to production with this? It’s a random URL for a specific deploy.” The only mutable part of the architecture is actually the naming layer. We need names for getting to a resource. In our case, we use DNS.
You can have twenty services in production that each does something different. You can stress test before you upgrade one and the upgrade process is merely changing that alias via DNS. If you’re debating between two versions of your website or if you sent a build via Slack, all the links still work. Rolling back is a simply updating that pointer to the previous build.
Live demos have always been a painpoint for open-source projects. We’re going after the GitHub model in that sense. If the code is public, the deployment is also free and public. For those projects, they also need a way to expose the demo site in a clean way. They can run `now alias` and update the URL.
What’s the end game? How do you take over the world?
We are focused on a few metrics that are very important to us. One of them is how long it takes for me to give you a URL that links to your code. You have to consider deploys from the very beginning. For us, you install now and then you run the `now` command. For any other cloud service…maybe if you invest a lot of time upfront, you get to the point where it approaches that sort of ease of use. In between, you’re going to have to set up accounts, instances, deployment strategies, integrations, etc.
We will continue to be very focused on making that gap as small as possible. In our introductory blog post, we said our benchmark is not other providers. It’s using localhost. For example, the installation process is very optimized for not needing to validate or fetch dependencies all the time. We will continue to try to be the fastest way to go to production with JavaScript code.
I think a big part of the appeal of static websites is that you put it in CloudFront and it works fast everywhere you go. I think we’re very far away from that for every other use-case. We try to see now as a CDN, but instead of just static files, it’s any sort of JavaScript-based project. Microservices, dynamic websites, etc. Anything we can put in the now cloud will be distributed across several data centers around the globe.
From a cost perspective, we’re not trying to compete with any existing cloud providers. We’re trying to abstract them away. From the perspective of the user, there’s a big motivation for using now because you can make your site faster and scalable without any additional effort. We take care of all the difficult work of fail-over to different providers and so on.
Our next goal is to communicate this with our community and our customers in a very direct way. We’re working on dashboards for data and statistics about your deployments. We want you to have a very direct picture of how fast we are from different points in the world, both for deployment and latency of your API endpoints.
There has to be a very great user experience for the cloud. When you deploy something with now, if you add /source to the url, and you’re logged in, we show you the code of your deployment. Moving forward, we will show statistics that are relevant to every line of code (e.g. hotness or errors). It’s very important that you understand exactly what we’re doing underneath the covers.
How can you get started with now?
In the beginning, we think it’s a great tool for anything you’re developing in JavaScript. Even if you already have a deployment solution for your production code, this is going to help integrate new services and share updates much faster. Rather than throw away everything you have, you can use now alongside your existing systems.
There’s another type of customer who’s starting a new project or a new API. For those people, we are removing all the setup and configuration required to deploy and it’s faster than anything else.
You’re going to be able to iterate on your entire infrastructure so quickly that I think a lot of new projects will think twice before trying to do all of this by hand. You don’t have to setup complicated routers and tie them into logging or debugging systems to understand how the code is performing in real-time.
You touched on global availability. What are you working on for enterprises?
We want to help businesses oversee their services and support developer productivity. For example, instead of using the now.sh domain name, you can configure it to yourcompany.com and deploy services under that umbrella. Even enterprises that have spent tons of money setting up their own integration systems will benefit from having their own dedicated now setup.
Also, we’re planning a path-aliasing feature for enterprise customers. One of the problems with microservices today is that consumers still have the expectation that it works as a monolith. You expect api.maps.google.com to handle a bunch of “/” methods. If you’re working with microservices, you would like all those to be independent services. Even from the performance standpoint, every time you change the domain name, it’s a brand new connection and there is no resource sharing. We’ll provide features to run `now alias` and even point a path towards a specific deployment.
Finally, we’re working on a lot of extra features that help manage an authentication system.
You’re clearly very passionate about the developer experience and community for Zeit. How do you support developers?
That’s a very good question and it’s something that we’re continuously iterating on. We have a lot of experience launching successful open source projects, but the best practices are continuously evolving.
Compared to how we were doing open-source five years ago, I’ve joked that launching an open-source project today is kind of like launching a startup. A lot of people think that if it doesn’t have a dedicated website and custom logo, maybe it’s not a good project.
I have a very strong focus on that initial experience, the getting started documentation. That’s really tricky to do. We try to have a single source of truth for documentation. This sounds like an obvious thing, but with a lot of projects, you go to GitHub and get one version of the documentation, then there’s an HTML version of the documentation, and finally there’s slightly different information on the project website. We’re making a very big effort to keep one authoritative version.
Also, we need to improve APIs without moving too quickly. I think it’s become a meme in JavaScript to complain about API fatigue. If you’re always trying to be on the cutting-edge, can you even finish a project? Probably not, because there is a new framework that comes out before you finish your project. You always have to rewrite and there is a constant feeling that you’re missing out.
We’re all about embracing many many different ways of putting code in production. We’re not trying to build a new framework, but we did release something we called micro. This is our personal take on how microservices can be written using async and some ES6 features.
We’re even agnostic in the documentation. We show how to use express or hapi.js or micro or anything else you want. We don’t tell you, “Okay, here’s the very best way to to write your JavaScript code or this is the best tool chain.”
Also, we try to be standards-compliant (e.g. using HTTP/2 instead of our own protocol). A lot of the serverless APIs out there introduce an API that passes an object with a bunch of properties to a function. As long as your service speaks HTTP, we can make it really fast and automatically expose it as a cloud service. This is something you can do without upgrading your entire infrastructure and codebase.
How do you view AWS?
AWS was in a class of its own at the beginning. A lot of their services weren’t even available anywhere else. Also, they provided really great abstractions for things that you would otherwise do manually. For example, S3 wrapped up the pieces for hosting static files. You used to put a file on a disk, scale the file system, and put an HTTP web server in front. Today, you wouldn’t think twice about throwing the file in a cloud bucket. AWS created the right abstraction and developer experience for that particular problem.
Once you bought into the platform, then you started progressively gaining all these amazing services around it. Personally, not all those services had the same effectiveness or results as that initial data layer did. A lot of them actually make it very hard to migrate away. If you use a particular API, you’re into the way that they do it.
We haven’t done anything differently than when you run your Node project with `node index.js`, but we call it now and it deploys to the cloud. If the service were to disappear or you wanted to migrate away tomorrow, you would still have an HTTP/2-compliant code base, which is an open standard, that you can run using Node.
Tell me more about the now API. What can developers do with it?
The API gives you access to every single piece of our architecture and infrastructure, everything from listing deployments to editing or removing them to even dynamically provisioning code in the cloud. You can have your existing services dynamically provision a JavaScript project that has dependencies that get resolved.
We think that this layer of dynamic JavaScript code execution will help a lot with people that are building their own continuous integration solutions. In the open source community, everyone uses Travis, but companies use lots of different continuous integration tools. Instead of trying to fight that and replace everything that’s out there, we want to help run that code a lot faster. You can say, “Here’s what my package looks like running in the cloud.”
If you like the post, please give it a 💚 below:
Want more API interviews in your inbox?
