I’ve spent the last 7 months working on a new programming language called MSON (pronounced Mason). Here is a nerdy post about why I did it, how it works and where I want to take it.

There are already a bazillion programming languages, why do we need another?

Software continues to eat the world and software development tools are evolving at lightning speeds, but it stills takes an in-depth knowledge of a programming language to develop software. Sure, there are millions of open-source projects that can be mashed together and robust cloud platforms that make the process easier, but it takes considerable skill to get all of these technologies working together. When I envision the future, I imagine the average person being able to make software, just like how the average person can create a spreadsheet today.

Creating a spreadsheet has become commonplace and this is largely a result of the intuitive user interface (UI) in MS Excel. Developing software; however, still remains something that is only done by a relative few and using complex programming languages. One could argue that it is somewhat trivial to create a UI that generates code, but it is almost infeasible to reverse the process and digest coded logic, as most programming languages have an almost infinite number of ways of representing logic. In other words, we don’t usually know how a piece of software will execute until we actually execute it. Creating a truly intuitive way of developing software requires a new paradigm, a new language that is easily read, modified and written by a UI. …

In Scalable CouchDB Replication and Change Listening with Spiegel, I summarized how you can use Spiegel to scale your replication and change listening. Be sure to read that introduction before exploring this follow-up article.

In this post we are going to use a basic example to step through how you would actually use Spiegel. This tutorial assumes that you have some familiarity with CouchDB and it will only focus on the replication and change-listening elements of these examples.

Example: A Blogging App

Consider an example where we have users posting blog entries. Let’s assume that we want to use PouchDB to sync data between the client and CouchDB. To streamline this syncing we’ll assume a design of a database per user where each user’s database is named user_<username>. We can then use PouchDB to sync with the user’s DB so that the user can create and edit blog posts via their own DB. Moreover, having a DB per user will allow us to restrict access to the user databases so that only the owner of a post can edit her or his posts. …

Editor’s Note: This article on Spiegel is intended for those who already have some familiarity with Apache CouchDB™, a JSON database known for its excellent offline sync capabilities. To learn more about CouchDB and Offline First, check out Geoff’s previous article on CouchDB, PouchDB, and Hoodie as a Stack for Progressive Web Apps.

Spiegel was originally designed to provide scalable replication and change listening for Quizster, a photo-based feedback and submission system. Spiegel is now an open source project available to everyone.

So, why do we need Spiegel? The short answer is that without it, you’ll have a hard time scaling your CouchDB replications and change listening as your user base grows. …

I recently had the privilege to attend Offline Camp, a 4-day retreat that brings together a select group of people to discuss offline-first development and design. One of the unconference sessions I participated in was CouchDB, PouchDB and Hoodie as a Stack for Progressive Web Apps. In this session a small but passionate group of us took turns sharing our views on this offline stack and asking questions like, why isn’t this stack more popular? The following is a write-up of what we discussed with my added two cents.

Let’s start with some background. An offline-first design enables an app that is mostly, if not completely, usable when offline, but which can then sync with the cloud when online. Most popular mobile apps, e.g. Google Maps, Facebook, and email implement at least some basic form of an offline mode, so it is easy for everyone to understand the importance of having at least some offline support. Going a step further and implementing an offline-first design takes work, but most people can agree that an offline-first design greatly improves the user experience because it reduces latency for users by keeping the majority of data changes local. And, in many cases where an Internet connection is not available or is unreliable, it is an absolute necessity to be offline-first. The next question is, what stack can you use to actually create an offline-first app? …

Imagine that you’ve got your new shiny CouchDB cluster working in production and then a node goes down. If your design is solid and you have enough CouchDB nodes still left running, your app should keep chugging along. You on the other hand, will want to know why a node went down so that you can determine whether you need to tweak things like the memory or CPU on your nodes. You may also want to receive notifications via email or slack so that you can diagnosis things in real-time.

Prometheus is an open source monitoring system that allows you to treat monitoring metrics as a data source for generating alerts and reports. It was originally built by SoundCloud and has since joined the Cloud Native Computing Foundation. There is a thriving community around Prometheus and a lot of developers have contributed plugins (known as exporters) for many web services, including CouchDB. …

AWS has an awesome firewall built into its core services which can easily be used to make sure that only certain ports are open to the outside world. One extra step that we can take is to run a VPN Server that serves as the gateway to our protected EC2 instances. We can then shutdown direct SSH access to our EC2 instances and also have the freedom to block access to our entire network just by revoking access via our VPN Server. The later is very useful if you need to revoke access for a former employee.

The following tutorial will take you through the steps of setting up an EC2 instance that will run the OpenVPN Server. It will then cover how to grant and revoke access through the VPN Server. …

Things are heating up in the CouchDB universe now that CouchDB 2 is an out-of-the-box multi-master database that can scale to store a lot of data!

Unfortunately, there is still a bit of a shortage in documentation when it comes to how to use CouchDB 2 in production. The point of this tutorial is to take you step by step through the process of setting up a CouchDB cluster in production using AWS and Docker. We’ve used a similar setup for Quizster, a digital dropbox and grading system, and it is working great!

The setup below uses open source software and therefore, it can easily be adapted to work for the Google Cloud Platform, Azure or any other hosting providers, i.e. no vendor lock-in. Moreover, because we are using open source software, you can also set up a local environment to develop against! (VirtualBox and Vagrant are great for…

With the recent release of CouchDB 2, the CouchDB ecosystem is on fire! There is now an out-of-the-box way of setting up a multi-master cluster of nodes that can scale to handle a ton of data. And, tools like PouchDB can allow your app to talk directly to CouchDB to enable your apps to be offline-first. It’s an exciting time to be using CouchDB at the data layer of your app!

Even if you focus on a frontend-heavy design, chances are, you’re going to have to do a lot of coding on your backend to communicate with your CouchDB instance, including subscribing to real-time changes. …