Connecting data is hard. What if it was easy?

I’m building Joiner, a radically easy data warehouse. Will you join me?

Michael J. Irvine
3 min readMar 12, 2018

As a data analyst, I want to spend my time answering important questions. You probably do too. We like designing dashboards to help the CFO understand what’s going on across the company, or figuring out the right way to A/B test a new feature. There’s nothing worse than spending hours struggling to join and clean separate data sources.

Unfortunately, that’s a huge part of what we do. According to a study by Blue Hill Research, analysts spend “between 40%-60% of their time just working to get their data to ‘play nice’.”[1]

Source: Blue Hill Research, August 2015

And for good reason. It’s hard. Think about the traditional ways of dealing with this. Imagine you want to join your product database with the Finance team’s spreadsheets. Or maybe you’re interested in understanding how marketing leads convert into sales and become your users.

If you’re lucky, you work at a company with an advanced data pipeline and an enterprise-wide data lake or data warehouse. One big database that everything else is copied into. It makes your life a lot easier. As long as you already have one.

If not? You’re looking at a few months or more to get it spun up, typically involving hiring data engineers or consultants. It’s the opposite of quick and easy.

If you don’t have a data warehouse, maybe you’d output your data into into a temporary database or Excel, or manually join it all with a Python script. Those methods can work (and I use them often). The problem is you have to do them over and over again. It wastes a lot of time.

Is there a better way?

These frustrations made me wonder: is there a better way? What if I could just spin up a small, easy data warehouse for myself or my team?

There is.

Over the past couple months, I’ve been building a radically easy data warehouse, Joiner, to do just that.

Joiner lets you set up a data warehouse in the cloud, connect your sources, and start querying without any overhead. Right now it only takes 5–10 minutes to go from creating an account to querying all your data in one database (including the couple minutes it takes for Amazon to get the instance ready).

5–10 minutes from creating an account to querying all your data in one database

Admittedly, it’s early — I’ve only built out support for Postgres, MySQL, SQL Server, Redshift, and CSV files. But if that’s where your data lives, Joiner makes it easy to bring it all together. If not? I’m working on it, so let me know what you need!

Get started

Sign up to beta test Joiner at www.getjoiner.com if this sounds useful. Or check out github.com/mjirv/JoinDB for open source, command-line, self-hosted fun.

I’m building Joiner to help analysts like me (and probably you if you made it this far), so let me know if you have suggestions or anything you would find useful. You can reach me at michael@getjoiner.com or on Twitter at @mjirv.

--

--

Michael J. Irvine

I do data things. My interests include public policy, ethical philosophy, and cats. @Mjirv