A lot of people use Google Analytics to track their website metrics. A lot people use Google Analytics because they don’t know of any alternatives. I was in that camp as well.
Out of pure frustration from how slow and over-complicated Google Analytics was for my simple use case of tracking pageviews and visitors, I started looking for other solutions. There are actually quite a few out there, such as Fathom, Plausible, Matomo and Simple Analytics. The problem though is that these are all paid solutions. Some are open-source as well, but they either had too many features, not enough features, or used a technology stack I wasn’t familiar with.
So I decided to write my own application, Umami, to provide the exact features I wanted. The end goal was to have a product that was simple, fast, and open-sourced. Plus, it was a fun opportunity to learn a few new things.
Starting with the data model
When starting any side-project, it’s often difficult to know where to start and where to go to next. In this case it was rather simple because as an analytics product, you have to have the data first.
I started building out the data model for the database tables in PostrgreSQL. Although familiar with databases, I had never used PostrgreSQL before so it was a fun learning experience. Trying to stick with the goal of being a simple product, I ended up with only 5 tables, account, website, session, pageview, and event.
account — Used to store logins to the application
website — Used to store individual website statistics
session — Information about individual visitors
pageview — Page URLs visited on a website
event — Events on a page such as clicks on buttons and elements
Collecting the data
In order to collect data for a website, you need to have a tracking script, a block of Javascript code that gathers metrics and sends them to your server to store in the database. A tracking script is usually embedded in the <head> section of a website and looks like this:
<script async defer data-website-id="83dd64c4-17ad-4e3e-a905-b3d8c867c1a5" src="http://app.umami.is/umami.js"></script>When creating the tracking script, I didn’t have anywhere to send the data yet. It started out with a bunch of console.log statements to make sure I was getting the data I needed. I would embed my in development tracking script on a couple of my sites, then click around to make sure everything was working correctly.
Storing the data
Now with the database up and running and a tracking script collecting metrics, I needed to get the data into the database.
I had used Next.js for a couple websites already, but only for static websites and never for a stand-alone application. Next.js has a great feature that allow you to create API endpoints in the same codebase that is hosting your application. So Next.js seem like the perfect choice for Umami.
The first thing I did was create an API endpoint called /api/collect that I would send my metrics to. For connecting to the database, I used Prisma.io, which is a great database toolkit that supports PostrgreSQL. I had never use Prisma.io before but it was simple to get set up and learn. It also allows you to write simple queries in Javascript like:
async function getWebsiteById(id) {
return prisma.website.findOne({
where: {
website_id: id,
},
});
}With the API endpoint created, I had my tracking script start sending POST requests with the gathered metrics and data started appearing in the database.
Showing the data
At this point, there was still no UI to show any data. I knew that I wanted to show a graph with basic metrics, pageviews and visitors. I researched a couple Javascript charting options like Recharts and D3.js, but landed on Chart.js. It has great documentation and lots of customization options.
I created a new page in Next.js and a new React component to display a Chart.js chart and had it pull data from the database.
One feature I found really lacking from Google Analytics was the ability to see a comprehensive view of all my websites and be able to quickly toggle the metrics individually. So I created a dashboard view for the entry point to Umami.
From the dashboard, you could then dive in a specific website to view more detailed metrics.
All the metrics fit on a single page. There are no complex menus to navigate and the screen updates dynamically as you click on different metrics. Great! Now I can say goodbye to Google Analytics forever.
So now we had simple and fast covered. Time to make it a real product for people to use and open-source it.
Making a complete product
So far, this was a great side-project. I had a product that had the features I needed, showed the metrics I wanted, and I learned a lot of things along the way. But to be an open-source project, it needed to be usable by others.
Often times when writing software, the last 10% of the project is the hardest and this was definitely the case. Even though I could successfully collect and display metrics, the application need to be usable and friendly.
The first step was get authentication working in Next.js so the user could log in and manage their websites. Using the Next.js built-in API functionality made things pretty easy. The front-end was simply React components that would call the API back-end for verification.
Once the user could log in, they needed to manage their websites with a simple admin interface.
This involved creating a lot of new React components, using Formik for managing forms, validation using JSON Web Tokens, and making sure all endpoints were secure.
Additionally, since Prisma.io was able to run MySQL in addition to PostgreSQL, I wanted to add support for MySQL as well. This required figuring out the equivalent functionality between the two databases and writing database specific queries for each.
Ship it!
From initial commit to first release, Umami took 30 days to complete. I initially posted it to the /r/reactjs group on Reddit and someone else posted on Hackernews and Producthunt. The reaction has been amazing! More than I ever expected for a simple side-project. This was the reason I got into open-source, to create great products for everyone to use for free and to learn new things along the way.
If you’re interested in contributing to Umami, the source code is on Github.
