How I built Defects AI a machine learning service to predict GitHub labels.

Sascha Heyer
Oct 6, 2018 · 4 min read

Defects AI is a fully automated service to predict labels for GitHub issues, using machine learning models that are trained in particular to that use case. Don’t waste your time on labeling issues, use it to solve them instead.

Update: I took the service down, it will now rest in peace as my first unsuccessfull side project. I decided to spend my time on my main project

Automatically labeled issue

How did everything start?

Defects AI began as a side project for a statistics based tool that provides insights about product quality. While digging deep into different data-sets which were based on varied GitHub repositories, I notice that many of the issues were not labeled.

After a couple of days and talks I realized why: For many projects, it is difficult to analyze issues in an acceptable amount of time.

That was the turning point, I paused working on the statistics tool and started working on the automated issue labeling for GitHub. Starting with the core feature, the machine learning model to predict the GitHub issue type.

A few first successful tests and further improvements of the machine learning model gave me the confidence to continue with that idea.

How does it work?

The setup is simple, register at Defects AI, install the GitHub Defects AI App and select the repositories which you want to enable for the automatic issue labeling.

Each time when a new issue is created, GitHub sends a request to our server and as a response, we send the predicted issue back to your repository. The predicted label is then set automatically on the issue.

Automatic Issue Labeling in action

How did I manage to build Defects AI while working full-time?

  • Reduced the features to the bare minimum

I spent a bit more than 3 months (95 Days) to build the initial release of Defects AI. To be exact I started on the 15–06–2018. Here is the first commit in the repository:

What’s next?

While collecting feedback from my user I continue working on Insights and improving the models further. Feel free to follow me on Twitter.

Supported currently are the labels: Bug, Feature, Documentation, and Question.

A few words about money

Like everything else also a SaaS costs money, the biggest expenses are the training costs for the models, hosting and collaboration tools. I spend around 12$ for each training, as you can imagine in the beginning I trained a lot.

I plan to publish my financial reports each month.


Defects AI provides also a simple and easy way to experiment with the prediction. Just follow the link and start predicting the labels of issues:

❤ Thank you for having you here. Please feel free to leave any comments if you have questions about building a SaaS.
I appreciate your feedback which helps me to get inspired for my next posts.


Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store