Don’t be a dev for a day and automate everything — an inventid case study

In every company, there will always be some kind of waste: Time spend badly. In general this is true no matter your size, there will always be something.

Obviously we are no different.

However I learned that solving waste can be easy. Identifying it is not.

Let me give you a picture

inventid is a completely distributed company. No office but Hipchat. Beers once in awhile. One Github to rule them all. Everyday sending an email beats a phone call. No managers and hardly meetings.

This is the perfect setup to let waste (or inefficiencies) pile up. And it was.

But we simply didn’t know it

First of all it turns out people are bad at identifying waste in their own work (including myself!). You do your job as you were once told, and keep doing that. If you are unfamiliar with other approaches, you are simply less likely to realize how something can be improved.

The first realization: support

When one of our supporters fell ill, I had to step in for a few days. Handling some tickets, I suddenly realized how long a simple action could take (what was the email address for this order, and was the email accepted by the remote server?).

As a developer I quickly became annoyed (yes I am severely impatient). I know this data was just a few joins away in the production database, but it wasn’t there in my UI. And it most definitely wasn’t there for other supporters!

So I quickly hacked a “follow the order” UI in the support admin interface. Based on an order number it would give you directly the associated account data, all customer messages sent, payment data, the delivery status and opening times of the messages, ticket downloads, and ticket usages. Every piece of information was available somewhere, but simply not in the location it was actually needed. And while at it: I knew the originating email address from the support request, so let’s fetch the data for all orders with that account. That way that is available upfront.

Work required: 4 hours. After making a quick screencast for the others I got back to my, now less annoying, job of helping our users. Quite happy since I thought I at least helped myself for the remainder of the afternoon.

Fast forward to next day numbers: our support time from an issue opened to the first fulfillment response (the response which should solve the problem) had gone down from 16 minutes to three. That is not a typo!

3 MINUTES!!!11!!

That is over 80% improvement! For a 4 hour investment!

Turns out I was even way above average in my response times. Bye bye ego!

But why?

Simply stated: support staff was never trained there was another way we could build stuff. Devs found the existing solution good enough. And the PM roadmap slide deck had no support admin in it.

Kidding, we have no PMs ;)

After a week I stopped thinking about the insight about building that UI. Some high concurrency locking issue needed my focus or something. Or some buttons needed re-alignment. Dunno

The second encounter: billing

As a ticket provider we have to handle an awful lot of billing (kind of the business so to say). No surprise some people are working on that.

However the view devs and accountants have on billing turns out to be really different. We’d send lots of CSVs to that team (yes we did that automatically) and dev-wise we were done.

However it turned out that was just the first 4% of the work. Everything needed to be invoiced, invoices needed to be sent, payments sent to the bank(s), payments needed to be collected etc. And after that there was the official accounting stuff!

I think just a few developers know how massive this billing exercise can be.

So when a developer happened to sit with somebody from the billing team, some eyes were opened.

First of all, we have all the data within some system (we sell something for some price after all). Secondly, accounting for the base case is an algorithm (ledgers, VAT, and whatnot). Thirdly it hurt his eyes to see CSV data was manually filtered, typed into a Google Doc, exported as PDF, attached to an email, and sent out.

Cue the music! Our invoice helper system started at that exact point.

Since it was a bit much to do everything in one go we split this one up (we do have these big projects sometimes). First was the automated creation of hundreds of PDF invoices at each billing run.

Instead of sending CSVs, we send billing a CSV and all precomputed invoices in PDF format, ready to be forwarded. We actually increased the workload for a while (two systems had to be in sync). Of course we got some bugs in the automation, or our formatting of invoices (so they were not legally valid). In those cases manual work took over and bugs were filed to the dev team.

Few weeks later we managed to get everything 100% correct for 25 working days straight. Victory on step 1! Time for number two. We simply dropped the CSV (and put the result table in the email for quick reference), and started emailing the invoices to customers directly. That was a huge time saver for the Invoice Squad.

Step 3 was to format nice files for the bank, allowing instant payouts.

The final step was the accounting software. Team Money still had to enter these invoices manually. And due to the lack of a comprehensible API we did not get around it. However their UI was great!

Fast forward some months (and multiple failed API attempts), and a non-negligible amount of beers we found the Holy Grail. We’d use the UI!

Instead of using the PDF data we could let a machine read the JSON for each billing run (we had saved the JSON for months, since it allowed us to figure out what was invoiced when, for how much etc. Think of it as a really cheap offline data lake).

One Windows VPS later, armed with RDP and a Java TypeRobot, we had a system which would login, type all invoices like a Money Guru would, check if everything summed to € 0.00, and click “Submit”. No human effort required!

What we learned

People don’t know their own limits, the limits of software or what is even possible. So we made a new rule:

Every developer takes one other job for one day a month.

The results were astonishing. Devs are really impatient, so in no time at all much more was automated. Stuff we would never think about to automate ourselves (as devs) or what we thought was good enough. Stuff others thought could not be automated. And in the meantime people became happier and we were able to grow significantly. No human effort required!

Among others, we automated the following:

1. Sales trajectories
2. Quick quotations
3. Hardware testing
4. Returns and refunds
5. Shipping!
6. Reseller setups
7. …

Take away

Adopt this rule in your organization. Also automate the small things. It will be worth it.

Every developer takes one other job for one day a month.

It costs a little but it brings you a mile. And a smile!