Last summer (2019), the Kapten data team organized its very first hackathon!
Struggle for creativity
There are many reasons as to why we did this hackathon. Many revolve around improving our creativity. I mention creativity, but it is deeply connected with curiosity, and the below sections apply to it as well.
Creativity is vital in data (to be honest, I believe it is crucial in all jobs if you want to excel). Indeed in data, we are facing new problems every day. While some of these can be solved with classical approaches, others require an out-of-the-box mindset to find the best solutions. Sometimes you have to use different statistical tools, to challenge the problem… This is why we hire people that are able to handle the unknown.
Let’s start with some background. Today, the Kapten Data team falls under the tech department and is composed of 3 units (analysts, engineers, and scientists) made up of 18 amazing people. This proximity allows each unit to be aware of the skills, needs, and mindset of the others. This is a breeding ground for ideas as some are looking for purpose when others are looking for means.
However, even if you have the best people (and we do), we can’t expect them to remain being creative all the time by themselves. Your company’s context may force you to take action. At Kapten, like in any other company that grows and conquers new markets rapidly, we have a continuous flow of questioning awaiting to be answered every day. And that is a risk on creativity for two reasons:
- With so many questions, it is difficult for any team to find the time to investigate their hunches and bring new, unexpected knowledge to the company. It can lead to discouragement and the team may stop trying to have new ideas or even worse, stop mentioning them.
- To answer the company questioning, they are always several methods. Most of the time, you can use your company’s usual approaches, but in some instances, it is better to try more exotic ones. However, after a while, it is natural to enter a routine-state ending in always using the same approaches (why? Complex subject, but you can start here). It decreases your team effectiveness as well as motivation.
“Insensibly one begins to twist questions to suit methods, instead of methods to suit questions” — Sherlock Holmes, data scientist @Kapten
Hackathon is a great way to fight this (though not the only one). But to maximize the impact, you need to prepare it.
First you need to optimize your time. Two days is a tight timing, so we had to plan everything in advance: subjects, teams, and the action plan. That’s how we were able to kick off the Hackaton within minutes after our arrival on the first day.
Second and foremost, we needed the right mindset: we want people to think differently than at work and break the routine. So we relocated into an external office for the hackathon, with a nice location and an enjoyable office. To get even more engagement from the team, we organized a nice restaurant at the end of the first day, and grabbed a beer (OK several) at the end of the second. Basically, we lived together for 2 straight days.
No need to mention that we cut all interactions with the rest of the company during those two days.
Creativity in progress — please wait
Kapten data team worked on 4 projects; we will discuss two of them here.
The first one was to gather information from open data sources to unlock new potentials for analysts and scientists. In this one, we gathered among others weather data (rain was the most anticipated information), road work locations and duration, places of interests (theaters, restaurants…).
More importantly we managed to automate the data ingestion from sites like opendatasoft (> 20k data sets) so that ingesting any new database is just a matter of configuration that can be done in minutes by analysts and scientists without any help from engineers. As a matter of fact, we are always pushing for autonomy at Kapten.
As a practical application, we tried to evaluate the number of rides starting from a road work area. In inner Paris, we had almost 2% of our rides impacted in the month before the hackathon. On these rides, we are more often late compared to a regular ride. Also, these rides are 20% more likely to be canceled than if they were booked from a non-disturbed location. They are also associated with a lower rating.
As we can see, there is a pain point there, and we were able to identify it. So the marketing, product, and tech teams will be able to discuss this unpleasantness and what to do about it.
In the second subject, we wanted to have a look at our driver fleet behavior. While before we were deciding on metrics and computing them, we wanted, during this hackathon, to get a visual evolution of the fleet on a map. Indeed, when looking at spatial and temporal data, the best tool to detect patterns, at least for a first-level analysis, is our brains and eyes. So rather than trying to find metrics, which relevance depends on the problem, we wanted to display the positions and their evolution. With it, our business (and us) can detect trends on their own and come to us for a comprehensive analysis if required. We make them more autonomous.
The strong point of this work is that we can generate such animation in a generic way for different metrics and with different granularity and/or filters. It will allow us to answer many questions with a unique tool. If we were to go along with this after the hackathon, we would further simplify/generalize its use.
As for the awesome third and fourth projects, you’ll have to apply for one of our job offers and be hired to see them ;)
Turning creativity into value
Creativity is not enough in itself: while Leonardo da Vinci was one of the most creative minds of our history, many of his discoveries had to be re-discovered centuries later because he did not share them and many of his ideas/inventions/paintings were never completed. (For instance, his idea on how the aorta valve closes when the heart pumps blood was only scientifically confirmed in the 1960's!!)
Once you are excited by some ideas, and worked on them for two days straight to produce a Proof Of Concept (POC), it is only natural to want to see them alive and adding value to your company.
This cannot be done just like this. During the hackathon, in order to deliver results in just a few days, you make compromises on code quality, and methodology, you left out hedge cases… You cannot take your work and put it in production after a few fixes. You almost have to start from scratch to have something production-grade.
You could then say that the value of the hackathon is only to help creativity thrive in your team, but in reality, it is more than this.
First, it allows you to get a grasp of a subject and solve its first issues.
Because you produce a working output, it also makes the rest of your company eager to use what you created. At this point, prioritizing the final version of your POC becomes a child’s play, and this allows you to bring the data one step further toward excellency.
You also show what you are capable of. Your company will realize your potential and will be more willing to let you some time for exotic ideas and new opportunities.
Finally, it is a great way to build cohesion and have fun while working.
A busy calendar and a busy mind will destroy your ability to do great things in this world — Naval Ravikant