From idea to product in 20 hours
I’d like to share the story of building a face-feature detection website, and how a very simple work flow helped me doing everything from idea generation to end product in less then 20 hours and makes me believe that short work hours are the way to go.
About 2 weeks ago, I (once again) came across the need to extract features from faces in images for an analysis I wanted to run. Conceptually, I wanted to input some image somewhere and get a set of variables returned that described the face, which could then be used as input for my own machine learning. While there are plenty of alternatives out there, no current commercial or open-source platform seemed to solve this need without some big road block. I thought I’d be a nice side project to implement using a personal scrum method that co-workers at Studyportals came up with. It should force you to work on the task that delivers most value with the least effort, and keep iterating based on things you find on the way. Seemed interesting, as most of my projects end up unfinished, with a lot of time spend on unimportant details.
After some quick thinking the idea for a MVP was straightforward: an API where users could send their photo’s, which would be analyzed by the platform and made accessible again on the API. The website would have some standard pages (home, login, contact, docs etc.) and a dashboard. I also wanted to include a payment solution as well, as it is something I spent countless hours on with KidUp, but should be much more trivial. After spending some time getting familiar with the options for the feature detection, I decided to go with an custom implementation of OpenFace, which saved the time of building an own Neural Network.
With the game plan known, the side of a cabinet served as scrum board, and I quickly jotted down and organized a number of tasks.
All the tasks are basically a description saying “As a user I want to do xxx”, forcing you to think in terms of value added to your product instead of technicalities.
Finding 1: get post-it’s that don’t curl up. Finding 2: don’t poker the points. While points is standard metric in scrum there are quite a number of counter arguments, and in general I did not found it that useful.
With 4 hours on the counter, I started on the first task: creating a dashboard.
Using Yii2 as my favorite PHP framework progress was quick, and within an hour I set up a quick website where users could login, register, visit all the (empty) pages and see their dashboard. I decided to do it manually instead of taking a prepped user module, as the latter choice almost always stabs you in the back when you want to change a small behavior and you have to go through the whole module anyway, which is usually more time consuming.
After setting up the basic API where users could upload & CRUD images & authenticate, I started looking into the “backend” of how images would be analyzed. I had OpenFace setup in a docker container and used AWS’s ECR as docker repository. While it was fine for a small project like this, it might be worth looking into a more streamlined process of deploying the docker container.
I also used the queuing service SQS from AWS. Each image that is uploaded to the API is placed in the “to-be-processed” queue, which is picked up by an available docker container. That reads the image (from S3), processes it and sends the results into another queue, which is read by the frontend application. This way, the backend is easily scalable, and the processing can take however long it needs. Took me a while to figure our this was a better alternative then some direct HTTP calls between the front- & backend, resulting in finding 3: use independent microservices as much as possible and don’t be afraid to throw some money at it. $3 per month for SQS > 3 hours of fiddling around.
About 13 hours into the project, the bare bones were functional, and I spend a delightful short amount of time to implement Stripe payments into the website. About 4 hours were spent on design (specifically a simple choice of fonts, color palette and some bootstrap tweaks) and the design of static pages and content like home, pricing and the documentation. I also came up with the name “face-features.com”, something that seemed quite right as it directly described the product and was still available.
Reflecting on this part, I still did find myself moving away from the post-its from time to time, effectively putting time into sub optimal features. In my experience, it is usually due to a lack of mindfulness of the high level overview of the project as you get too much dragged into the details. And the counterintuitive thing is that working on predefined tasks is just as pleasurable as “going freestyle”, perhaps even more as it gives you small satisfaction tasks you can tick of once done. So it’s more fun, a lot more productive but quite hard to keep yourself on, leading to finding 4: force yourself to be mindful of what you are doing, be it by sticking the post-it with your current task on your monitor or by setting an alarm every 20 minutes forcing you to do a quick reflection. Finally, take some time to reflect on the current status every now and then. As your experience in the project grows, you get a better feeling for what is important and what is difficult, making it easier to improve your scrum board.
I finished off with some deployment debugging, final tweaks and a result analysis, with just over 19.6 hours spent on this project. Seeing that the result is a full website which performs quite a complex task and serves a real (though very niche) need, I consider it quite a success. Overall it confirms my gut feeling that mindful, quality programming time beats late night, 12-h a day cramming, something that seems to be backed more and more by science and other programmers these days.