Work in the kitchen.
TLDR: Use your product, and talk to your users, as a developer there is no better way to work on the things that matter. Almost every other path leads to institutional imperative¹ and deepest technical debt that is almost impossible to pay.
Hi,
I joined REKKI 8–9 months ago, REKKI is an ordering app that looks like a chat app, that chefs use to order from their suppliers (kind of like inverse Deliveroo). “WhatsApp for chefs” is how Ronen described it when we met (best fucking CEO I have ever seen, it is pointless to talk here about his vision and etc., but if you get a chance to meet him, don’t miss it).
My mom has a small grocery shop in Sofia, and I know how much pain it is for her to order from her suppliers, and also I believe that the whole producer>supplier->restaurant->consumer market is completely non transparent and everything can be improved.
So as I said, I joined 8–9 months ago, and then I really did not understand what is the actual difference between WhatsApp and REKKI, so I asked Ronen if he can set me up to work in a restaurant for 2 weeks. “2 weeks? are you crazy, who will write code?” he said, so he gave me 2 days.
My first 10 hour shift was few days after that.
My expectations were fairly low, besides me being in complete panic and outside of my comfort zone, I thought I would wash the dishes and observe the dynamics.
I wanted to not be a “minus” to the staff, so I pre-trained myself how to use the industrial dishwashers watching youtube videos. BTW those things have the best UX ever, the colours they use to convey information and the way everything works is just incredible.
PANIC! The day has arrived! I calmed myself down by watching more dish-washing videos from the metro on my way to Restaurant X. I was there at 14:00 (as agreed).
The restaurant is fairly big, and the kitchen is in the middle, so everyone can see you cooking, and the dishwasher was in the back in smaller kitchen. You can’t imagine my smile when it was one of the dishwashers I have pre-trained for.
The staff was very welcoming, and when I told them I trained for the dishwasher they were happy for me to wash the dishes while they do some veg-prep (that’s kitchen slang for “vegetable preparation” like chopping potatoes). Shortly after that they gave me a knife and some carrots to peel, I am fairly good with the knife (I do leatherworking² and woodworking as a hobby), but in no way I am comparable with a chef, for them knifes are more like body parts than anything else, still I didn’t do bad, and when they saw I didn’t draw any blood they gave me more and more veg-prep work.
The restaurant opens at 18:00, and the guests arrive around 18:30, we ate around 18:00 and then the guests started coming, I was in the kitchen (wearing apron and everything, also I have just as many tattoos as the chefs, so on the outside there was no question if I am a chef or not).
Because of my veg-prep and dish-washing performance (I suspect) they
gave me an actual dish to cook: mushroom ravioli.
I think it involved these things:
- long mushroom stems (the ones I cut earlier)
- some other mushrooms
- truffle butter
- chicken broth (one of the tricks I learned, everything tastes better if you use chicken broth instead of water)
- some sauce I forgot how it is called
- stirring until mushrooms get golden
- some olive oil in the end
BTW, you can also check out my soup recipe³ if you are interested in my cooking abilities.
Turns out all people order food at the same time, I am telling you I have been dealing with distributed system consensus for the last 6–7 years, and I never thought unorganized entities can act in this way without behavior emerging from a simple rule (like ducks following), its CHAOS, I have to do so much mushroom ravioli.. PANIC…You know how all chefs stir the pan by tossing the food in the air? Well I have to use a spoon to stir; also all the guests can see me using a spoon.. Anyway I did ok, nobody returned their food! At 22:00 it is the first time I can step outside for a breath of fresh air for 5 minutes.
Then things get easier, from 22:00 to 00:00 guests only drink, and we started cleaning the kitchen around 23:30, my last metro leaves at 00:15, so if I don’t leave at 00:00 I won’t catch it, and of course there are guests that won’t leave, despite restaurant closing.
My legs hurt, from 14:00 to 00:00 nonstop, bloody hell this is some brutal work. I couldn’t take notes, or do anything, just chop chop chop, stir stir stir.
BTW If the people of Restaurant X happen to read this post, thank you so much for letting me cook! You rock!
Few weeks after that I got my second shift, in a small Michelin star restaurant. Things went in quite a similar way, but I was more confident, I did some dishes, a lot of veg-prep, which they thought was boring, but I kind of liked it, there aren’t that many times in my life when I know exactly what to do.
Again from 14:00 to 00:00, and again people did not leave when restaurant is closing, fuck people just leave! Chefs are tired!
I will do a small divergence here to tell you about the tech stack we had when I joined:Elixir was chosen, because WhatsApp uses Erlang, and we need presence and “who is typing” feature as any serious chat app.In order to build a chat app, elixir makes a lot of sense, hot code reloading can keep the sockets alive during deploys, you have native presence with phoenix channels, etc. also super easy to build channels and joining and leaving and everything.
So I came back to work, and let me tell you REKKI is not a WhatsApp for chefs app. It is an ordering app that looks like a chat, but there is no way that two chefs are in the app at the same time, if one is ordering from suppliers, the others are cooking, or veg-prepping or anything but not ordering. Oh and I forgot to mention: the internet in the kitchen is total shit, constantly switching wifi and 3G, and in some parts you have neither.
Phoenix channels (abstraction on top of WebSocket) are full duplex, but serial, which means you basically can’t block on your endpoints because no new requests can come through the channel, so the payload was sent to RabbitMQ and then something else executed it. This lead to incredibly interconnected and complex (especially with retries) system in order to write a record in a database, but it was a requirement if we want “someone is typing”, except we don’t! And guess how good WebSockets perform with the kitchen’s internet.
Heartbeats, reconnects, flushing queues, pending actions etc. etc., all had to be taken care of. This is true technical debt, you can only solve by doubling down on the problem you already have, and of course, you make it work, and you improve it, but you can only make incremental improvements.
You see, if you work in the kitchen, you see what your technology is doing, and what your users do.
Now we are moving from Elixir to Go, we moved from channels to endpoints, pulled up the queue actions in the endpoints so the chef gets immediate feedback if there is any kind of error with their request. I have to say I love the app now, how snappy and clean it is. We moved away from Elixir because we can’t get devs, also everyone can read and write Go (it has like ten reserved words), and also empowered frontend devs to become full stack devs and make their endpoints (also it’s super annoying to write business logic in a functional language).
There is no better way to code than to work in the kitchen.
Fri 7 Feb 19:38:07 CET 2020
Borislav Nikolov (https://github.com/jackdoe)References:¹ https://www.berkshirehathaway.com/letters/1989.html
BERKSHIRE HATHAWAY 1989 shareholders letter² https://txt.black/~jack/jar-to-mug.txt³ https://txt.black/~jack/soup.txt---Actual post is hosted at:
https://eng.rekki.com/kitchen/kitchen.txtThis medium post was copy pasted from the original and some images were embedded, you can see the rest of pictures at https://eng.rekki.com/kitchen/pictures/REKKI's engineering blog:
https://eng.rekki.com
