My Startup Was Attacked by Rockets
Continuous Deployment Under Fire
I think I hear the sirens. A second ago I was completely in the Zone, adding the last touches to the code based on customer feedback. The air-conditioning is a bit loud in our office, the sirens not very loud.
“Sirens”? I ask standing up from my laptop.
“Yep” Noam says. “Let’s go”.
There are rockets from Gaza heading in our direction and the Iron Dome radar and control software extrapolated their course to our location, hence the localized sirens. Gaza is 46.4 KMs from us (there is an app for that). We have 90 seconds until they hit.
In Eclipse I click on Mandatory Tests and we all leave the office.
Our startup office has many windows, so we head towards the heart of the building which happens to be the stairwell. Many other people are already there, others joining every second.
Some kids that came to work with their parents (summer school vacation) are here as well and it is getting a bit crowded.
“Running Mandatories” I said to Noam.
“Good Luck” he said.
“Not Really. Not done coding”. I’m sure I’m going to fail several integration tests for which I haven’t finished the code. But when we are back I’ll have a better understanding of how far I am from ending the task by seeing how many tests fail. I’m a fanatic of Test Driven Development, writing tests first and coding later. I finished coding the tests a couple of hours ago and for some reason I find it comforting that while we are wasting time outside the office my laptop is running the complete test suite we lovingly call “Mandatories”.
We hear a distant Boom. Then another one. People stop talking. The building shakes a tiny bit. Not an Earthquake shake, more like a super sonic jet flew by and rattled the windows, except there are no windows around us, only a concrete office building stairway. The sirens stop wailing. 30 seconds pass.
“That’s it” someone says. A kid is crying.
“Let’s go back to the office” I say to Noam and we head out.
We notice a large group of people standing outside the main entrance and curiosity grabs hold. Walking outside we see about 10 people with their smartphones taking pictures of the sky (this is Israel and everyone has a high-end smartphone, for some value of everyone and some other value of high-end). There are 2 Interception Mushrooms over our heads.
“Cool”. We walk back to the office.
I look at my laptop monitor. “23 failures”, I say. Damn.
My name is Tal Weiss and I’m the CTO and Co-Founder of Evature. This is my first blog post. I’m not even sure why I’m writing it. I guess I’m tired of reading. There were 84 rocket attacks on Israel today.
Some very cool animation of the attacks, overlaid on a map of Paris.
…and the Intertubes are ablaze. So many words. So much hate. So much ignorance. Almost no perspective. So little hope.
What do I aim to accomplish here? Tell my story, add a little perspective. Give back some hope. I have to.
Not an anonymous coward
Born in Israel. 43 years old. Married + 3 boys.
I spent 6 years in one of the greatest tech units in the army building cool stuff. I met my wife in the army. She was also an officer. I met Barry in the army, my Evature co-founder.
I live in Nes Ziona, a jewish village founded in 1883 and proud of having been the place where the first modern Israeli flag was waved, 113 years ago. My wife’s grandmother’s grandmother is buried in an old cemetery not 5 KM from our house.
I come from a family of survivors. Well, my parents were both born after The War and lived in Israel their whole lives, but my grandparents…
GP1 had a wife and kids that didn’t make it. He served in the Czechoslovakian army, escaped from the Germans to the Soviet Union, joined the army and spent a few fun winters fighting the Germans. GM1 was in Auschwitz and lived to tell. She is 98.
GP2 and GM2 spent the war hiding underground in a hole in the ground dug in a friend’s remote farmhouse. Their family saved ours and we are eternally grateful to them.
I am not, nor can I be, objective.
I’m writing this as I’m running my tests again. 20 failed tests to go.
Evature has a vision: We will replace travel agents.
A vision needs to be very easy to understand. Can my 10 year old son comprehend it? Can my parents? The vision is based on our prescient view of the future, a future in which technology is just “there” all the time and you can interact with it just like talking with a fellow human.
“What time is my flight?”
“I need the Hilton Garden Inn in Sioux Falls”
“Call my wife and tell her I’m going to be 30 minutes late for dinner”
We are not all the way there yet, but we have a working product, an incredible team and paying customers, 3 of them are billion dollar companies. The future is looking great.
Sadly, reality is very complicated and executing on that vision has not been perfected. Yet.
There are many excuses. As with most of the middle east, the semi-arbitrary lines drawn on a map at the end of world war 2 failed to address the complexities of geography, religion, language and culture. Suffering ensued.
But I’m an optimist.
As long as we manage to hold on to and execute towards the vision.
My tests all pass and I Commit my code to source control and Push it. This is a very rough sketch of what happens next (technical mumbo jumbo, feel free to skip):
Github notifies Amazon Web Services (aka the Cloud). New EC2 servers are spawned, installed with my code, and tested. Going beyond our regular Integration tests, a random test set of about 100,000 past logs is reevaluated, generating performance statistics, gathering metrics and scanning for crashes. We have zero tolerance for crashes and we fail the release if memory leaks are found or if it crosses performance thresholds. Optimizing for cost and overall responsiveness, the tests are all completed before an hour is over and I get a detailed email report with all the metrics.
Since I passed the tests EC2 images are automatically generated.
Clicking another button (a second click! The horror!) worker EC2 instances are spawned with the newly created image. They are sanity tested and added to the elastic load balancers, while the old servers are removed. The entire system is tested across 2 regions and 4 availability zones.
I’ve heard continuous deployment equated to fixing an airplane mid flight.
That’s exactly what it is like. The entire process takes 5 seamless minutes.
We do this about 5 times per day and practice makes perfect. Very cool.
Beka just Skyped me. He received a “Tsav 8", basically a no-warning recruitment for reserve duty in the army. Beka is our lead algorithms developer. I hope he is alright.
Most of the Israeli army is built on a big reserve. It allows people to live their lives and only calls upon them (usually with ample warning) when they are needed. This goes on until they are about 45. Not everyone likes doing reserve duty — in fact many people try to avoid it. These are not kids anymore. They are doctors and lawyers, students, teachers and nurses, putting on a uniform, leaving their families, risking their lives for weeks every year.
But now things are different. They share their thoughts in infinite streams of social media posts, Facebook Twitter and WhatsApp being the most popular. As the rockets continue they are being called upon to protect their home and they are eager and willing. And they are showered with love and respect from their families, friends and fellow Israelis.
Many Israeli soldiers have died in this war. Some young. Some old. Some with families. Some without. Just married. Just graduated from school. Israel is so small that everyone knows a soldier who died or was wounded. My school WhatsApp group shared a photo of a fallen soldier. I will not repost it here — too painful. It is a happy family photo of him with his wife and 2 beautiful children, a boy and a girl. And everyone is smiling.
It is 7:58am, 2 minutes before the 6th (?) ceasefire is scheduled to begin, and the sirens start their horrifying wailing. You would not believe how many everyday things sound like a rocket siren. Songs on the radio. Motorcycles accelerating. Sometimes we hear distant sirens from neighboring towns, but this time the volume is too loud and there is no question that they are heading towards my house. I was just about to leave home taking my 10 year old son to tennis camp, but we drop everything, I pick up the baby and we all head toward the safe-room. Every house in Israel has one, built like a mini-bunker, usually used as a normal room. The boys cover their ears. We just manage to shut the steel window shade when we hear the boom. It is the loudest we have heard in a while, and we have heard plenty. Several house / car alarms start wailing, but other than that there is a tensed silence. 3 or 4 minutes pass and I say “It’s over”.
I grab my laptop, say goodbye and take my son to tennis camp, continuing to the office.
On the radio I hear a man crying. His daughter’s room has been hit while she was luckily away. He is one of 250,000 Israelis living in the villages next to Gaza — no Iron Dome, no early warning system. His life has been living hell for the past 15 years as mortar fire rained on them non-stop disrupting their life. And it has gotten worst since the Hamas takeover of Gaza in 2007.
He describes teenage kids shitting their pants, whole families in psychological therapy, businesses lost, families destroyed.
And no hope. He does not believe our government, he does not believe the Hamas; He does not believe in peace nor war. He has been through this exact cycle before, again and again and again.
Do I believe in a better future?
I do. I said before that I’m an optimist. I would not be an entrepreneur if I wasn’t. But we need leaders who are visionaries as well.
Most Israelis I know viewed the disengagement from Gaza as an experiment. An Alpha program for the Palestinian state. Israel left Gaza, evacuating all the settlements and the military outposts. The Gaza border with Egypt has been completely left to Egyptian control. The Israeli border has been, hmmm, well it is complicated — Israel still supplies Gaza with electricity, water, fuel and communication infrastructure but very few people pass through, some humanitarian supplies, and Israel tries to stop weapons from pouring in (unsuccessfully, obviously).
I have no idea how the Palestinians in Gaza feel. They do not have free press and speaking against the Hamas can cost you your life. They might all be pacifist hippies for all I know, but the fact is that Gaza did not become Singapore . But it can become a paradise. It really is possible.
Imagine international industrial parks with access to the Mediterranean and to Europe. Image the countries of the world really helping the Palestinians, building infrastructure, establishing free trade zones, improving education, health, employment, giving the poor people of Gaza hope. Rebuilding Gaza with a future unclouded by war.
Will happy Palestinians in Gaza stop firing rockets at Israel? I have no idea, but I’m pretty sure the miserable souls living there now will not.
Will Israel support promote and support peace? I think so. We already have peace with our 2 most important neighbors, Jordan and Egypt. As an Israeli citizen I have toured the pyramids and visited Petra with my family and in both countries I have been welcomed and embraced by the locals. Many years will pass before the hatred subsides, but we owe it to our children to try.
There are no easy solutions. Super complex problems (challenges?) stand in our way. But just like a startup tackling huge technical obstacles, you need to hold on to your vision and start executing on it, one baby step at a time.
James Altucher (I’m a fan) says he never hits Publish unless he is scared. I’m a bit scared. Evature is in the middle of a financing round. Will this post hurt us? Can people comprehend our vision beyond the sensational news they see on their TV sets? Will our homepage be Slashdotted? (what if it does, it’s just a homepage). I don’t know.
Barry and I have bootstrapped Evature for years while we were building the first prototype, so the company is already Agile, lean and mean. Evature will be ok. I’ll be ok.