9 Things I Learned as a Software Engineer
…that I wish I had known when I started grad school
Three years ago I was working in a neuroscience lab in Barcelona, busy putting electrodes on people and teaching classes on cognitive systems. Today I design and write software for a living.
Of course back in science I wrote a lot of software — if you want to make any sense of 40 GB of brain scan data you’ll have to roll up your sleeves and write scripts to crunch those numbers, and I was always a good programmer. But it wasn’t until I quit my job (and possibly my future) in academia and started working for a small and ambitious start-up that I understood what being a software engineer — and more importantly, being in the business of software engineering — is really about. It’s not knowing more programming languages, libraries, algorithms, and design patterns. It’s a mindset.
That mindset would have made my work a lot easier if only I had known and adopted it before I started grad school. This is a note to my younger self, a list of things I learned, sometimes painfully, in the past three years.
1. Intelligence is overrated
When you’re young, being smart gets you a long way. You’re a big fish in a small pond. Doubly so if you have a knack for expressing yourself half-way eloquently. In fact, being intelligent and a smooth talker will get anyone through high-school and most of college without learning much at all (you’ll have to study for physics though. Can’t just talk an equation away.) — congratulations, you’re lucky. And also, very unlucky. Because while you were effortlessly rushing through school, picking things up as you were going, others had to learn what would be much more important later on: Diligence. Persistence. Networking. And probably some of the eight things further down the list.
Our society values intelligence beyond proportion. When I tell people that I used to work in neuroscience, the first response if often: “Wow, you must be super smart”. I’m not dumb, but I know a lot of people who are probably less intelligent than I am, but far better neuroscientists.
Intelligence is certainly still a door-opener. But it will never get the job done on its own. Diligence, rigour, a reliable network, and finally not being a dick are essential qualities of not just software engineering but any profession that’s outside the little bubble called grad school.
2. Take pride in your craft
That mantra may be over-used, but it still holds an important lesson for you, dear younger self: whatever you do, consider it to be an honourable craft. Nothing should ever be just a means to an end. We all love seeing our names on publications, but the actual craft is to come up with and invalidate dozens of hypotheses, to work with your subjects — whether human or floating in a test tube — and tend to their needs, to rigorously analyse your data and validate your statistics, and to start over again because at some point you will notice an embarrassingly stupid mistake you made earlier on. If you write software, that means planning your features, researching existing open source code, learning new paradigms and programming languages, fixing your bugs, refactoring code and maintaining it. If you take no pleasure in these steps and just consider them to be what has to get done in order to publish your paper or ship your product, then you will never become truly good at it. If you have no ambition to become truly good at your craft, then maybe being a scientist, or an engineer, or whatever you’re doing right now is a waste of your time.
A good sign that you’re honouring your craft is that you’re taking on pet projects: silly little projects that don’t necessarily serve any immediate need that you’re doing just for the project’s sake. Because you enjoy working on them. Interestingly, these seem to be fairly common in the software community — many products we use every day started as someone’s pet projects — but are much more rare in scientific circles. One of my favourite quotes comes from Konrad Lorenz:
“It is a good morning exercise for a research scientist to discard a pet hypothesis every day before breakfast.”
If that sounds stupid to you, maybe you shouldn’t be a research scientist.
3. Learn new tools
As a continuation of the last point: devote time to learning new tools. Not just to expanding your abstract knowledge, but to actually learn tools that will help you get things done. It will pay off soon enough.
A great way of learning new tools is with “pet projects” mentioned above. Every time you build something new, also build it in a new way. Remember, pet projects are about failing. You don’t invest much, you learn a bit, and if it doesn’t take off or you lose interest or you realise that the challenge was a little too much: no harm done. No ego hurt.
Great tools that I highly recommend learning if you’re in academia:
- Git and Github. Git helps you to manage your work and never worry about backups again, and there’s a ton of great code on Github so you don’t have to reinvent the wheel over and over again. Oh, and please do code reviews with your peers. Don’t ever use code to analyse data that no-one has read but you (I can’t even believe I even have to tell you this, younger self. You’ve always been a good programmer, but the mistakes I still make that would go unnoticed if it wasn’t for code reviews makes me believe that 30% of all results in science are probably bogus because of bugs).
- An illustration software: I personally prefer Inkscape, but the industry standard Adobe Illustrator or newcomer Sketch are just as good. Use it to post-process your plots and graphs; it’s often much easier than writing plotting directives in Matlab or matplotlib.
- Learn how to use your text and code editor efficiently. Sublime Text is a great editor with a much lower learning curve than Vim or Emacs. Learn the shortcuts. It will save you an enormous amount of time.
- Learn how to speak. Watch TED talks and pay attention how the more seasoned speakers can engage the audience for 15 minutes while telling a compelling story. Practice in front of a mirror. Your body and voice are tools, too.
- Knowing the fundamentals of Python, R, HTML and Javascript will get you a long way. If you’re no stranger to programming already, learn a new aspect or library. Play with computer vision. Natural language processing. Web scraping. Music synthesis. Robots!
The solutions you can see to a problem will always be limited by the tools you know. Learning new tools means looking at problems from other angles.
If you’re in college, I strongly suggest you set apart one day every week only for learning new tools. When you start doing your own research as a PhD: make that two days a week. You will save a lot of time in the long run and people will be astounded by your efficiency. If that sounds like a lot and you think that you don’t have the time and there’s too much pressure to get other things done, talk to your older peers and ask them for advice on what to really spend your time on.
4. Be a stake-holder and make your agenda known.
It’s a common assumption that your supervisor or CEO will always act in the institute’s or company’s best interest; that’s her job.
But, neither a company nor a lab is a conscious entity, and as such has no intrinsic interest. When we speak about a company’s best interest, we actually mean the best interest of the stakeholders. The real question now is: who does your CEO or supervisor think those stakeholders are, and how important are their interests?
If your boss thinks she or he is the only stakeholder (get as many publications as possible; aim for a quick and profitable exit): get out as fast as you can. You will be thrown under the bus. Who else? Your investors or grant givers? The employees? Students? Humanity? The point is: find out as soon as possible. If you’re not seen as a stakeholder, get out. As much as you may love your work, it’ll be a one-sided, abusive relationship.
5. Shipping it
“Shipping it” has become a very fashionable term in tech. It means getting your product out of your warehouse and to the consumer. But more than an action, it is a mentality. It means that your work is worthless until it ends up in the hands of the consumer, and that this should always be your main goal.
In academia, most software I wrote had to work exactly once on exactly one system. Writing production-ready code that will work for half a million users is a completely different kind of animal, and when I started to write code professionally my work often fell short of that.
But it also means that there’s no point in iterating for years until you have the perfect product: make a small point, and get it across. Write the simplest paper you can possibly get accepted. Worry about making a more complex study later. Get the basics right quickly, and get them out there as soon as possible. Just ship it.
6. Know the 80/20 rule
The 80/20 rule basically states that it will take you 20% of a project’s time to achieve 80% of the desired effect, and then the remaining 80% of time to just get the last 20% right. It’s like driving from the suburbs to the city: in 20% of the time, you’ll cover 80% of the distance. But once you hit urban traffic, the last 20% of distance will take you much longer.
Why is that important to know? Because people constantly underestimate the time a project needs. Scientists and engineers are particularly prone to that. That’s partly a matter of experience: the more you know, the better you can anticipate what will go wrong later and what the funny edge cases will be that nobody thought of when you started out.
If you don’t have that experience yet, just multiply the time you need for a project by 5, and expect to be “almost there” after a fifth of your estimated time.
7. You didn’t sell your soul.
I started my PhD for all the wrong reasons. One of them is what I now call “academic guilt”. I believed that if I didn’t pursue a PhD programme I would be wasting my talent. And I felt that I owed it to the people who went out of their way to support my academic career — professors and the people who paid for my scholarship — to do research. I didn’t. They may have invested in my academic future and may be disappointed that their investment didn’t pay off and produce a great scientist, but that’s their problem, not mine.
The same holds for any other job. People will always invest in you and it’s often in their best interest to do so. But that doesn’t mean they own your soul.
8. Leave your comfort zone.
Here’s how I look at the world:
There’s your comfort zone. You know every fish in that pond. You belong. You know how to deal with problems. Nothing new under the sun. If you want to learn something new and grow, you’ll have to leave your comfort zone. This is where learning starts. That’s where interesting things happen. That’s where you don’t immediately have a response to everything.
Of course, there’s also the point where you’re just overwhelmed. That’s the panic zone. That’s where you’ll black out. That’s where all you can do is to try to keep your head outside the water hope somebody will save you.
The sweet spot is right before your panic zone. That’s where the challenges are where you’ll learn the most, grow the most, and change the most. Go there.
“Forget safety.
Live where you fear to live.
Destroy your reputation.
Be notorious.”- Rumi
9. Tame your monkey mind
Sit comfortably, close your eyes and just continue breathing normally. Focus on how the air you exhale through your nostrils feels on your skin above your upper lip. Nothing else. Just focus on that.
How long before your mind wanders off? Five minutes? Probably not. A minute? Good. Twenty seconds or less? Congratulations, you’re normal. Your mind is like a monkey and it will grasp whichever branch is closest. I would probably phrase that slightly differently in academic settings… the buzzword is associative thinking. Associative thinking is great if you want to do something creative, but it’s the killer of focus. Good news is: you can learn how to focus. There are a bajillion “productivity techniques” out there, but all of them just scratch on the surface. You don’t want pasta timers and distraction free writing software. You want to tame your monkey mind once and for all.
What works for me might be grossly different than what works for you. I get great results with meditating regularly (which has a number of other beneficial side effects), but even then there are so many different styles and traditions, and I can’t possibly recommend one that suits everybody. What I do recommend is keeping your mind in great shape, and taking that seriously. Think meditation is a waste of time? You go to the gym to pump up on your body. You should spent at least twice as much time on mental work-outs: Lose a few pounds of distracting thoughts. Improve your mental eye-sight. Strengthen your back to be able to keep your mind upright longer.
Thanks to Benedict, Judith, Gabriel, and Peter for feedback and discussion.
