Half-baked life lessons from a twenty-something data engineer
If the title didn’t give it away, allow me to be explicit in the spirit of the Zen of Python. Don’t expect much out of reading this post.
I’d originally planned to write an year-in-review blog like I’d done last year. However, reminiscing on two significant personal improvements of 2018 — becoming better at execution and note-taking — made me change the post’s format from prose to a big-a#@ list. Some of you may view this as a downgrade. Truth be told, it may very well be one. But it is easier to construct and gets the job done. So that’s that.
The list reflects topics that captured my interest in 2018 and were relatively new for me. Hence, don’t expect a point such as ‘learn basics of machine learning’ or ‘improve writing’. You should definitely do both though. The topics covered are health, problem solving, software engineering, data, finance, people, business, science, public speaking, travel and music.
For the technically inclined and data aficionados, there’s a caveat. Although my work and, to an extent, life revolves around working with data, 2018 saw me improving more on the engineering front as compared to data science. That was a conscious decision and reflects in the below list. That being said, understanding the nuances of software and data engineering makes you nothing but better at solving data science problems.
All set? Here we go!
Health
We’re running a marathon, not a sprint. Don’t burn out.
- Eat healthy and track what you eat. At the very least, be mindful. Make sugar and caffeine luxuries instead of dependencies. Drink water whenever you get a craving.
- Exercise or play a sport. Do whatever regimen suits you but do something! When starting out, try the 7-minute workout or something similar. You can easily find an app that may help.
- Practice meditation or at least, try. Headspace is a great app for getting started.
- Running is amazing — give it a shot. Joining an online or local running club can help with initial motivation.
- Pick up an individual sport. Squash, badminton and tennis are good options.
Problem solving
The art of problem solving cuts across domains.
- Take notes and make lists. Whether you’re in a meeting, learning something or reading, note-taking and list-making will help more often than not. Use an app such as Evernote or simply scribble on a notebook. Do whatever works for you.
- Understand problems before thinking of a solution. Break problems down into sub-problems as much as possible.
- Learn to reason using first principles. Strike a balance between concepts and context. Charlie Munger and Elon Musk are two of the most famous exponents of first-principle reasoning. Farnam street has an excellent blog about the same.
Software engineering
There’s much more to creating software than learning frameworks and languages.
- There’s a difference between platforms and products. In the long run, platforms can be much more valuable than a mere product. AWS, Microsoft and Android are great examples. Read this amazing internal note back from 2011 by an ex-Amazon and now Xoogler about Amazon’s success with building the AWS platform.
- Understand DevOps and cloud computing. If possible, work on infrastructure. Read this blog — a tale of cloud, containers and kubernetes — that I’d written about the evolution of cloud computing and DevOps over the last decade and a half.
- Security is paramount. This can’t be stressed enough. Learn about SSL/TLS/HTTPS, authentication and firewall rules.
- Logging, monitoring and alerting are important, in that order. If you’ve bandwidth, set them up for your projects.
- Don’t re-invent the wheel. Just don’t do it! There’s a reason open source and third-party providers exist.
- Learn some shell (bash) scripting. To begin, get familiar with basics such as regex (grep), working with files (ls, less, chmod) and text processing (sed, awk).
- Optimize your workspace and development environment but don’t obsess over them. Learn about your favorite IDE’s features. Keyboard shortcuts can be handy.
- Ambient music or a song on loop can counter visual and auditory distractions.
- Use jargons to simplify conversations, not complicate them. Understand your audience.
Data
In this wave of information age, being able to work with data is one of the most critical skills one can possess.
- Learn about DataOps and why it is poised to have a huge impact. Read ‘From DevOps to DataOps’ by Andy Palmer of Tamr.
- Understand data engineering and key concepts including pipelines, batch workflows and streaming. This blog post is a great introduction to DE.
- Learn about and try out graph databases. Neo4j is a great entry point.
- Research about end-to-end ML workflows and bottlenecks. Understand where ML can be applied and more importantly, where it won’t add value. The paper, Hidden Technical Debts in Machine Learning, provides some great insights from Google’s own lessons operating a global-scale ML infrastructure. Google provides good points in Rules of ML: Best practices of ML Engineerings. I’m yet to read it but have high hopes from Andrew Ng’s Machine Learning Yearning book.
Finance
Don’t go crazy about money but avoid going broke too.
- Understand what wealth is and how it is different from money. Seek ways to create wealth — investments, recurring income. Read this amazing twitter thread — How to get rich (without getting lucky) — by Naval Ravikant. This is harder said than done. I’m still trying to figure this out.
- Learn about money management. If you are into books, ‘Let’s talk money’ by Monika Halan is a beginner-friendly introduction.
- Understand basics of investing, equity, debt, financial instruments. I’ve gone through portions of Varsity — a set of free stock-market lessons by Zerodha — and liked what I saw. If they’re too thorough, basics are just a Google search away.
- Start investing. Even small amounts contribute to a better financial future.
People
Unless you’re a once-in-a-generation genius, you’ll need to collaborate with people. Besides, it can be fun.
- Don’t force conversations — learn to live with silence.
- Look for genuine conversations. There’s something interesting to share and learn with almost everyone.
- Aim to empower those around you instead of just going about your work.
- Don’t be afraid to seek help when emotionally disturbed.
Business
Even if you don’t want to indulge whole-heartedly, no harm in being aware about some nuances of doing business. Capitalism is a thing after all.
- Aim to solve pain points instead of just improving lives for your customers. “Be a painkiller, not a vitamin.” — Aneel Bhusri on Masters of Scale.
- User experience is paramount when designing products.
Science
No matter what your experience with the education system was, adopting the scientific method can improve your life!
- Learn basic Astronomy. History of our cosmos is yours too. Go stargazing. It will be a beautiful experience. Watch Wonders of the Universe, hosted by Brian Cox. It is amazing!
- Read about your brain and learn basic neuroscience.
Public speaking
Get on a stage or podium sometime. It may turn out to be one of the most satisfying experiences of your life.
- Be honest with your content. Admit if something is experimental or was a failure.
- Give the audience a story/narrative instead of a cold presentation.
Travel
Traveling won’t provide you with all the answers. What it’ll provide you with is enough time and peace of mind to ask yourself some much-needed questions.
- Pack light and learn about techniques to do it better. Read this article which compares various packing techniques.
- Visit the mountains and go for a trek. Indiahikes is great if you’re looking to do Himalayan treks.
- Get a good camera or smartphone if you can afford it — photos can turn out to be precious.
Music
I’m not sure what to put here. Music is one of humanity’s most beautiful creations. That’s about it.
- Find good live music online/offline and watch/attend it. Attend NH7 Weekender or Ziro festival if you’re in India.
- Share and receive good and interesting music with your family, friends and colleagues.
- Read about various genres and listen to the classics. Try out new genres and artists.
Hope some of the above is useful for you. That’s it for now. Have a great 2019!
If you read and liked the article, sharing it would be a good next step. Additionally, you can check out some of my open source projects on Github.
Drop me a mail, or hit me up on Twitter or LinkedIn in case you want to get in touch.
This post was originally published on Tech and Mortals.