My Journey to Data Engineering

About Me

My name is Jennifer Ebe. I am a Data Engineer with about four years of experience in data engineering and two years of data analysis experience who loves to travel, try out new restaurants and eat (obviously)! I have a track record of warehousing billions of rows of data and cloud computing, creating custom ETLs with python and, Airflow, Dagster and managing and administrating databases and data warehouses. I have worked with some of Nigeria's largest Fintechs and one of Nigeria’s biggest Banking institutions.

How I Got into Data Engineering

I started as a data analyst, just wanting to learn more python and do more. I read up a little and decided data science was the most logical step for me then. I got into learning using DataCamp and completed about 42% of the data science learning path before I tried to build a single personal project. This was a massive mistake on my part, and I learned that building alongside learning was the only way to learn.

After building my first few projects, I realized I spent more time cleaning data than building models. I made a post online, and someone responded with: "if you're so stressed by data cleaning, join a company that has a data engineer, so you have access to already cleaned data".

It was my first time hearing about Data Engineering, and I got excited after some research. There are few data engineers, so I volunteered as a tribute.

I Volunteer As Tribute

That same day I changed my Track (Datacamp folks get it) to the Data Engineer Track.

I soon started off building lots of fun projects and collaborating with friends. One of the most prolific projects I have worked on to date was the sex for grades project with Emeka Boris Ama (which I still haven't published 😭). I also had the best opportunity to learn from seasoned colleagues who shared articles, courses, and their time to help me grow.

What I would do differently

Thinking back, there are a few things I wish I had done differently. I will share some of those so you can learn from them too.

1. Document Everything! — Have a physical notebook or on your system to document interesting new things you learn about Pandas, Numpy, Airflow Etc
2. Build more projects — The truth is you will never be less busy; work is going to get tasking and tasking, and you have to make time for what is essential — projects are critical to your growth, so create them!
3. Comment your Code — Everyone should know how important this especially when it's been a while since you wrote the code. Comment on your code because it's good practice and helps you remember what you were trying to achieve.
4. Be patient and kind to yourself — It takes a while to learn new technologies or a new and different way of thinking. It will take time to master everything you need. Numerous new tools will be released in data engineering before you finish reading this article, as it is a developing field, and discoveries come up every day.

Finally, I am grateful for how far I have come, Rome wasn’t built in a day, and neither were any of the skills I have now. You can do this! I am rooting for you!