You’ve Completed Lots Of Data Science Courses? Now, What's Next?
Don´t head to Kaggle only. How to make the transition into the real world.
After getting my specialization in Data Science from Coursera and Data Analysis Nanodegree from Udacity, I took many more courses online, developed many skills, finished different assignments, and wrote a variety of successful toy code. Did you notice that? “toy code”!
They were all code on virtual problems, where most of the data was served on a platter, nothing in real!
The question was always there; “What’s after courses and certifications?” The field of Artificial Intelligence is a field to apply, to benefit from, to prosper.
The answer was simple but yet great; “Omdena”.
Omdena — The Experience Builder
Before I tell you how experience is built there, let me first tell you how I got there.
Luckily, I had met an amazing person in one of the scholarships I attended, who enlightened me with his own Omdena experience. I hesitated for months — I would go to their website, read about all the projects they have accomplished, and others that were to kick-off, but every time I failed to click the “APPLY” button, obviously because of my old friend that is “the Imposter Syndrome”.
And why wouldn’t my “friend” haunt me. An Egyptian mother, middle-aged, got a Bachelor's degree in Electrical Engineering a while ago, and all of a sudden, I want to make a career shift to Data Science. There were fresh graduates all around me with their blinding degrees and many more Data Scientists with all that experience under their belt.
On one such lonely visit to Omdena’s website, a project caught my attention and almost as if some mysterious force made me hit that button — “APPLY”. The application was very smooth.
A few days later, I got an email to set a schedule for interviewing on Skype. Yay!
The kick-off was on the same day that I had my interview, and boy I was sweating. Amazingly, it turned to be so friendly and within a few hours, I got the good news that I am accepted along with my Slack invitation.
Till that point, I kept asking myself, “What are you going to do? How is this going to work?”
To be honest, I did not have much experience in AI practical implementations, so developing a full project pipeline seemed like a mammoth task. You could say I know the Math, but here in Omdena, I practiced the Math and all the theory I had learned.
The Kick-Off
Then came the Kick-off meeting, and let me tell you, that’s when I felt the essence of Data Science and Artificial Intelligence. I was so so excited, yet scared. In spite of the fact I was already selected among 51 collaborators from all over the world, I still wondered who am I to participate in such a community. My imposter syndrome was taking over.
As soon as I was added to the Slack channel, we were sent the project statement, and then we had an “Ice-Breaker” Zoom meeting, so we can introduce ourselves and get familiar with the tasks ahead of us.
Believe it or not, after I introduced myself, got acquainted with everyone, and got familiar with the problem statement, on our second Zoom meeting, I volunteered to be a task manager! Hey, where did your old friend go? As if the community really broke the ice and made me merge into the AI society.
The Task
In my dreams, I would not imagine, that we can (and I can) collect a dataset to work on in the project.
Well, let me explain the previous sentence — in all the online courses out there, the tutor provides you with a dataset, mostly clean, labeled, and ready to ignite the engine of your ML model. But here, World Resources India (WRI) in association with Omdena gave us the chance to weigh, analyze, estimate, predict, and evaluate India’s economic well-being.
We had to read papers, collect data, implement those papers in code, clean, process, label data, build a model, evaluate, tune then re-evaluate.
See! That’s why I said Omdena brought the essence of a real-world AI project.
How that could be accomplished by newbies?
Well, here is how. We had full support, experienced collaborators, resources, previous studies, weekly Zoom meetings with other collaborators, and another meeting with the WRI team to follow along and to make sure everything is on track. We started off with two tasks — to collect satellite imagery of India’s landscape and also to find a panel data reference on the web for labeling. A technical post to walk through all the steps is here. link.
Back to my chores as a task manager (Well, I still feel thrilled whenever I remember I got the courage to be a task manager on my first project *wide-grin*).
I worked on panel-data scraping and analysis. I and my colleagues explored several survey-based data available on the web as Human Development Index (HDI), Demographic and Health Surveys (DHS), Socioeconomic High-Resolution Rural-Urban Geographic dataset (SHRUG), and others. Guided by the WRI team, we focused on Census data provided by India’s government, which was quite reasonable as there is a Census planned in 2021, so our project will serve a purpose in the future as well.
As a task manager, I had to be on top of how things were going. I was also coding, documenting, preparing for the presentation, communicating, and updating the project’s Wiki page. All this was new, and well beyond my own expectations of me. We built a web scraper using Selenium to get house listing data based on states and districts with a set of 146 features, for example, roof-top materials, water sources, kinds of households, assets owned, and sources of light.
Along the way, I learned plenty of new things;
- Professional web scraping
- GeoPandas
- Principal Component Analysis (PCA)
- Feature Engineering
- Data Pre-processing
I learned how to work in a team, a really big team, a multi-national team, a very talented and competitive team.
Looking Back
Eight weeks flew by (that’s how long most Omdena’s projects last), and when the time for the final presentation approached, I looked back to see how we started with a genuine idea, accompanied by some talented collaborators and how we explored, experimented, analyzed, read a lot, implemented, evaluated and got final results. I realized this is what machine learning is all about, this is how a real project is done, and a sincere experience is built.
I will not deny the importance of certificates and specializations but to get your hand dirty with a real project is something we cannot find easily unless you get lucky and read a post like this, and then don’t hesitate to click the “APPLY” button on Omdena web page.
