Career Development — My experience gearing up from Business Analyst to Data Scientist
Written by Ayrton Barros — Business Analyst at AMARO
In the new world of Big Data and Analytics, traditional Business Analysis positions are passing through a huge transformation. The goals are still the same: extract the best insights from all available information, allowing precise decision making that leads companies to achieve their objective. So what have changed?
The ever-growing number of information sources and huge amount of data being poured into databases every second together with all the hype from the media, end up creating high expectations on stakeholders. Not surprisingly, demand for deeper insights that can lead to game changer decisions has never been higher.
With that in mind, traditional skills (aka spreadsheets) coupled with BI visualization tools alone are not enough anymore. Though spreadsheets are great tools to analyze small sets of data, they are not made to support millions of data points. I bet that everyone can remember a simple task that took too long because a spreadsheet was crashing when making calculations in a 100k+ rows dataset. Even though handy, traditional BI data visualization tools offer limited customization possibility needed for diverse point of views without the aid of a specialist. Who likes to open a ticket to the Data & Analytics team and wait several weeks to get the requested vision? Also, this tools enable us to analyze the past, and have some insights of the present, but they still lack the capacity of giving us glances about the future.
At AMARO, a data-driven culture means not to make decisions only guided by analytics, but also to be open minded to learn new skills that will empower our analytical capacities. As we discuss with more detail in this article about data science projects within AMARO, several challenges that we face are related to the future, and giving our best at predicting it.
In general, the underlying math of aggregated predictions is well-known and useful to forecast how many jackets we are going to see this year, but has its limitations on a granular level (aka how many pieces of a white jacket with polka dots are going to sell on the 3rd week of May). Also, in the classic process, the experience of the analyst is hugely influential on the forecast outcome. With all that data, this approach doesn’t seem the most data driven we can do, does it?
With those things in mind, I was asked if I would be willing to take part on a data science project aiming to create a prediction model for sales forecasting. As a Business Analyst, I could contribute with some business and market knowledge. On the other hand, it was necessary to embrace the challenge and get to learn a whole new set of tools that definitely were going to get me out of my comfort zone.
It seemed hard, but could be fun. I chose to go for it!
First, I truly had to learn or revisit some things:
- Statistics: sorry, but the concepts you remember from college are not enough. You’ll have to brush it a little, but there are plenty of awesome online courses out there.
- Machine Learning: before learning to code ML, it’s very important to understand how it works, what are the problems that it solves and the approaches to each one of them. ML isn’t magic, in its basics, it solves classification problems. And you need to understand what the logic is all about.
- Learn Basic Coding: At AMARO, we chose Python as our (main) weapon of choice. Again, there’s plenty of good stuff on those basic online courses, and no previous knowledge is required (trust me, I didn’t have any). The suggestion is to learn the basics of pure Python and its logic. You don’t need to become an expert to start, but it’s important to get comfortable.
- Learn How to Code with Jupyter Notebook and Python Libraries:
This was the moment which I first thought: I might be able to do it! Coding with Jupyter and Libraries is surprisingly easier and user friendly than on a Python shell. - Understand Data Science Framework: Now that we’ve learned the basic tools, we need to understand how to use them. Studying a framework like CRISP-DM, understanding what does its steps mean and the challenges of them will help you figure out how the job gets done.
Now it’s time to put all this to practice. I won’t lie, at the beginning I struggled at several moments to get simple things done, but it will pass!
Here at AMARO, I had the opportunity to put my skills to test on the real world as soon as I got them, but in case you don’t find this opportunity within your company at first, don’t worry. My suggestion is to take a look at Kaggle Competitions, it’s easy to find challenges for beginners with problems to be solved and datasets being provided. It’s very interesting, and there will be plenty of people looking to help and discuss problems. A gold tip: Kaggle provides the commented Python notebooks on several challenges, it’s extremely helpful to see how the most different kind of problems can be solved.
Final tip: always have in mind that someone already passed through your problem, and have published it in the internet where Google can see it. Make good use of search queries, there are no dumb questions either unsolved ones. The community is there for you, and Google is the librarian.
Conclusion
The past decade has seen the rapid development of computational storage, speed, cloud service providers, benchmark datasets and availability of publicly available APIs and open source projects to perform state of the art machine learning tasks nearly for free, providing high-level accuracy for many tasks. Traditional Data Analysis is very important to companies. However, it is when you put your “Data comfort zone” behind and invest on complex analytical projects that the game changing ideas are born. On the current pace of the tech environment, getting comfortable is not a viable option if you’re willing to be on the top notch in the future. If you’re curious about the data science world, but lack the skills, I invite you to embrace this challenge! You’re going to learn a lot, and it is a rewarding experience (:
