Jan 23
Why I won’t switch to Data Science?
A story of an unpopular opinion derived from experience
Unpopular opinion: In the day and age of data science being the most popular career choice, I don’t want to make a switch to data science.
Don’t get me wrong here, I have immense respect for data science as a field and I am in complete awe of it, but honestly(presently), I feel like it is not the thing for me. Explained below are some of the reasons why…
First, let me start by introducing myself, I am a software engineer working in data or a data engineer. Though the domain of data is the same for both a data engineer and a data scientist(also the word, data), both are different though sometimes there are overlaps too.
While working with data, I have also done my fair share of data science projects for learning or proof of concept purposes or sometimes simply because the use case required it.
During those times, I started courses over courses, trying out Kaggle notebooks to enhance the learning. I am a notetaker and I have around three(or maybe four?) notebooks filled with the notes of those courses as well as during learning.
With that learning, and also from multiple blogs available on the internet, I was able to do the projects on machine learning which were around using machine learning to find similarities or writing a machine learning model for inventory management, pretty interesting stuff, right? So why didn’t I pursue it?
While working with machine learning projects or while doing those courses or even some of the blogs I read, I realized one thing, they just taught me the application of machine learning not the internal workings for the same. Now, I am someone who needs to get an understanding of something completely before getting confidence in it, otherwise, it gives me a lot of anxiety(I know I need to work on it!). Quoting my friend(I wish), Sir Einstien here:
If you can’t explain something simply, you don’t understand it well enough.
Though I tried my best to understand some of the math behind algorithms, for most of the cases, I didn’t understand half of it. (I know it’s on me, I am working on it)
Well, this was the first reason I kind of lost interest, now let’s move on to next.
Over time, I realized I am a person who loves to code, the feeling of code compiling and running successfully and I can mark the task as done and it gives me immense completion and satisfaction.
But for some reason, while doing ML projects, I never got that feeling. Mainly because no model can be 100% accurate, right? No matter how much time I spend, I found some improvement or changes that can be done or approaches can be changed and results varied. How do you define accuracy in real life?
If you do not at all relate to this, maybe you have mostly worked with test data for learning data science which is pretty much clean but the data for real use cases and scenarios is in no way clean.
It may or may not follow a pattern, it may or may not have anomalies, it may or may not be seasonal, one has to spend a lot of time cleaning the data to actually run the model over it. Though I did enjoy the data cleaning portion, exploration, analyzing, and cleaning or finding anomalies.
Next thing, now if data is clean, at present there are some managed ML services that can also be utilized which chooses the model for you and trains it for forecasting. I think, for now, I am going to stick with such solutions if the use case really requires it.
Concluding, I want to say that this is in no way a rant, it is something of a personal preference based on what I love doing and what I have realized over time. So as of today, I don’t want to switch to data science, but I don’t know what the future holds! Also, I will be more than happy to be proven wrong.
Lastly, my advice is don’t jump to data science just because it is the hottest career choice, evaluate carefully and make the choice by choosing what’s right for you.
And if you do choose, wish you all the best and lots of $$$ :)
Happy learning,
JD
