The boring part of working with Data Analysis — and that no one talks about
In this article, I tell you what is the most boring part about working with data analysis and that is little talked about out there
Portuguese version this article can be read here.
But before…
If even after reading the title of this article, you still have a warrior spirit and want to move forward in this area, here are some very good and didactic books to guide you:
First, an introduction
I start this article by confessing that I wasn’t completely honest when I said that “no one talks about it”.
In some posts on communities such as Quora and Reddit, it is possible to find testimonials about the reality of working in the field of data analysis.
However, the amount of down-to-earth stories that debunk the glamor of data doesn’t come close to the volume of content about:
- How to enter the data area
- How to become a Data Scientist
- Data Science Course
- Migrate to Data
- How much does a Data Scientist make?
And, in fact, with the amount of data being produced at every moment, more and more companies will need professionals in this area, even more considering all the value that can be extracted from this generated data.
The question I pose here is, those who work in other areas may be tempted to migrate to this area not necessarily due to a genuine interest, but rather because the person is constantly bombarded with influencers, journalistic articles and topics on social networks. selling the idea that working in data is perfect and intensely exciting.
And the truth is, it isn’t.
Work is work and, as much as the end result can be rewarding, the process to reach it can be long, time-consuming and dense.
My idea is not to discourage anyone who wants to work in the data area (I include myself in that group), but to remember that it is no use looking for an impeccable professional area, but at least one with problems that the person is willing to deal with, coexist and resolve.
That said, I list here the most annoying situations when working with data analysis.
I made the list based on my experiences and testimonials I read around.
Boring situations in data analysis
1.It doesn’t matter what data you want to present, the problem is the color of the chart
I’ve been in this situation a few times (I say “a few” to be nice).
The book Storytelling with Data (which I can’t stop recommending as it completely improves one’s perspective on how to present numbers) has a full explanation of how colors should be used strategically in charts and visualizations.
The idea is to reduce the cognitive load and highlight what is relevant.
However, it has happened to me that I have been presenting numbers in meetings (including regarding an unfavorable scenario in the company’s sales) and the board’s biggest problem was that the charts were not the color theywanted.
I swear.
And I got called out in front of everyone for it.
So, although you use color theory to enhance your visualizations, be aware that there can be schisms over the fact that you used orange instead of yellow in a line chart.
Even in situations where it is not necessary to use the colors of the company’s brand identity or respect some standards (for example, green for positive numbers and red for negative ones).
2. Data cleaning consumes most of your time
This boring part bears repeating.
Even more so when it comes to data science, the illusion is still too often propagated that these professionals spend most of their day making predictions and working with super cool machine learning models.
The reality is that collecting, organizing and cleaning data takes up most of the daily journey of those who work with it.
Mainly because real-life data are very different from those presented in courses and tutorials out there. They’re messy, can be difficult to access, and don’t necessarily have adequate documentation.
As a result, those who finish studying and start working for real can easily become disillusioned.
The Python Data Cleaning Cookbook is a very practical manual on how to deal with this task, by the way.
3. It’s not just programming, statistics and machine learning: it’s communication
For obvious reasons, content and tutorials on analytics and data science focus mostly on the technical skills that a person needs to develop to work in the area.
And indeed, working with data requires a lot of knowledge and practical experience.
However, this creates a fantasy that the person will devote himself/herself to creating codes, beautiful graphics and complex statistical models.
However, the fact is that, as in virtually any area of corporate life, a good portion of the time involves meetings and emails.
There’s no other way.
In addition to presenting your data and conclusions, you will still need to:
- Align expectations
- Negotiate deadlines and deliveries
- Argue with managers
- Face customers
And for those who are more passionate about the technical side of working with data, communication is often by far the dullest and most difficult part.
Moreover for those who work remotely with global teams, given that the language we use has a strong cultural load.
But it is possible to train and develop clearer and more objective communication, as explained in The First Minute: How to Start Conversations That Get Results (it is a class on how to improve oratory, both in person and remotely).
4. Your dashboard can be spectacular, but people will still ask for more charts and also if you can export them to Excel
This is one of the boring parts that hurts me the most.
I worked at a company where, due to tool limitations, we didn’t have our own BI platform and the CRM had very limited functionality. Soon, I was building most of the charts and reports in Excel.
So far, it wasn’t ideal, but I accepted that.
What bothered me was that, at each meeting with management and the board, there was a request for a new chart, and the report already had several charts that were barely consulted.
And I could see that these weren’t seen when I was asked for a chart that already existed.
Or worse, they asked for a chart because of a specific situation, I created it and showed it at the next meeting, and then that chart was never used again.
It caused me a horrible feeling of wasted time.
The other situation I pointed out in the title of this section is quite famous for those who work with BI.
The person spends years studying PowerBI, Tableau, Metabase, dominates the best methods to build visualizations and make the dashboard interactive and practical, and uses the ideal colors. Beautiful.
After the person presents the final result and everything that the dashboard can do, there is always someone who stands up and asks:
-Can I export it to Excel?
This question hurts in the soul, but it tells a true story bout users of data products: many of them don’t know what they want.
It is common, even more so in business areas, to propagate the idea that buying a BI tool is the solution to be able to carry out all the analyzes that the company needs and that, when implemented, all the insights will magically appear.
However, at “hands on” time, the end user still wants his spreadsheet with a pivot table summarizing the numbers.
This brings a sense of control to whoever is consulting the data, while dashboards can end up requiring contact with a new tool and not everyone is willing to interact with what they don’t know.
Especially when you need quick answers.
Storytelling with Data: Let’s Practice!, an expansion of the Storytelling with Data I mentioned earlier, is a hands-on guide full of examples that can help reduce this friction, as it has several exercises on how to create more effective and easier visualizations to understand.
5.Documentation is essential, but creating it is extremely tedious
Programmers know that having complete documentation is very important to find out how certain libraries and functions work.
However, creating such a document is quite boring.
No wonder many people neglect this step. Allied to this is the fact that the person constantly has other, more urgent tasks to deliver.
A correct documentation includes recording the processes, methodologies and results achieved, in addition to keeping a detailed follow-up of all the steps carried out in a given project.
This is necessary to have a history of the work done that the current team and possible “future heirs” can refer to.
However, creating this document with all its details is very tiring and monotonous. I can’t deny.
Final considerations
These are the parts I find least interesting when working with data analysis.
If you identified yourself or want to mention another not so exciting situation in working with data, leave your comment.
