Generative AI for data analysis
I have a good one for you, can I replace my data analyst intern with Generative AIs?
Turning Generative Ai into a data analyst
At “fifty-five — the Data Company”, my current company, we are curious enough to invest the required time in order to explore the idea of turning Generative AIs into media and analytics data analysts.
Well, did it work? Yes, we were able to create a functional POC where by using natural language, Generative AIs (Azure OpenAi & Google Generative Ai) can understand the intention and generate SQL queries and graphics adapted to the need.
What have we learned?
Here is some information I can share with you:
1 — Data cleaning, governance, and cataloging are requirements for this to work. The AIs need an unambiguous context and only these 3 practices make it easier, the principle is always the same: garbage in, garbage out.
2 — Concerned about data security, we provided Generative AIs only with metadata and not the data content and… it works.
3 — It wasn’t done without a lot of work, I still have my founder's voice in my head repeating to us: He doesn’t know how to correctly manage date windows, it’s inconsistent. Thank you for having been our awesome beta tester. We did a lot of testing and worked a lot on the pre-prompts, keeping in mind the complexity in terms of the maximum tokens allowed by the APIs.
4 — Knowing how to correctly express needs (specifications) has always been critical for data projects. Here too, you have to know how to clearly say what you want without ambiguity… Generative AI cannot read minds.
5 — It’s better to calibrate the AI only for a specific task, rather than making it a Swiss army knife.
6 — Even though many SQL queries are both syntactically correct and work, at the moment this bot CANNOT REPLACE HUMAN.
Because Generative Ais needs precise contexts and unambiguous expressions of demand, they can only help professionals in their daily work by helping them save time in many ways. Technologies are human extensions that help transcend their limits, not replace them.
Is your organization prepared to admit they don’t know what will work with Generative Ais and are you willing to invest resources in finding out what you don’t know?
