Agentic AI: A Buzzword or a real deal? Why should you care?
What makes an AI agent? Why data engineers should learn about orchestrating AI agents?
Agentic AI (aka AI agents) seems to be the keyword of the year. Companies such as NVIDIA, Gartner, IBM, Google, Blue Prism, and UI Path called Agentic AI as the top technology trend to watch out for in 2025. Researchers like Chip Huyen and Andrew Ng are particularly enthusiastic about the potential of AI agents.
Earlier this year, OpenAI announced Operator — an agent that can use its own browser to perform tasks for you. Later on, OpenAI released Deep Research — an AI system that utilizes an Agentic framework to autonomously conduct complex research tasks on the internet, essentially mimicking human research by actively searching, analyzing, and interpreting information across multiple sources to generate comprehensive reports, all within a structured, multi-step process.
Before we could realize the power of Deep Research, Perplexity AI and Google have both released AI-powered deep research tools that can quickly answer complex questions. While OpenAI and Google made Deep Research available to paid users, Perplexity AI made it free for all users.
Do I live in a bubble? Is Agentic AI a buzzword or a real deal? As data engineers, data scientists, and data practitioners, why should we care?
To top it all, my friends, family, colleagues and every single soul I know in Silicon Valley are working for/on an AI startup that builds agents.
Do I live in a bubble? Geographically, maybe!
Is Agentic AI a buzzword or a real deal? Let’s break it down!
History of AI agents
AI agent as a theoretical concept existed since the 1950s as long as the field of artificial intelligence has. Earlier versions of AI or AI agents were rule-based systems that operated on a set of predefined rules to make decisions. But in the 1990s with the rise of machine learning techniques, AI agents levered data-driven approaches rather than solely relying on hard-coded rules.
What is an AI agent?
IBM says:
“An artificial intelligence (AI) agent refers to a system or program that is capable of autonomously performing tasks on behalf of a user or another system by designing its workflow and utilizing available tools.”
Perplexity AI says:
“An AI agent is an autonomous software system that perceives its environment, makes decisions, and takes actions to achieve specific goals without constant human intervention.”
Google Gemini says:
“AI agents are software programs that perceive their environment, make autonomous decisions, and take actions to achieve specific goals. They often utilize machine learning to adapt and improve their performance over time.”
What makes an agent an agent?
Three key phrases emerge –
1. Autonomous:
An agent needs to run autonomously. Automating a workflow is not something new to the world of technology. However, most of the workflow automation tools were rule-based and they cannot adapt to the changing environment or information.
But an agent needs to run autonomously without human intervention while adapting to the environment.
2. Make decisions
An agent needs to make decisions on the perceived information or environment. For example, when you ask ChatGPT a question, it returns an answer. It’s not making any decisions based on the answer. You do. ChatGPT or Google Search is not an agent.
3. Take Actions
An agent, after making a decision, also needs to take action based on that decision to achieve a particular goal. For example, when you look up a book you want to order on Amazon, you go through the list of books and prices, and make a decision on which book to buy. Making the decision alone is not enough. You will follow it up with an action which is to make a purchase. Similarly, an agent needs to be able to take action.
Alright, we know what an agent is. An AI agent is simply an agent that uses AI in any or all of these steps to achieve a specific goal.
Why is it a big deal?
The big promise of AI agents is that it helps us automate (boring) tasks in an intelligent way without human intervention.
What’s new you ask?
Process automation is not new. Digital transformation and automation have revolutionized businesses for decades now.
Before AI agents hysteria happened, there were different ways of automating digital workflows.
- You could call third-party software API (application programming interfaces) to communicate between applications and systems.
- In cases where you don’t have API access, web-scraping tools were used as part of automation workflows.
- Another interesting way is Robotic Process Automation (RPA). It automates repetitive, manual tasks that are rule-based and highly structured. These tasks often involve interacting with applications or systems through user interfaces, much like a human would. Companies such as UI Path, MuleSoft, and Blue Prism have been building Robotic Process Automation (RPA) tools for decades now.
But what happens when there are API changes, UI changes, or when tasks become dynamic and require judgement?
Let’s understand this using an example!
Example Scenario 1: Automating Tennis Court Reservations 😭😭
The apartment I live in has Tennis courts that need to be reserved through an App. Reservations for everyday open up at 12:00 am, and are available as 1-hour slots.
What fun it is to wake up at 12:00 am to reserve courts. No kidding!
Of course, I had to write a Selenium script to automate the booking process. It worked just fine… until they changed the app interface and the script broke. Not fun y’all.
These automation frameworks and tools work by simulating user interactions like button clicks, form filling, mouse movements, etc. Say if the position of the “reserve button” changed, the tool would have no idea and would keep clicking on the same position in the UI where the button was and would crash. i.e., the automation would break.
An AI agent, on the other hand, is intelligent enough to adapt to these changes. How? The agent would potentially invoke a multimodal AI model. These state of the art AI models are capable of understanding where the reserve button on the web app is, using advanced object recognition techniques and natural language understanding. The AI models, and in turn, an AI agent is able to read and see (perceive) the contents of the web application just like a human and act accordingly.
In short, AI agents can handle more complex, cognitive tasks with unstructured data, often requiring adaptability. AI agents become immensely powerful when you use it for repetitive, high-volume tasks in an enterprise setting!
Example Scenario 2: AI agent for Data Analysis!!
The architecture of an AI agent for data analysis could look like this.
There are multiple agents tasked with different expertise. Manager agent oversees the individual task agents to accomplish a goal you defined.
If you come from the world of data engineering, an agentic system is very similar to tasks and DAGs in a data pipeline. Agents can also be orchestrated over a periodic schedule.
Why should you care, as a data practitioner?
Look at the agentic architecture and compare it with a data pipeline.
The key distinction lies in the nature of the tasks performed. Data pipelines typically involve deterministic tasks with predictable outcomes, while agentic architectures incorporate intelligent tasks capable of decision-making and adaptability. Despite these differences, both systems utilize similar tools and frameworks for implementation.
Data engineers have an advantage in this evolving landscape, as their experience with scheduling and orchestrating complex systems translates well to managing AI agents.
As the field progresses, data engineers may find themselves building and overseeing numerous AI agents, necessitating a solid understanding of generative AI terminology, the fundamentals of agent construction and orchestration, and effective strategies for integrating AI into existing workflows.
What next?
Build your first AI agent — Coming soon!
Thanks for Reading!
If you like my work and want to support me…
