The AI Agents are Coming

12 Predictions for 2024

A High-Level Take from a Low-Level AI Implementer

Change is coming with the advent of AI agents. Below, I discuss what AI agents are, and why they are coming now. I also discuss how they will be developed. Finally, I make 12 big predictions regarding AI agent impacts on businesses and consumers in 2024, with the goal of helping people get a head start on what is shaping up to be another wild ride.

The Dawn of AI Agents

The year 2023 has been the year of generative AI (GenAI) and large-language-models (LLMs) for generating content in ways never seen before. Indeed, there was an explosion in the marketplace of new models and new applications. However, LLMs are just the preamble to the GenAI revolution that has begun.

LLMs are impressive at generating text, images, and code–outputs that humans can use in their work (or just to have fun). But what if we could go a step further, and use LLMs not just to generate as a final step, but as the glue to accomplish more complex work. For example, what if we could have LLMs do work that requires sequential steps, specialized tools, up-to-date information, and special skills? This is where AI agents are born.

An AI agent is a program that can pair LLM models with various data and tools to complete tasks, and get work done. Agents move beyond content generation to decision making and action taking. A simple AI agent has the ability to interpret high-level instructions, make a plan, and then execute steps towards achieving a goal. With each step, an AI agent may be able to assess its progress and adjust its strategy. Crucially, these agents have access to sets of enabling tools and data that they can choose to use.

AI agents will have the means to complete actions. Envision an email agent that will be able to not only generate email text, but also to automatically converse with a recipient towards achieving a goal. Or imagine a scheduling agent with the ability to converse with multiple parties to get availability and interest information, and then schedule (or reschedule) an event based on feedback. The marrying of intelligent decision-making with real-world actions in the form of AI agents will produce powerful and valuable outcomes.

They’re Already Here, Some Assembly Required

In a way, the popular OpenAI chatbot, ChatGPT, is an AI agent with the goal of providing valuable conversation, and the action of responding to prompt inputs, while using recent memory as context. Recently, chat tools like ChatGPT have also been paired with tools like web search and database connections, to augment queries with up-to-date information.

The truth is, the building blocks for future AI agents are already here. First, thanks to 2023, now we have a variety of exceptional LLMs, and coding tools to work with them (langchain, Hugging Face, etc.). Second, there are already many existing software programs and functions that are connected to — and do work in — the real world. Third, there is valuable data everywhere. We just lack experience putting these pieces (LLMs, tools, and data) together, but that will develop quickly. As businesses understand the value and see the impact around them, the sense of need for implementing AI agent technology will grow fast.

While the pieces are here, there is a lot of thought that will need to go into the technical design of AI agents. Developers will need to decide what program tools an agent will have access to, and how the agent will interact with them. They will also need to decide what data is available, and how it is presented. For example, is it in JSON, SQL, or a vector database? Is there accompanying metadata? Is the data chunked intelligently? How is the data updated? And so on.

Even after figuring out tools and data, there are still many technical decisions to be made. What prompts will the AI agent use, and which LLMs will the agent be able to call? What will be done with the output? How will agent performance be evaluated so that objective improvement decisions can be made?

On the business side, assembly is also required. Product teams are new to GenAI and AI agents, just like everyone else. Managers will need to gain experience building teams with the right expertise, which will be a mix of domain understanding, product experience, software development, and AI chops. It’s likely that companies will also need to experiment with proof-of-concepts before gaining their balance in production applications.

Many companies already have the vision. Leaders are preparing their moves, and developers are gaining experience. As an AI/ML consultant, I see this in conversations. The ideas are here already. The implementations are coming soon. Over time, more specific tools will develop, along with business experience. Users will gain confidence as they interact with AI agents, and the wheels of iterative improvement will continue to turn.

Envisioning the Future of AI Agents in 2024

When I began devising this article a few weeks back, Google hadn’t yet announced its new multi-modal model Gemini, which is able to work adeptly with various types of media (text, images, audio, etc.). The story of what Gemini can do is just at its beginning, and I feel as if the ground is shifting beneath my feet faster than I can write about it. Pairing multi-modal models like Gemini with specialized data and specialized tools, will create even bigger breakthroughs for AI agents in the coming months.

I anticipate 2024 will be another big year for AI, as innovative companies learn to implement AI agent technology. Here are a few of my AI agent predictions for 2024.

1. AI agents do simple work: AI agents will quickly graduate from novelty toy status, to doing simple, boring, routine work. Tasks like updating documents, scheduling, and auditing will all see newly capable AI agents. These are the low-hanging fruit that companies will use to dip their toes in AI agent waters. Wins here may appear trivial, but they will be a big step from hyped-up AI talk, to concrete evidence that shows businesses are in the game.
2. AI agent specialization: the foundation generative models built and trained by large tech companies are general, and the tools they access out of the box (like web-search) are often general as well. In 2024, I expect to see more specialized models, as well as specialized tools and datasets meant for AI agent use. I expect to see some modularity emerge here as well, particularly in the case of complex tools and proprietary data. Some of these may be offered through APIs as services, or in datasets.
3. Multi-agent development: Don’t expect all AI agents to be lone workers in silos. The natural step after specialization is to have specialized agents work together to complete more complex tasks than any one could do alone. Also, multi-agent frameworks will begin to leverage hierarchies, in which some agents focus on high level objectives, while others take care of task-specific work, and then report up.
4. AI agents as consumers: What does it mean to get more page views from agents? Will there be websites or APIs optimized for AI agent discovery and use? Right now, the web and many other software tools are generally thought of through the lens of human consumption, but this will change as AI agents consume more content, and use more tools.
5. More trust and power to choose and do: AI agents will progress from simple choices, to more impactful decisions. As businesses experience progress, they will learn to trust their AI Agents. These agents may be given budgets and money to spend. Perhaps they will take on tasks like purchasing stocks, or posting personalized ads with time-sensitive interactive deals. For example, imagine haggling with an AI ad agent that can make decisions.
   Ad Agent: “Hey Dan, enjoy 20% off on our new gadget if you purchase right now.”
   Me: ”I’ll do it if you can ship within 3 days, and you also send me some of your product stickers free.”
   Ad Agent: “Agreed! I’ll send 3 gadget stickers in addition. Shipping will cost $x, and we’ll ship to your address. Confirm to initiate payment.”
6. AIAaaS (AI Agents as a Service): Amazon Web Services for some years now has offered a service called Mechanical Turk, where businesses can hire armies of (human) workers to perform repetitive routine technical tasks like data labeling. Imagine being able to rent AI Agent hours to do similar things on-demand and at-scale. These agents may even become teachable, or tunable, so that they can perform better in chosen areas or tasks. Are you ready for an AI agent to review your competition submission? Or an AI tax preparer that works with your W2s and 1099s? Or how about an on-demand financial advisor that you can bounce ideas off of, and that can quickly move your funds to match your evolving goals?
7. Some jobs will face pressure: There is no doubt, AI agents will continue the trend that GenAI has started, taking some work away from some people. Consider the degree to which,
   1. Your work is done at a computer.
   2. Your work is routine.
   3. You are easily interchangeable with other people with your same skill set.
   The higher you score on these categories, the more your job may begin to receive pressure from AI agents. Many white collar jobs are on the list. Roles like auditor, technical writer, and customer service agent can be automated to some degree. However, look to AI Agents to augment, rather than outright replace many jobs in 2024. On the other hand, consider work that requires,
   1. Significant tasks away from a computer or digital interface
   2. Many varied tasks
   3. Highly specific knowledge or skills that few people have
   In these cases, your job is relatively more insulated from AI agent automation.
8. New job roles will emerge: As companies begin to use AI agents, they will need employees who can take care of them, and interact with them. Will the next hot job be Agent Engineer? Agent Manager? Agent Trainer? Agent Evaluator? Agent Auditor? Perhaps AgentOps? It’s hard to tell what exact titles will be, but AI agents will require human counterparts who know how to interact with them, implement them, maintain them, and improve them. AI agents will also require monitoring and oversight.
9. Hype and doom. If there is one thing you can depend on not changing in 2024, it will be wild claims about AI. The talkers and writers in all their exuberance to gain attention (and get paid by advertisers) will continue to stoke the flames that excite us. We’ll read about how AI agents are solving all of our problems, while simultaneously also bringing about the end of the human race. If it sounds too good (or bad) to be true, it probably is. Unfortunately, this will make it hard (as it did in 2023) to tell where the real substance is.
10. Calls for governance. Also already a feature of 2023, calls for governance will continue in 2024. The power of AI agent technology will become more apparent, and so too will the need to organize the way we work with it. However, governments will continue to struggle to guide the development of AI. They will lead somewhat blindly, and be prone to influence from lobbyist groups who are seeking to outmaneuver their own competition. Any entity that acts like they are in control of GenAI is either delusional or wishful. As I mentioned above, the building blocks of AI agents are already freely available. Many government officials don’t understand tech, and many tech leaders don’t understand AI, so on governance we may be several degrees woefully removed from sanity. Unfortunately, 2024 probably won’t be the year of effective AI governance.
11. Lots of competition. LOTS! In 2023, major cloud providers and AI companies competed fiercely for attention in the GenAI space. Expect this to continue like a raging wildfire with more AI companies releasing even better products in 2024. Look to see some helpful AI agents from major tech companies. We’ll also see competition between companies who benefit as implementers and consumers, as they convert AI agent work into business value that they can pass downstream to their customers.
12. AI Agent powered devices: Finally, expect to see some physical devices become more capable and interactive due to AI agent technology. Smart devices will become smart 2.0. (The cynic in me predicts they will be 2.0 times more expensive as well.) For example, I currently have a smart vacuum that cleans some rooms in my house. I don’t talk to it now because it’s not THAT smart. And it doesn’t really talk to me either, other than reporting errors. With AI agent technology I may be able to tell a smart 2.0 vacuum, “hey vac, can you clean this room in an hour, I’m busy here right now”. Or “can you push all the toys there into a pile to help me clean up, and then vacuum around them?” I’m just ideating here. Maybe the first widely available AI-agent-backed physical devices won’t be vacuums, but I’m quite certain they are coming. I’m excited to see them, use them, and understand their capabilities and limitations.

Did I say 12 predictions? As I was doing final edits and thinking deeper, a few more ideas crossed my mind. I just couldn’t resist that imaginative urge. These next ideas may not surface in 2024, but it will be interesting to see if they show up.

1. AI agent assisted malware: Computer viruses evolve in so many ways, so it stands to reason we’ll see one that is AI-agent-powered. Also, AI agents are coming to identity theft. We may need better ways to spot them, as they will be more convincing than many human criminals out there. To be clear, I’m not predicting that these agents spawn on their own, but rather that human actors will create them.
2. Self-destructing AI agent: We have to create this, because it sounds cool. But seriously, there may be applications where an on-demand API doesn’t make sense, and an AI agent needs to be downloaded to a local device. Once a task is completed successfully, the agent can self-destruct to free up space. Another application may be to enforce a license agreement, and keep customers paying periodically.
3. Self-improving AI agent: There are various ways to improve LLM responses — few-shot prompting, fine-tuning, RAG, RLHF — but these improvements generally have a manual implementation component. A self-improving AI agent will recognize when it has learned something new that is worth internalizing; it will be able to automatically tune it’s own LLM model(s) to improve its native performance.
4. AI agent scientists: In contrast to an AI agent with a business goal, an AI agent scientist may have a learning mandate, coupled with access to tools and data. The agent would devise hypotheses, and test them. Maybe the agent will even be able to request access to new tools. Connected to a real lab (in say materials science, or biotech), this could supercharge experimentation and R&D.

What about you? What ideas do you have about where AI agents are headed? I’d love to hear about it in the comments.

Conclusion

LLM apps stole the show in 2023. I anticipate AI Agents will steal the show in 2024. Right now many agents (and LLMs for that matter) are like toys–fun to play with, but not necessarily useful in the real world. However, the building blocks are already here, and we’re figuring out how to put them together. Companies building these toys will graduate soon, releasing production-grade AI agents that are sophisticated, and capable of working in real-life scenarios. Like it or not, AI agents will become part of our experience.

Thinking about the future with AI agents can bring both a sense of excitement and terror at the same time. The future is coming so quickly, and it is hard to keep up. The power that AI agents bring to perform tasks and complete work represents great opportunity for many people and businesses. We will be able to do more, create more, and enjoy more with less effort. However, these innovative new ideas will undoubtedly disrupt business as we know it, and all of us will need to learn and adapt to the new technology.

About the Author

Daniel Dowler has consulted on industry projects in healthcare, financial services, entertainment, software, and retail with some of the best companies in the world. He is an AI/ML Principal for Eviden, leading exciting technical implementations of AI/ML. He enjoys using the best technology to solve problems, connecting the dots from the data, to the math, to the business.