What problems should LLM Agents be used for?
We conducted customer discovery sessions with the operators of multi-million dollar revenue businesses across verticals like media, finance, and utilities; only to discover that all their productivity problems clustered around a core theme. All these businesses experienced severe pain when they attempted and failed to automate high-impact tasks that require significant manual effort with timescales ranging from hours to days to complete. More often than not, these problems were a perfect match for agentic applications as they required both data interpretation, tool use, and human approval in some cases.
With LLMs becoming cheaper, more complex automation use cases become viable, including the rapid emergence of agent architectures for the most advanced tasks. Despite advancements in technology, it is still unclear how businesses are expected to consume and implement agents that work on their private data or require access to internal tools.
Finding the right problems for LLM Agents to solve
Before diving deeper, it is important to acknowledge the spectrum of tasks that are available for automation within a business, as well the relative value that automation may hold:
Looking at this graph of Impact vs Cost, we discovered that businesses have been having significant problems with Zone 3 and Zone 2. These tasks often need to be completed on a daily or weekly basis and involve extracting large amounts of unstructured information from contractual documents, invoices, and emails and then entering that data into downstream systems such as CRMs and CMSs’.
Zone 3 represents relatively low-impact tasks that don’t directly affect one’s revenue, but have to be completed regularly with a high cost of doing so. Often this takes the form of bank reconciliations or invoicing. Depending on the size of the business this work sometimes needs a full team assigned to it to ensure that the sheer volume of data is processed in time. In smaller cases this can be a person’s full-time job, easily spending 20+ hours just on this task instead of other more productive activities related to the job description.
Moving to Zone 2, these are critical tasks that affect multiple areas of the business and require significant due diligence to avoid errors that could derail downstream tasks. This work might require multiple senior leaders’ full attention, and its cost can be measured in the time taken to complete it; either due to the volume of work or the input required from other sources of data.
Even more pressing, for both Zones 2 & 3, is the opportunity cost relating to this work. Time spent completing these tasks is time taken away from other activities that could be growing the business and finding new opportunities for savings or revenue generation.
For completeness, Zone 1 represents tasks that are usually carried out once, or quite sporadically, usually of strategic importance. Think UX personas, competitor analysis, or roadmap planning.
Positioning your LLM Agent
There are numerous agentic startups (QoQo.ai, Taskade, Devin) that have positioned their product as part of a growing productivity toolset to be used in conjunction with business software such as Microsoft Word, Outlook, and even more specialized tools like Jira and Figma. While these tools go some way to alleviate the pain, often they solve problems in Zones 1 & 3 and fall short of the type of solutions that could make a difference in Zone 2.
The aforementioned agent products focus on generating an end result, such as UX personas, support tickets, and technical specifications. While they have the potential to provide immense value, they have yet to address the problems described above. Instead, businesses need tools that can help them analyze their existing data, extract relevant details, and finally enter those into other systems.
As a builder, your most valuable asset is the subject matter expertise you bring to the table for the particular niche your agent occupies.
For instance, consider the following decisions to make when building a news summarising agent:
- Which news sources should you use?
- Which elements of the news articles are particularly relevant?
- How should that summary be used, or combined with other software systems?
The correct answers to all these choices change depending on your customer, sometimes even based on the subcategory. Think real estate vs finance, private equity vs VCs, growth funds vs seed funds; the list goes on.
It’s on the next step where the builders that we’ve spoken to get stuck, however. While agents present a neat solution with the ability to shrink a 3-hour process down to 3 seconds, the following areas prove difficult:
- Obtaining customers who understand the value proposition of an agent for their organization
- Identifying the right use cases across different industries where the agent excels at providing value
- Building a common experience that can be adapted to work with each customer’s unique internal tooling environment
Building an agent that satisfies a customer’s expectations for robustness, reliability, and consistency is already hard enough without the need to satisfy the above to build a sustainable business. Not to mention additional requirements around enterprise-style features that customers often request including data residency, multi-user access (RBAC), notifications, and alerting.
Recalling Zone 2 in the previous diagram, the most valuable problems within businesses involve sensitive data that are present in their internal tools such as Outlook for email, Salesforce for CRM, and SAP or Microsoft Dynamics for their ERP to name a few. This raises further questions:
- How should agents securely connect to these tools?
- Which LLM API providers are compatible with the customer’s data locality requirements?
- How and where should the customer’s data be stored, before and after it is processed by the agent?
These are all non-trivial engineering problems that need to be addressed per locality to successfully serve a consistent experience to many different kinds of customers.
This brings us to the Solitude platform. We want builders to focus on what they do best. Builders should focus on developing a deep understanding of the customer problems within the verticals they want to address, and then selecting the right combination of technologies and algorithms to power the agentic solution.
There is an immense amount of value in solving these problems which are unique to each business. Solitude takes care of the common elements beneath them like infrastructure, data handling, and secure application access on behalf of your agent for tools like SAP, Outlook, Gmail, and Hubspot.
Tradeoffs to make when building LLM agents
Similar to traditional software, each combination of technologies used to build an agent comes with its tradeoffs. When building an agent, consider the following constraints:
LLM Agent Memory
How should the agent’s memories be stored?
- GraphDB
- RDMS
- In-Memory Cache
For how long?
- Time-based schemes
- Priority-based schemes
How do you tell the difference between useful memories and the rest?
- Relevance vs Importance vs Recency
LLM Agent Planning
Which planning algorithm should you use?
- Reasoning as Planning
- Reflection
How complex does the plan have to be?
- Linear plans (ReAct)
- Tree-based planning (Tree-of-thought)
3. Graph-based planning (Graph-of-thought)
Will this impact the latency requirements for the customer?
- “Deep” planning requires several cycles, this usually incurs a ~30s response time (such as multi-level trees).
- “Short” planning, which may be one-shot or two-shot plans, has a much faster ~1–3s response time (a simple ReAct execution pattern).
LLM Agent Tools
- Which tools should the agent have access to?
- What actions is the agent allowed to take on this tool? Can they read, write, or do both?
- Will an RAG system be required to gather the correct inputs for that tool?
While Solitude lets you configure all these options, it is up to you to decide what is best for the problems you’re trying to solve; which largely depends on the feedback you get from customers.
So you’ve built your agent, what next? In our next blog series, we’ll dive into how Solitude promotes your agent through our marketplace and other distribution channels to help buyers find your solution. We take care of details like payments and transaction so that you can get paid monthly for the work your agent carries out.
If you found this compelling, we encourage you to join our early access program where you can stay up to date with out progress and get an early invite to our platform: https://www.solitude.ai/
