AI agents: Exploring the potential and the problems

As generative AI systems evolve from chat-based tools into agents, understanding the potential and limitations of AI agents becomes increasingly important. 'Agents' are claimed by many to be the next big thing in AI. We're told that these AI-powered autonomous agents will act on our behalf and book our holidays, writing our code or completing mundane work tasks to enable us to spend our time on more worthy things. So at the start of 2025, the BBC R&D’s Advisory Team decided to look more closely at the technology and the issues raised by this rapidly developing technology to guide future research efforts and explore whether and how we might use agents in the BBC.

It's important to note that this is a fast-changing landscape with new releases coming regularly from many different research labs. During our work, we saw major releases such as OpenAI’s Deep Research and Operator reshaping expectations of what AI agents can and cannot do. As a result, rather than attempt to capture the state-of-the-art, this article offers a working definition of AI agents, explores what makes them distinctive, and highlights the challenges and limitations the technology may encounter in the near and long term. 

What is an agent anyway?

Before turning to AI agents specifically, it’s helpful to define what we mean by 'agent' more generally. The word has been used in computer science for decades, but definitions vary widely. The simplest option is to think of agents as systems that have the power or authority to act on the world in some way, but this leaves many questions. Do agents have to act on behalf of someone? Do they need actuators or sensors?

Given this complexity, we have taken a user-centric approach and considered when and how someone might use an agent, looking for a definition by purpose rather than by function. From this perspective, an agent is something a user reaches for when they have a goal but lack sufficient resources (time, knowledge, skills, etc.) to complete it. They trust the agent to achieve the goal independently, and ideally, to do it faster, more cheaply, or more accurately than they could themselves.

Based on this, we see that for something to be an agent, it has to:

• Be given a goal (e.g. a task to complete, a decision to make),
• Be given the authority (and trust) to act for or on behalf of another entity,
• Act with some autonomy (otherwise, it is simply a tool),
• Free up resources or improve accuracy (otherwise, there is no reason to use it).

Agency is not about AI

We want to be clear: AI is not a requirement for agents. What distinguishes an agent is not the technology it uses, but the authority it has to make decisions and take actions on behalf of the user. The difference is not in the capability but in the authority given to the system. Authority becomes a finite resource shared between the user, the agent, and its creator. Many non-AI computer programs can be agents: they already achieve complex goals with authority and autonomy and enable humans to focus on something else. In fact, much of what is marketed today as AI agents (e.g. hotel booking assistants or meeting organisers) could be accomplished with regular scripts and without the help of machine learning.

Defining AI agents

Given this, it becomes counter-productive to define AI agents solely by their capabilities or technical underpinnings. Instead, following the thinking of the likes of HuggingFace, we think agents become AI agents based on how they determine and execute their workflow. The more an agent plans and executes its steps independently to achieve a goal (i.e. without pre-defined logic and processes) the further it is on the AI agent spectrum.  This means that the majority of an AI agent's behaviour must be independently self-determined at time of the request. An AI agent not only has authority to act, but also has authority over the process.

Current AI agents typically have two phases which reflect this:

• Planning: Determining the steps required to achieve a goal.
• Execution: Taking those steps and evaluating outcomes.

As a result, an agent may not follow the same plan or take the same steps twice in a row for the same request. Critically, an agent also decides when a step is complete and whether the result is satisfactory — the programmer may not be in control of this. One consequence of this is that many AI agents function by generating and executing code directly, because code is easily verifiable with a compiler or interpreter. Syntax errors and failures provide strong feedback that can be used to correct issues.

Sharing agency

Ultimately, AI agents depend on our willingness to give up control over processes and execution. The key question is not what AI agents can do, but what we are willing to let them decide for us. Because agency and authority are finite resources, the design of AI agents will always involve negotiating how much agency users, programmers and agents each retain and in what contexts we are comfortable ceding control.

In many cases, traditional programming and scripting may remain preferable to AI agents. Indeed, a human-written script tends to have more desirable traits in complex systems, such as being predictable, understandable, repeatable, and testable. However, because their processes are dynamic and self-directed AI agents excel when there is a lot of uncertainty. For example, when it wouldn’t be possible to predict all use cases in advance, or when workflows are too dynamic to manually script. As a rule of thumb, we think AI agents will be most useful when:

• It is difficult or impossible to design a reasonable user interface because of complexity.
• It would be prohibitive to manually write all necessary program logic.
• The task is highly tailored, a one-off, or user-specific (and users cannot write their own code).
• A workflow relies heavily on third parties or changes frequently.

This means that agents are good for tasks a human doesn’t know how to do or cannot accurately describe the steps to accomplish them. It’s much more difficult to write a computer program when the task is poorly defined or there is a wide range of different input or data that would be expected.

The more uncertainty there is in how the program is going to be used or what it’s supposed to do, the more likely an AI agent will be useful.

Challenges and limitations

Throughout our research, we've encountered many challenges and barriers to both the making and adoption of AI agents. Some of these problems may prove temporary, others may be much harder to solve, and a few may spell a different future altogether if they can be overcome.

Interfacing with agents

Using chat interfaces with natural language prompts can be frustrating. While it seems intuitive on the surface, natural language is often imprecise or not discoverable enough. In UX terms, there are very few affordances which means that the agent's capabilities are hidden — that makes it hard to know what it can and cannot do. This lack of visibility echoes issues found in the first few generations of voice-based devices and interfaces, where broad promises ("just talk to it") clashed with their narrow and brittle capabilities.

As agents become more capable, the need for better, more discoverable design becomes critical. With previous voice-based interfaces, users were left guessing. With AI agents, this may be less of an issue as the answers don’t have to be binary, but it comes with its own crop of issues.

Issues with Large Language Models (LLMs)

LLMs are the current core of AI agents and provide much flexibility and power with one glaring problem dubbed 'hallucinations': the confident fabrication of information, sources, facts, and references that don't exist. In the realm of code, this is less of a problem because code can be checked and executed. But when it comes to facts, there is no compiler. No simple check can confirm whether an LLM's claim about a legal precedent, medical treatment, or financial strategy is correct. Mistakes can slip by unnoticed and depending on the context could cause real harm. There is good reason to believe that these issues may not disappear as the technology grows but are in fact symptoms of the limitations of the transformer architecture which underpins many of the popular LLMs. Some researchers believe that no amount of scaling will overcome this bottleneck and that entirely new architectures will be needed to achieve truly reliable AI. One view is that agent frameworks are the way to keep LLM behaviour within acceptable error limits, but this has yet to be shown. Right now, even the most advanced LLMs remain fundamentally unreliable, casting doubt on their suitability for critical tasks and require an expert to assess whether the output is correct or not.

The AI-expert gap

A major barrier to adopting AI agents is the gap between what these systems can do and what users can verify. Not only are AI agents likely to need the user to review and validate their outputs (undercutting the promise of automation), LLM hallucinations may introduce problems only experts could figure out.

To establish more trust, some AI systems now try to show their 'thought processes' or intermediate reasoning steps to increase transparency in exchange for the loss of control. Still, understanding these steps requires domain knowledge.

While this may be an acceptable trade-off for experts, it creates a dilemma for those seeking help outside of their area of expertise, since they are not equipped to audit the results. They have to choose between accepting some risk by uncritically trusting the agent or spending time second-guessing it. Either way, this negates some of the benefits of using AI agents in the first place.

Until agents can reliably demonstrate not just that they’ve completed a task, but that they’ve done it correctly, they may remain safest in the hands of experts. However, experts are also likely to expect a level of reliability and repeatability.

Repeatability

Because agent processes are generated on the fly their outputs are not guaranteed to be repeatable. Running the same prompt twice can yield different results, sometimes subtly, sometimes dramatically.  For creative tasks, this variability can be a feature, but for operational tasks, users depend on consistent behaviour. Without it, agents will be difficult to trust and likely relegated to peripheral or low-risk use cases. However, if we start reducing the dynamic nature of the system by baking in approved behaviours, we start to chip at what makes AI agents distinct and useful. Until AI systems can guarantee some reliability and repeatability, all their actions will need reviewing and to be able to be undone with minimal cost. One way to bypass this would be if these systems could accept some responsibility for their actions.

Responsibility

When a human makes a mistake, there are legal, contractual, and ethical systems in place for assigning blame, demanding restitution, or making improvements. None of these structures exist for AI agents yet. Today, if an agent makes an error, the ultimate responsibility falls on the user. The onus of verification is placed on the user as LLM providers typically deny any liability with consistent warnings that these systems “can make mistakes”. In theory, future regulations could mandate accountability frameworks for AI. Companies might need to certify their agents, carry insurance, or offer some form of warranty. But there is little incentive for them to move in this direction voluntarily: building trustworthy systems is expensive and denying responsibility is cheap. Until responsibility can be clearly assigned and consequences enforced, users will shoulder all the risk of deploying agents which is likely to limit adoption in high-risk domains.

Proprietary resources

Compounding the responsibility issue is the proprietary nature of most LLMs. The dominant players (e.g. OpenAI, Anthropic, Google) control the models that underlie many of today's AI agents. Relying on proprietary models means that agents are subject to changes in access, pricing, functionality, and alignment at the whim of their corporate owners. An agent that works today might break tomorrow as models are changed and updated to satisfy their own needs. Open-source models are a promising alternative. Projects like LLaMA, DeepSeek, Mistral, and others provide tools for the community to build upon. However, the access to the massive datasets that fuel the largest models is still limited, and operating LLMs at scale still requires significant infrastructure and expertise. Additionally, the issues around copyright in the training data are still largely unresolved. In the long term, reliance on closed systems could leave AI agent providers vulnerable, but running their own infrastructure may prove dissuasive.

Data access

To reach their full potential, AI agents will likely need access to more personal and proprietary data. This may be another barrier as companies have strong incentives to build up their own customer data to decrease the chances of a user leaving their platform. Sharing these results with third-party agents for free seems unlikely. This concentration of data can create a unique advantage as well as monopoly issues, so regulators will need to take this into account to ensure fair competition.

Users themselves may be wary of entrusting sensitive information to systems they do not fully understand and would benefit from more control over their data by storing it for themselves (e.g. in a Solid pod). However, it’s a big responsibility and the public’s interest in such detailed control may not be significant enough yet. The economics of AI services also hint at a future where free access becomes unsustainable. Running large models is expensive, and companies will need ways to recoup costs, so this may lead to user data monetisation or re-use for further AI training.

Economic consequences

As AI agents grow more capable, they become more likely to automate some people’s job wholly or in part. Some workers, particularly knowledge workers, may find themselves displaced faster than anticipated if AI agents become viable professional alternatives. Anyone from writers to software engineers could see core parts of their work taken over by autonomous agents. Unlike previous waves of automation, which often unfolded over decades, AI could transform these industries in a few short years and take people by surprise. Societies may need to respond with radical economic measures (such as Universal Basic Income) to compensate the fallout, but implementing such programs is a political and fiscal challenge in itself. Without proactive planning, AI agents could widen inequality and leave displaced workers with few options or losing parts of what makes their work interesting to them. Employers may have chosen to outsource tasks to AI agents for efficiency or cost-saving reasons, but the consequences may be far-reaching.

Environmental impact

We should not think of agents as being somehow separate from the computing infrastructure that underpins the modern world. In the end, every agent is just some code running on hardware, and that running code carries an environmental payload.  AI requires specialised hardware and sometimes even data centres, all of which consume significant amounts of electricity and water. Even as researchers make models more efficient, overall energy consumption is likely to grow as efficiency gains tend to drive up demand rather than reduce total impact. If AI agents see widespread adoption, their ecological footprint could become a serious concern.

Conclusion

As AI agents become a reality and take on increasing responsibility, the key question isn’t just what they can do, but also what humans want to let go of and retain. Alongside this, there are serious obstacles to overcome: managing risks, ensuring accountability, building trust, and balancing openness with safety.

Making powerful, reliable agents is not cheap, and should they become popular, the environmental cost will not be negligible. They’re also likely to have wide-ranging human and economic consequences if they eventually join the workforce. Users turn to agents when they lack time, knowledge, or skills, but in doing so, they hand over their own agency. Since agency is a finite resource, it’s crucial we divide it between users, agents, and their creators with care.

How we design, deploy, and regulate these systems may not just shape the future of work or technology, but also everyday life.  This hints at a more subtle human cost: automating mundane tasks might sound liberating, but it could also chip away at small pleasures and moments of agency in daily life. Choosing what to read, where to eat, or what project to pursue can sometimes be perceived as a mild annoyance but all of these things also offer opportunities for creativity, learning, and discovery. If agents handle every minor choice, human life could become more efficient, but also more monotonous. By shielding us from friction, agents may lead to fewer opportunities to explore, craft, learn and master, or to simply stumble onto new interests serendipitously.

Ultimately, the future of AI agents is not just a question of what they can do, but of what kind of human experience we want to preserve.