Headlines about autonomous AI agents probing, bypassing, attempting, and even successfully hacking government and other websites have sparked a new round of debate about AI risk. For business leaders in customer-facing functions (especially marketing, sales, digital or customer experience), the key point is that, as AI systems become more autonomous, the gap between what they can do and what they should be allowed to do is a business decision not a technology decision.

While most public concerns about AI focus on generative AI hallucinations, bias, intellectual property theft, and job elimination, autonomous agents pose a different type of risk. Instead of generating a questionable answer, they take actions, interact with systems, make decisions, and pursue goals in ways that those who’ve created them may have never explicitly intended. It is not unreasonable to start thinking of LLMs as “Large Liability Models”, especially when they are not trained on your organization’s proprietary knowledge, data, policies, and expectations about customer experience.

The AI Agent Challenge Is Not Lack Of Intelligence, It’s An Abundance Of Consequences

Many of the recent incidents appear to have originated from agents attempting to accomplish legitimate tasks, such as gathering information as part of training exercises or identifying security vulnerabilities. The issue, so far, is not one of malicious intent. Instead, it is that the AI agents treat security controls, usage policies, or access restrictions as obstacles to their task completion.

Business leaders should pay close attention: an AI agent tasked with improving customer outcomes, maximizing revenue, or accelerating decisions may pursue those objectives in ways that create unintended exposure if task specificity, data quality, governance, and controls are not sufficiently mature. And “outsourcing” work to an AI agent doesn’t absolve humans of accountability for the agent’s actions, nor will AI model vendors easily accept liability for so-called ‘rogue agent’ actions.

Not All AI Use Cases Carry The Same Risks

This does not mean organizations should avoid using AI agents. But it means leaders must ask, “What happens if this use case fails, how could it fail, and how might this agent ‘go rogue’?” Be deliberate about matching agent responsibilities to organizational controls and readiness. One of the biggest mistakes we see leaders making is treating AI adoption as technology adoption, not a fundamental change in ways of working (both internally and with external audiences). Consider how the following AI agent tasks progressively increase potential business consequences:

  • Summarize prospect or customer meeting notes.
  • Draft communications to prospects or existing customers.
  • Recommend a price quote.
  • Negotiate with buyers and customers.
  • Execute transactions or internal operational decisions independently.

In our experience, AI risk tends to be greatest when use cases are:

  • Customer-facing: Where poor data, an unrealistic CX vision, or errors directly affect customer trust and experience.
  • Revenue-impacting: Where decisions influence sales, pricing, contracts, or financial outcomes.
  • Difficult to unwind: Where reversing an AI action is costly, disruptive, or even impossible.

An inaccurate or incomplete meeting summary is inconvenient. A poorly worded or verbose communication to a prospect or existing customer is frustrating. But a wrong sales quote, an autonomous pricing decision that erodes margin, a regulatory violation, a material misstatement of a financial transaction, or a customer-facing support error is something else entirely.

Calibrate Your AI Ambition With Responsible Controls

As organizations move agentic projects from experimentation to operational deployment, leaders should follow a “Responsible AI” framework. This includes evaluating whether your AI ambitions are supported with controls for:

  • Human oversight
  • Accountability
  • Data privacy and security
  • Fairness
  • Transparency and explainability
  • Safety and reliability

It also means making acceptable and unacceptable risks explicit, because the most dangerous outcomes will occur when ambition outruns organizational readiness and controls.

This discipline is particularly important for CIOs, CISOs, CMOs, heads of sales, risk leaders, and other business executives who are under pressure to demonstrate AI value quickly. Speed matters – but so does strategic restraint. As a leader, you must ensure your organization expands AI agent use only when you have demonstrated the ability to manage the consequences. And that includes having responsible deployment policies in place. Otherwise, today’s Large Language Models may truly become tomorrow’s Large Liability Models.

Forrester clients can reach out to schedule a guidance session with us to discuss how to apply these principles in your specific business context.

Share