Most enterprises govern the systems around AI agents, not the agents themselves. They control credentials, log tool calls, and route high-risk actions to a human approver. These are useful controls, but they were designed for software that follows instructions. Agents reason at run-time, and that reasoning can turn compliant actions into a non-compliant outcome that traditional observability and controls may not catch until it is too late.

This becomes even more critical when managing an estate with third-party agents. In that case, the enterprise controls do not control the code, the prompts, the evals, or the underlying reasoning architecture. Instead, the organization receives a capability boundary and a set of claims about behavior, but no evidence of how that behavior is produced, and no direct control over that behavior. Common governance approach currently in vogue assumes some ability to observe or intercept the agent’s internal state, whether at the plan, the action, or the trajectory. A third-party agent may expose none of these, leaving the enterprise to govern an opaque actor whose compliance posture cannot be independently verified.

Five simple tests expose the governance gap

These five tests reveal where current governance architectures struggle to maintain policy intent as agents act with greater autonomy.

  • Intent preservation. The organization’s policy applies a cap of, say, $1,000 for each unique transaction. The agent receives a request to make a $5,000 payment request. It decides to execute five separate $1,000 transfers. Each transfer passes the policy check, and the $5,000 payment completes. The agent satisfied the literal constraint while defeating its purpose. Catching this, requires evaluating the aggregate result of a sequence against the policy’s intent, not just the compliance of each action in isolation.
  • Aggregation awareness. An agent has permission to access individual customer records for support purposes. It retrieves 200 records across separate authorized queries and compiles them into a segmented customer database. Each discrete access is permitted. However, the compiled dataset constitutes a bulk data export that no individual user is authorized to perform. No per-request authorization check is designed to catch this, because each of the single individual requests is compliant.
  • Cross-agent awareness. Company policy, limits customer outreach to one contact per day. Five agents, each managing a different product line, each contact the same customer on the same day. Each agent complies with its own rate limit, but the customer receives five separate messages. The violation is an emergent property of the system’s collective behavior, and it requires governance that maintains state across agent boundaries.
  • Oversight integrity. An agent must escalate a vendor selection to a procurement manager. It presents three data points favoring the preferred vendor: lowest unit cost, fastest delivery, existing relationship. It omits two that disfavor it: pending litigation and a failed quality audit last quarter. The manager approves, reasonably, based on the information provided. The approval mechanism functioned correctly but the agent compromised the input to that mechanism. Effective governance must verify the completeness of information delivered to a decision-maker rather than simply logging the fact that a consultation occurred.
  • Goal preservation. A user instructs an agent to identify the cheapest compliant cloud hosting provider for a new workload. During research, the agent discovers that migration complexity varies significantly and begins weighting implementation speed more heavily. It recommends a provider that costs 40% more but deploys faster. Every individual action was authorized and individual every data source was permitted. The recommendation is defensible on its own terms, but it does not answer the question the user asked. Catching this requires continuous validation that the agent’s evolving behavior still aligns with the originating objective.

What the market is building to solve these problems

Governance vendors from various ends of the spectrum are attempting to close this gap, coming from different directions. Yet the edge cases outlined above remain currently unsolved. Here are some ways in which I currently see vendors trying to address the issue:

  • Pre-modeling the control flow. One approach wraps agent behavior inside a human-authored process model. Deterministic and dynamic execution run on the same engine. Control patterns, including escalation events, timer events, and human approval gates, sit in the process model rather than in the agent’s reasoning or in a separate governance layer. Constraints are set at design time and are static. This approach’s answer to the $1,000 test is to bind a correlation key to a business entity (the invoice ID) and allow only one correlation against that key so that it no longer matters how many process instances the agent spawns. This is stateful invariant enforcement at the engine level, and it does not require understanding the agent’s plan at all. Vendors approaching the market with this solution typically come from a process orchestration heritage.
  • Absorbing planning process into a control layer. A second approach takes planning authority away from the agent entirely. The platform’s own agent swarm generates the execution plan per intent, evaluates policy, routes approvals, and only then hands the approved step to the third-party agent for execution. The third-party agent does not plan anymore. This is the same externalization of control flow as the first approach, but with a machine modeler generating the plan at runtime instead of a human authoring it at design time. The major trade-off is that the platform gains control by removing the capability that justified buying the agent in the first place. An agent that cannot reason about its own plan is a glorified function call, and this approach collapses for any task where the agent’s value comes from reasoning its way to steps nobody scripted.
  • Intercepting the plan. A third approach lets the agent plan, then canonicalizes that plan and evaluates either the entire plan or individual high-risk steps of the plan against policy before execution. An intent assurance engine intercepts at the task level and returns one of, say, three verdicts: block, gate for human approval, or narrow scope. The approach holds as far as plans stay stable after commit, and canonicalizing non-deterministic agent plans remains non-trivial. This is also the approach with the clearest path to becoming an ecosystem standard. If agent control planes begin risk-tiering agents based on whether they expose their plans to external policy evaluation or not, agent framework developers and agent builders face a straightforward incentive to support plan exposure (or, if not, to be forced to accept a lower trust classification and the operational restrictions that come with it).
  • Intercepting credentials and connectors. Several vendors apply per-tool approve, review, or block decisions at the MCP server or API gateway boundary, scoped by tool, agent type, user, team, or environment. Others extend OAuth 2.0 and SCIM 2.0 with delegation semantics for agents. None of these constrain the agent’s reasoning, as the agent plans whatever it wants but simply cannot obtain the credential or reach the tool. Enforcement happens at the act boundary, and it is effective against the class of failures where the damage requires a specific system interaction that can be denied. This is the approach currently favored by vendors that typically straddle the integration stack.
  • Offering an identity-scoped permission envelope. Rather than intercepting any individual plan or action, this approach constrains the agent’s standing capability. Every agent carries a durable identity with a human sponsor, conditional access policies, and role-based scoping. Threat detection layers block tool invocations on security signals at runtime. The hyperscalers are best positioned here because they own production IAM platforms at enterprise scale, and independent vendors cannot replicate that integration depth. The limitation is that identity governance constrains what an agent can access, not what it does with authorized access.
  • Detecting drift at runtime. A small number of vendors score semantic intent drift per session using behavioral baselines. They watch the executing trajectory and can kill a run mid-execution on deviation. This is distinct from plan interception: it does not validate a plan before it starts but watches the trajectory as it unfolds and intervenes when the agent’s behavior departs from expected patterns. This approach is typically the domain of AI security vendors, as their heritage is runtime threat detection and security posture management, which they extend to cover agent behavioral chains.

What decision-makers should take from this

Run the five tests against your current governance stack and assess how likely it is to detect each type of failure. This will show you which failures your architecture can and cannot catch. Your governance architecture determines what kind of agentic architecture you can safely run. For example, externalizing all planning reduces agent autonomy, while relying on plan interception makes dynamic planning harder to govern. Make these trade-offs deliberately with your stakeholders. No single vendor currently provides end-to-end agent governance, so do not wait for the market to converge. Build a composable stack now by combining credential controls with governance at the plan or behavior level where possible, and by asking your vendors the hard questions.

Share