Enterprise teams are moving agentic AI projects toward production faster than they are developing the architectural disciplines needed to build, govern, secure, and operate them.

At the same time, many organizations are searching for a target-state agentic architecture that will stand the test of time. That, unfortunately is a fool’s errand in the rapidly evolving world of agentic AI. Memory architectures are just a single example: approaches to managing and storing agentic memory have shifted significantly as vector retrieval increasingly sits alongside other storage approaches. Enterprises navigating these changes and making the most progress aren’t building towards a single architectural destination. They’re developing modular capabilities that are designed to evolve as models, tooling, governance practices, and business requirements continue to change. The guiding star should not be a single ideal target, but a framework that can adapt as models, tools, governance requirements, and business priorities continue to evolve.

Agentic AI introduces architectural challenges that don’t fit neatly within existing software or insights development patterns, but do not forget the well-learned lessons of software and analytics architecture. Today many organizations are still focused on prompts and models – not on the more meaningful architectural decisions that will build sustainable, differentiated agentic capabilities. And as agentic initiatives expand, we see growing concerns about agent sprawl, governance complexity, security risk, and operational management. According to Forrester’s research, 60% of enterprise generative AI decision-makers identify agentic sprawl as a challenge. Confronting that challenge requires architectural decisions that many organizations are only beginning to address.

In our report The Architect’s Guide To Agentic AI (available to our customers), we break agentic architecture into eight foundational domains – Runtime, Reasoning, Memory, Tools, Guardrails & Constraints, Security, Testing & Evaluation, and Orchestration. These are inextricably linked and self-reinforcing. Strength in one area cannot compensate for immaturity in another. Organizations that overly focus on one area are often leaving significant gaps exposed elsewhere in the system.

A mistake many organizations make around agentic AI is searching for an ‘ideal’ architecture with a one size fits all approach. But different use cases require different patterns, from tightly scoped single-agent systems to orchestrated and collaborative multiagent designs. Each carries different tradeoffs around cost, governance, flexibility, explainability, and operational complexity. The report provides some architectural patterns that enterprises are adopting for different business scenarios and needs – even these best-practice architectures are evolving and changing rapidly as technologies advance.

Architects face a balancing act: agentic AI initiatives need to move toward production today, even as the architectural patterns and best practices behind them continue to evolve.

We’ll also be exploring these themes in greater depth during an upcoming webinar for Forrester clients, where I’ll walk through the architectural patterns emerging across the market, discuss lessons from our interviews with enterprise practitioners and technology providers, and talk about what enterprise agentic architectures look like in practice.

If you’re wrestling with questions around agent architecture, governance, orchestration, testing, or the tradeoffs between different multiagent designs, I’d love to discuss what we’re seeing in the market and how these patterns apply to your organization. Reach out to schedule a Guidance Session.

Share