It’s prediction season again and 2027 will bring a reality check to infrastructure strategies. AI is moving into mission-critical production, geopolitical uncertainty is testing global operations, and organizations are pushing more applications and intelligence to the edge. For infrastructure and operations (I&O) leaders, that means rethinking how they source AI infrastructure, build resilience, and invest at the edge. Three I&O predictions stand out:   

  • A quarter of AI infrastructure RFPs will require non-NVIDIA accelerator options. NVIDIA will remain the default platform for frontier model training and large-scale inference, but enterprises will increasingly seek alternative AI accelerators to reduce costs and mitigate supply chain, sovereignty, and geopolitical risks. Rather than replacing NVIDIA, organizations will segment workloads across accelerator types based on specific requirements. Hyperscalers are already expanding AI infrastructure options with proprietary silicon such as AWS Trainium, Google Ironwood TPU, and Microsoft Maia. Infrastructure leaders should incorporate heterogeneous AI infrastructure requirements into their sourcing and architecture plans. 
  • Thirty percent of large enterprises will build private AI infrastructure for inference. As agentic AI moves into production, enterprises will need dedicated AI infrastructure for sensitive, latency-critical, and high-volume inference workloads. Rising token costs and more complex agentic workflows will further drive demand for infrastructure optimized for AI inference economics. Leaders should start now by developing an enterprise inference strategy that separates production inference from training and aligns infrastructure with workload needs. 
  • Infrastructure investments will shift by 20% to industry-specific edge devices. Organizations in industries such as manufacturing, healthcare, and retail will increasingly run AI workloads and analytics at the edge, closer to where decisions need to happen. Small language models, domain-specific AI models, robotics, and vision language action systems will move onto operational technology devices to address latency, resiliency, and connectivity challenges in environments such as factories, hospitals, ships, and oil rigs. Infrastructure decision-makers should build an industry-specific computing strategy and invest in smaller form factors and capabilities that can improve workload efficiency while bringing AI closer to the point of action. 

AI will become just another part of I&O. Getting there will require infrastructure that is more distributed, specialized, and adaptable. Technology leaders must decide where workloads run, which technologies support them, and how to keep infrastructure resilient as business and geopolitical conditions change. 

Take a deeper dive into these three predictions as well as two others, “Sixty percent of organizations will fund a context infrastructure initiative’ and “Fifteen percent of large enterprises will establish agent portfolio rationalization” in the full Predictions 2027: Technology Infrastructure And Operations report.  

Forrester clients can also explore the rest of our Predictions 2027 research on the Predictions 2027 hub. 

Share