GPU Flexibility Is Vital for Neoclouds and Enterprise AI in a Competitive Inference Market
The AI industry landscape is shifting as new players spin up inference compute to meet a ceaseless demand. JLL reports that neoclouds make up 11% of data center capacity demand in 2026,1 and Gartner projects that number to rise to 20% by 2030.2
Neocloud providers are betting that low-cost, inference-driven capacity will create profitable AI factories that can disrupt the existing cloud AI market. They want to offer their customers predictable, outcomes-based pricing around tokens per watt. At the same time, enterprises are standing up data centers to manage their own AI needs with secure, cost-controlled, dedicated capacity.
Both of these groups find themselves bottlenecked from scaling by compute availability. Production-grade architectures are limited to a few closed rack-scale designs at price points unsustainable for businesses of this size, and building rack-scale systems with alternative compute typically involves lengthy, costly validation timelines. For neoclouds, this threatens the viability of their inference-delivery business model. For enterprises, this delays the availability of their in-house AI infrastructure.
Everybody needs inference capacity ASAP today, especially organizations who are standing up cloud infrastructure from scratch. In this environment, flexibility is the greatest competitive advantage: Getting AI inference capacity online fast must mean decoupling rack designs from accelerator selection.
The Orion Rack-Scale Reference Architecture is our way of meeting this moment. We welcome neocloud and enterprise customers to bring their own XPU, choose their components, and get hyperscale-grade inference online fast.
The Orion Vision: Bring Your GPU, Build and Deploy Fast
Orion is a set of reference designs that span board-level to rack-scale, designed around a simple principle: Infrastructure should adapt to compute, not the other way around.
By separating the rack architecture from any single CPU, GPU, or XPU, Orion creates a common foundation across compute, fabric, memory, and management. Neocloud and enterprise customers can choose their accelerator based on suitability to specific workloads (or simply based on availability) while preserving a consistent path to integration, qualification, deployment, and operations.
Orion is available as a full, deployment-ready Orion Rack system or as any combination of six modular designs that span the compute, memory, connectivity, and software layers of the rack-scale system. At the heart of each of these designs are the industry-leading interconnect components on which we’ve built the Astera Labs name through millions of hyperscale deployments.
How Orion Is Changing the Game for Inference-Ready Infrastructure
These open, modular reference designs chart a new course for AI infrastructure at rack scale, moving the industry:
- From lock-in to choice: A compute-agnostic architecture allows customers to select best-of-breed accelerators based on workload, availability, and economics.
- From repeated integrations to rapid deployment: Production-grade reference designs reduce the need to conduct rack-level integration and validation for every new XPU.
- From device management to rack intelligence: Enhanced COSMOS Rack and Fabric Manager extends visibility and control from silicon and fleet diagnostics to performance monitoring and management at the rack and fabric level.
- From raw performance to token economics: Scalable fabric, In-Network Compute, Hypercast™, and fabric-attached memory are designed to improve utilization and support more efficient inference.
The Business Impact on Neoclouds and Enterprise AI
Orion changes the strategic equation for newer cloud AI operators in four key ways:
- Faster time to deployment: A validated, repeatable architecture can compress the path from accelerator availability to production.
- Greater supply optionality: Organizations gain flexibility to deploy across a broader set of accelerators instead of waiting on a single platform.
- Improved capital productivity: Better fabric efficiency and memory placement can help extract more tokens per dollar from deployed compute.
- A scalable operating model: Unified rack and fabric management simplifies the transition from individual systems to infrastructure at scale.
A Roadmap to Deployment at Scale
Orion began with a practical observation: AI compute demand – driven by inference workloads increasing in quantity and token consumption – is moving faster than infrastructure can adapt. Neocloud and enterprise AI customers trying to scale at this precise moment need more compute, more choice, and faster deployment, while fragmented architectures kept adding time, complexity, and threats to their bottom line.
In keeping with our consistent commitment to an open AI ecosystem, our solution to this challenge is an open-by-design, modular approach that leverages the advantages created by ecosystem diversity. Orion is brought to market through co-development and collaboration between Astera Labs and our ODM/OEM partners to create a powerful mix of industry-leading interconnect technology built to rack scale and delivered on predictable timelines.
Our ambition is clear: Enable customers to choose the compute they need, deploy it faster, and extract more value from every rack.
Power Performant, Profitable AI Your Way With GPU-Agnostic Architecture
Each of Orion’s component parts is a modular unit capable of accelerating time to market for a rack deployment with production-grade, hyperscale-tested, industry-leading interconnect.
But taken as a whole, the Orion Rack-Scale Reference Architecture is itself a roadmap for rapid deployment at scale of the token-efficient inference capacity necessary to meet the coming era’s skyrocketing demand. With compute continuing to diversify and specialize for emerging workloads, neoclouds and enterprise AI operators can’t afford to start from scratch every time they want to bring new compute online.
AI performance is scaling. Inference demand is scaling. Our vision as a company, and an industry, must scale to keep pace.
To see a full build of the Orion Rack-Scale Reference Architecture and learn more about how it can be deployed at AI factory scale, come visit us in Booth B34 at the 2026 OCP Global Summit.
References:
- JLL, “North America Data Center Report,” accessed October 2026, https://images.hello.jll.com/Web/JLLAmericas/%7B8e00602e-eb6f-4411-bfa9-1653154a936e%7D_JLL_North_America_Data_Center_Report_-_Midyear_2026_(Condensed).pdf.
- Gartner, “Gartner Predicts Neocloud Providers Will Capture 20% of the $267 Billion AI Cloud Market by 2030,” accessed October 2026, https://www.gartner.com/en/newsroom/press-releases/2026-06-23-gartner-predicts-neocloud-providers-will-capture-20-percent-of-the-267-billion-dollar-ai-cloud-market-by-2030.