Compute-Agnostic Designs for Token-Efficient AI Factories
Read more Read more 
Orion Rack-Scale Reference Architecture
Choose Your GPU/CPU. Scale Your Inference Platform.
Modular Rack Architecture for 8 to 160 GPUs
Get inference racks online fast with six production-grade reference designs.
Orion Rack
Reference Architecture- Compute-agnostic reference design; multiple GPUs/XPUs/CPUs
- PCIe® 6 scale-up architecture that aggregates up to 160 accelerators per pod
- Modular architecture that supports scale-up for accelerators with fabric-attached memory for KV cache acceleration
Orion-CT
Compute Tray- Joint design co-developed by Astera Labs and ODM/OEM partners
- Compute-agnostic; tailor accelerator selection to your workloads
- 8 accelerators scaled up within platform
Orion-FT
Fabric Tray- Common scale-up fabric tray for accelerators and memory pooling
- Non-blocking bandwidth across fabric
- Enables low-latency solutions through In-Network Compute and Hypercast™
Orion-MT
Memory Tray- Expands cache and data resources for AI systems
- Fabric-attached memory for low-latency, high-volume KV cache offload close to the GPU
- Enables data locality, workload-aware resource placement, and software control
Orion-CB
Compute Board- Scale-up 8 accelerators + peripherals under a single fabric switch
- Supports scalable fabric expansion from platform to rack-scale
- Enables low-latency solutions through In-Network Compute and Hypercast™
Orion-LB
Link Board- Extends fabric connectivity from tray to rack-scale
- Supports modular and scalable expansion across compute and memory trays
- Leverages Astera Labs’ industry-leading signal connectivity portfolio
COSMOS Rack and Fabric Manager
COSMOS orchestration software manages the connectivity system across device, link, fabric, rack, and fleet levels.
Together, COSMOS Rack Manager and COSMOS Fabric Manager enable discovery, telemetry, diagnostics, RAS, health monitoring, fabric routing, firmware/lifecycle management, and optimization APIs.
Resources
Read more Read more First Look: Orion Rack-Scale Reference Architecture
Get the first look at Orion, a compute-agnostic, production-grade reference architecture for token-efficient inference racks.
FAQ
Customers can choose a best-of-breed accelerator and CPU based on their needs (workloads, availability, budget, etc.) then pair them with Orion’s deployment-ready and customizable COSMOS Rack and Fabric manager stack. To build a rack or components from there, they work with an Astera Labs OEM/ODM partner to purchase their desired build. Interested parties may connect with Astera Labs for commercial inquiries.
Neoclouds and enterprise AI operators both need to source inference compute at a controlled price point. The neocloud business model depends on being able to offer inference-driven cloud AI capacity to their customers at a low cost, while enterprises are standing up their own data center to take greater ownership of their AI spend. Inference workloads can run efficiently across many different accelerators, but these two customer segments often struggle to secure or afford the newest, high-end compute. Mixing and matching compute across their deployment can help them consistently deliver competitive tokens per dollar, regardless of which accelerator is easiest to secure at a given time.
While the highest-performing single-vendor racks deliver excellent performance, they often carry a high cost, are supply-constrained and difficult to source, and often include more memory or specialized hardware than lighter-weight inference deployments require. Orion is purpose-built for enterprise inference economics, letting neocloud and enterprise operators pick accelerators based on cost, availability, and optimized workload, deploy at a much lower price point than a fully-loaded single-vendor rack, and move toward outcomes-based, dollar-per-token pricing rather than committing to a single, fixed hardware configuration.
No. Astera Labs designs and validates the Orion reference architecture and its component sub-systems. Complete racks and individual systems are manufactured, licensed, and sold through OEM/ODM partners.