Buy the Rack Once, Pick the Accelerator Later: How Orion and Hyve Aim to Speed Up AI Inference Capacity

Brian Deng, Product Marketing, AI Platform Solutions

Inference capacity is one of the most sought-after resources in the data center, yet supply chain constraints make it difficult for neoclouds and enterprises to secure accelerators in time. The challenge is sequencing. Operators commit to power and floor space long before they know which accelerator and rack architecture they will use. Tie that early investment to a single platform, and the bet gets expensive.

Orion, the compute-agnostic rack-scale reference architecture Astera Labs unveiled at the 2026 Open Compute Project Global Summit, starts from a different premise: Establish a common rack architecture while keeping accelerator choices open. At the summit, Sanjay Gajendra (President and COO, Astera Labs) and Dr. Dimitris Ziakas (CTO, Hyve Solutions) walked Brendan Burke of Futurum through how it works on SixFive Media’s “On the Road” series. Hyve Solutions is one of the first partners to build the Orion rack, so the conversation also explores what it takes to turn a reference design into a customer-ready system.

Watch the full conversation above

What to Take Away

1. Treat the rack as the constant and the accelerator as the variable.

If you run a neocloud or build enterprise inference capacity, this is the idea to hold onto. Orion provides a common architecture for connectivity, power, cooling, and software, with pre-validated CPU and accelerator configurations. Customers can select supported options based on cost, lead time, and workload, including PCIe add-in-card configurations. Built on open standards, Orion is designed to work across leading inference accelerators and CPU ecosystems.

Sanjay framed the goal as making the rack GPU-agnostic so customers can upgrade accelerators over time. The result: teams can build familiarity with a common architecture while keeping sourcing options open.

2. Look at who builds the rack as well as who designs it.

Astera Labs co-designs and co-validates Orion with ecosystem partners. Racks and systems are manufactured, licensed, and sold through OEM and ODM partners, with availability expected to begin in Q1 2027.

Hyve’s pitch is that it already does this work every day: It designs and ships rack-scale AI systems, with storage, compute, and fabric under one roof, and its TD SYNNEX parent brings procurement scale across vendors. For customers, that could mean working with a partner to configure an Orion system around their workload and sourcing needs.

3. Count the integration work you will not have to repeat.

Orion leverages several production-grade reference designs that stem from the rack architecture and ship as six validated modular building blocks: the rack, compute tray, fabric tray, memory tray, link board, and compute board. COSMOS Rack Manager and COSMOS Fabric Manager software come pre-bundled and handle orchestration, telemetry, and diagnostics. Every new accelerator platform requires another round of integration and validation for ODMs and OEMs. Starting from a validated design shortens that path, and accelerates time to deployment.

4. Pay attention to memory, because long agentic sessions can push KV cache into terabytes.

Orion-MB, the Memory Board, enables fabric-attached memory that sits on the same PCIe® fabric as the GPUs, so memory is reachable without protocol translation. Dimitris pointed out that serving KV cache over a large-radix PCIe fabric can free up GPU cycles, a direct lever on tokens per dollar.

Sanjay said that depending on the workload, the architecture can improve inference performance by 25 to 80 percent, and he credited three design choices:

5. Plan for growth on open standards.

Orion is designed to scale from 8 to 160 accelerators per pod. Sanjay described how ten compute systems, each with eight GPUs, can form an 80-GPU rack; connect multiple racks together to create a 160 GPU pod. The roadmap extends to additional open interconnects, including UALink™, and he pointed to liquid-cooled versions and larger clusters. And, as the saying goes: “We’re just getting started.”

FAQ: The Hyve Solutions and Astera Labs Partnership

Q: What does Hyve bring to Orion?

A: System design, build, and delivery at rack scale. Hyve combines storage, compute, and the Astera Labs fabric into a finished rack, and its TD SYNNEX parent adds global procurement across vendors.

Q: What does Astera Labs bring?

A: Key building blocks, including validated boards, the fabric tray, and enhanced COSMOS Fabric Manger and Rack Manager software. This gives customers and partners the flexibility to design a low-latency inference solution and KV cache acceleration architecture.

Q: How does Hyve turn the reference design into a customer rack?

A: Hyve starts from the pre-validated Orion fabric, pairs it with the accelerator and CPU, and composes the rack around customer needs. Dimitris described the result as scale with a fast time to market.

Q: What mission do the two companies share?

A: Getting inference capacity online faster on open, interoperable infrastructure. Dimitris said opening up scale-up “really unlocks how you can compose a rack solution,” combining the best of different compute and storage vendors.

Q: How does the partnership help with part availability?

A: Orion’s accelerator flexibility and Hyve’s multi-vendor procurement give customers more ways to source capacity. Dimitris also noted that KV cache over a PCIe fabric can unlock GPU cycles in parts customers already have.

Q: When can customers get Orion racks?

A: Orion reference and system designs are on track for availability through Astera Labs’ OEM and ODM partners beginning in Q1 2027. Interested parties can connect with Astera Labs for commercial inquiries.

Learn more on the Orion product page and in the Orion announcement press release.