Nearly every assumption in conventional storage topology design comes from an era when the CPU issued the reads. NVIDIA’s SCADA architecture removes the CPU from the storage metadata queue management, allowing thousands of GPU threads to issue fine-grained I/O requests directly to storage at massive parallel scale.
That raises a more practical question for AI infrastructure: If thousands of GPU threads are issuing I/O directly, what must the fabric provide to keep AI workloads running efficiently? The fabric has changed from a passive interconnect to an observable, congestion-aware data-movement layer that must deliver high bandwidth and telemetry across GPUs, SSDs, and switches. This visibility allows developers to correlate fabric behavior with storage performance and identify the bottlenecks limiting throughput.
Why GPU-Initiated Storage Changes the Fabric Requirement
SCADA stands for Scaled Accelerated Data Access,1 a programming model that lets GPU threads initiate and manage storage I/O from inside CUDA kernels, moving the main data path off the CPU. In practice it implements an NVMe driver inside the GPU, takes ownership of the block devices, coalesces random I/Os into a read-through cache in HBM / GDDRx, and batches whatever remains across PCIe® to local NVMe or a remote SCADA server.
SCADA is designed for workloads that generate massive numbers of small, random storage accesses.2 CPUs are optimized for relatively serial control flow and can become the bottleneck when trying to issue millions of fine-grained I/O requests fast enough to fully utilize high-bandwidth PCIe and NVMe infrastructure. GPUs are built for massive parallelism. By allowing thousands of GPU threads to drive storage requests concurrently, SCADA turns the I/O problem into a parallel workload, keeping more requests in flight, increasing achievable IOPS, and reducing the risk of the GPU waiting on data. The smaller and more random the accesses, the greater the potential advantage.
That request rate also puts pressure on the drive side, demanding IOPS well beyond what conventional storage stacks were built to deliver. A PCIe 5 x16 GPU pulling 50GB/s at 512-byte granularity implies roughly 100 million IOPS, which today requires 15 to 20 drives to feed a single accelerator. This exposes a significant opportunity to unlock untapped SSD bandwidth through more efficient small-block I/O, maximizing storage performance and GPU utilization.
The AI Workloads That Actually Drive SCADA
AI applications that benefit from SCADA architecture share one signature: irregular, fine-grained, high concurrency reads against a dataset too large for HBM / GDDRx or host DRAM. Two such workloads deserve special callouts:
Graph Neural Network (GNN) Training
This is the clearest SCADA use case. The graphs involved carry enormous numbers of edges and embedding tables that dwarf available memory, and training walks that structure one hop at a time, so fine-grained fetches3 retrieve only the slices needed instead of loading the graph wholesale.
Retrieval-Augmented Generation (RAG)
Query vectors and stored chunk vectors for vector search applications sit at unpredictable locations across a very large index, which is precisely the random-access pattern SCADA was designed around. Retrieval-augmented generation presents an intensified version of this problem: Its knowledge bases and embedding indexes routinely exceed what GPU and host memory can hold and augmentation must stay real-time rather than bottlenecking on host-mediated lookups.
Both of these use cases illustrate why the pattern is so unforgiving. Both follow a structure whose next access depends on the result of the current one, limiting the effectiveness of prefetching and making it harder to keep the GPU fed.
The same signature shows up in other AI workloads. Petabyte-scale data lakes benefit from view-based access that lets a GPU work against the data without staging it into memory first. Real-time inference pipelines need fine-grained access spread across very large datasets without leaning on host memory to stage it. And graph analytics and recommendation systems, with their irregular pointer-chasing traversals, were among the original motivating workloads for the BaM (Big Accelerator Memory) research that SCADA builds on.
One Switch Can Do It All: Scorpio 320L P-Series at the Center of SCADA-Style Architecture
We assembled in our lab a GPU-accelerated memory and storage server that unifies compute, storage, and networking under a single switch. Inside it:
- 1 AMD EYPC Turin
- 48 Micron 9650 PRO E1.S PCIe 6 SSDs
- 2 NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs
- 2 NVIDIA ConnectX-8, and, at the center of the topology
- A single Astera Labs Scorpio 320L P-Series Fabric Switch
Figure 1: Astera Labs SCADA validation platform
The Scorpio 320L PCIe switch enables more GPUs and SSDs to connect through a single switch, expanding storage bandwidth and capacity beyond what native host PCIe connectivity can support.
Figure 2: Dual switch vs. high-radix switch
The dual-switch approach (Figure 2, right) is a valid way to run SCADA: two smaller-radix switches each pair a CPU with its own GPU and SSD pool, and because SCADA maps requests client-side, every GPU-to-SSD path stays within one switch domain. This yields a deterministic GPU-to-SSD mapping and predictable performance.
A single Scorpio 320L switch eliminates that partitioning, creating one non-blocking fabric. All 48 SSDs, both GPUs, and both NICs connect through a single coherent, software-defined switch; every drive-to-GPU path is one hop, and capacity can be pooled and reassigned across GPUs as workloads change instead of being locked into fixed domains at build time. That flexibility matters most when accesses can’t be pinned to a local domain. In GNN training and RAG, datasets are striped across many drives, and each fetch depends on the previous result.
Proving It: End-to-End Observability With COSMOS and Nsight
Performance claims are only as credible as the telemetry behind them. That is why we’ve developed an Astera Labs COSMOS software plugin for NVIDIA Nsight Systems, paired with our COSMOS Developer Kit, so customers can fine-tune performance themselves rather than taking our word for it.
The integration lets performance engineers correlate telemetry from the host, GPUs, storage, networking, and our PCIe switch on a single unified timeline. We validated the plugin in our lab against the Scorpio 320L switch, CPU, GPUs, and SSDs, so the numbers reflect real fabric behavior under load rather than idealized benchmarks. It surfaces:
- Deep PCIe link observability, including per-port throughput and utilization
- End-to-end SSD-to-GPU data-path visibility with GPU initiated storage access
- Corresponding storage I/O, CUDA activity, and GPU utilization
Figure 3: NVIDIA Nsight enabled with COSMOS plugin
This matters more for SCADA-style platforms than for conventional storage servers. When the GPU owns the control path, the interesting failure modes hide in the interaction between CUDA activity and fabric behavior, and a profiler that sees only one side of that boundary cannot find them. Putting switch-level counters on the same timeline as GPU and storage activity gives users a granular, verifiable view of data movement under load, which lets them identify bottlenecks, optimize data paths, and maximize system performance.
A Clear Case for Single-Switch, Full-Visibility Fabric
When we talk about SCADA, we’re no longer asking whether GPU-initiated storage works. We’re asking what an ideal fabric underneath looks like, and how that fabric should be monitored and managed.
Our testing points to a single-switch fabric to eliminate interswitch latency, with enough radix to keep every drive a single hop from every accelerator and headroom left to grow. And a COSMOS plugin for Nsight is the difference between asserting that a single-switch fabric removes the interswitch bottleneck and showing it, port by port, under real load.
References:
- NVIDIA, “Expanding AI Storage Access with NVIDIA cuObject and the NVIDIA SCADA Server SDK,” 2026.
- Z. Qureshi et al., “BaM: A Case for Enabling Fine-Grain High Throughput GPU-Orchestrated Access to Storage,” ASPLOS, 2023.
- S. Park et al., “Accelerating Sampling and Aggregation Operations in GNN Frameworks with GPU Initiated Direct Storage Accesses,” PVLDB, 2024.
Appendix:
This appendix provides a summary of COSMOS plugin and Nsight Systems capabilities.
End-to-End Observability with COSMOS and NVIDIA Nsight Systems
SCADA-style architectures can make performance bottlenecks difficult to isolate. Low GPU utilization may result from storage latency, PCIe congestion, switch contention, or an imbalanced drive-to-GPU data path.
The Astera Labs COSMOS plugin for NVIDIA Nsight Systems correlates telemetry from the CPU, GPUs, NVMe SSDs, networking devices, and PCIe fabric on a shared timeline. It visualizes:
- Per-port PCIe bandwidth and utilization
- PCIe link errors and switch temperature
- SSD-to-GPU data movement
- Storage I/O and request activity
- CUDA activity and GPU utilization
- Fabric traffic during active workloads
By aligning switch-level counters with CUDA and storage activity, engineers can identify GPU stalls, locate congested or underutilized links, compare traffic across drives and accelerators, and optimize topology, workload placement, and I/O behavior.
Nsight Systems Plugin Documentation
For more information, see the Nsight Systems Plugins documentation. The documentation identifies the Astera Labs Scorpio plugin as providing per-port bandwidth, PCIe link-error counters, die temperature, and GPU-workload correlation.