Breaking Through the Memory Wall

The term “memory wall” was first coined in 1994 to define what was becoming an obvious problem at the time: processor performance was outpacing memory interconnect bandwidth. In other words, memory access was limiting compute performance. Almost 30 years later this statement still holds true, especially in memory-intensive applications such as artificial intelligence (AI) where training Large Language Models (LLMs) on big datasets in a reasonable timeframe requires ultra-fast memory bandwidth.

The “Wall”

Many attempts have been made to break through the memory wall. These have failed to gain widespread deployment because they lacked one or more key attributes:

• Memory bandwidth and capacity did not scale efficiently
• Latency inferior to local CPU memory
• Not deployable at scale
• Not easily adopted by existing applications

Memory Wall Breakthrough

Astera Labs’ Leo Memory Connectivity Platform is the first to break through the memory wall with CXL-attached memory. Its hardware-based heterogeneous interleaving capability enables CPUs to spread memory access across the native CPU memory and the CXL-attached memory, dramatically increasing bandwidth while maintaining low latency. For example, a 5^th Gen Intel^® Xeon^® Scalable processor and two Leo controllers can create a single high-bandwidth node of 12, low-latency DDR5-5600 memory channels.

 

Astera Labs’ Leo Memory Connectivity Platform:

1. Increases total server memory bandwidth and capacity by 50% The Leo hardware and firmware have been finely tuned for high performance to efficiently scale server memory with the additional CXL-attached memory channels.
2. Reduces latency by 25%  Native CPU memory latency increases under load, so Leo’s extra memory bandwidth enables lower latency across the heterogenous memory channels when compared to native memory.
3. Uses standard DRAM technology already deployed at scale  Data center integrators don’t need to qualify a new memory technology. Leo uses the same DDR5 memory modules used for native CPU memory.
4. Expands memory in both new and existing systems seamlessly  Leo hardware-based interleaving doesn’t require software changes to manage different memory nodes. Existing applications can expand memory capacity—just plug & play.

Proven Performance

In benchmarks performed in partnership with industry leaders, Astera Labs’ Leo Memory Connectivity Platform boosts memory capacity and bandwidth up to 50% while lowering latency by up to 25% in memory-intensive applications such as AI, computational fluid dynamics, data warehousing, and EDA.

Astera Labs leads the way in memory connectivity and PCIe/CXL interconnect technology. Discover how Leo CXL Smart Memory Controllers as well as Aries PCIe/CXL Smart Retimers can help you break the memory wall and advance your designs.

Resources:

• Video: Demonstration of Leo Breaking Through the Memory Wall at Intel Innovation
• Press Release: Astera Labs First to Break Through the Memory Wall with the Industry’s Highest Performance CXL Memory Controllers

 

References:

1. Wulf WA, McKee SA. Hitting the memory wall: Implications of the obvious. ACM SIGARCH computer architecture news. 1995 Mar 1;23(1):20-4.
2. Gholami A, Yao Z, Kim S, Mahoney MW, Keutzer K. AI and Memory Wall. Medium [Internet]. 2021 Mar 29. [cited 2023 Sep 27]