In traditional servers, memory is directly connected to a specific CPU or GPU (i.e., locked behind the host) and can result in over-provisioning of memory resources when applications are not using the available memory. When the memory is over-provisioned to a specific host, the memory is now stranded and cannot be accessed by other hosts, thereby increasing data center costs. In addition, when memory is locked behind a host, the data being processed by the application needs to be copied through high latency interconnects if a different CPU or GPU needs access to the data.

Memory pooling allows multiple hosts in a heterogenous topology to access a common memory address range with each host being assigned a non-overlapping address range from the “pool” of memory resources. Memory pooling allows system integrators to dynamically allocate memory from this pool, which reduces costs by reducing stranded memory and increasing memory utilization. Memory pooling is part of a growing trend for resource disaggregation or composability for heterogeneous solutions.

Memory sharing allows multiple hosts in a heterogeneous topology to access a common memory address range with each host being assigned the same address range as the other host. This improves memory utilization similar to memory pooling, but also provides an added benefit of data flow efficiency since multiple hosts can access the same data. With memory sharing, coherency needs to be managed between the hosts to ensure data is not overwritten by another host incorrectly.