Distributed Shared Memory for Multi-GPU Computing

GPU applications, abstractions, and APIs today are primarily designed for a single GPU. Both major discrete GPU vendors support multiple GPUs in a system with special-purpose connectors (Crossfire and SLI), but these communication mechanisms are neither general nor programmable.

For large-scale GPU applications such as visualization and GPU computing, scaling to multiple GPUs is essential. Existing multi-GPU programs are primarily limited to either trivial parallelization (in which the problem is partitioned across multiple GPUs) or a fixed pipeline of GPUs. Our work proposes a more generalized method of communication.

We propose a distributed shared memory abstraction in which each GPU can see a global address space that is distributed across all GPUs. If one GPU needs data from another GPU, it can request it from that GPU, and if one GPU writes data to its local memory, it invalidates it in other GPUs. Our implementation makes this process (mostly) transparent to the user so that existing single-GPU applications can run across multiple GPUs (for example, by partitioning the display across multiple GPUs).

Unfortunately this abstraction is not performance-optimal, but the most important part of the work is identifying the hardware and software features and mechanisms necessary to make it more efficient.

Adam Moerschell and John D. Owens. Distributed Texture Memory in a Multi-GPU Environment. Computer Graphics Forum, 27(1), March 2008.
http://graphics.idav.ucdavis.edu/publications/print_pub?pub_id=918.

Adam Moerschell and John D. Owens. Distributed Texture Memory in a Multi-GPU Environment. In Graphics Hardware 2006, pages 31-38, September 2006.
http://graphics.idav.ucdavis.edu/publications/print_pub?pub_id=886.