In traditional CC-NUMAs, e.g. DASH / FLASH, Alewife and NUMA-Q, the DSM is managed with a cache line size, and data in other clusters is copied into a cache attached to each processor. The consistency protocol is a simple invalidate policy, the interconnection network is a simple mesh or ring, and the directory scheme is based on one-to-one data transfer.
Although such a mechanism works efficiently in those systems with a limited number of processors, it is not suitable for a system with thousands of processors. For example, a large amount of memory for cache and directory is required. The invalidate policy based on one-to-one data transfer often causes a network congestion when many processors share the same data.
In order to address these problems, the following methods are used in JUMP-1.
- Each processor (SuperSPARC+) shares a global virtual address space with two-stage TLB implementation, and the directory is attached not to every cache line but to every page, while the data transfer is performed by a cache line. Some parts of cluster memory are available as L3 (Level-3) cache which stores the copies of other cluster memory.
- Various types of cache coherence protocols can be utilized dynamically, including not only an invalidate policy but also an update policy. In traditional CC-NUMA systems, an update policy has never been implemented since it requires a lot of packet transfers. However, it can be useful for applications which require frequent data exchange by accessing shared data aggressively. For efficient implementation of such a update protocol, the MBP-light provides dedicated hardware mechanisms which multicast network packets and collect acknowledgment packets.
- Reduced Hierarchical Bitmap Directory schemes (RHBDs) are introduced to manage efficiently directory. In the RHBD, the bitmap directory is reduced and carried in the packet header for quick multicasting without directory access in each hierarchy. The hierarchical structure of the RDT is suitable for efficient implementation of the RHBD.
- Each processor provides a custom sophisticated snoop cache as the L2 cache. Various cache protocols including cache injection are supported by this chip. A relaxed consistency model is implemented using write buffers in the L2 cache. Moreover, a virtual queue message transfer scheme on the shared memory can be efficiently implemented.
