This is KMR, a high-performance map-reduce library. KMR-1.0 is available since 2013-04-26. KMR works on ordinary clusters as well as large-scale supercomputers. KMR source code is available under the BSD license.
Latest release is KMR-1.8.1 (2016-04-25).
KMR is a set of high-performance map-reduce operations in the MPI (Message Passing Interface) environment. It makes programming for data-processing much easier by hiding low-level details of message passing. Its main targets are large-scale supercomputers with thousands of compute nodes. KMR provides utilities other than map-reduce operations to address issues such as accessing very large file-systems, on platforms K and Fujitsu FX10.
KMR is designed to work in-memory and exploit large amount of memory available on supercomputers, whereas most map-reduce implementations are designed to work with external (disk-based) operations. So, data exchanges in KMR occur as message passing instead of remote file operations. The KMR routines work in bulk-synchronous and the most part of the code is sequential, but the code inside the mapper and reducer are multi-threaded.
KMR comes with ABSOLUTELY NO WARRANTY. This wiki also comes with ABSOLUTELY NO WARRANTY. Contents are liable to change.
KMR is a product of RIKEN R-CCS. Part of the results is obtained by using K computer at RIKEN R-CCS.