Welcome to KMR (K Map-Reduce)#
This is KMR (K Map-Reduce), a high-performance map-reduce library. KMR-1.0 is available on the K Computer in "/opt/aics/kmr" now (2013-04-26).
KMR is a set of high-performance map-reduce operations in the MPI (Message Passing Interface) environment. It is to make programming for data-processing much easier by hiding low-level details of message passing. Its main target is large-scale computers with thousands nodes, K and Fujitsu FX10. On those platforms, KMR provides utilities for the map-reduce operations to address issues such as accessing large file-systems. But, KMR works on clusters as well.
KMR assumes large amount of memory and designed to work on-memory, whereas most map-reduce implementations are designed to work with external (disk-based) operations. So, data exchanges in KMR occur as message passing instead of remote file operations. The KMR routines work in bulk-synchronous and the most part of the code is sequential, but the code inside the mapper and reducer are multi-threaded.
- Project Overview and other Activities of the Team:
- Overview and API Document:
- It is a Doxgen generated document, included in the installation.
- Overview and API Document (Newer, corrected, for the next release):
- Documentation in the distribution is late, and it is placed here for late breaking.
- KMR Issue Tracker:
- Please make a new user by the login-page to report a new issue.
- KMR source is available with LGPL-2.1.
KMR comes with ABSOLUTELY NO WARRANTY. This wiki also comes with ABSOLUTELY NO WARRANTY. Contents are liable to change.
KMR is the product of AICS, Riken. Part of the results is obtained by using K Computer at AICS, RIKEN.
Are you lost?, for wiki on mt65 try: https://mt65.aics.riken.jp/wiki/