Welcome to KMR#

This is KMR, a high-performance map-reduce library. KMR-1.0 is available on the K Computer in "/opt/aics/kmr" now (2013-04-26). KMR works on ordinary clusters as well.

KMR is a set of high-performance map-reduce operations in the MPI (Message Passing Interface) environment. It makes programming for data-processing much easier by hiding low-level details of message passing. Its main targets are large-scale supercomputers with thousands of compute nodes, such as K and Fujitsu FX10. On these platforms, KMR provides utilities other than the map-reduce operations which address issues such as accessing very large file-systems.

KMR is designed to work on-memory and exploits large amount of memory available on supercomputers, whereas most map-reduce implementations are designed to work with external (disk-based) operations. So, data exchanges in KMR occur as message passing instead of remote file operations. The KMR routines work in bulk-synchronous and the most part of the code is sequential, but the code inside the mapper and reducer are multi-threaded.

Project Site#

• Project Overview and other Activities of the Team at RIKEN:
  • http://mt.aics.riken.jp

Documents#

• Overview and API Document:
  • http://mt65.aics.riken.jp/kmrdoc/kmr-1.0/html/index.html
  • It is a Doxgen generated document, included in the installation.
• Overview and API Document (Newer, corrected, for the next release):
  • http://mt65.aics.riken.jp/kmrdoc/next/html/index.html
  • Documentation in the distribution is late, and it is placed here for late breaking.

Issue Tracker#

• KMR Issue Tracker:
  • https://mt65.aics.riken.jp/jtrac/
  • Please make a new user by the login-page to report a new issue.

Downloading#

• Download:
  • https://mt65.aics.riken.jp/kmr/Wiki.jsp?page=Download
  • KMR source is available with LGPL-2.1.

DISCLAIMER#

KMR comes with ABSOLUTELY NO WARRANTY. This wiki also comes with ABSOLUTELY NO WARRANTY. Contents are liable to change.

Acknowledgment#

KMR is a product of RIKEN AICS. Part of the results is obtained by using K Computer at RIKEN AICS.

Are you lost?, for wiki on mt65 try: https://mt65.aics.riken.jp/wiki/