Welcome to KMR (K Map-Reduce)#
This is KMR (K Map-Reduce), a high-performance map-reduce library. KMR-1.0 is available on the K Computer in "/opt/aics/kmr" now (2013-04-26).
KMR is a set of high-performance map-reduce operations in the MPI (Message Passing Interface) environment. It is to make programming for data-processing much easier by hiding low-level details of message passing. Its main target is large-scale computers with thousands nodes, K and Fujitsu FX10. On those platforms, KMR provides utilities for the map-reduce operations to address issues such as accessing large file-systems. But, KMR works on clusters as well.
KMR assumes large amount of memory and designed to work on-memory, whereas most map-reduce implementations are designed to work with external (disk-based) operations. So, data exchanges in KMR occur as message passing instead of remote file operations. The KMR routines work in bulk-synchronous and the most part of the code is sequential, but the code inside the mapper and reducer are multi-threaded.
- Project Overview and other Activities of the Team:
- Overview and API Document:
- It is a Doxgen generated document, included in the installation.
- Overview and API Document (Newer, corrected, for the next release):
- Documentation in the distribution is late, and it is placed here for late breaking.
- KMR Issue Tracker:
- Please make a new user by the login-page to report a new issue.
- KMR source is available with LGPL-2.1.
KMR comes with ABSOLUTELY NO WARRANTY. This wiki also comes with ABSOLUTELY NO WARRANTY. Contents are liable to change.
Part of the results is obtained by using K Computer at AICS, RIKEN.
Are you lost?, for wiki on mt65 try: https://mt65.aics.riken.jp/wiki/