IHK/McKernel is a lightweight multi-kernel operating system designed for high-end supercomputing, and is developed by RIKEN R-CCS. There are two main components of the software stack, a low-level software infrastructure, called Interface for Heterogeneous Kernels (IHK) and a lightweight co-kernel called McKernel.

IHK provides capabilities for partitioning resources in many-core environments (e.g., CPU cores and physical memory) and it enables management of lightweight kernels. IHK is capable of allocating and releasing host resources dynamically and no reboot of the host machine is required when altering configuration. The latest version of IHK is implemented as a collection of Linux kernel modules without any modifications to the Linux kernel itself. This enables straightforward deployment of the multi-kernel stack on a wide range of Linux distributions. Besides resource and LWK (Light Weight Kernel) management, IHK also facilitates an Inter-kernel Communication (IKC) layer, which is used for implementing system call delegation.

McKernel is a lightweight co-kernel developed on top of IHK. It is designed explicitly for high-performance computing workloads, but it retains a Linux compatible application binary interface (ABI) so that it can execute unmodified Linux binaries. There is no need for recompiling applications or for any McKernel specific libraries. McKernel implements only a small set of performance sensitive system calls and the rest of the OS services are delegated to Linux. Specifically, McKernel provides its own memory management, it supports processes and multi-threading, it has a simple round-robin co-operative (tickless) scheduler, and it implements standard POSIX signaling. It also implements inter-process memory mappings and it offers interfaces for accessing hardware performance counters. Applications are executed on the lightweight kernel, and only performance insensitive operations are offloaded to Linux, therefore OS noise is significantly reduced. In particular, the performance of collective communication by IHK/McKernel is better than that of Linux on OFP. Finally, many applications attained efficient and robust performance on OFP with may nodes.

In this talk, outline of IHK/McKernel is described, and experiences in parallel multigrid solver with IHK/McKernel on the Oakforest-PACS system (OFP) are shown.