Introduction to the Oakforest-PACS Supercomputer System
Characteristics of the system
The Oakforest-PACS system is a supercomputer system for shared use from the Joint Center for Advanced High Performance Computing (JCAHPC), operated jointly by the Information Technology Center, the University of Tokyo and the Center for Computational Sciences, University of Tsukuba. It is the largest massively parallel cluster supercomputer in Japan, equipped with the next-generation Intel Xeon Phi processor, an ultra-high performance many-core processor produced by U.S. Intel Corporation, and 8,208 compute nodes mounted with Intel Omni-Path Architecture.
Hardware configuration
Overall configuration
| Item | Specifications | ||
|---|---|---|---|
| Total theoretical calculating performance | 25.004 PFlops | ||
| Total number of nodes | 8208 | ||
| Total main storage capacity | 897 TByte | ||
| Network topology | Full-bisection Fat Tree | ||
| Parallel file system | System name | Lustre file system | |
| Server (OSS) | DDN SFA14KE | ||
| Number of servers (OSS) | 10 | ||
| Storage capacity | 26 PB | ||
| Storage data transmission speed | 500 GB/sec | ||
| Fast file cache systems | Server | DDN IME14K | |
| Number of servers | 25 | ||
| Capacity | 940 TB | ||
| Storage data transmission speed | 1,560 GB/sec | ||
Node configuration
| Item | Specifications | |
|---|---|---|
| Machine name | Fujitsu PRIMERGY CX1640 M1 | |
| CPU | Processor name | Intel® Xeon Phi ™ 7250 （code name: Knights Landing） |
| Number of processors (number of cores) | 1 (68) | |
| Frequency | 1.4 GHz | |
| Theoretical calculating performance | 3.0464 TFlops | |
| Memory | 96 GB(DDR4)＋ 16 GB(MCDRAM) | |
| Interconnections | Intel ® Omni-Path network (100 Gbps) |
|
Software configuration
| Item | Configuration |
|---|---|
| OS | Red Hat Enterprise Linux 7, CentOS 7 |
| Compiler | GNU Compiler Intel Compiler(Fortran77/90/95/2003/2008, C, C++) |
| Message communication library | Intel MPI, Intel Omni-Path Fabric Software |
| Library | Intel’s Math Kernel Library (MKL), BLAS, LAPACK, ScaLAPACK, FFTW, GNU Scientific Library, NetCDF, Parallel netCDF, Xabclib, ppOpen-HPC, ppOpen-AT, MassiveThreads, SuperLU, SuperLU MT, SuperLU DIST, METIS, MT-METIS, ParMETIS, Scotch, PT-Scotch, PETSc and Boost |
| Applications | mpijava, omnicompiler, OpenFOAM, ABINIT-MP, PHASE, FrontFlow/blue, FrontISTR, REVOCAP, OpenMX, xTAPP, AkaiKKR, MODYLAS, ALPS, feram, GROMACS, BLAST, R, bioconductor, BioPerl and BioRuby |
| Free software | autoconf, automake, bash, bzip2, cvs, emacs, findutils, gawk, gdb, make, grep, gnuplot, gzip, less, m4, python,perl, ruby, screen, sed, subversion, tar, tcsh, tcl, vim, zsh, cmake, HDF5 and git etc. |