Supercomputing Division, Information Technology Center, The University Tokyo

Introduction to the Oakbridge-CX Supercomputer System

Characteristics of the system

The Oakbridge-CX system is a supercomputer system operated by the Information Technology Center, The University of Tokyo.It is a large-scale massively parallel cluster supercomputer consisting of Intel's latest Xeon processor (development code name: CascadeLake) by U.S Intel Corporation and 1,368 computing nodes equipped with Intel Omni-Path Architecture.In addition, 128 nodes out of all 1,368 nodes are equipped with an SSD, which supports processing that requires particularly high file input / output performance.

System Diagram

Hardware configuration

Overall configuration

Item Specifications
Total theoretical calculating performance 6.61 PFlops
Total number of nodes 1,368(1,240+128)
Total main storage capacity 256.5 TiB
Network topology Full-bisection Fat Tree
Parallel file system System name Lustre file system
Server (OSS) DDN ES18K
Number of servers (OSS) 2
Storage capacity 12.4 PB
Storage data transmission speed 193.9 GB/s

Node configuration

Item Specifications
Machine name Fujitsu PRIMERGY CX2550 M5 Fujitsu PRIMERGY CX2550 M5
Number of nodes 1240 128
CPU Processor name Intel® Xeon® Platinum 8280
(code name: CascadeLake)
Number of processors (number of cores) 2 (28+28)
Frequency 2.7 GHz
Theoretical calculating performance 4.8384 TFlops
Memory 192 GiB(DDR4)
Interconnections Intel ® Omni-Path network
(100 Gbps)
SSD Capadty - 1.6 TB(NVMe)
Reading performance 3.20 GB/s
Writing performance 1.32 GB/s

Software configuration

Item Configuration
OS Red Hat Enterprise Linux 7, CentOS 7
Compiler GNU Compiler
Intel Compiler(Fortran77/90/95/2003/2008, C, C++)
Message communication library Intel MPI, Open MPI, Intel Omni-Path Fabric Software
Library Intel’s Math Kernel Library (MKL)(BLAS, CBAS), etc(LAPACK, ScaLAPACK, SuperLU, SuperLU MT, SuperLU DIST, METIS, MT-METIS, ParMETIS, Scotch, PT-Scotch, PETSc, Trillinos, FFTW, GNU Scientific Library, NetCDF, Parallel netCDF, HDF5, CMake, Anaconda, Xabclib, ppOpen-HPC, ppOpen-AT, MassiveThreads)
Applications mpijava, OpenFOAM, ABINIT-MP, PHASE, FrontFlow/blue, FrontISTR, REVOCAP-Coupler, REVOCAP-Refiner, OpenMX, xTAPP, AkaiKKR, MODYLAS, ALPS, feram, GROMACS、BLAST, R packages, bioconductor, BioPerl, BioRuby, BWA, GATK, SAMtools, Quantum ESPRESSO, Xcrypt, Paraview、Vislt, POV-Ray
Free software autoconf, automake, bash, bzip2, cvs, emacs, nndutils, gawk, gdb, make, grep, gnuplot, gzip, less, m4, perl, ruby, sed, ubversion, tar, tcsh, tcl, zsh, FUSE, git etc.
Container virtualization singularity (docker Image available)