Introduction to the Oakbridge-CX Supercomputer System
Characteristics of the system
The Oakbridge-CX system is a supercomputer system operated by the Information Technology Center, The University of Tokyo.It is a large-scale massively parallel cluster supercomputer consisting of Intel's latest Xeon processor (development code name: CascadeLake) by U.S Intel Corporation and 1,368 computing nodes equipped with Intel Omni-Path Architecture.In addition, 128 nodes out of all 1,368 nodes are equipped with an SSD, which supports processing that requires particularly high file input / output performance.
Hardware configuration
Overall configuration
| Item | Specifications | ||
|---|---|---|---|
| Total theoretical calculating performance | 6.61 PFlops | ||
| Total number of nodes | 1,368(1,240+128) | ||
| Total main storage capacity | 256.5 TiB | ||
| Network topology | Full-bisection Fat Tree | ||
| Parallel file system | System name | Lustre file system | |
| Server (OSS) | DDN ES18K | ||
| Number of servers (OSS) | 2 | ||
| Storage capacity | 12.4 PB | ||
| Storage data transmission speed | 193.9 GB/s | ||
Node configuration
| Item | Specifications | ||
|---|---|---|---|
| Machine name | Fujitsu PRIMERGY CX2550 M5 | Fujitsu PRIMERGY CX2550 M5 | |
| Number of nodes | 1240 | 128 | |
| CPU | Processor name | Intel® Xeon® Platinum 8280 （code name: CascadeLake） |
|
| Number of processors (number of cores) | 2 (28+28) | ||
| Frequency | 2.7 GHz | ||
| Theoretical calculating performance | 4.8384 TFlops | ||
| Memory | 192 GiB(DDR4) | ||
| Interconnections | Intel ® Omni-Path network (100 Gbps) |
||
| SSD | Capacity | - | 1.6 TB(NVMe) |
| Reading performance | 3.20 GB/s | ||
| Writing performance | 1.32 GB/s | ||
Software configuration
| Item | Configuration |
|---|---|
| OS | Red Hat Enterprise Linux 7, CentOS 7 |
| Compiler | GNU Compiler Intel Compiler(Fortran77/90/95/2003/2008, C, C++) |
| Message communication library | Intel MPI, Open MPI, Intel Omni-Path Fabric Software |
| Library | Intel’s Math Kernel Library (MKL)(BLAS, CBAS), etc(LAPACK, ScaLAPACK, SuperLU, SuperLU MT, SuperLU DIST, METIS, MT-METIS, ParMETIS, Scotch, PT-Scotch, PETSc, Trillinos, FFTW, GNU Scientific Library, NetCDF, Parallel netCDF, HDF5, CMake, Anaconda, Xabclib, ppOpen-HPC, ppOpen-AT, MassiveThreads) |
| Applications | mpijava, OpenFOAM, ABINIT-MP, PHASE, FrontFlow/blue, FrontISTR, REVOCAP-Coupler, REVOCAP-Refiner, OpenMX, xTAPP, AkaiKKR, MODYLAS, ALPS, feram, GROMACS、BLAST, R packages, bioconductor, BioPerl, BioRuby, BWA, GATK, SAMtools, Quantum ESPRESSO, Xcrypt, Paraview、Vislt, POV-Ray, HyperWorks, MATLAB |
| Free software | autoconf, automake, bash, bzip2, cvs, emacs, nndutils, gawk, gdb, make, grep, gnuplot, gzip, less, m4, perl, ruby, sed, ubversion, tar, tcsh, tcl, zsh, FUSE, git etc. |
| Container virtualization | singularity (docker Image available) |