Supercomputing Division, Information Technology Center, The University Tokyo

Oakbridge-CX FAQ

Application for us

I would like to apply for the use of the Oakbridge-CX system. Could you tell me the start date for application and other details?
Application for use is accepted at any time as long as there are computational resources available. Please fill in the necessary fields in the application form for use of the Oakbridge-CX system.If you wish to use the facilities for other than general use, please apply through the respective application sites.
Are there any points I should check (take note of) before applying for and using the Oakbridge-CX system?
Various manuals are available on the application method, method of use, and other information pertaining to the Oakbridge-CX system. Please refer to these for details about the application method and service contents. (Manuals and guides provided by the manufacturers are only available to users.)

* Courses of use for Oakbridge-CX
* Overview of Oakbridge-CX tokens
* Cost of use for Oakbridge-CX (Applicable from April 1, 2022)
* Cost of use by month for Oakbridge-CX (Applicable from April 1, 2022)

(Explanations are provided on the overview of how to use the Oakbridge-CX system, etc.
For details on how to use the system, please refer to the guides and manuals provided in the User Support Portal.)

How is the quota for disk space within a group set?
In the disk space within a group, directories have been prepared for each individual user registered in the group (in the following example, /work/gz00/z3000*), as well as for shared use by all the users within the group (in the following example, /work/gz00/share). A quota is set for the users and the group for all the directories.
In the case where 4 sets are applied for, the quota for users and the group is both 16TB respectively.

Group course
Tokens
(8,640 node-hours allocated per 1 set applied for)
Disk space for the group
(4TB allocated per set applied for)
/work/gz00/share
/work/gz00/z30000
/work/gz00/z30001
/work/gz00/z30002
It is stipulated that each group must have a group administrator. What kind of administration does the group administrator carry out?
For the Oakbridge-CX system, the number of tokens and nodes that are used to execute batch jobs is allocated to the entire group. However, the group administrator has the authority to set and change the maximum amount of these computational resources allocated to each user. Note that it is not possible to change the maximum number of tokens or nodes in excess of the maximum number that the group has been allocated with.
Functions of the group administrator
* Change the maximum number of tokens allocated to users within the group (Default value: Maximum number allocated to the group)
* Change the maximum number of nodes allocated to users within the group (Default value: Maximum number allocated to the group)
Is it possible to add users from a personal (group) course to my own group? In such cases, will new User IDs be assigned?
It is possible to add users to the group. However, when adding existing users of the Center to the group, such users are registered as additional users of the group based on the User IDs that they are currently using. Hence, new User IDs will not be assigned.
When adding users from a personal (group) course to my own group, how will the resources allocated to users in the group (tokens, disk storage) be handled?
For new users who are added and registered to the group, approval for the use of tokens available for the group will be granted, and a user directory will be created in the space allocated to users within the group.
When adding user "z40000" to the group

Group course
Tokens
(8,640 node-hours allocated per set applied for)
z40000 can use the tokens
Disk space within the group
(4TB allocated per set applied for)
/work/gz00/share
/work/gz00/z30000
/work/gz00/z30001
/work/gz00/z30002
/work/gz00/z40000 Directory created for z40000's use
When deleting user "z40000" from the group
Group course
Tokens
(8,640 node-hours allocated per set applied for)
z40000 cannot use the tokens
Disk space within the group
(4TB allocated per set applied for)
/work/gz00/share
/work/gz00/z30000
/work/gz00/z30001
/work/gz00/z30002
/work/gz00/z40000 Directory for z40000's use deleted
Is it possible to register for multiple groups?
Yes, it is. The representative of the group to be added should submit the application for use.
I was a group user, but use by the group I was registered will be cancelled. I would like to continue using the supercomputer system. What should I do?
Please submit a new application for use under the group course or personal course before the period of use of the current group you are registered with expires. Please note that if you submit an application for use under the group course or personal course after the period of use expires, you will not be able to retain the same User ID and keep the files that you had been using.
Is it possible to add users for a group course at any time?
Yes, it is. Fill in the necessary fields in the application form for changes, and submit it to the Research Support Team, Information Strategy Group, Information Systems Department, The University of Tokyo.
Is there a maximum number of users who can be registered under the group course?
There are no limits to the number of users who can apply (be registered) for the group course. However, regardless of the number of users, the number of tokens that are allocated to the group is fixed. Hence, this fixed number of tokens will be used (consumed) by all the users registered in the group.
There are descriptions about the number of nodes applied for, and the maximum number of nodes. Could you tell me the difference between these?
Under the Oakbridge-CX system, it is possible to use up to the maximum number of nodes in excess of the number of nodes applied for. However, when executing a batch job in excess of the number of nodes applied for, the number of tokens consumed will increase.
Is there a limit to the total number of files and directories that can be stored on the system?
Each group can store 450,000 files/directories per 1 TB of disk storage on Oakbridge-CX.

▲ Return to the top of the FAQ for the Oakbridge-CX system

Token

Could you tell me about the tokens?
For explanations and FAQ about tokens, please refer to the webpage about "Tokens".

▲ Return to the top of the FAQ for the Oakbridge-CX system

Overall system, service contents

What types of uses does educational use refer to?
The Information Technology Center, the University of Tokyo (hereafter, "the Center") provides supercomputing resources for practices in graduate and undergraduate classes.
There have been moves to use supercomputers as a practice tool for the use of methods applied to a wide range of research fields, including climate, fluid analysis, structural analysis, molecular science, nanotechnology, and aerospace. Supercomputers are already being used for practices related to structural analysis, Earth science, and fluid-related subjects in specialized undergraduate courses as well as graduate classes.
The use of supercomputers in education is anticipated to contribute to human resource development and the expansion of use of such systems. Hence, the Center accepts applications, at any time, for use in lectures and workshops (including intensive lectures) conducted by teaching faculties or teachers at graduate schools, undergraduate schools, and vocational institutes.
Use for educational purposes is not limited to use within the University of Tokyo. Applications for use outside of the University of Tokyo are also accepted. For details, please refer to "Educational Use".

Available resources
Oakbridge-CX
Maximum execution time per job of 15 minutes, and maximum of 8 nodes (488 cores)
Is it possible to change the login shell?
Changes can be made using the "chsh" command. The standard login shell is set to "bash".

▲ Return to the top of the FAQ for the Oakbridge-CX system

Compilers

How do I check for version updates for the compilers?
Information on version updates for compilers is provided on the User Support Portal. Please check the portal for more details.
Are debug profilers, etc. available?
Intel VTune Amplifier XE, Allinea DDT, etc. are available. For details, please refer to the User's Guide for the Oakbridge-CX system and other documents available in the User Support Portal (-Documents)

▲ Return to the top of the FAQ for the Oakbridge-CX system

Libraries

Could you tell me about the versions of the libraries and applications provided?
The versions of the libraries and applications provided by the Center are listed as follows (as of October 2019). It is possible to obtain the version information using the "module" command. For more information on how to use the "module" command, please refer to the User's Guide.
Library Version
Boost
FFTW
GNU Scientific Library
Intel library (MKL)
MassiveThreads
METIS
MT-METIS
NetCDF
Parallel netCDF
ParMETIS
PETSc
ppOpen-APPL/BEM
ppOpen-APPL/DEM-util
ppOpen-APPL/FDM
ppOpen-APPL/FEM
ppOpen-APPL/FVM
ppOpen-AT
PT-Scotch
Scotch
SuperLU
SuperLU DIST
SuperLU MT
Xabclib
1.53
3.3.8
2.5
2019 update5
0.99
5.1.0
0.6.0
4.7.0
4.7.0
4.0.3
3.11.2
0.5.0
1.0.0
0.3.1
1.0.1
0.3.0
1.0.0
6.0.6
6.0.7
5.2.1
6.1.1
3.1
1.03

Application Version
ABINIT-MP
AkaiKKR
ALPS
bioconductor
BioPerl
BioRuby
BLAST
feram
FrontFlow/blue
FrontISTR
GROMACS
MODYLAS
mpijava
OpenFOAM
OpenMX
PHASE
R
REVOCAP
xTAPP
open1_rev15
cpa2002v010
2.3.0
3.9
1.007002
1.5.2
2.9.0
0.26.04
8.1
4.5
2019.2
1.0.4
1.2.7_ojdk1.8
6 or v1812
3.8
2019.01
3.6.0
2.1
rc-150401
I would like to use the library (xxxx). Could you tell me how to use it?
Please refer to the User's Guide for the Oakbridge-CX system and other documents available in the User Support Portal (- Browse documents).

▲ Return to the top of the FAQ for the Oakbridge-CX system

Applications

I would like to use the application (xxxx) provided. Could you tell me how to use it?
Please refer to the User's Guide for the Oakbridge-CX system and other documents available in the User Support Portal (- Browse documents).
Can I use software that I have bought separately?
You may install and use your own software in the user directory after checking the license and other details.

▲ Return to the top of the FAQ for the Oakbridge-CX system

Job administration system

What are the types of queues (resource groups/job classes) available?
Please refer to "Queues (Resource groups/Job classes)".
Are there any commands I can use to check the queues that I can submit jobs to?
You can check this using the "pjstat--rsc" command. Details about the "pjstat" command are provided in the User's Guide for the Oakbridge-CX system (- 5.8. Display job status) available in the User Support Portal.
What is the maximum number of jobs that can be submitted, and the maximum number of jobs that can be executed simultaneously?
The number of jobs differs depending on the number of sets applied for. Please refer to the following table.
Group Course
User unit Group unit
Maximum number of jobs that can be executed simultaneously No limit 16 when the number of sets applied for is 4
20 when the number of sets applied for is 8
Thereafter, add 4 jobs per 8 sets
Maximum number of submissions No limit Maximum number that can be executed simultaneously ?? 4
64 for 4sets
80 for 8sets
Maximum number of nodes that can be used simultaneously 256 256
Maximum number of nodes
(Job unit)
256 256

You can check the maximum number of jobs that can be executed simultaneously and the maximum number of jobs that can be submitted using the following command.

$ pjstat --limit
PROJECT     ACCEPT       RUN  BULK_RUN      NODE
gXXX        0/  64    0/  16    0/  16    0/ 256
pXXXXX      0/  16    0/   4    0/  16    0/ 256
Could you tell me how to start up a batch job and an interactive job?
A batch job is submitted using the "pjsub" command. For details, please refer to the User's Guide for the Oakbridge-CX System (- 5.4. Batch job execution, 5.5. Interactive job execution) available in the User Support Portal.
Could you tell me how to delete a job?
The "pjdel" command is used to delete a job. For details, please refer to the User's Guide for the Oakbridge-CX System (- 5.9. Job cancellation) available in the User Support Portal.
Could you tell me how to check the execution status, etc. of a job?
The "pjstat" command is used to check the execution status, etc. of a job. For details, please refer to the User's Guide for the Oakbridge-CX System (- 5.8. Display job status) available in the User Support Portal.
Job execution does not start.
Please check the following points.

1. There are cases where the system is waiting to secure the computational resources required for the job awaiting execution. You can check the overall usage status of the system using the "pjstat--nodeuse" command.

$ pjstat --nodeuse
RSCGRP                                                  Ratio Used/Total
debug/interactive             -------------------------    0%     0/ 192
short                         -------------------------    0%     0/  72
regular                       *------------------------    3%    37/1088

2. As job scheduling is carried out through FIFO operations, there are cases where the system is waiting to finish a job submitted earlier.

3. The time required for computation specified for a job awaiting execution may sometimes be longer than the time remaining until service suspension. You can check the time remaining until service suspension using the "pjstat" command.

4. There are cases where the system has reached its maximum limit on the number of jobs that can be executed simultaneously. Please check if other jobs are being executed, or if another user from the same group is executing the job. You can use the "pjstat --limit" command to check the number of jobs that are being executed within the group, and the maximum number of jobs that can be executed simultaneously.

5. A system problem may have occurred. Please inquire via the webpage for consultation and inquiries on system use.

▲ Return to the top of the FAQ for the Oakbridge-CX system

User Support Portal/Key registration

What can I do through the User Support Portal?
The User Support Portal allows users to register the public keys that are used to login to the Oakbridge-CX system, and to browse through the User's Guide, manuals provided by the manufacturer, and information about version updates for compilers, etc. Some of the group administrator functions can also be carried out through the User Support Portal. For details, please refer to the User's Guide for Oakbridge-CX Group Course Project Administrators.
I tried to register the public key through the User Support Portal, but could not login (authentication failed).
If you received the registration notice in writing, please read the notes in red outside the initial password field and try again. If you received the registration notice by e-mail, you can reset your password by clicking the URL in the body of the e-mail. After resetting your password, please try again (it may take up to 15 minutes for the change to take effect).
If the above does not help, please contact the reception (Email address for inquiries).
I have changed the permissions in the .ssh directory, and cannot login anymore. What should I do?
Please contact us by providing the necessary information via the webpage for consultation and inquiries on system use. You will not be able to login if there are any errors in the permissions for the directory or files in question, or in the saved key. Hence, please take great care when editing the information. (Please back up any files, etc. beforehand, and check if you can connect from other terminals before logout.)

▲ Return to the top of the FAQ for the Wisteria/BDEC-01 system