Supercomputing Division, Information Technology Center, The University Tokyo

Wisteria/BDEC-01 FAQ

Application for use

I would like to apply for the use of the Wisteria/BDEC-01 system. Could you tell me the start date for application and other details?
Application for use is accepted at any time as long as there are computational resources available. Please fill in the necessary fields in the application form for use of the Wisteria/BDEC-01 system.If you wish to use the facilities for other than general use, please apply through the respective application sites.
Are there any points I should check (take note of) before applying for and using the Wisteria/BDEC-01 system?
Various manuals are available on the application method, method of use, and other information pertaining to the Wisteria/BDEC-01 system. Please refer to these for details about the application method and service contents. (Manuals and guides provided by the manufacturers are only available to users.)

* Courses of use for Wisteria/BDEC-01
* Overview of Wisteria/BDEC-01 tokens
* Cost of use for Wisteria/BDEC-01
* Cost of use by month for Wisteria/BDEC-01

(Explanations are provided on the overview of how to use the Wisteria/BDEC-01 system, etc.
For details on how to use the system, please refer to the guides and manuals provided in the User Support Portal.)

How is the quota for disk space within a group set?
In the disk space within a group, directories have been prepared for each individual user registered in the group (in the following example, /work/gz00/z3000*), as well as for shared use by all the users within the group (in the following example, /work/gz00/share). A quota is set for the users and the group for all the directories.
In the case where 4 sets are applied for, the quota for users and the group is both 8TB respectively.

Group course
Tokens
(8,640 node-hours allocated per 1 set applied for)
Disk space for the group
(2TB allocated per set applied for)
/work/gz00/share
/work/gz00/z30000
/work/gz00/z30001
/work/gz00/z30002
It is stipulated that each group must have a group administrator. What kind of administration does the group administrator carry out?
For the Wisteria/BDEC-01 system, the number of tokens and nodes that are used to execute batch jobs is allocated to the entire group. However, the group administrator has the authority to set and change the maximum amount of these computational resources allocated to each user. Note that it is not possible to change the maximum number of tokens or nodes in excess of the maximum number that the group has been allocated with.
Functions of the group administrator
* Change the maximum number of tokens allocated to users within the group (Default value: Maximum number allocated to the group)
* Change the maximum number of nodes allocated to users within the group (Default value: Maximum number allocated to the group)
Is it possible to add users from a personal (group) course to my own group? In such cases, will new User IDs be assigned?
It is possible to add users to the group. However, when adding existing users of the Center to the group, such users are registered as additional users of the group based on the User IDs that they are currently using. Hence, new User IDs will not be assigned.
When adding users from a personal (group) course to my own group, how will the resources allocated to users in the group (tokens, disk storage) be handled?
For new users who are added and registered to the group, approval for the use of tokens available for the group will be granted, and a user directory will be created in the space allocated to users within the group.
When adding user "z40000" to the group

Group course
Tokens
(8,640 node-hours allocated per set applied for)
z40000 can use the tokens
Disk space within the group
(2TB allocated per set applied for)
/work/gz00/share
/work/gz00/z30000
/work/gz00/z30001
/work/gz00/z30002
/work/gz00/z40000 Directory created for z40000's use
When deleting user "z40000" from the group
Group course
Tokens
(8,640 node-hours allocated per set applied for)
z40000 cannot use the tokens
Disk space within the group
(2TB allocated per set applied for)
/work/gz00/share
/work/gz00/z30000
/work/gz00/z30001
/work/gz00/z30002
/work/gz00/z40000 Directory for z40000's use deleted
Is it possible to register for multiple groups?
Yes, it is. The representative of the group to be added should apply for the change of registered user from the change application form.
I was a group user, but use by the group I was registered will be cancelled. I would like to continue using the supercomputer system. What should I do?
Please submit a new application for use under the general course before the period of use of the current general use you are registered with expires. Please note that if you submit an application for use under the course after the period of use expires, you will not be able to retain the same User ID and keep the files that you had been using.
Is it possible to add users for a group course at any time?
Yes, it is. Fill in the necessary fields in the application form for changes, and submit it to the Research Support Team, Information Strategy Group, Information Systems Department, The University of Tokyo.
Is there a maximum number of users who can be registered under the group course?
There are no limits to the number of users who can apply (be registered) for the group course. However, regardless of the number of users, the number of tokens that are allocated to the group is fixed. Hence, this fixed number of tokens will be used (consumed) by all the users registered in the group.
There are descriptions about the number of application sets, and the maximum number of nodes. Could you tell me the difference between these?
Under the Wisteria/BDEC-01 system, the maximum number of nodes can be used regardless of the number of application sets.
Is there a limit to the total number of files and directories that can be stored on the system?
Each group can store 500,000 files/directories per 1 TB of disk storage on Wisteria/BDEC-01.

▲ Return to the top of the FAQ for the Wisteria/BDEC-01 system

Token

Could you tell me about the tokens?
For explanations and FAQ about tokens, please refer to the webpage about "Tokens".

▲ Return to the top of the FAQ for the Wisteria/BDEC-01 system

Overall system, service contents

What types of uses does educational use refer to?
The Information Technology Center, the University of Tokyo (hereafter, "the Center") provides supercomputing resources for practices in graduate and undergraduate classes.
There have been moves to use supercomputers as a practice tool for the use of methods applied to a wide range of research fields, including climate, fluid analysis, structural analysis, molecular science, nanotechnology, and aerospace. Supercomputers are already being used for practices related to structural analysis, Earth science, and fluid-related subjects in specialized undergraduate courses as well as graduate classes.
The use of supercomputers in education is anticipated to contribute to human resource development and the expansion of use of such systems. Hence, the Center accepts applications, at any time, for use in lectures and workshops (including intensive lectures) conducted by teaching faculties or teachers at graduate schools, undergraduate schools, and vocational institutes.
Use for educational purposes is not limited to use within the University of Tokyo. Applications for use outside of the University of Tokyo are also accepted. For details, please refer to "Educational Use".

Available resources
Wisteria-O
Maximum execution time per job of 15 minutes, and maximum of 12 nodes (576 cores) Wisteria-A
Maximum execution time per job of 15 minutes, and maximum of 2 nodes (GPU 16)
Is it possible to change the login shell?
Changes can be made using the "chsh" command. The standard login shell is set to "bash".

▲ Return to the top of the FAQ for the Wisteria/BDEC-01 system

Compilers

How do I check for version updates for the compilers?
Information on version updates for compilers is provided on the User Support Portal. Please check the portal for more details.
Are debug profilers, etc. available?
Intel VTune Amplifier XE, Allinea DDT, etc. are available. For details, please refer to the User’s Guide for the Wisteria/BDEC-01 system and other documents available in the User Support Portal (-Documents)

▲ Return to the top of the FAQ for the Wisteria/BDEC-01 system

LIBRARY

I would like to use the library (xxxx). Could you tell me how to use it?
Please refer to the User's Guide for the Oakbridge-CX system and other documents available in the User Support Portal (- Browse documents).

▲ Return to the top of the FAQ for the Wisteria/BDEC-01 system

Applications

I would like to use the application (xxxx) provided. Could you tell me how to use it?
Please refer to the User's Guide for the Wisteria/BDEC-01 system and other documents available in the User Support Portal (- Browse documents).
Can I use software that I have bought separately?
You may install and use your own software in the user directory after checking the license and other details.

▲ Return to the top of the FAQ for the Wisteria/BDEC-01 system

Job administration system

What are the types of queues (resource groups/job classes) available?
Please refer to "Queues (Resource groups/Job classes)".
Are there any commands I can use to check the queues that I can submit jobs to?
You can check this using the "pjstat--rsc" command. Details about the "pjstat" command are provided in the User's Guide for the Wisteria/BDEC-01 system (- 5.8. Display job status) available in the User Support Portal.
What is the maximum number of jobs that can be submitted, and the maximum number of jobs that can be executed simultaneously?
The number of jobs differs depending on the node group used (Odyssey, Aquarius). Please refer to the following table.
・Wisteria-O (Odyssey)
Group Course
User unit Group unit
Maximum number of jobs
that can be executed simultaneously
No limit 4 when the number of sets applied for is less than 3
8 when the number of sets applied for is less than 5
16 when the number of sets applied for is less than 9
Thereafter, add 2 jobs per 8 sets
Maximum number of submissions No limit Maximum number that can be executed simultaneously × 8
Maximum number of nodes
that can be used simultaneously
2304
Maximum number of nodes
(Job unit)
2304
・Wisteria-A (Aquarius)
Group Course
User unit Group unit
Maximum number of jobs
that can be executed simultaneously
No limit 8 when the number of sets applied for is less than 25
Thereafter, add 2 job per 12 sets
Maximum 20
Maximum number of submissions No limit Maximum number that can be executed simultaneously × 2
Maximum number of GPU
that can be executed simultaneously
64
Maximum number of GPUs
(Job unit)
64

You can check the maximum number of jobs that can be executed simultaneously and the maximum number of jobs that can be submitted using the following command.

$ pjstat --limit
SYSTEM: Odyssey
PROJECT     ACCEPT       RUN  BULK_ACCEPT  BULK_RUN      NODE
gXX0        0/ 128    0/  16         0/ 8    0/  16   0/ 2304
gXX1        0/  32    0/   4         0/ 8    0/   4   0/ 2304

SYSTEM: Aquarius
PROJECT     ACCEPT       RUN  BULK_ACCEPT  BULK_RUN     GPU
gXX0         0/  4    0/   2         0/ -    0/   -   0/ 64
gXX1         0/  4    0/   2         0/ -    0/   -   0/ 64

Could you tell me how to start up a batch job and an interactive job?
A batch job is submitted using the "pjsub" command. For details, please refer to the User's Guide for the Wisteria/BDEC-01 System (- 5.4. Batch job execution, 5.5. Interactive job execution) available in the User Support Portal.
Could you tell me how to delete a job?
The "pjdel" command is used to delete a job. For details, please refer to the User's Guide for the Wisteria/BDEC-01 System (- 5.9. Job cancellation) available in the User Support Portal.
Could you tell me how to check the execution status, etc. of a job?
The "pjstat" command is used to check the execution status, etc. of a job. For details, please refer to the User's Guide for the Wisteria/BDEC-01 System (- 5.8. Display job status) available in the User Support Portal.
Job execution does not start.
Please check the following points.

1. There are cases where the system is waiting to secure the computational resources required for the job awaiting execution. You can check the overall usage status of the system using the "pjstat--nodeuse" command.

$ pjstat --nodeuse
SYSTEM: Odyssey
RSCGRP                                                      Ratio Used/Total
debug-o/interactive-o             -------------------------    0%     0/ 768
short-o                           -------------------------    0%     0/ 384
regular-o                         *******************------   76%  4096/5376
priority-o                        ******-------------------   25%   288/1152

SYSTEM: Aquarius
RSCGRP                                                       Ratio Used/Total
debug-a/interactive-a             ************-------------   50%     1/  2
short-a                           ******-------------------   25%     1/  4
regular-a                         ********************-----   80%    20/ 26

SYSTEM: Prepost
RSCGRP                                                  Ratio Used/Total(Node)
prepost                       -------------------------    0%     0/   9

It is necessary to specify --gpuuse option in order to show the information of resource groups occupied GPU.

2. As job scheduling is carried out through FIFO operations, there are cases where the system is waiting to finish a job submitted earlier.

3. The time required for computation specified for a job awaiting execution may sometimes be longer than the time remaining until service suspension. You can check the time remaining until service suspension using the “pjstat” command.

4. There are cases where the system has reached its maximum limit on the number of jobs that can be executed simultaneously. Please check if other jobs are being executed, or if another user from the same group is executing the job. You can use the “pjstat --limit" command to check the number of jobs that are being executed within the group, and the maximum number of jobs that can be executed simultaneously.

5. A system problem may have occurred. Please inquire via the webpage for consultation and inquiries on system use.

▲ Return to the top of the FAQ for the Wisteria/BDEC-01 system

User Support Portal/Key registration

What can I do through the User Support Portal?
The User Support Portal allows users to register the public keys that are used to login to the Wisteria/BDEC-01 system, and to browse through the User’s Guide, manuals provided by the manufacturer, and information about version updates for compilers, etc. Some of the group administrator functions can also be carried out through the User Support Portal. For details, please refer to the User’s Guide for Wisteria/BDEC-01 Group Course Project Administrators.
I tried to register the public key through the User Support Portal, but could not login (authentication failed).
If you received the registration notice in writing, please read the notes in red outside the initial password field and try again. If you received the registration notice by e-mail, you can reset your password by clicking the URL in the body of the e-mail. After resetting your password, please try again (it may take up to 10 minutes for the change to take effect).
If the above does not help, please contact the reception (Email address for inquiries).
I have changed the permissions in the .ssh directory, and cannot login anymore. What should I do?
Please contact us by providing the necessary information via the webpage for consultation and inquiries on system use. You will not be able to login if there are any errors in the permissions for the directory or files in question, or in the saved key. Hence, please take great care when editing the information. (Please back up any files, etc. beforehand, and check if you can connect from other terminals before logout.)

▲ Return to the top of the FAQ for the Wisteria/BDEC-01 system