Tokens, which refers to the duration that compute nodes are available for use, are allocated for the Wisteria/BDEC-01 system based on the course of use and the number of sets applied for. There are no restrictions to the number of nodes and GPUs that can be used within the constraints of the allocated tokens (with the exception of some courses and services), and batch jobs can be executed up to the maximum number of nodes and GPUs that can be used. This page explains the concept of tokens, and the important points to be noted.
Overview
The tokens are needed (consumed) during the execution of batch jobs. The amount of allocated tokens is determined beforehand based on the course and number of sets applied for. Batch jobs can be executed up to the maximum number of nodes and GPUs regardless of the course of use applied for. Please refer to the following chart on the relationship between the amount of tokens allocated based on the course of use applied for, and the maximum number of nodes and GPUs.
Amount of tokens allocated for Wisteria/BDEC-01
Number of usable nodes and GPUs
Amount of allocated tokens (per year)
Consumption coefficient
General Use (Per set)
Wisteria-O : Maximum of 2304 nodes
Wisteria-A : Maximum of 64 GPUs
(Regardless of number of sets)
8,640 tokens
Wisteria-O : 1.00 (per node)
* 1.50 for priority nodes (about 15% of the total)
Wisteria-A : 3.00 (per GPU)
Wisteria-A Fixed nodes (Per set)
207,360 tokens * Only 1 set can be applied for.
Wisteria-A Reserved GPU (Per set)
25,920 tokens * Only 1, 2, or 4 set can be applied for.
The tokens are consumed corresponding to the number of nodes, GPUs and execution time of each batch job. Hence, it will no longer be possible to execute a batch job at the point when the tokens run out (or at the point when it becomes clear that the job will exceed the amount of remaining tokens even before the execution of the job). However, even after the tokens have run out, it will still be possible to login to the login nodes and execute an interactive job.
Calculation of tokens
As explained in the overview, tokens are consumed for the execution of each batch job. The system allows for the execution of batch jobs above the number of sets applied for. Please refer to the calculation formula and specific examples of calculation shown below.
Calculation formula for tokens
Tokens = [Lapsed time] × [Requested No. of nodes / GPUs] × [Consumption coefficient]
* As for jobs per node on Wisteria-A, it is calculated assuming that 8 GPUs are requested per node.
* Example 1: Tokens for the job (24 hours and 128 nodes) on Wisteria-O
* Example 3: Tokens for the job (24 hours and 2 nodes, 16GPU) on Wisteria-A
Tokens = 24 hours × 16 GPUs × 3.00
= 1152 Tokens
Token sharing with group members
The tokens allocated are shared and consumed among all the users registered in the group. However, it is possible either for just one user registered in the group to consume all the tokens allocated to the group, or to set a maximum amount of tokens that can be used by each user in the group.
Points to note when using tokens
The token system allows executing batch jobs flexibly for more than the number of sets applied for. However, it will no longer be possible to execute batch jobs once all the tokens have been consumed. (Please be sure to check the amount of remaining tokens.) For that reason, the addition of tokens is accepted (additional application) only in cases where there are tokens that can be provided. However, there is no guarantee that all tokens can be used within the period in which use is permitted. As use of the tokens is only allowed during the period in which use is permitted (effective validity of the tokens), the remaining tokens cannot be carried out to the next fiscal year or refunded.
FAQ
Is there a way of verifying the status of use of tokens?
You can verify this using the “show_token” command.
With regard to submitted jobs, is there a way of verifying which tokens had been used for the job? In addition, is it possible to verify jobs that were submitted using the same tokens?
These can be verified using the “pjstat” command.
What should I do if I wish to change the number of tokens allocated to users within a group?
The group administrator can set the number of tokens for users within a group. Please consult with the administrator for your user group (group administrator).
I submitted an application for 8 sets under the group course, but can I change this to xxx sets?
It is possible if there are system resources to spare (number of tokens that can be added).
Does it mean that I will not be able to login once the tokens have run out?
Tokens are consumed only when a batch job is executed. Even if the tokens run out (become "0"), you will be able to login (interactive job execution).
Is it possible to add tokens?
It is possible to add tokens only if system resources (the amount of tokens that can be added) are sufficient.
How can tokens be added?
Please submit an application form for changes. However, depending on the contents of the application, there may be cases where the addition is not approved.
I added tokens as my tokens had been used up. Although 8,640 tokens were added, the available remain token is below 8,640 token. Why is that so?
This is because the tokens were used in excess prior to the token addition. For example, if the excess usage had been 640 tokens at the point of application for additional tokens, the number of usable tokens would be 8,000 tokens.
The amount of tokens used in excess is deducted from the tokens added, and the remaining tokens is the number of tokens available for use.
I added the number of sets applied for (addition of tokens as all the tokens had been used up). Will there be any changes to the maximum number of tokens allocated to individual users within the group?
There will be no changes. The maximum number of tokens (value set by the group administrator) can only be changed by the group administrator. A change in the number of maximum tokens through the addition of tokens needs to be made by the group administrator.
Is there a validity period for the tokens?
The validity period is the period in which the use of computers is permitted (processing day at the end of the month of expiration of use, or the day for the end of services at the end of the fiscal year in question). Hence, even if there are any remaining tokens, they cannot be carried over to the next fiscal year or refunded.
I could not finish using the tokens during the period in which use is permitted. As I am submitting an application for continued use, can I continue to use the remaining tokens?
The validity period of allocated tokens is within the initial period in which use is permitted (processing day at the end of the month of expiration of use, or the day for the end of services at the end of the fiscal year in question). Hence, even if you submit an application for continued use, they cannot be carried over beyond the initial validity period nor refunded.
I am a member of several groups, but am unable to finish using my tokens within the validity period for the group. Can I use these tokens under other groups after the validity has expired for the group in question?
The validity period for tokens is the period in which the use of computers is permitted (processing day at the end of the month of expiration of use, or the day for the end of services at the end of the fiscal year in question). Hence, even if your User ID remains effective, tokens that have been allocated to a group for which validity has expired cannot be used in another group.
