12.9 HEPS¶
Overview¶
| Resource type | Resource amount | Notes |
|---|---|---|
| Login nodes | ||
| HTCondor | ||
| Slurm | ||
| Home directory | ||
| Work directory |
12.9.1 New Users¶
- Apply for a computing cluster account.
Refer to the account application chapter. After approval from the HEPS experiment contact, users can obtain an AFS account.
- Initialize the computing environment.
After logging in for the first time, run:
12.9.2 Computing Resources¶
HEPS computing resources include the following nodes:
| Node | Logical CPU cores per node | Node type | Scheduler | Notes |
|---|---|---|---|---|
| accap002.ihep.ac.cn | 28 | Login node | ||
| accap001.ihep.ac.cn - accap009.ihep.ac.cn | 56 | Compute node | Slurm | Excluding accap002 |
| accap010.ihep.ac.cn - accap015.ihep.ac.cn | 72 | Compute node | Slurm | |
| accap016.ihep.ac.cn - accap017.ihep.ac.cn | 88 | Compute node | Slurm | |
| accap019.ihep.ac.cn - accap053.ihep.ac.cn | 36 | Compute node | Slurm | SSH login is not allowed |
After the 2025 summer system upgrade, all compute servers were upgraded to ALMA-9. Users need to reset the login configuration:
Log out and log in again for the setting to take effect.
For VNC access toaccap002, disable screen blanking after login:Applications -> System Tools -> Settings -> Power -> Blank Screen Never.
12.9.3 Storage Directories¶
| Directory | Quota / file limit | Quota command |
|---|---|---|
| /afs/ihep.ac.cn/users/a-z/username | 500 MB / none | fs listquota /afs/ihep.ac.cn/users/a-z/username |
| /workfs2/heps/username | 5 GB / 50,000 files | lfs quota -u username /workfs2 |
| /scratchfs/heps/username | 500 GB / none; data kept for two weeks | lfs quota -u username /scratchfs |
| /sharefs/heps/user/username | 3 TB / none | lfs quota -u username /sharefs |
Some users may have personal directories under/workfs2/csns/username,/workfs2/others/username,/scratchfs/csns/username, or/scratchfs/others/username.
Usage recommendations:
- Keep only important personal configuration files under
/afs. /workfs2can be read and written on login nodes, but Slurm jobs on compute nodes cannot create or modify files there.- Use
/sharefsfor important results and data that need to be read and written by login and compute nodes. - Use
/scratchfsfor temporary large files and intermediate results. - Back up important results to your personal computer in time.
12.9.4 Programs and Examples¶
HEPS computing programs are installed under/sharefs/heps/data/APcodesand/sharefs/heps/data/opt/. See/sharefs/heps/data/APcodes/README.mdfor a detailed program list.
12.9.4.1 Run Parallel Programs Through Slurm¶
Submit jobs after logging in tolxlogin.ihep.ac.cnoraccap002.ihep.ac.cn. The recommended number of parallel cores is a multiple of 36.
Examplesbatch.sh:
#! /bin/bash#SBATCH --partition=heps#SBATCH --account=heps#SBATCH --job-name=DA#SBATCH --qos=regular#SBATCH --output=slurm-%j.txt#SBATCH --ntasks=36mpiexec-np36PelegantparTrack2.ele
Submit the job:
For debugging, use thehepsdebugqueue. It allows jobs with no more than 36 tasks and a maximum run time of 30 minutes:
Useful commands:
12.9.4.3 Run Licensed MATLAB¶
HEPS MATLAB is installed under/afs/ihep.ac.cn/soft/hepsap/matlab. Log in toaccap002.ihep.ac.cnby SSH or VNC and run:
The installed MATLAB supports parallel computing with up to 512 parallel cores. MATLAB parallel jobs do not need Slurm; request parallel cores inside MATLAB:
12.9.4.4 Run Licensed Mathematica¶
HEPS Mathematica is installed under/sharefs/heps/data/opt/Mathematica/12.0:
The license supports up to 17 concurrent Mathematica main programs and up to 136 cores for parallel computing. Examples are available under:
For single-node parallel jobs onaccap002, add the following before the parallel section:
For cross-node parallel jobs, submit resources through Slurm. Non-interactive Wolfram Language scripts can be run with:
Interactive jobs can first allocate resources:
Then launch Mathematica onaccap002and run the parallel computation.
12.9.5 FAQ¶
12.9.5.1 Cannot Write to the AFS Personal Directory¶
Users need to renew tokens about every 10 hours:
You can also create a scheduled task:
If the AFS directory is full, check quota and disk usage:
MATLAB users should clear cache files regularly:
12.9.5.2 Cannot Log In by SSH from a Linux Host¶
If SSH reportsHost key verification failed, remove the lines containing the target host, such asaccap002, from~/.ssh/known_hosts.
12.9.5.3 MATLAB Plots Do Not Display Correctly on accap002¶
Start MATLAB with software OpenGL:
有问题?输入后跳转到对话页继续交流