3. Filesystems
All users are allocated space on the filesystems /home, /work and /scratch.
Each storage location has different size, speed, backup setting, and availability.
You as the user need to balance between the various features.
The filesystems for data storage that are available on the tux cluster are:
Name |
Path |
Availability |
Backup |
FS-type |
Purpose |
|---|---|---|---|---|---|
Home |
|
all nodes |
Yes |
NFS |
Home-directory: user specific files, no calculation data. |
SimLab |
|
all nodes |
No |
NFS |
Project area for long-term-storage. |
Work |
|
all nodes |
No |
NFS |
Personal work (scratch) space for every user. Calculation data etc. |
Scratch |
|
single node |
No |
ext4 |
Primary (and usually fastest) place for single-node calculation data. Removed once user's jobs are finished on the node. |
tmp |
|
single node |
No |
tmpfs |
Ramfs; fast in-memory filesystem |
3.1. Home
Your home directory /home/$USER is private to you , and is where your sessions begin by default.
Its intended use is for storing scripts, notes, final products (e.g. figures), etc.
In particular, you should NOT run jobs out of your home directory --- run your jobs from the work or scratch filesystem; these provide better read and write performance to improve the speed of your job. After the job is finished, you might wish to copy the more critical results files back to your home directory. The home directory gets backed up nightly (unlike /work and /scratch) and it is shared with the linux system and IFY-HPC system at Department of Physics.
3.2. SimLab
The 20TB filesystem /simlab is intended for shared or project storage. It is not backed up, but it is a RAID-5 system which provides some basic protection. The filesystem is not optimized for speed, and it should not be used for running jobs. Normal users do not have write permission to all parts of this filesystem while reading permissions is for the most part granted.
Current the filesystems are organized into the following directories:
/simlab/backups
/simlab/data
3.3. Work
/work is the main global temporary storage (or global scratch) area on the tux cluster. Every user has a folder dedicated to them at /work/$USER.
We recommend that people use this filesystem as their primary job working area, as this area is optimized for cluster use (raid-0).
Use this area for processing large files, but realize that this filesystem is not backed up.
3.4. Scratch
/scratch is the local on-node temporary storage (or local scratch) on tux.
When running batch jobs, /scratch is a very fast filesystem for files created while a job is running.
This space is on the node's local hard drive.
It is a good place for temporary files created while a job is executing because the disks are local to the node that is performing the computation making access very fast. However, data are only accessible from that node during the time the job runs. Therefore, you cannot directly retrieve data after calculations are finished since the job scheduler slurm cleans up this storage area just before the job finishes. If you use /scratch, you should therefore make moving any data off it and onto another storage system part of your job script. The size of a nodes /scratch areas are listed by executing sinfo -Nl and inspecting the column TMP_DISK.