3  Practical Examples

Author

Sean Taylor, Lindsay Clark, Yeji Bae, and Glenn Morton

Published

May 7, 2026

3.1 Using slurm

3.1.1 Basic slurm syntax

3.1.1.1 Requesting a single CPU core

To request 1 CPU, 7.5G of memory, and 1 hour walltime:

sbatch --partition=cpu-core-sponsored --account=cpu-core-sponsored \
  --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=7.5G \
  --time=0-01:00:00 myscript.py

3.1.1.2 Requesting multiple CPU cores

To request 4 CPU cores, 30G of memory (4 * 7.5G), and 2 days walltime:

sbatch --partition=cpu-core-sponsored --account=cpu-core-sponsored \
  --nodes=1 --ntasks=1 --cpus-per-task=4 --mem-per-cpu=7.5G \
  --time=2-00:00:00 my_multi_cpu_script.sh

3.1.1.3 Requesting a GPU

To request 1 GPU, 32 CPU cores, 480G of memory, and 30 minutes walltime:

sbatch --partition=gpu-core-sponsored --account=gpu-core-sponsored \
  --nodes=1 --ntasks=1 --gpus-per-task=1 \
  --cpus-per-gpu=32 --mem-per-gpu=480G \
  --time=0-00:30:00 my_gpu_script.py

3.1.1.4 Requesting multiple GPUs and CPU cores

To request 2 GPUs, 64 CPU cores (2 * 32), 960G of memory (2 * 480G), and 15 min walltime:

sbatch --partition=gpu-core-sponsored --account=gpu-core-sponsored \
  --nodes=1 --ntasks=1 --gpus-per-task=2 \
  --cpus-per-gpu=32 --mem-per-gpu=480G \
  --time=15:00 my_multi_gpu_script.sh

3.1.1.5 Requesting an Interactive Session

To request an interactive session with 1 CPU, 7.5G of memory, and 1 hour walltime:

srun --partition=cpu-core-sponsored --account=cpu-core-sponsored \
  --nodes=1 --ntasks=1 --cpus-per-task=1 --mem-per-cpu=7.5G \
  --time=01:00:00 --pty /bin/bash

See Chapter 11 for more details.

3.1.2 Slurm batch files

3.1.2.1 Full sbatch Example

Create a slurm batch file from this template:
file example.slurm

#!/bin/bash

#SBATCH --account=cpu-test-sponsored
#SBATCH --partition=cpu-test-sponsored
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=7.5G
#SBATCH --time=0-00:01:00
#SBATCH --mail-type=ALL

# ^ Resource request
#------------------------------------------------------------------------------
# v Execution script

# log the current working directory
pwd

# run script
echo "hello world"

echo "FINISHED!"

command:

sbatch --mail-user=mickey.mouse@company.org example.slurm

See Section 11.5 for more information.

3.1.3 System info

3.1.3.1 Checking available accounts and partitions

To view a summary of the accounts and partitions you can use:

sshare -o "Account%40,Partition%40"

3.1.3.2 Cluster and Partition status

To see a summary of all partitions, including their availability and the number of nodes in each state (e.g., idle, allocated, down):

sinfo --summarize

3.1.3.3 Queue status

To see a list of jobs sorted by priority, which can help you estimate your own job’s wait time:

squeue -l --sort=-P

3.1.3.4 Job status

To get more detailed information about your pending jobs, including estimated start times:

squeue -u $USER -l --start

To view information for a running or recently completed job:

scontrol show job <JOB_ID>

To view real-time statistics for jobs that are currently running:

sstat --jobs=<JOB_ID> --format=JobID,AveCPU,MaxRSS,NTasks

To see CPU time and maximum memory usage for a specific completed job:

sacct -j <JOB_ID> --format=JobID,JobName,State,CPUTime,MaxRSS,Elapsed

To cancel a pending or running job

scancel <jobid>

See Chapter 12 for more information.

3.2 Using Apptainer on Sasquatch

3.2.1 Running an executable or script within apptainer

To run a script from inside a container with home and association bound:

apptainer exec --bind /data/hps/assoc <container_path> <command script>

3.2.2 Submitting a job with apptainer

To submit a CPU job that runs my_cpu_script.py inside the my_container.sif container:

sbatch --partition=cpu-core-sponsored --account=cpu-core-sponsored \
  --nodes=1 --ntasks=1 --cpus-per-task=8 --mem-per-cpu=7.5G \
  --time=0-04:00:00 \
  --wrap="apptainer exec --bind /data/hps/assoc my_container.sif python3 my_cpu_script.py"

To submit a GPU job that runs my_gpu_script.py inside the my_container.sif container:

sbatch --partition=gpu-core-sponsored --account=gpu-core-sponsored \
  --nodes=1 --ntasks=1 --gpus-per-task=1 \
  --cpus-per-gpu=32 --mem-per-gpu=480G \
  --time=0-04:00:00 \
  --wrap="apptainer exec --nv --bind /data/hps/assoc my_container.sif python3 my_gpu_script.py"
  • --nv This is a crucial flag for GPU jobs. It enables the container to access the NVIDIA GPU drivers and libraries from the host system. It is not necessary for CPU jobs.

3.2.3 Using a batch script for apptainer jobs

Create a file named job_script.sh with the following content

#SBATCH --account=cpu-core-sponsored
#SBATCH --partition=cpu-core-sponsored
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=8
#SBATCH --mem-per-cpu=7.5G
#SBATCH --time=0-04:00:00

# ^ Resource request
#------------------------------------------------------------------------------
# v Execution script

apptainer exec --bind /data/hps/assoc my_container.sif python3 my_cpu_script.py

Then submit the job using the batch script:

sbatch job_script.sh

See Chapter 20 for more information.

3.3 Running common software

3.3.1 R

Note: It is recommended to use RStudio in Posit Workbench for interactive workloads with R. R is provided inside the Posit Apptainer container located at /data/hps/assoc/public/bioinformatics/container/posit, and it’s execution follows the basic form for any Apptainer command:

CONTAINER=/data/hps/assoc/public/bioinformatics/container/posit/posit-base-20250501.sif
apptainer exec --bind /data/hps/assoc ${CONTAINER} /opt/R/4.5/R

See Chapter 16 for more details on best practices for launching R and running R scripts.

3.3.2 Cell Ranger

Cell Ranger is provided as a system module:

module load cellranger - load latest version of cellranger
module unload cellranger - unload the loaded cellranger module
module load cellranger/8.0.0 load cellranger/8.0.0 explicitly

3.4 Install and run software using mamba

3.4.1 Building a simple conda environment

To build a mamba environment called fastqc with FastQC installed from the bioconda channel:

mamba create --name fastqc -c bioconda fastqc

3.4.2 Building a conda environment from yml:

Create a file called fastqc.yml with the following content:

name: fastqc
channels:
  - bioconda
dependencies:
  - fastqc

Then you can use this to create the environment from the file.

mamba env create --name fastqc --file fastqc.yml

3.4.3 Activating/Deactivating environments

To activate an environment named fastqc

mamba activate fastqc

To deactivate an environment

mamba deactivate

Please refer to Chapter 18 for details on installing and using conda.

3.5 Accessing your files

3.5.1 Files on RSS (helens or baker)

RSS is mounted only on the login nodes. To move data from RSS to HPC storage:

rsync -rzP /data/rss/helens/mouse_m/thingtocopy /data/hps/assoc/private/mylab/user/mmouse

3.5.2 Files on your computer

From your Company imaged laptop, desktop, or Windows VDI, you can use the scp command to move files back and forth to Sasquatch. You will need to be connected to the Company network or on Zscalar. Open a Terminal in Mac or Linux, or the Command Prompt in Windows. Use the cd command to navigate to the directory on your computer where you want to send or receive files. (Hint: on Windows you might find your Documents folder is actually "OneDrive\Documents".)

To copy a file from your computer to Sasquatch do

scp myfile.txt username@login-1.hpc:/data/hps/assoc/private/mylab/user/mmouse

Likewise to copy a file from Sasquatch to your computer, do

scp username@login-1.hpc:/data/hps/assoc/private/mylab/user/mmouse/myfile.txt .

You can copy a whole folder with

scp -r myfolder username@login-1.hpc:/data/hps/assoc/private/mylab/user/mmouse

See Chapter 8 for more details.

3.6 Using tmux

To start a new tmux:

tmux new -s mycoolsession

To reattach to an existing session:

tmux a -t mycoolsession

To list all sessions:

tmux ls

To scroll up/down:

Ctrl + b [ and then use the up and down arrows to scroll.

Press q when you are done.

See Tmux Cheat Sheet for more help.