The Cluster File System

This section covers how storage is organized on the Yens: where your files live, how much space you have, and what software is available. Knowing this early helps you avoid running out of quota mid-job or being unable to find the software you need.

Exercise

Before you work with data files, learn the layout: what storage is yours, how much space you have, and what software the cluster already provides.

Task: Explore the Yens file system — find out where your data lives, how much space you have, and what software is available.

The file system layout:

Rule of thumb: the project itself — scripts, data, and outputs — lives in projects; your personal files live in home (both are backed up); scratch is for big temporary files you don’t need to keep.

How to organize your work on the Yens:

A project — its scripts, data, and outputs → /yen/projects/faculty/your_project/. This is the shared, backed-up home for the project itself; keep raw data and outputs in separate subfolders (e.g. data/ and output/) so they never get mixed up. Access is controlled by the project’s workgroup: everyone in it can read and write, which is how you, your PI, and collaborators share the same files. You may belong to several project workgroups at once, each with its own folder under /yen/projects/faculty/ (or /yen/projects/students/). See Workgroups on RCpedia for who gets access and how it’s managed.

Personal files → your home, /home/users/SUNetID/. Things that are yours, not any one project’s: authentication tokens, R or shell preferences, quick one-off experiments. Backed up, and only you can see it.

Large, temporary things → /scratch/users/SUNetID/. Fast and roomy, but not backed up and periodically cleared. Use it for things you don’t need to keep or that won’t fit in your quota — a big public dataset you’re exploring, or an LLM you’re testing out. Copy anything worth keeping back to /yen/projects/.

Local disk: /tmp

Those three locations — home, projects, and scratch — are all on the shared file system: every node sees the same files. Each node also has its own local disk that is not shared with other nodes, and /tmp lives there — it’s where programs often write temporary files while they run.

Two things to know about /tmp: it’s private to that node (a file at /tmp on yen1 isn’t visible on yen2), and it’s temporary and not backed up (cleared automatically). Keep anything you care about — data, results — on the shared file system.

Check your quota:

gsbquota                             # shows home and scratch usage for your account
gsbquota /yen/projects/faculty/your_project  # append a path to check usage for a project folder

Browse storage in a visual file manager:

gsbbrowser                # opens an interactive file size browser in the terminal
# navigate with arrow keys, q to quit

See what software modules are available:

module avail
module avail python
module load python/3.11
python3 --version
module list
module unload python/3.11

module avail — list all available software modules
module avail python — filter that list by name
module load python/3.11 — load a specific version (adjust to what’s available)
python3 --version — confirm it loaded. Note it’s python3, not python — on the Yens the plain python command does not exist
module list — see what’s currently loaded
module unload python/3.11 — unload it

🟢 Green sticky = I’m done and ready 🔴 Red sticky = I need help

Put a sticky note on your laptop lid so instructors can see where you are.

Exercise complete