Day 2 — The Yen-Slurm Cluster
Today you’ll move from running your code interactively — one piece at a time, watching it go — to handing bigger jobs off to the scheduled Yens and letting them run on their own. You’ll pick up the ideas and habits for that step by step: how to size up what a job needs before you run it, how to send it off and check how it went, how to fix it when something breaks, and how to write down what you did so you (or a labmate) can run it again.
By the end you will have done it on real work: nearly a thousand SEC filings processed by a job you sized yourself, then checked against what it actually used.
Here’s what we’ll do today:
| Part | Clock | What it delivers |
|---|---|---|
| Part 1 — Profile & Submit a Job | 9:00–10:30 | Measure what the script needs, declare it, and submit |
| Part 2 — Submit a Job Array | 10:30–noon | One submission, many tasks — up to ~992 filings |
Day 1 Recap
Where yesterday left you:
- Built a virtual environment on the Yens and installed dependencies from
requirements.txt - Ran
scripts/extract_form_3_one_file.pyinteractively, pulling fields out of one SEC Form 3 filing and validating them with Pydantic
Before You Start
Four things need to be in place. Say so now if any of them are missing:
- You can log in to a Yen through JupyterHub with your SUNetID
- Your fork, cloned to the Yens at
~/yens-onboarding-2026 - A virtual environment at
~/yens-onboarding-2026/.venv, withrequirements.txtinstalled - A
.envholding yourANTHROPIC_API_KEY, and.envin.gitignore
If you finish a section early, check whether anyone at your table is stuck before you move on to the bonus material. Explaining a thing you just learned is the fastest way to find out whether you actually learned it.
Work through the sections in order — they build on each other, and the numbers you write down in one are the inputs to the next. Each page ends with bonus work, folded away, for when you finish early.