Moving Data with Globus

In this section you copy your training files from the Yens to Marlowe with the Globus command-line tool, with Claude Code on the Yens running the commands.

Globus copies files between storage systems directly: the data goes from the Yens to Marlowe without passing through your laptop, it checks every file when the copy finishes, and it retries if the connection drops. Each storage system appears in Globus as a collection. The Yens are the GSB/DARC yen collection; Marlowe’s /scratch is a collection you find by searching for Marlowe.

You will copy $WORK/data/ (about 270 MB) from the Yens to the same place on Marlowe:

	Path
From (Yens)	/scratch/users/<sunet>/ceo-turnover/data/
To (Marlowe)	/scratch/m000106/gsb_hackathon/<sunet>/ceo-turnover/data/

Jev’s labeled filings and the Kev-27B weights are already on Marlowe in the hackathon’s shared folders. You only copy your own training files.

---

Step 1: Start Claude Code on the Yens

ssh <sunet>@yen.stanford.edu
cd /scratch/users/$USER/marlowe-hackathon
ml globus-cli claude-code
claude

If this is your first time using Claude Code on the Yens, sign in with Continue with SSO and your Stanford address, as in the Yens onboarding course.

Step 2: Log in to Globus

Ask Claude to start the login:

> Run `globus login --no-local-server` and show me the URL it prints.

There is no browser on the Yens, so the login runs across both machines. Open the URL in the browser on your laptop, choose Stanford University, sign in with your SUNet ID and Duo, and click Allow. Globus shows an authorization code: paste it back into the terminal.

The login is saved in your home directory, so you do this once.

Step 3: Find the two collections

> Use the globus CLI to find the collection IDs for "GSB/DARC yen" and for Marlowe's /scratch, and save them as YEN and MARLOWE in ~/.globus-hackathon.env. Then list /scratch/m000106/gsb_hackathon/ on Marlowe to check access.

Read the commands Claude proposes before you approve each one. They should look like this:

globus endpoint search "GSB/DARC yen"
globus endpoint search "Marlowe"
globus ls "$MARLOWE:/scratch/m000106/gsb_hackathon/"

If globus ls says the collection needs more authentication, Globus prints a globus session consent ... command. Run it, sign in again in your laptop’s browser, and retry.

Step 4: Copy your training files

> Start a Globus transfer of /scratch/users/$USER/ceo-turnover/data/ on YEN to /scratch/m000106/gsb_hackathon/$USER/ceo-turnover/data/ on MARLOWE, recursive, labeled "ceo-turnover data", then wait for it to finish and show me the task status.

The commands behind it:

globus transfer "$YEN:/scratch/users/$USER/ceo-turnover/data/" \
  "$MARLOWE:/scratch/m000106/gsb_hackathon/$USER/ceo-turnover/data/" \
  --recursive --label "ceo-turnover data"
globus task wait <task-id>
globus task show <task-id>

A finished transfer shows Status: SUCCEEDED. Globus also emails you when a transfer finishes.

Step 5: Check on Marlowe

ls -lh /scratch/m000106/gsb_hackathon/$USER/ceo-turnover/data/
wc -l /scratch/m000106/gsb_hackathon/$USER/ceo-turnover/data/*.jsonl

   1000 .../test1000.jsonl
  54426 .../train.jsonl
   2000 .../valid.jsonl

The same steps copy results back, for example a model you trained on Marlowe, to run it on the Yens.