HF upload tooling: env-only auth, scoped-token guidance, optional ladder auto-upload

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GkgLsACEF6CCP7EUfA5fZe

4 files changed, 49 insertions, 0 deletions

diff --git a/.gitignore b/.gitignore
index 8cc215d..f46af0e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -5,3 +5,5 @@ __pycache__/
 *.pyc
 *.bin
 *.pt
+*.token
+.env
diff --git a/README.md b/README.md
index 128eed0..7b4f7a3 100644
--- a/README.md
+++ b/README.md
@@ -53,3 +53,15 @@ git add results && git commit -m "ladder results" && git push
 Analysis (anywhere): `python scripts/plot_ladder.py --results results/h200node1` -> per-run table, loss
 curves, and the gap-vs-scale figure (the paper's part-2 headline). If a specific checkpoint is needed for
 the estimator audits, scp just that `runs/<name>/ckpt.pt`.
+
+## HF upload & security (shared nodes)
+Results (and optionally checkpoints) can go to a **private** HF repo: `HF_UPLOAD=1 [HF_CKPT=1] ./scripts/run_ladder.sh`
+or manually `python scripts/upload_hf.py --results results/<tag> [--with-ckpt runs]` (default repo
+`<whoami>/zbp-scaling-runs`, created private if missing).
+
+Uploads authenticate ONLY via the `HF_TOKEN` environment variable or a standard `hf auth login`; tokens are
+never CLI arguments (argv is world-readable via /proc on shared machines), never written by our scripts, and
+`.gitignore` excludes token-like files. On a shared node, mint a **fine-grained HF token scoped to the single
+private repo** (write permission only), `export HF_TOKEN=...` per session, and revoke it after the campaign.
+Zero-token alternative: push only the small JSONL results to GitHub (a repo-scoped deploy key suffices) and
+upload checkpoints from a trusted machine.
diff --git a/scripts/run_ladder.sh b/scripts/run_ladder.sh
index 17219bd..1d412a0 100755
--- a/scripts/run_ladder.sh
+++ b/scripts/run_ladder.sh
@@ -66,4 +66,9 @@ for size in $SIZES; do
     touch "$dir/DONE"
   done
 done
+if [ "${HF_UPLOAD:-0}" = 1 ]; then
+  TAG=${HF_TAG:-$(hostname)-$(date +%Y%m%d)}
+  python scripts/collect.py --runs "$OUT" --out "results/$TAG"
+  python scripts/upload_hf.py --results "results/$TAG" ${HF_REPO:+--repo "$HF_REPO"} ${HF_CKPT:+--with-ckpt "$OUT"}
+fi
 echo "== ladder complete: $OUT"
diff --git a/scripts/upload_hf.py b/scripts/upload_hf.py
new file mode 100644
index 0000000..284b9a9
--- /dev/null
+++ b/scripts/upload_hf.py
@@ -0,0 +1,30 @@
+"""Upload collected results (and optionally checkpoints) to a private HF repo.
+
+AUTH: read STRICTLY from the environment -- the HF_TOKEN env var or a standard `hf auth login`.
+Never passed as a CLI argument (argv is world-readable on shared nodes), never written to any file.
+On shared nodes use a FINE-GRAINED token scoped to this single repo with write-only permission,
+exported per session:  export HF_TOKEN=hf_...   (see README "Security").
+
+    python scripts/upload_hf.py --results results/h200node1 [--repo user/zbp-scaling-runs] [--with-ckpt runs]
+"""
+import os, glob, argparse
+from huggingface_hub import HfApi
+
+p = argparse.ArgumentParser()
+p.add_argument("--results", required=True)
+p.add_argument("--repo", default=None, help="default: <whoami>/zbp-scaling-runs")
+p.add_argument("--with-ckpt", default=None, help="runs dir: also upload runs/*/ckpt.pt (large!)")
+a = p.parse_args()
+api = HfApi()                       # token from env / login cache only
+repo = a.repo or f"{api.whoami()['name']}/zbp-scaling-runs"
+api.create_repo(repo, private=True, exist_ok=True)
+tag = os.path.basename(os.path.normpath(a.results))
+api.upload_folder(folder_path=a.results, path_in_repo=f"results/{tag}", repo_id=repo,
+                  commit_message=f"results: {tag}")
+print(f"uploaded results/{tag} -> https://huggingface.co/{repo}")
+if a.with_ckpt:
+    for c in sorted(glob.glob(os.path.join(a.with_ckpt, "*", "ckpt.pt"))):
+        name = os.path.basename(os.path.dirname(c))
+        api.upload_file(path_or_fileobj=c, path_in_repo=f"ckpts/{tag}/{name}.pt", repo_id=repo,
+                        commit_message=f"ckpt: {tag}/{name}")
+        print(f"uploaded ckpts/{tag}/{name}.pt")