From 8acf3f94c630c0ef2cc89c56d39464c42a0d5e3a Mon Sep 17 00:00:00 2001 From: yurenh Date: Mon, 31 Aug 2026 18:20:12 -0500 Subject: gitignore data/runs; data prep documented as on-node Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01GkgLsACEF6CCP7EUfA5fZe --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'README.md') diff --git a/README.md b/README.md index 9c038d5..c89bb5c 100644 --- a/README.md +++ b/README.md @@ -16,8 +16,9 @@ product are digital. See the main zobp repo for the method, theory (NSR ≈ c·d - `configs/` — model sizes (60m/124m/350m/1b) x training arms (bp / zbp_n16 / zbp_n4) ## Run +Data prep runs **on the training node** (H200), not on a dev machine; `data/` and `runs/` are gitignored. ``` -python scripts/prepare_data.py --dataset fineweb-edu --tokens 3e9 --out data/fineweb +python scripts/prepare_data.py --tokens 3e9 --out data/fineweb # on the H200 node torchrun --nproc_per_node=8 scripts/train.py --model configs/model/m124.yaml --train configs/train/zbp_n16.yaml ``` Global batch is fixed in the train config; per-rank micro-batch and accumulation adapt to world size. -- cgit v1.2.3