diff options
| author | yurenh <blackhao0426@gmail.com> | 2026-08-31 18:20:12 -0500 |
|---|---|---|
| committer | yurenh <blackhao0426@gmail.com> | 2026-08-31 18:20:12 -0500 |
| commit | 8acf3f94c630c0ef2cc89c56d39464c42a0d5e3a (patch) | |
| tree | c3cec35990ce885b4d6000b2c13240cca7c419c3 | |
| parent | 17a81b9c86cfedd70812a0e83f33798b64c1678e (diff) | |
gitignore data/runs; data prep documented as on-node
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GkgLsACEF6CCP7EUfA5fZe
| -rw-r--r-- | .gitignore | 6 | ||||
| -rw-r--r-- | README.md | 3 | ||||
| l--------- | data/wikitext103 | 1 |
3 files changed, 8 insertions, 2 deletions
diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..3b68828 --- /dev/null +++ b/.gitignore @@ -0,0 +1,6 @@ +data/ +runs/ +__pycache__/ +*.pyc +*.bin +*.pt @@ -16,8 +16,9 @@ product are digital. See the main zobp repo for the method, theory (NSR ≈ c·d - `configs/` — model sizes (60m/124m/350m/1b) x training arms (bp / zbp_n16 / zbp_n4) ## Run +Data prep runs **on the training node** (H200), not on a dev machine; `data/` and `runs/` are gitignored. ``` -python scripts/prepare_data.py --dataset fineweb-edu --tokens 3e9 --out data/fineweb +python scripts/prepare_data.py --tokens 3e9 --out data/fineweb # on the H200 node torchrun --nproc_per_node=8 scripts/train.py --model configs/model/m124.yaml --train configs/train/zbp_n16.yaml ``` Global batch is fixed in the train config; per-rank micro-batch and accumulation adapt to world size. diff --git a/data/wikitext103 b/data/wikitext103 deleted file mode 120000 index fbd5f64..0000000 --- a/data/wikitext103 +++ /dev/null @@ -1 +0,0 @@ -/home/yurenh2/zobp/data/wikitext103
\ No newline at end of file |