diff options
| author | yurenh <blackhao0426@gmail.com> | 2026-08-31 18:26:01 -0500 |
|---|---|---|
| committer | yurenh <blackhao0426@gmail.com> | 2026-08-31 18:26:01 -0500 |
| commit | 76652293b39fd97a2e7870e46ad3af63f9457cc1 (patch) | |
| tree | 5cc28e87545847a3fae3b010c62232180269f5f7 /configs/train | |
| parent | 98d05c2f52527d31cf1872f8eda19d1b6d91bc5b (diff) | |
add zbp_n64 train config; guard missing configs in the ladder runner
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GkgLsACEF6CCP7EUfA5fZe
Diffstat (limited to 'configs/train')
| -rw-r--r-- | configs/train/zbp_n64.yaml | 16 |
1 files changed, 16 insertions, 0 deletions
diff --git a/configs/train/zbp_n64.yaml b/configs/train/zbp_n64.yaml new file mode 100644 index 0000000..b5f87f6 --- /dev/null +++ b/configs/train/zbp_n64.yaml @@ -0,0 +1,16 @@ +mode: cd +n_probes: 64 +probe: rademacher +eps: 0.1 +probe_chunk: 8 +measure_fp32: true +tokens: 2e9 +global_batch_tokens: 524288 +lr: 6e-4 +warmup: 2000 +optimizer: adamw +wd: 0.1 +betas: [0.9, 0.95] +grad_clip: 1.0 +eval_every: 500 +seed: 0 |