diff options
| author | yurenh <blackhao0426@gmail.com> | 2026-08-31 18:22:10 -0500 |
|---|---|---|
| committer | yurenh <blackhao0426@gmail.com> | 2026-08-31 18:22:10 -0500 |
| commit | d60d87e6cd3c1caf2187600b2c73035abf66dde6 (patch) | |
| tree | 1c0fbbeb2c948376484b1dbede65f9bf3499f700 /README.md | |
| parent | 8acf3f94c630c0ef2cc89c56d39464c42a0d5e3a (diff) | |
package: pyproject, Apache-2.0, install docs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GkgLsACEF6CCP7EUfA5fZe
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 9 |
1 files changed, 9 insertions, 0 deletions
@@ -5,6 +5,11 @@ Scaling study for **ZBP (zeroth-order backpropagation)** vs BP on decoder-only l forward queries only (score-space attention core + per-token SwiGLU FFN); linear maps and the score product are digital. See the main zobp repo for the method, theory (NSR ≈ c·d/n) and part-1/3 results. +## Install +``` +pip install -e . # or: pip install -e .[dev] && pytest tests/ +``` + ## Layout - `src/zbp_scaling/zbp/` — vendored ZBP package (probes, estimators, ZBPBlock autograd) - `src/zbp_scaling/model.py` — OLMo2-style transformer (RMSNorm, SwiGLU, untied embeddings; learned @@ -28,3 +33,7 @@ BP 312 ms/step; ZBP n=16 **8.7x**, n=64 **29x** (FLOPs-bound: probe batching is probe_chunk 8; the score core uses its own chunk <= 4 since its memory goes as chunk*B*H*T^2). Ladder arms: **bp / zbp_n16 / zbp_n64** per size (n=4 optional). Headroom if needed: torch.compile on the query path; forward differences (n+1 instead of 2n queries) as a cheaper biased arm. + +## Citation +Paper in preparation ("Backpropagation Without Jacobians"); see the main zobp research repo for the +method, theory and part-1/part-3 results. License: Apache-2.0. |
