diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 02:13:33 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 02:13:33 -0500 |
| commit | 7e8d314c0fb8e82730e24da153e8b73c3af4ecd6 (patch) | |
| tree | 49b57e987648bd9352aa2f65ee8d12ee208160c1 | |
| parent | b9a9524ca559cc30a584779ba39f7764887d7fcf (diff) | |
results: freeze predictor timescale choice
| -rw-r--r-- | RESULTS.md | 27 | ||||
| -rw-r--r-- | ROADMAP.md | 5 | ||||
| -rw-r--r-- | sdil/core.py | 8 |
3 files changed, 36 insertions, 4 deletions
@@ -131,6 +131,33 @@ SDIL matches DFA on easy MNIST and **beats DFA on FashionMNIST (+1–2 pts) and On CIFAR the SDIL−DFA gap GROWS with depth (+1.8 → +2.7 → +5.0 at d5/10/20): DFA degrades with depth, SDIL holds. +## Endogenous top-down traffic timescale screen (validation only) + +This development screen used one model seed, a fixed 5,000-example stratified MNIST validation +split drawn only from training data, and never evaluated the test set. All 18 runs share validation +index hash `c4cec216...2e3d`, clean source revision `17bcd97`, and the same 15-epoch training +protocol. The grid crossed `eta_P in {0.002, 0.01, 0.05}`, neutral warmup steps in +`{0, 200, 1000}`, and endogenous top-down traffic scales in `{0.2, 0.5}`. + +| eta_P | warmup steps | rho=0.2 last val (%) | rho=0.5 last val (%) | mean (%) | worst (%) | +|------:|-------------:|---------------------:|---------------------:|---------:|----------:| +| 0.002 | 0 | 91.50 | 77.24 | 84.37 | 77.24 | +| 0.002 | 200 | 91.02 | 89.10 | 90.06 | 89.10 | +| 0.002 | 1000 | 93.10 | 88.82 | 90.96 | 88.82 | +| 0.010 | 0 | 94.94 | 92.40 | 93.67 | 92.40 | +| 0.010 | 200 | 92.88 | 91.12 | 92.00 | 91.12 | +| 0.010 | 1000 | 95.04 | 91.06 | 93.05 | 91.06 | +| 0.050 | 0 | 96.52 | 91.40 | 93.96 | 91.40 | +| **0.050** | **200** | **94.20** | **95.26** | **94.73** | **94.20** | +| 0.050 | 1000 | 94.70 | 93.62 | 94.16 | 93.62 | + +The selected development setting is `eta_P=0.05`, 200 warmup steps: it has the highest mean +last-epoch validation accuracy and the highest worst-traffic endpoint. Its final mean traffic R2 is +0.802 at rho=0.2 and 0.999 at rho=0.5. This reverses the initial assumption that `P` must be much +slower than forward learning: once teaching and traffic are separated by neutral periods, fast +tracking is beneficial, while a long warmup is not consistently useful. This is a frozen +validation choice for the multi-seed C1 experiment, not evidence that C1 has passed. + ## Legacy credit assignment / depth utility Per-layer alignment cos(r_l, −∇_{h_l}L), early-third average, CIFAR no-BN residual: | depth | DFA early-align | SDIL early-align | @@ -36,6 +36,11 @@ held-out accuracy by at least 2 percentage points over norm-matched raw feedback traffic level, without harming the no-traffic setting by more than 0.5 points. The predictor must remove measurable soma-predictable traffic while retaining causal teaching alignment. +The validation-only timescale screen froze `eta_P=0.05` and 200 neutral warmup steps by the highest +mean last-epoch accuracy across top-down traffic scales 0.2 and 0.5 (and the best worst-scale +endpoint). C1 must use this choice without further test-set tuning. Traffic projection seeds remain +separate from model/data seeds and must be recorded explicitly. + ### C2. SDIL scales when depth is useful Flattened CIFAR is retained as a preservation result, not as evidence that added depth is used. diff --git a/sdil/core.py b/sdil/core.py index cece194..8b3f923 100644 --- a/sdil/core.py +++ b/sdil/core.py @@ -18,7 +18,7 @@ For hidden layer l: Three learning rules, all local (no weight transport, no reverse-mode graph): dW_l = eta * (r_l ⊙ phi'(u_l)) h_{l-1}^T (three-factor: pre * gain * innovation) - dP_l = eta_P * (a_l - P_l h_l) h_l^T (slow; only on neutral periods) eta_P << eta + dP_l = eta_P * (a_l - P_l h_l) h_l^T (identified only on neutral periods) dA_l = eta_A * (q_l - r_l) c_l^T (on perturbation trials; q_l = causal node-pert estimate) q_l is a node-perturbation estimate of the causal descent direction @@ -513,9 +513,9 @@ def sdil_step(net, x, y, y_onehot, cfg, step, prev_error=None): dA = (qs[l] - r_list[l]).t() @ c / B # (n_l, n_classes) net.A[l] += cfg.eta_A * dA - # ================= predictor P (slow) ============================= - # KEY timescale-separation condition. P must learn the soma->apical - # coupling WITHOUT absorbing the teaching signal. On "neutral periods" + # ================= predictor P (neutral) ========================== + # KEY identification condition. P must learn the soma->apical coupling + # WITHOUT absorbing the teaching signal. On "neutral periods" # the apical compartment carries only the soma-predictable nuisance drive # (top-down teaching c is absent), so we fit P to the c=0 apical. This # makes P -> nuisance map (rho*Bnuis) and leaves the innovation intact. |
