EVIDENCE_SUMMARY: add §3.5 sensitivity, §3.6 cross-width, §4 separability, figures section

1 files changed, 41 insertions, 8 deletions

diff --git a/protocol/EVIDENCE_SUMMARY.md b/protocol/EVIDENCE_SUMMARY.md
index 517f7c7..395d875 100644
--- a/protocol/EVIDENCE_SUMMARY.md
+++ b/protocol/EVIDENCE_SUMMARY.md
@@ -43,6 +43,32 @@ and the file or memory entry where the result is recorded.
 | Cross-architecture temporal figure | 3×3 grid: ResMLP/ViT/StudentNet × ‖h_L‖/‖g_L‖/acc | `python -m protocol.examples.plot_temporal_cross_arch --seed 42` |
 | Training-monitor early-stop demo | DFA: protocol fires at ep 4, saves 96% compute, 0pp acc loss; BP: never fires, runs to completion | `python -m protocol.examples.training_monitor_demo` |
 
+## §3.5 Threshold sensitivity (calibration evidence)
+
+| evidence | result | reproduce |
+|---|---|---|
+| Threshold sensitivity sweep | (a) **63× separation gap**, (b) **24,338× separation gap** between healthy and degenerate | `python -m protocol.examples.threshold_sensitivity` |
+| Default thresholds | sit cleanly in the middle of substantial margins | (in sensitivity output) |
+
+## §3.6 Cross-width validation (d=512)
+
+| evidence | result | reproduce |
+|---|---|---|
+| 4-method audit at d=512 | BP trustworthy on 3/3 seeds; DFA/SB/CB walked back on 3/3 (same as d=256) | `python -m protocol.examples.audit_d512` |
+| Width effect | max-per-block growth is HIGHER at d=512 (6e3-7e4 vs ~1e3 at d=256) | (in d=512 output) |
+
+## §4 Two failure modes
+
+| evidence | result | reproduce |
+|---|---|---|
+| Penalty rescue (3 seeds, λ=1e-2) | DFA acc 0.308 → 0.363, ‖h_L‖ 4e8 → 4e4, ‖g_L‖ 5e-10 → 1e-6 | `dfa_residual_penalty_test.py --lam 1e-2` |
+| **Penalty partial protocol audit** | Penalized DFA: (a)+(b) **PASS** (penalty fixes scale), but (d) **STILL FIRES** on 3/3 seeds (margin 1.38 ± 0.05 pp < 2 pp) | `python -m protocol.examples.penalty_partial_audit` |
+| Vanilla DFA per-layer cosine (3 seeds) | layer 0: cos = +0.42 (high), layers 1-4: cos ≈ 0 (range -0.03 to +0.03). Headline +0.07 is entirely from layer 0. | `python experiments/measure_direction_quality_existing_ckpt.py --seed 42` |
+
+The two failure modes are mechanistically separable: the penalty fixes the
+scale failure (a+b pass) but not the direction failure (d still fires).
+This is the cleanest possible separability evidence.
+
 ## §5 Pipeline pitfalls reproducers
 
 All 6 bugs from `protocol/CHECKLIST.md` have a reproducer:
@@ -90,19 +116,26 @@ All 6 bugs from `protocol/CHECKLIST.md` have a reproducer:
 The penalty rescues by +5.5 pp over vanilla DFA but only +1.4 pp over the
 shallow baseline; mechanism is necessary but not sufficient.
 
+## Figures (paper-ready)
+
+- `results/protocol_audit/figure_audit_5method.png` — §2/§3 hero figure: 4-panel horizontal bar chart of the 5-method audit, color-coded healthy/degenerate, threshold lines drawn
+- `results/protocol_audit/figure_cross_arch_temporal_s42.png` — §4 cross-architecture temporal evolution: 3 architectures × 3 metric rows (‖h_L‖, ‖g_L‖, acc) with floor line
+
 ## Status of evidence
 
 - §1 protocol package: **DONE**, committed
-- §2 audit findings: **DONE** for ResMLP single + 3-seed; ViT audit waiting on checkpoint training
-- §3 decision utility: **DONE**, ablation table + sensitivity analysis ready
+- §2 audit findings: **DONE** for ResMLP at d=256 (3 seeds, single seed) and d=512 (3 seeds); ViT audit waiting on checkpoint training
+- §3 decision utility: **DONE**, ablation table + threshold sensitivity ready
+- §3 hero figure: **DONE**
 - §4 temporal validation: **DONE** for 3 architectures × 3 seeds (ResMLP, ViT, StudentNet)
+- §4 cross-arch figure: **DONE**
+- §4 two failure modes (separability): **DONE** via penalty partial audit + vanilla cosine measurement
 - §5 pipeline pitfalls: **DONE**, all 6 reproducers
-- §4 figure: **DONE** for cross-arch single-seed
-- Direction-quality direct test: **RUNNING** (vanilla phase silent)
-- ResNet experiment: **RUNNING** (ep 1 of 60)
-- ViT checkpoint training: **RUNNING** (just started)
-- λ=1e-1 penalty data: **RUNNING** (ep 30 of 100)
-- Penalized DFA protocol application: **WAITING** for direction-quality checkpoint
+- Direction-quality direct test on penalized checkpoint: **RUNNING** (gives mechanistic explanation of (d) failure on penalized net)
+- ResNet experiment: **RUNNING** (ep 1 of 60, slow due to GPU contention)
+- ViT checkpoint training: **RUNNING** (ep 1 of 60)
+- λ=1e-1 penalty data: **RUNNING** (ep 40 of 100)
+- ViT 5-method audit: **WAITING** for ViT checkpoint
 
 ## Open questions for paper writeup