docs: track reviewer score trajectory

1 files changed, 63 insertions, 0 deletions

diff --git a/REVIEW_SCORECARD.md b/REVIEW_SCORECARD.md
new file mode 100644
index 0000000..4cae7a1
--- /dev/null
+++ b/REVIEW_SCORECARD.md
@@ -0,0 +1,63 @@
+# ICLR-style reviewer scorecard
+
+This is a deliberately adversarial paper-level assessment, not an average of the best runs. It is
+updated only when a frozen experiment is complete and audited. Development pilots, partial jobs,
+and post-hoc subsets cannot raise the score. Failed gates remain visible and narrow the claim.
+
+## Scoring convention
+
+The overall score uses a 1--10 ICLR-like scale: 1 strong reject, 3 reject, 5 borderline reject,
+6 borderline accept, 8 accept, and 10 strong accept. Component scores use 1--4. The overall score
+is a reviewer judgment rather than the arithmetic mean of the components. Confidence uses 1--5.
+
+Every formal result report records:
+
+- the previous and new overall score;
+- soundness, novelty, significance, empirical support, and reproducibility;
+- which frozen evidence changed the assessment;
+- the strongest remaining reviewer objections;
+- whether the evidence is development, validation, or untouched confirmation.
+
+## Current paper-level assessment (2026-07-22)
+
+| Dimension | Score | Strict reviewer assessment |
+|:--|--:|:--|
+| Soundness | 3/4 | Theory, local-gradient checks, causal diagnostics, cost accounting, and frozen stop rules are unusually careful. The learned apical vectorizer remains an unresolved failure mode. |
+| Novelty | 2/4 | Learned node-perturbation feedback is prior art. The defensible novelty is the per-cell innovation operation under mixed apical traffic, together with its causal and scaling analysis. |
+| Significance | 3/4 | Near-flat performance over 12x depth while DFA alignment collapses is potentially important, but the current flattened-CIFAR task does not benefit from depth. |
+| Empirical support | 2/4 | Five-depth scaling and the residualization ablation are strong. Useful-depth C2, broad endogenous C1, and oral-B gates failed; standard ResNet-scale confirmation is still pending. |
+| Reproducibility | 4/4 | Code, exact provenance, seed panels, costs, failed branches, frozen selectors, and staged test-access rules are retained in git. |
+| **Overall** | **5/10** | **Borderline reject: a strong core result without a completed standard-scale or useful-depth demonstration.** |
+| Confidence | 4/5 | High confidence in the assessment because the positive and negative branches are both extensively audited. |
+
+### Evidence already carrying the paper
+
+- On flattened CIFAR-10, SDIL changes by only `-0.214 +/- 0.349` accuracy points from depth 5 to
+  depth 60, while DFA's early-layer alignment falls from `0.514` to `0.047`.
+- Under strong soma-predictable apical traffic, raw and norm-matched raw learning fall to about
+  chance (`10.38%` and `10.31%`) while innovation learning retains `97.35%` accuracy.
+- The local update has a proved descent condition and an explicit query/MAC/memory audit; direct
+  node perturbation isolates the learned vectorizer as the useful-depth bottleneck.
+
+### Objections currently preventing acceptance confidence
+
+1. The depth result is preservation on a depth-flat task, not evidence that SDIL uses added depth.
+2. The amortized apical vectorizer failed the frozen useful-depth gate; direct causal targets work
+   but cost `68.4x` ordinary forward-equivalent work.
+3. Innovation was not uniformly beneficial for arbitrary endogenous top-down traffic, so the
+   supported mechanism is narrower than the initial claim.
+4. The frozen oral-B screen falsified the desired-velocity and Harnett error-derivative claims.
+5. Native BurstCCN/Dual Prop endpoints and the standard CIFAR ResNet panel are not yet complete.
+
+## Score trajectory and prospective gates
+
+| Checkpoint | Overall | What changed | Remaining ceiling |
+|:--|--:|:--|:--|
+| Current audited package | 5 | Strong depth-preservation and residual-necessity evidence; negative gates retained | Standard useful scale is absent |
+| Native baselines complete | pending | Can close fairness/completeness objections, but cannot by itself establish the main claim | Usually no automatic score increase |
+| Oral-A A3 passes | pending | A full-data ResNet20 SDIL run is within 5 points of BP, beats DFA by 2, stays aligned, finite, and locally cost-audited | Plausible 6 if the evidence is clean |
+| Oral-A A4 passes | pending | Five untouched seeds across ResNet20/32/56 pass the frozen accuracy, scaling, alignment, and Pareto gates | Plausible 7--8; oral discussion becomes credible |
+
+These are conditional reviewer forecasts, not promised scores. A failed stage leaves its negative
+result in the record and can lower the score if it invalidates a current claim. Oral-B does not get
+retroactively reopened by success on standard vision benchmarks.