diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-23 08:24:43 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-23 08:24:43 -0500 |
| commit | 6ebf1f1858590b53b47f67708cbd79aa57f79a67 (patch) | |
| tree | 07ffa9397133e2f696ddca92c0fe1ac276820ab5 /REVIEW_SCORECARD.md | |
| parent | 75a64888cb3e117ee6d232adc7b48974647fbc1e (diff) | |
paper: integrate calibrated oral-B-v2 evidence
Diffstat (limited to 'REVIEW_SCORECARD.md')
| -rw-r--r-- | REVIEW_SCORECARD.md | 40 |
1 files changed, 29 insertions, 11 deletions
diff --git a/REVIEW_SCORECARD.md b/REVIEW_SCORECARD.md index 1b23968..36da020 100644 --- a/REVIEW_SCORECARD.md +++ b/REVIEW_SCORECARD.md @@ -22,14 +22,20 @@ Every formal result report records: | Dimension | Score | Strict reviewer assessment | |:--|--:|:--| -| Soundness | 3/4 | Theory, local-gradient checks, causal diagnostics, cost accounting, and frozen stop rules are unusually careful. The learned apical vectorizer remains an unresolved failure mode. | +| Soundness | 4/4 | Theory, exact local-update checks, causal lesions, clustered uncertainty, cost accounting, and retained frozen failures make the implemented claims unusually well identified. | | Novelty | 2/4 | Learned node-perturbation feedback is prior art. The defensible novelty is the per-cell innovation operation under mixed apical traffic, together with its causal and scaling analysis. | -| Significance | 3/4 | Near-flat performance over 12x depth while DFA alignment collapses is potentially important, and dynamic innovation now reaches near-BP accuracy on a standard ResNet-20. Added-depth utility and broader biological generality remain absent. | -| Empirical support | 4/4 | Five-depth scaling, residual necessity, the frozen 91.18% ResNet-20 validation endpoint, and an untouched five-seed paired test confirmation at 91.584% are strong. Useful-depth C2, broad endogenous C1, oral-B, A3, and the original MT-1 remain disclosed failures. | +| Significance | 3/4 | Dynamic innovation reaches near-BP ResNet-20 accuracy and a separate local actor--critic reproduces role-vectorized outcome surprise. Added-depth utility and original-data biological validation remain absent. | +| Empirical support | 4/4 | Five-depth preservation, residual necessity, untouched ResNet-20 confirmation, and a complete 30-record task-clustered BCI confirmation are strong. Useful-depth C2, broad endogenous C1, old oral-B, A3, and the original MT-1 remain disclosed failures. | | Reproducibility | 4/4 | Code, exact provenance, seed panels, costs, failed branches, frozen selectors, and staged test-access rules are retained in git. | -| **Overall** | **7/10** | **Weak accept: untouched five-seed test confirmation establishes that dynamic innovation is robust and noninferior to strong clean KP on ResNet-20. Positive added-depth utility and the biological signature remain oral-level gaps.** | +| **Overall** | **8/10** | **Internal accept milestone: untouched confirmations establish both load-bearing ResNet-20 innovation and role-vectorized TD outcome surprise under the declared synthetic BCI paradigm.** | | Confidence | 4/5 | High confidence in the assessment because the positive and negative branches are both extensively audited. | +The conservative external-review forecast is **7/10**, not 8: a reviewer can +reasonably discount the synthetic BCI because terminal reward is supplied and +the psychometric target range is calibrated per policy. The 8/10 value is the +repository's predeclared evidence milestone; the external forecast is the +recommendation I would actually submit as a reviewer today. + ### Evidence already carrying the paper - On flattened CIFAR-10, SDIL changes by only `-0.214 +/- 0.349` accuracy points from depth 5 to @@ -39,6 +45,10 @@ Every formal result report records: - On untouched CIFAR-10 test endpoints, dynamic innovation reaches `91.584%` versus clean KP's `91.388%` over five paired ResNet-20 seeds; its paired deficit upper bound is only `0.131` points and every frozen mechanism/cost invariant passes. +- In the untouched six-task by five-model BCI panel, final task success is + `100%`, terminal residual outcome decoding is `99.83%`, the acute + outcome-lesion separation drop is `0.400`, critic expectedness is `0.319`, + and all 30 causal signs are positive after label-free calibration. - The local update has a proved descent condition and an explicit query/MAC/memory audit; direct node perturbation isolates the learned vectorizer as the useful-depth bottleneck. @@ -49,9 +59,10 @@ Every formal result report records: but cost `68.4x` ordinary forward-equivalent work. 3. Innovation was not uniformly beneficial for arbitrary endogenous top-down traffic, so the supported mechanism is narrower than the initial claim. -4. The temporal-difference recovery solves the task, passes the plasticity lesion, and yields - 30/30 positive sign inversions, but its untouched R2 panel fails residual outcome advantage and - longitudinal prediction. The broader Harnett-like population signature remains unsupported. +4. The passed BCI is synthetic: reward is directly supplied, causal roles are + experimenter-defined for diagnostics, and target quantiles are calibrated + on a separate cursor split. Longitudinal prediction remains failed, and no + original Francioni/Harnett event-level data are tested. 5. The standard-network result inherits reciprocal KP and pays for a paired neutral microphase. D4 establishes the innovation operation, not a new credit-transport mechanism or a literal cortical implementation. @@ -60,7 +71,7 @@ Every formal result report records: | Checkpoint | Overall | What changed | Remaining ceiling | |:--|--:|:--|:--| -| Current audited package | 7 | The untouched D4 panel reaches 91.584% dynamic versus 91.388% clean KP over five paired test seeds, with every mechanism and cost invariant passing | Added-depth utility and oral-B biology remain absent | +| Current audited package | 8 | D4 confirms load-bearing ResNet-20 innovation; calibrated oral-B-v2 confirms role-vectorized TD outcome surprise over 30 untouched records with all cluster bounds passing | Added-depth utility and original-data biological validation remain absent | | Native baselines complete | 5 | BurstCCN is below its published endpoint; Dual Prop reproduces 92.46% versus 92.41%, with strict provenance and cost semantics | Fairness objection narrows, but SDIL gains no standard-scale evidence | | Oral-A A1/A2 | 5 | BP reached 91.62%; short channel-gated SDIL reached 41.98% versus tuned DFA at 37.16% | Development screening alone cannot raise the score | | Oral-A A3 fails | 5 | Full ResNet-20 SDIL became nonfinite at epoch 89 and ended at 10%; DFA ended finite at 33.06% | Standard-scale and oral-A claims are closed; A4 remains untouched | @@ -83,14 +94,17 @@ Every formal result report records: | Dynamic projection D3 | 6 | All 19 frozen checks pass at 91.18%, within 0.44/0.08 points of BP/clean KP, with 0.9994 early alignment and 1.326x BP MACs | One validation seed cannot establish robustness | | Dynamic projection D4 | 7 | All ten untouched records pass: dynamic 91.584% versus clean KP 91.388%, paired upper deficit bound 0.131 points, early alignment 0.999687, and no invariant failures | Establishes ResNet-20 robustness/noninferiority, not positive depth utility | | Oral-B recovery R1/R2 | failed at R2 | R1 selects eta 0.1 with 98.05% worst-task success; untouched R2 retains 99.53% mean success and 30/30 positive signs but fails outcome-vectorization and longitudinal gates | Score remains 7; the joint oral-B claim is not established | +| Oral-B-v2 initial grid | failed at development | All 24 records preserve role learning and residual identification but fail from a one-quarter dense-signal cold start | Failure retained; no confirmation touched | +| Oral-B-v2 fixed-target recovery | failed at development | Two seeds pass 18/18; the third passes 17/18 but its fixed target ladder has 98.96% success | Mechanism works, absolute assay scale does not generalize | +| Oral-B-v2 calibrated R1/R2 | 8 | Three fresh development seeds pass, then all 30 untouched records and every task-cluster bound pass under independent label-free calibration/evaluation splits | Establishes synthetic outcome surprise; does not establish cortex or added-depth utility | | Oral-A dynamic depth recovery | closed | A 60-cell ResNet-20/32/56 BP/DFA/clean-KP/dynamic panel was frozen before any new endpoint | Its oral-B R2 prerequisite failed, so none of the 50 new cells may run | | Oral-A A4 | not opened | The prerequisite A3 gate failed | No oral-A confirmation claim is available | These are conditional reviewer forecasts, not promised scores. A failed stage leaves its negative result in the record and can lower the score if it invalidates a current claim. The original -oral-B branch remains failed and cannot be retroactively reopened by vision results. The -plasticity-only recovery passes R1 but fails its separately frozen R2 joint gate, so oral-A -remains closed. +oral-B branch and both v2 development failures remain failed. The calibrated +v2 pass does not reopen the old oral-A panel; it permits only a new +independently frozen oral-A-v2 protocol. ## Evidence-to-score log @@ -127,6 +141,10 @@ remains closed. | 2026-07-23 / `03c94a1` oral-B recovery R2 | Thirty untouched records retain 99.53% mean success, 90.45-point gain, 30/30 positive signs, and strong decorrelation, but fail seven population-vectorization/longitudinal checks | 7 → 7 | The recovery fixes learning, causal role, and sign but not the broader Harnett-like signature; oral-B and oral-A close without threshold repair | | 2026-07-23 / `2a6f72e` audited D4 main figure | The strict renderer independently rechecks the ten D4 records and visualizes paired test accuracy, layerwise raw-versus-innovation direction, all 200 tracking epochs, and explicit neutral/MAC/memory/wall costs | 7 → 7 | Makes the accept evidence reviewable without adding or selecting data; presentation improves, but visualization alone cannot repair oral-B or justify score inflation | | 2026-07-23 / `87cfb93` evidence-bound manuscript | A 3,238-word working draft binds 34 central numbers and all four figures to source manifests, retains the passed D4 gate and all seven failed R2 checks, and is re-audited by the accept finalizer | 7 → 7 | Substantially improves submission readiness and guards against claim drift; it adds no empirical evidence, so soundness and recommendation do not inflate | +| 2026-07-23 / `eb021a6` oral-B-v2 initial R1 | The complete 24-record grid learns causal roles and identifies innovations but reaches at most 0.78% evaluation success because terminal reward remains unreachable | 7 → 7 | Localizes a cold-start created by scaling the only pre-reward drive to one quarter; confirmation remains untouched | +| 2026-07-23 / `378e68d` fixed-target recovery R1 | Unit dense velocity restores 100% task learning and 17--18 biological checks per seed, but one fresh seed has 98.96% challenge success | 7 → 7 | Confirms the algorithmic repair while falsifying an absolute target ladder as a model-independent assay | +| 2026-07-23 / `70e180c`, `9a8c057` calibrated oral-B-v2 R1/R2 | Three new development seeds pass all 18 gates; all 30 untouched confirmation records then pass every clustered learning, innovation, decoder, lesion, and expectedness bound | 7 → 8 | Establishes role-vectorized TD outcome surprise in the synthetic paradigm and raises the formal milestone; ecological validity and added depth remain the external-review ceiling | +| 2026-07-23 / `75a6488` audited oral-B-v2 figure | The strict renderer visualizes all 30 records, task-cluster learning, residualization, independent psychometrics, and acute lesions | 8 → 8 | Improves reviewability without adding evidence or inflating the score | Future rows are appended only after an audited frozen stage. A score staying flat is informative: engineering, theory exposition, or visualization may make the paper more defensible without |
