summaryrefslogtreecommitdiff
path: root/RESULTS.md
diff options
context:
space:
mode:
authorYurenHao0426 <Blackhao0426@gmail.com>2026-07-22 12:10:39 -0500
committerYurenHao0426 <Blackhao0426@gmail.com>2026-07-22 12:10:39 -0500
commit04de9326e60bb9e3592c0ddb5474265d228cbce0 (patch)
tree2066af6e8b164ea825934a416581f73dbaa13c58 /RESULTS.md
parent09e8ccdf2404ba0691bffeab00cf223fcbd13222 (diff)
results: record failed Oral-A full ResNet gate
Diffstat (limited to 'RESULTS.md')
-rw-r--r--RESULTS.md26
1 files changed, 26 insertions, 0 deletions
diff --git a/RESULTS.md b/RESULTS.md
index cd74107..5ef402a 100644
--- a/RESULTS.md
+++ b/RESULTS.md
@@ -586,6 +586,32 @@ strong modern EP variant remains necessary. DFA is also strong on easy MNIST, so
comes from the five-depth CIFAR panel; this comparison establishes that SDIL need not trade away
accuracy to obtain a large cost advantage over canonical EP.
+### Frozen standard-ResNet development funnel
+
+The prospective Oral-A funnel used a 45k/5k training-only CIFAR-10 split; no
+test metric was evaluated. The exact-BP ResNet-20 reference passed A1 at
+`91.62%` validation accuracy. In the 10k-example, 20-epoch A2b screen, selected
+channel-gated SDIL reached `41.98%` with mean teaching alignment `0.1021`,
+versus `37.16%` for tuned DFA and `74.94%` for BP. This was a positive
+development result, not confirmation.
+
+The full 200-epoch A3 gate failed:
+
+| method | final val acc (%) | finite | early-third alignment | estimated MACs |
+|:--|--:|:--:|--:|--:|
+| BP | 91.62 | yes | n/a | 1.095e15 |
+| DFA | 33.06 | yes | 0.000473 | 7.469e14 |
+| SDIL | 10.00 | no | NaN | 9.165e14 |
+
+SDIL reached `35.26%` at epoch 20, fell to `22.82%` at epoch 80, and became
+nonfinite at epoch 90. It passed only the BP-reference and MAC checks; it failed
+the within-5-point, DFA-advantage, alignment, and all-finite checks. The failure
+is specifically for the frozen recipe selected under the short cosine screen
+and transferred to the long step schedule. It is not evidence that every
+possible SDIL optimizer must diverge, but it does falsify the preregistered
+standard-ResNet scaling claim. A4 was not opened and confirmation seeds 10--14
+were not touched.
+
## How to run
`experiments/run.py --mode {bp,fa,dfa,sdil} --dataset {mnist,fmnist,cifar10} --depth D --residual {0,1} --act {tanh,gelu,silu,relu}`
Batteries: `experiments/run_v2.sh <ds> "<depths>" <res> <act> "<seeds>" <ep> <pfx>`.