summaryrefslogtreecommitdiff
path: root/results/native_baseline_audit.md
blob: 73262db718f8474dbc8da30470416bd547c0d656 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# Native author-code baseline audit

Both records passed the strict C4 artifact, source, protocol, environment,
dataset, completeness, finiteness, selection, and cost-definition checks.
These are method-native reproductions, not equal-compute comparisons with SDIL.

| method | reproduced seed | primary test (%) | validation-selected test (%) | test-selected test (%) | published (%) | audited wall (s) |
|:--|--:|--:|--:|--:|--:|--:|
| BurstCCN | 0 | 80.070 | 80.100 | 80.250 | 82.97 ± 0.21 | 15451.3 |
| Dual Prop | 1988 | 92.460 | 92.460 | — | 92.41 ± 0.07 | 23119.8 |

BurstCCN's primary metric is its final epoch. Its validation-selected metric is
the test accuracy from the minimum-validation-error epoch; its best test value
is explicitly test-selected because the native trainer evaluates test every
epoch. The `n_epochs=400` author configuration performs an epoch-0 evaluation
and 399 optimization epochs. Dual Prop evaluates test once after restoring the
best-validation checkpoint, so its primary and validation-selected values are
identical and no test-selected value exists.

The BurstCCN wall value is W&B run time through the final epoch log. The Dual
Prop wall value sums author-measured train, validation, and final test time and
excludes setup, checkpoint I/O, and diagnostic plotting. One reproduced seed
establishes executable fidelity; published multi-seed uncertainty is retained
for context and is not replaced by a one-seed error bar.