diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 12:19:44 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 12:19:44 -0500 |
| commit | 9763a8ecf4d614370cfde3ff926c5070ab766433 (patch) | |
| tree | 4ed0b067bfcf4a16f811f37b5723d728ffb5db74 /BASELINES.md | |
| parent | 04de9326e60bb9e3592c0ddb5474265d228cbce0 (diff) | |
baselines: complete native author-code audit
Diffstat (limited to 'BASELINES.md')
| -rw-r--r-- | BASELINES.md | 26 |
1 files changed, 16 insertions, 10 deletions
diff --git a/BASELINES.md b/BASELINES.md index 5051fca..9c3a320 100644 --- a/BASELINES.md +++ b/BASELINES.md @@ -4,7 +4,8 @@ This ledger separates three comparison types: 1. matched architectures/budgets inside this repository; 2. native author protocols reproduced without tracked source edits; -3. published author numbers used for context while reproduction is pending. +3. published author numbers retained as multi-seed context beside the completed + one-seed executable-fidelity reproductions. Native protocols are not equal-compute comparisons. Their value is to prevent weak in-house reimplementations from defining the state of the art. @@ -80,10 +81,11 @@ WANDB_DIR=/scratch/yurenh2/burstccn-wandb CUDA_VISIBLE_DEVICES=5 \ ``` The native trainer evaluates test every epoch and records both latest and best -test error. It has a validation loader but model saving is disabled in this -configuration. We will report final test accuracy as the primary reproduction, -native best-test accuracy separately, and explicitly label the latter as -test-selected. No run will be removed for a poor endpoint. +test error. The completed seed-0 record reports `80.07%` final test accuracy, +`80.10%` at the validation-selected epoch, and `80.25%` test-selected, versus +the published `82.97 +/- 0.21%`. Audited W&B wall time is `15451.3 s`. The lower +reproduced endpoint is retained rather than removed; the best-test value is +explicitly labelled test-selected. ## Dual Propagation native CIFAR-10 @@ -132,9 +134,13 @@ TF_FORCE_GPU_ALLOW_GROWTH=true \ The actual command also prepends the environment's NVIDIA wheel `lib` directories to `LD_LIBRARY_PATH`; this changes runtime linkage, not author source. A one-epoch full-VGG smoke completed without OOM/NaN in 203.4 training -seconds, reaching 20.48% validation and 20.04% test accuracy. The 130-epoch -seed is therefore scheduled as a long native reproduction rather than treated -as a cheap matched baseline. +seconds, reaching 20.48% validation and 20.04% test accuracy. The completed +130-epoch seed-1988 reproduction reaches `92.46%` test accuracy from the +best-validation checkpoint, versus the published `92.41 +/- 0.07%`. Test was +evaluated exactly once. Author-measured training, validation, and test time +sums to `23119.8 s`; this excludes setup and checkpoint I/O. The near-exact +accuracy reproduction establishes executable fidelity, not equal compute with +SDIL. ## Reporting rules for external runs @@ -155,5 +161,5 @@ freeze, the wrong physical GPU, and missing dataset hashes. Completed records live under `results/native/<protocol-id>.json`. The separate strict aggregator `experiments/audit_native_baselines.py` distinguishes final, validation-selected, and test-selected metrics and generates -`results/native_baseline_audit.md`. `experiments/finalize_accept.sh` remains -red until both author runs have passed this C4 audit. +`results/native_baseline_audit.md`. Both author runs pass this C4 audit, and +`experiments/finalize_accept.sh` is green. |
