path: root/BASELINES.md

# Native-baseline reproduction ledger

This ledger separates three comparison types:

1. matched architectures/budgets inside this repository;
2. native author protocols reproduced without tracked source edits;
3. published author numbers retained as multi-seed context beside the completed
   one-seed executable-fidelity reproductions.

Native protocols are not equal-compute comparisons. Their value is to prevent
weak in-house reimplementations from defining the state of the art.

## In-repository baselines

The audited implementations and completed results are in `RESULTS.md`:

- BP, FA, and DFA share the SDIL forward architecture and data order;
- convolutional hierarchical FA (`hfa`) mirrors the ResNet residual DAG with
  independent random 3x3 feedback tensors, exact parameter-free shortcut
  adjoints, and local ReLU/BatchNorm Jacobians; it never copies or reads a
  downstream forward convolution and is the mandatory baseline for any learned
  hierarchical extension. Its frozen validation-only screen is specified in
  `HFA_BASELINE.md`; the selected short endpoint is 43.52% versus matched DFA's
  37.16%, but its 50% full-run gate failed;
- normalized response mirroring (`wm`) is an inherited weight-mirror/weight-
  estimation control: Gaussian parent probes pass through ordinary local
  forward synapses, and a separate update consumes only parent/child activities
  to estimate each feedback kernel. It is deliberately treated as a strong
  learned-FA baseline, not as SDIL novelty;
- residual response mirroring (`rrm`) replaces WM's noisy estimate-then-average
  step with local normalized LMS on the child-response prediction error. Its
  stochastic update vanishes sample-by-sample at Q=W; the extra feedback
  prediction convolution is included in MAC accounting. It is also an
  inherited learned-alignment control. Its short gate passes, but its sole
  full run ends at 10% with NaN validation loss despite 0.999998 endpoint Q/W
  cosine, closing intermittent RRM as a scalable baseline;
- modified Kolen--Pollack (`kp`) is the reciprocal local-plasticity baseline of
  Akrout et al. Forward and feedback synapses independently recompute the same
  correlation from their own local activity pair and use matched optimizer and
  decay dynamics. The feedback update never reads W or its update. A separate
  reciprocal correlation is charged in the MAC audit; `KP_BASELINE.md` freezes
  its staged ResNet gate before any KP task endpoint. The frozen KP-1 record
  passes at 82.66% validation accuracy versus matched BP's epoch-20 81.02%,
  with 0.886 early alignment and 1.326x BP MACs. The frozen KP-2 record also
  passes at 91.26% versus matched BP's 91.62%, with 0.9994 early alignment,
  0.9997 late feedback cosine, zero loss queries, and the same 1.326x MAC
  ratio;
- direct node perturbation uses a causal target on every hidden update and no
  learned vectorizer;
- the no-traffic, `P=0` SDIL backbone is learned direct feedback by node
  perturbation in the sense of Lansdell, Prakash, and Kording (2020);
- PEPITA and Forward-Forward use paper/reference-code schedules;
- canonical EP uses persistent free particles and the published shallow/deep
  relaxation schedules.

The main ledger reports accuracy, wall time, scalar loss observations,
forward-equivalent examples, peak memory, and any test-selection differences.

## BurstCCN native CIFAR-10

- Paper/model: Greedy et al., *Cell-type-specific cortical feedback coordinates
  hierarchical credit assignment* (2026),
  <https://www.biorxiv.org/content/10.64898/2026.06.16.732595v1>.
- Author repository: <https://github.com/neuralml/BurstCCN-journal>.
- Frozen revision: `9170e110b4d18729dc5bc5d517fc52eefcd04d69`.
- External checkout: `/scratch/yurenh2/sdil-reference/BurstCCN-journal` on
  `timan107`.
- Environment: `/scratch/yurenh2/venvs/burstccn`.
- Dataset root: `/home/yurenh2/sdrn/data`.

The repository's top-level `cifar10_burstccn_fa` default currently resolves to
sigmoid/MSE with fixed random feedback. That configuration stayed at chance in
an initial compatibility run and does not match the paper's strongest CIFAR
method. The paper Methods and repository plotting labels specify ReLU with
divisive RMS normalization, softmax output, and plastic W/Y/Q pathways. The
paper-faithful Hydra resolution used here is:

```text
--config-name=cifar10_burstccn_kp
dataset.root=/home/yurenh2/sdrn/data
model.hidden_activation_function=relu_rms
model.output_activation_function=softmax
model.W.optimiser.lr=0.002
model.Y.mode=sym_W
model.Q.mode=sym_Y
```

The inherited configuration supplies three convolutional layers plus a
1500-unit fully connected layer, batch size 32, 400 epochs, label smoothing
0.05, SGD momentum 0.9, weight decay `1e-4`, two warmup epochs and cosine
decay. The resolved author config uses softmax outputs with MSE loss; Y and Q
use learning rate `0.001`. The current author loop interprets `n_epochs=400`
as an epoch-0 evaluation followed by 399 optimization epochs, and the audit
records both counts rather than silently relabelling them. The paper reports
`17.03 ± 0.21%` top-1 error (`82.97 ± 0.21%` accuracy) for symmetric-plastic
BurstCCN.

Reproduction command (seed 0):

```bash
WANDB_MODE=offline WANDB_SILENT=true \
WANDB_NAME=burstccn-cifar10-symplastic-paper-seed0 \
WANDB_DIR=/scratch/yurenh2/burstccn-wandb CUDA_VISIBLE_DEVICES=5 \
/scratch/yurenh2/venvs/burstccn/bin/python train.py \
  --config-name=cifar10_burstccn_kp \
  dataset.root=/home/yurenh2/sdrn/data training.seed=0 \
  model.hidden_activation_function=relu_rms \
  model.output_activation_function=softmax \
  model.W.optimiser.lr=0.002 model.Y.mode=sym_W model.Q.mode=sym_Y \
  hydra.run.dir=/scratch/yurenh2/burstccn-native/cifar10_symplastic_paper_seed0
```

The native trainer evaluates test every epoch and records both latest and best
test error. The completed seed-0 record reports `80.07%` final test accuracy,
`80.10%` at the validation-selected epoch, and `80.25%` test-selected, versus
the published `82.97 +/- 0.21%`. Audited W&B wall time is `15451.3 s`. The lower
reproduced endpoint is retained rather than removed; the best-test value is
explicitly labelled test-selected.

## Dual Propagation native CIFAR-10

- Paper/model: Høier and Zach, *Two Tales of Single-Phase Contrastive Hebbian
  Learning* (ICML 2024), <https://arxiv.org/abs/2402.08573>.
- Author repository: <https://github.com/Rasmuskh/dualprop_icml_2024>.
- Frozen revision: `7b2595b34421e1483a721dbfdeff8cdabda3a1ff`.
- External checkout: `/scratch/yurenh2/sdil-reference/dualprop_icml_2024` on
  `timan107`.
- Environment: `/scratch/yurenh2/venvs/dualprop` with JAX/JAXlib 0.4.14 and
  CUDA 11.8/cuDNN 8.6 runtime wheels.

The official CIFAR archive already present in `/home` used Python batches,
whereas TensorFlow Datasets requested the official binary layout. It was
losslessly converted into
`/scratch/yurenh2/cifar10-binary-converted-v1.tar.gz`; TFDS then generated the
author-code records under `data/cifar10/3.0.2`. The verified splits contain
50,000 train and 10,000 test examples. No image or label values were changed.

The paper's strongest stable CIFAR-10 row is DP-transpose with `alpha=0`,
`beta=0.1`, reporting `92.41 ± 0.07%`; the paper BP reference is
`92.36 ± 0.16%`. The exact author script uses VGG16, 130 epochs, batch size
100, 16 requested `fwK` nudged passes (17 forward-only layer passes including
the initial pass), 90/10 train/validation selection, peak learning rate 0.025,
10 warmup epochs, decay at epoch 120, momentum 0.9, and weight decay `5e-4`.
It saves the best-validation checkpoint and evaluates test once after training.

The complete seed-1988 reproduction uses the official command with only the
experiment name added:

```bash
CUDA_VISIBLE_DEVICES=7 XLA_PYTHON_CLIENT_PREALLOCATE=false \
TF_FORCE_GPU_ALLOW_GROWTH=true \
/scratch/yurenh2/venvs/dualprop/bin/python train.py \
  --model VGG16 --dataset cifar10 --num-epochs 130 --batch-size 100 \
  --alpha 0.0 --beta 0.1 --loss sce --inference-sequence fwK \
  --inference-passes-nudged 16 --learning-rate 0.025 \
  --learning-rate-final 0.000002 --warmup-learning-rate 0.001 \
  --learning-algorithm dualprop-lagr-ff --activation relu \
  --decay-epochs 120 --warmup-epochs 10 --momentum 0.9 \
  --weight-decay 5e-4 --dtype float32 --param-dtype float32 \
  --percent-train 90 --percent-val 10 --seeds 1988 \
  --experiment-name sdil-dp-full-a0-b0p1-seed1988-v1
```

The actual command also prepends the environment's NVIDIA wheel `lib`
directories to `LD_LIBRARY_PATH`; this changes runtime linkage, not author
source. A one-epoch full-VGG smoke completed without OOM/NaN in 203.4 training
seconds, reaching 20.48% validation and 20.04% test accuracy. The completed
130-epoch seed-1988 reproduction reaches `92.46%` test accuracy from the
best-validation checkpoint, versus the published `92.41 +/- 0.07%`. Test was
evaluated exactly once. Author-measured training, validation, and test time
sums to `23119.8 s`; this excludes setup and checkpoint I/O. The near-exact
accuracy reproduction establishes executable fidelity, not equal compute with
SDIL.

## Reporting rules for external runs

- Record repository URL/revision, tracked-source cleanliness, full resolved
  configuration, environment package freeze, dataset identity, GPU, wall time,
  and output hashes.
- Preserve failed compatibility runs, but do not mix them into a paper-method
  aggregate when their resolved protocol differs from the paper.
- Report native final, validation-selected, and test-selected metrics under
  separate labels.
- One reproduced seed is evidence of executable fidelity, not a replacement for
  a published multi-seed uncertainty interval. Expand seeds only when GPU time
  permits and never select the reproduced seed by its accuracy.

`experiments/import_native_baseline.py` refuses incomplete histories, protocol
or revision drift, tracked source changes, nonfinite values, a missing package
freeze, the wrong physical GPU, and missing dataset hashes. Completed records
live under `results/native/<protocol-id>.json`. The separate strict aggregator
`experiments/audit_native_baselines.py` distinguishes final,
validation-selected, and test-selected metrics and generates
`results/native_baseline_audit.md`. Both author runs pass this C4 audit, and
`experiments/finalize_accept.sh` is green.