path: root/COLLABORATOR_ONBOARDING.md

# A6000 collaborator onboarding: complete 81-cell crossover

## Assignment

Run one independent, failure-retaining copy of the complete validation matrix
on one otherwise-idle physical NVIDIA RTX A6000:

| family | sizes | methods | formal cells |
|---|---|---:|---:|
| plain CNN | miniCNN, VGGlike, VGG16 | 9 | 27 |
| residual CNN | ResNet-20, ResNet-32, ResNet-56 | 9 | 27 |
| decoder Transformer | 4, 8, 12 blocks | 9 | 27 |
| **total** | **9 architecture-size points** | **9** | **81** |

The methods are BP, ordinary FA, DFA, PEPITA, Forward-Forward, equilibrium
propagation, Dual Propagation, clean reciprocal Kolen-Pollack (clean KP), and
SDIL. This is not an SDIL-only sweep and not a request to fill selected missing
cells. Every architecture-size point must contain all nine methods on the same
A6000. Divergence, timeout, chance performance, and out-of-memory are retained
results; do not silently restart with a new learning rate or remove a cell.

The complete contract is
[`CROSS_ARCHITECTURE_CROSSOVER.md`](CROSS_ARCHITECTURE_CROSSOVER.md).
Family-specific semantics and schedules are frozen in
[`PLAIN_CNN_CROSSOVER.md`](PLAIN_CNN_CROSSOVER.md),
[`RESNET_CROSSOVER.md`](RESNET_CROSSOVER.md), and
[`TRANSFORMER_CROSSOVER.md`](TRANSFORMER_CROSSOVER.md).

After accepting the private-repository invitation:

```bash
git clone https://github.com/YurenHao0426/sdil.git
cd sdil
```

## What SDIL is

SDIL tests a narrow algorithmic reading of the Harnett et al. result: the
teaching quantity is the component of apical activity that is unexpected
under a neuron's ordinary soma-apical relation, not raw apical activity.
For hidden population \(l\),

\[
a_l=A_lc_l+n_l,\qquad
\widehat a_l=P_l(h_l),\qquad
r_l=a_l-\widehat a_l,
\]

\[
\Delta W_l
=\eta\left(r_l\odot\phi'(u_l)\right)h_{l-1}^{\mathsf T}.
\]

The update is local once \(r_l\) is present: presynaptic activity,
postsynaptic gain, and the same neuron's dendritic innovation. \(P_l\) is
identified from neutral observations. The standard-network implementation
also uses a counted paired-neutral microphase that projects remaining affine
neutral residual off current soma before plasticity. \(A_l\) is calibrated
with antithetic causal perturbations and amortized by reciprocal KP feedback;
that learned-feedback substrate is prior art and is shared with `clean_kp`.
The claimed SDIL contribution is the per-neuron neutral residualization under
mixed apical traffic, its stability/control implementation, and the resulting
scaling behavior. `clean_kp` is therefore the load-bearing matched control:
same reciprocal credit transport, but no nuisance traffic, predictor, or
subtractive innovation. See [`NOVELTY.md`](NOVELTY.md) for the precise
prior-art boundary.

## Evidence already obtained

The complete matched Plain-CNN P2 audit is checked into
[`results/plain_cnn_p2_audit.json`](results/plain_cnn_p2_audit.json). Values
below are best validation accuracy; parenthesized values are measured
GTX-1080 wall hours. Nonfinite cells remain in the table at their last finite
metric.

The audited visual summary is available as
[`figure7_plain_cnn_pareto.png`](results/figs/figure7_plain_cnn_pareto.png)
and a vector
[`PDF`](results/figs/figure7_plain_cnn_pareto.pdf).

| method | miniCNN | VGGlike | VGG16 |
|---|---:|---:|---:|
| BP | 83.62 (0.08) | 90.52 (0.28) | 92.68 (0.71) |
| FA | 68.14 (0.12) | 57.94 (0.35) | 33.80 (0.67) |
| DFA | 70.52 (0.12) | 57.70 (0.29) | 9.40 nonfinite (0.02) |
| PEPITA | 30.24 (0.09) | 9.40 nonfinite (0.01) | 14.92 nonfinite (0.04) |
| Forward-Forward | 10.14 (0.07) | 20.44 (0.27) | 19.16 (1.42) |
| EP | 10.88 (0.34) | 9.40 (3.00) | 9.40 (6.22) |
| Dual Propagation | 83.26 (0.36) | **90.98 (3.43)** | 92.38 (5.97) |
| clean KP | 82.60 (0.15) | 89.46 (0.55) | **90.86 (1.08)** |
| SDIL | **82.90 (0.18)** | **89.58 (0.60)** | 90.70 (1.24) |

The positive result is real but bounded: SDIL preserves useful optimization
from miniCNN to VGG16 while FA/DFA collapse, and it is much cheaper than Dual
Propagation or EP in this panel. It is not globally dominant. BP is both more
accurate and cheaper on VGG16; clean KP is 0.16 points more accurate and
faster than SDIL there; Dual Propagation is 1.40 points above SDIL but roughly
4.8 times slower. EP is finite but at chance, so this particular adapter does
not support the expectation that EP is automatically the strongest baseline.

Separate five-seed standard-ResNet evidence—not a substitute for the 81-cell
matrix—shows positive depth utility:

| depth | BP | DFA | clean KP | SDIL | SDIL early alignment | SDIL/BP MACs |
|---:|---:|---:|---:|---:|---:|---:|
| 20 | 91.624 | 31.878 | 91.388 | 91.584 | 0.999687 | 1.326x |
| 32 | 92.302 | 32.684 | 92.332 | 92.254 | 0.999613 | 1.329x |
| 56 | 92.632 | 30.850 | 92.670 | **92.760** | 0.999423 | 1.331x |

All five SDIL ResNet-20-to-56 pairs improve, with a mean gain of 1.176
points. The main unresolved question is whether the same scaling/cost pattern
survives a fully matched crossover across plain CNNs, residual CNNs, and
Transformers. That is exactly what this A6000 run measures.

Important retained negatives are documented rather than hidden: the first
long standard-ResNet SDIL branch became nonfinite at epoch 89; fixed-margin,
representable-subspace, vectorizer-space, response-mirroring, broad endogenous
traffic, desired-velocity online-control, and broad Harnett-like population
decoding branches missed their gates. Current evidence supports innovation for
traffic predictable from the chosen somatic statistic, not arbitrary
top-down context, and clean-task scaling does not by itself prove that
residualization is necessary.

## Frozen-run state

The Plain-CNN selector and its 27-cell endpoint are frozen. As of 2026-07-30,
the ResNet selector has 18/19 records and the Transformer selector has not yet
opened; therefore the current commit can start the Plain-CNN 27 cells but the
complete launcher intentionally refuses to start ResNet or Transformer
endpoints until their passed selector JSON files are committed. Wait for the
maintainer to announce the exact A6000 run-packet commit/tag before launching
`--family all`. Never construct or edit a selector manually.

## Machine and environment setup

Use a local scratch filesystem with enough space, not a quota-limited home
directory. Record the physical GPU index shown by `nvidia-smi`; the launcher
requires `CUDA_VISIBLE_DEVICES` to contain that one numeric physical index and
verifies the exact device name `NVIDIA RTX A6000`. Keep the GPU otherwise idle
for meaningful wall-time and peak-memory comparisons.

The main repository needs a CUDA-enabled PyTorch environment plus NumPy. The
Plain-CNN family uses the Dual Propagation authors' JAX/TensorFlow environment
and should be a separate Python 3.10 environment. Its frozen
`requirements.txt` is inherited from the author repository. Before a formal
run, both of these must show a GPU:

```bash
/absolute/path/to/main-env/bin/python -c \
  'import torch; print(torch.__version__, torch.cuda.get_device_name())'
/absolute/path/to/plain-env/bin/python -c \
  'import jax; print(jax.__version__, jax.devices())'
```

Reconstruct the Plain-CNN code from the frozen author revision plus the 19
tracked patches:

```bash
cd /absolute/path/to/sdil
experiments/bootstrap_plain_cnn.sh \
  /local/scratch/dualprop-sdil-a6000
```

The main ResNet code expects the extracted CIFAR-10 Python archive:
`$SDIL_DATA_DIR/cifar-10-batches-py/data_batch_1` through `data_batch_5`
and `test_batch`. The Plain-CNN author code independently prepares the same
dataset through TFDS under its own `data/` directory. Prepare the frozen
Tiny-Shakespeare bytes with:

```bash
/absolute/path/to/main-env/bin/python \
  experiments/prepare_shakespeare_char.py \
  --out-dir /local/scratch/shakespeare_char
```

Then set the run environment:

```bash
export CUDA_VISIBLE_DEVICES=0
export SDIL_HARDWARE_PROFILE=a6000
export SDIL_DATA_DIR=/local/scratch/cifar10-python
export SDIL_TRANSFORMER_DATA_DIR=/local/scratch/shakespeare_char
```

Replace `0` with the physical A6000 index on that machine. Do not pass a
remapped logical list such as `0,1`.

## Launch, resume, and audit

First inspect the full command list without starting training:

```bash
/absolute/path/to/main-env/bin/python \
  experiments/run_a6000_matrix.py \
  --plain-root /local/scratch/dualprop-sdil-a6000 \
  --plain-python /absolute/path/to/plain-env/bin/python \
  --family all --dry-run
```

Start the formal run inside `tmux` or another persistent session:

```bash
/absolute/path/to/main-env/bin/python \
  experiments/run_a6000_matrix.py \
  --plain-root /local/scratch/dualprop-sdil-a6000 \
  --plain-python /absolute/path/to/plain-env/bin/python \
  --family all 2>&1 | tee a6000-matrix.log
```

Before the selector-release commit, the collaborator may start only the
already-frozen Plain-CNN block by replacing `--family all` with
`--family plain`. Later, rerunning `--family all` preserves every completed
manifest and continues with ResNet and Transformer. A manifest is the resume
boundary. If an output exists without its manifest, the runner stops and asks
for an audit; do not delete the output just to make the queue continue.

The launcher runs equation/mechanics smokes, all 81 endpoints, each
family-specific audit, and finally the global joiner. Success means
`results/crossover_81_audit.json` reports exactly 81 audited cells and every
architecture has all nine methods. A failed method may reduce the finite-cell
count but not the audited-cell count. The final A6000 table remains a separate
hardware replication from the GTX-1080 table; never merge their wall times
cell by cell.

Budget several uninterrupted GPU-days until an A6000 timing calibration is
available. The completed Plain-CNN panel alone used 27.64 sequential
GTX-1080 GPU-hours. Do not shorten epochs, relaxation passes, validation, or
the method set to meet a calendar estimate.

## What to return

Return the exact run-packet commit, `a6000-matrix.log`, the three family audit
JSON files, `results/crossover_81_audit.json`, launch locks and per-cell
manifests, `nvidia-smi` output, both environment freezes, and any retained
failure trace. Keep raw run directories until the maintainer confirms hashes
and backup. Do not commit multi-gigabyte raw checkpoints or rewrite published
history; use a separate results branch or agreed artifact storage.

Report any code or protocol issue before changing it. Hardware-path fixes are
allowed only in a new commit and require restarting the affected family under
a new launch lock; learning-rate, schedule, method, data split, and
architecture changes define a different experiment.