|
Round 19's #5 recommendation. Major new finding for the paper:
| lam | acc | ||h_L|| | ||g_2|| | deep cos | deep rho |
|-------|------:|--------:|--------:|---------:|---------:|
| 0 | 0.308 | 4e8 | 5e-10 | -0.008 | -0.003 |
| 1e-4 | 0.359 | 2.4e4 | 6.3e-7 | -0.022 | -0.004 |
| 1e-2 | 0.363 | 4e4 | 1e-6 | +0.155 | +0.080 |
| 1e-1 | 0.349 | 1.2e4 | 1.6e-6 | +0.131 | +0.067 |
KEY: at lam=1e-4 the residual stream is contained AND ||g|| is healthy
(mode 1 ALLEVIATED), but deep cos and rho are still essentially zero
(mode 2 NOT alleviated). This is independent dissociation of the two
modes via penalty strength: at weak penalty you get mode 1 fix WITHOUT
mode 2 fix.
Both metrics (cos, rho) agree at every lambda. Penalty strength has a
non-monotonic effect on mode 2 alleviation:
- lam=1e-4: too weak, mode 2 not alleviated (cos ~0)
- lam=1e-2: sweet spot, cos +0.16, rho +0.08
- lam=1e-1: slightly over-constrained, cos +0.13, rho +0.07
This is the 7th independent validation of the two-mode separation, and
the strongest one because it shows mode 1 alleviation WITHOUT mode 2
alleviation — the modes do not even respond to the same intervention
strength.
|
