diff options
| -rw-r--r-- | MIXED_TRAFFIC.md | 10 |
1 files changed, 10 insertions, 0 deletions
diff --git a/MIXED_TRAFFIC.md b/MIXED_TRAFFIC.md index 0bc0ee2..521d108 100644 --- a/MIXED_TRAFFIC.md +++ b/MIXED_TRAFFIC.md @@ -37,6 +37,16 @@ matched: u_l = ||r_l||_example / ||a_l||_example * a_l, innovation: u_l = r_l. ``` +The scale was chosen from a pre-endpoint signal calculation, not an accuracy +pilot. If instruction and traffic are uncorrelated and the RMS ratio is +`rho=4`, the instruction contributes only `1/(1+rho^2)=1/17=5.88%` of raw +signal power. Ideal normalized LMS leaves coefficient fraction +`(1-0.1)^20=0.12158` after warmup, so residual traffic has only +`4*0.12158=0.4863` times the instruction RMS and innovation's instruction +fraction rises to `1/(1+0.4863^2)=80.88%`. Ongoing neutral updates reduce it +further. These are falsifiable scale predictions under the stated +orthogonality idealization, not guaranteed accuracy outcomes. + Norms are taken over every non-batch coordinate separately for each example and layer. Consequently the matched control has the innovation's update-signal norm but the raw signal's direction at the same network state. Every condition |