diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-06-05 14:55:48 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-06-05 14:55:48 -0500 |
| commit | 44fc9af8034ce58ba61299ebfeb79725d0ca02e7 (patch) | |
| tree | ef6ab45ce484a9f2d897305ac2993bf0a294e2fa /notes/18_contribution_roadmap.md | |
| parent | 29b92fb3146f57f4d0ff7cb76f073b387a0d4879 (diff) | |
Clarify soft erosion scaling claim
Diffstat (limited to 'notes/18_contribution_roadmap.md')
| -rw-r--r-- | notes/18_contribution_roadmap.md | 11 |
1 files changed, 11 insertions, 0 deletions
diff --git a/notes/18_contribution_roadmap.md b/notes/18_contribution_roadmap.md index 4805a19..be4d775 100644 --- a/notes/18_contribution_roadmap.md +++ b/notes/18_contribution_roadmap.md @@ -71,6 +71,17 @@ So the FA/BP gap need not wait until redundant directions are fully exhausted. Redundancy suppresses the expected task-relevant alignment burden, but does not make it exactly zero. +The scaling distinction is: + +```text +k fixed, P grows -> k/P -> 0, erosion vanishes +k proportional to P -> k/P stays constant, relative erosion persists +``` + +For equal-width MLPs, both parameter count and feedback matrix dimensions scale +like `Θ(L n^2)`, so a fixed per-matrix alignment burden can remain a constant +fraction of capacity under width scaling. + Hard functional rank loss remains a limiting sanity check: ```text |
