diff options
Diffstat (limited to 'notes/18_contribution_roadmap.md')
| -rw-r--r-- | notes/18_contribution_roadmap.md | 11 |
1 files changed, 11 insertions, 0 deletions
diff --git a/notes/18_contribution_roadmap.md b/notes/18_contribution_roadmap.md index 4805a19..be4d775 100644 --- a/notes/18_contribution_roadmap.md +++ b/notes/18_contribution_roadmap.md @@ -71,6 +71,17 @@ So the FA/BP gap need not wait until redundant directions are fully exhausted. Redundancy suppresses the expected task-relevant alignment burden, but does not make it exactly zero. +The scaling distinction is: + +```text +k fixed, P grows -> k/P -> 0, erosion vanishes +k proportional to P -> k/P stays constant, relative erosion persists +``` + +For equal-width MLPs, both parameter count and feedback matrix dimensions scale +like `Θ(L n^2)`, so a fixed per-matrix alignment burden can remain a constant +fraction of capacity under width scaling. + Hard functional rank loss remains a limiting sanity check: ```text |