summaryrefslogtreecommitdiff
path: root/notes/18_contribution_roadmap.md
diff options
context:
space:
mode:
authorYurenHao0426 <Blackhao0426@gmail.com>2026-06-05 14:55:48 -0500
committerYurenHao0426 <Blackhao0426@gmail.com>2026-06-05 14:55:48 -0500
commit44fc9af8034ce58ba61299ebfeb79725d0ca02e7 (patch)
treeef6ab45ce484a9f2d897305ac2993bf0a294e2fa /notes/18_contribution_roadmap.md
parent29b92fb3146f57f4d0ff7cb76f073b387a0d4879 (diff)
Clarify soft erosion scaling claim
Diffstat (limited to 'notes/18_contribution_roadmap.md')
-rw-r--r--notes/18_contribution_roadmap.md11
1 files changed, 11 insertions, 0 deletions
diff --git a/notes/18_contribution_roadmap.md b/notes/18_contribution_roadmap.md
index 4805a19..be4d775 100644
--- a/notes/18_contribution_roadmap.md
+++ b/notes/18_contribution_roadmap.md
@@ -71,6 +71,17 @@ So the FA/BP gap need not wait until redundant directions are fully exhausted.
Redundancy suppresses the expected task-relevant alignment burden, but does not
make it exactly zero.
+The scaling distinction is:
+
+```text
+k fixed, P grows -> k/P -> 0, erosion vanishes
+k proportional to P -> k/P stays constant, relative erosion persists
+```
+
+For equal-width MLPs, both parameter count and feedback matrix dimensions scale
+like `Θ(L n^2)`, so a fixed per-matrix alignment burden can remain a constant
+fraction of capacity under width scaling.
+
Hard functional rank loss remains a limiting sanity check:
```text