From 44fc9af8034ce58ba61299ebfeb79725d0ca02e7 Mon Sep 17 00:00:00 2001 From: YurenHao0426 Date: Fri, 5 Jun 2026 14:55:48 -0500 Subject: Clarify soft erosion scaling claim --- notes/18_contribution_roadmap.md | 11 +++++++++++ 1 file changed, 11 insertions(+) (limited to 'notes/18_contribution_roadmap.md') diff --git a/notes/18_contribution_roadmap.md b/notes/18_contribution_roadmap.md index 4805a19..be4d775 100644 --- a/notes/18_contribution_roadmap.md +++ b/notes/18_contribution_roadmap.md @@ -71,6 +71,17 @@ So the FA/BP gap need not wait until redundant directions are fully exhausted. Redundancy suppresses the expected task-relevant alignment burden, but does not make it exactly zero. +The scaling distinction is: + +```text +k fixed, P grows -> k/P -> 0, erosion vanishes +k proportional to P -> k/P stays constant, relative erosion persists +``` + +For equal-width MLPs, both parameter count and feedback matrix dimensions scale +like `Θ(L n^2)`, so a fixed per-matrix alignment burden can remain a constant +fraction of capacity under width scaling. + Hard functional rank loss remains a limiting sanity check: ```text -- cgit v1.2.3