diff options
Diffstat (limited to 'notes')
| -rw-r--r-- | notes/01_theory_notes.md | 8 | ||||
| -rw-r--r-- | notes/02_experiment_notes.md | 25 |
2 files changed, 33 insertions, 0 deletions
diff --git a/notes/01_theory_notes.md b/notes/01_theory_notes.md index d524000..7a54374 100644 --- a/notes/01_theory_notes.md +++ b/notes/01_theory_notes.md @@ -357,6 +357,14 @@ d-d_{\mathrm{hard}} \max(0,k-(P-d)). \] +Interpretation: + +- \(P-d\) is the local redundant parameter dimension. +- If \(k\le P-d\), generic alignment constraints can be absorbed by redundant directions without reducing hard local function rank. +- If \(k>P-d\), every additional generic constraint reduces hard local function rank one-for-one. + +This gives the proposed FA/BP functional-gap onset: parameter-volume cost can grow before the model loses hard function rank, but once the alignment burden exhausts redundancy, FA should separate from BP more sharply. + Soft overlap model. Let \(E\) be the alignment constraint subspace and \(S\) be the task-sensitive subspace: \[ diff --git a/notes/02_experiment_notes.md b/notes/02_experiment_notes.md index 05b108b..889d1bd 100644 --- a/notes/02_experiment_notes.md +++ b/notes/02_experiment_notes.md @@ -284,3 +284,28 @@ Random-target means remain close to \(1/D\) for all distributions, but worst-cas - subspace and axis initializations leave entire orthogonal directions uncovered. This empirically illustrates the prior-free minimax theorem: without target or weight prior information, anisotropic feedback cannot improve the worst-case angular bound. + +## Functional Capacity Overlap Run Log + +Script: + +```bash +python scripts/functional_capacity_overlap.py --parameters 96 --task-rank 24 --constraint-ranks 0 24 48 72 84 96 --trials 100 --seed 5 --plot +``` + +Setup: + +- parameter dimension \(P=96\) +- task-sensitive rank \(d=24\) +- redundant dimension \(P-d=72\) + +Result: + +- \(k=0\): hard loss `0`, theory `0`; soft overlap `0`, theory `0` +- \(k=24\): hard loss `0`, theory `0`; soft overlap `6.0315`, theory `6` +- \(k=48\): hard loss `0`, theory `0`; soft overlap `12.0018`, theory `12` +- \(k=72\): hard loss `0`, theory `0`; soft overlap `18.0018`, theory `18` +- \(k=84\): hard loss `12`, theory `12`; soft overlap `21.0029`, theory `21` +- \(k=96\): hard loss `24`, theory `24`; soft overlap `24`, theory `24` + +This validates the redundancy-exhaustion interpretation: hard functional rank remains intact until alignment constraints exceed the redundant dimension \(P-d\), while soft overlap grows linearly as \(kd/P\). |