From a6d88a18a4bfd46ab0dd487730b72dcbc3b8b5b8 Mon Sep 17 00:00:00 2001 From: Yuren Hao Date: Wed, 15 Jul 2026 10:32:26 -0500 Subject: assets/: organize into decks|figs|hw|refs|misc; scripts + gitignore repointed Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn --- assets/figs/fig_primer_phases.svg | 925 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 925 insertions(+) create mode 100644 assets/figs/fig_primer_phases.svg (limited to 'assets/figs/fig_primer_phases.svg') diff --git a/assets/figs/fig_primer_phases.svg b/assets/figs/fig_primer_phases.svg new file mode 100644 index 0000000..da9ed6a --- /dev/null +++ b/assets/figs/fig_primer_phases.svg @@ -0,0 +1,925 @@ + + + + + + + + 2026-07-15T09:44:53.517354 + image/svg+xml + + + Matplotlib v3.10.8, https://matplotlib.org/ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Free phase (= inference) + + + + + + g + i + v + e +   + e + a + c + h +   + l + a + y + e + r +   + a +   + s + t + a + t + e +   +   + ( + 3 +   + o + f +   +   + d + r + a + w + n + ) + ; +   + s + e + t + t + l + e +   + t + h + e +   + d + i + s + a + g + r + e + e + m + e + n + t +   + e + n + e + r + g + y + z + L + l + + + + + + + + E + z + z + f + z + ( + ) + = + ‖ + − + ( + ) + ‖ + ∑ + l + l + l + l + 1 + 2 + − + 1 + 2 + + + + + + + + + z + f + z + 3 + 3 + 2 + = + ( + ) + + + + + term = 0 + + + + + + z + f + z + 2 + 2 + 1 + = + ( + ) + + + + + term = 0 + + + + + + z + f + 1 + 1 + = + ( + e + m + b + ) + + + + + term = 0 + + + + + + + + + + + + + + + + + + e + m + b + ( + ) + x + + + + + + + + T + h + e +   + m + i + n + i + m + u + m +   + i + s +   + e + x + a + c + t + : +   +   + = + 0 + , +   +   + s + t + a + t + e + s +   + ≡ +   + f + o + r + w + a + r + d +   + a + c + t + i + v + a + t + i + o + n + s + , + E + + + + + and one bottom-up pass reaches it. + + + The free phase adds nothing and changes nothing at inference. + + + Nudged phase (training only) + + + + + + a + d + d +   + t + h + e +   + l + o + s + s +   + a + t +   + s + t + r + e + n + g + t + h +   + ≪ + 1 +   + a + n + d +   + s + e + t + t + l + e +   + a + g + a + i + n + : + β + + + + + + + + E + z + β + z + y + ( + ) + + + ⋅ + C + E + ( + l + o + g + i + t + s + ( + ) + , + ) + 3 + + + + + + + + p + u + l + l +   + − + ∇ + C + E + β + + + + + (the only place the label enters) + + + + + + + + + + z + z + d + β + 3 + 3 + 3 + = + + + + + + + + + + z + z + d + β + 2 + 2 + 2 + = + + + + + + + + + + z + z + d + β + 1 + 1 + 1 + = + + + + + + + + + + + + + + + + + + + + + + e + m + b + ( + ) + x + + + + + + + + + + + + d + J + d + 2 + 3 + ⊤ + 3 + = + + + + + + + + + + + + d + J + d + 1 + 2 + ⊤ + 2 + = + + + + + + + + T + h + e +   + t + o + p +   + s + t + a + t + e +   + i + s +   + p + u + l + l + e + d +   + t + o + w + a + r + d +   + l + o + w + e + r +   + l + o + s + s + ; +   + e + a + c + h +   + l + a + y + e + r + ' + s +   + m + i + s + m + a + t + c + h +   + d + l + + + + + transmits DOWN through the same weights, and the stack re-settles. + + + + + + ( +   + = +   + t + h + e +   + t + r + a + n + s + p + o + s + e +   + r + e + a + d +   + a +   + b + i + d + i + r + e + c + t + i + o + n + a + l +   + p + h + y + s + i + c + a + l +   + d + e + v + i + c + e +   + p + r + o + v + i + d + e + s + ) + J + ⊤ + + + + + The update — a difference measurement between the two settled states + + + + + + ̂ + = + ∂ + ( + ) + − + ∂ + ( + ) + / + ( + ∂ + ( + ) + ≡ + 0 + h + e + r + e + , + s + i + n + c + e + = + 0 + ) + g + E + z + E + z + β + E + z + E + [ + ] + θ + β + θ + θ + 0 + 0 + + + + + + + + p + e + r +   + l + a + y + e + r + : +   +   + Δ + ∝ + ⟨ + , + ∂ + ( + ) + / + ∂ + ⟩ + / + θ + d + f + z + θ + β + l + l + l + l + l + − + 1 + + + + + Each layer updates from its own boundary mismatch — no global backward graph, no global tape, no loss derivatives except the top nudge. + + + + + + + + + -- cgit v1.2.3