Add AAAI-27 paper scaffolding

main.tex (AuthorKit 27 wrapper), preview.tex (kit-free preview build),
shared defs. body.tex is a draft to be REWRITTEN from the evidence
ledger (notes/36): it omits most of the validation chain and cites the
retired closed-form number; do not treat its content as authoritative.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

5 files changed, 620 insertions, 0 deletions

diff --git a/.gitignore b/.gitignore
index 6dd4b78..12c5ea2 100644
--- a/.gitignore
+++ b/.gitignore
@@ -39,3 +39,13 @@ Thumbs.db
 
 # Logs
 *.log
+
+# Paper build artifacts
+paper/*.pdf
+paper/*.aux
+paper/*.out
+paper/*.bbl
+paper/*.blg
+paper/*.fls
+paper/*.fdb_latexmk
+paper/*.synctex.gz
diff --git a/paper/body.tex b/paper/body.tex
new file mode 100644
index 0000000..3dad469
--- /dev/null
+++ b/paper/body.tex
@@ -0,0 +1,521 @@
+
+\begin{abstract}
+Feedback alignment (FA) trains a neural network with \emph{fixed random}
+matrices in place of the transposed weights that backpropagation (BP) uses
+to send errors backward. A decade of work has established the surprising
+fact that this \emph{works}. It has not established what it \emph{costs}.
+We settle that question for multilayer perceptrons, exactly and without a
+single fitted parameter. First, the geometric mismatch between a random
+feedback matrix and the matrix BP would have used obeys an exact
+$\Beta(\tfrac12,\tfrac{D-1}{2})$ law, which we turn into a capacity cost
+that adds up to $\Theta(Ln^2)$ across an $L$-layer width-$n$ network.
+Second, this cost cannot be engineered away: with no prior on the task, no
+feedback initialization beats isotropic random feedback. Third---and this
+is the crux---the resulting FA/BP optimization gap is \emph{always
+strictly positive}, equals \emph{exactly} the hidden-layer share of BP's
+learning speed at initialization, and, contrary to the natural guess,
+never undergoes a phase transition: it is a \emph{soft ramp} that we derive
+in closed form and predict to log-space correlation $0.98$ from
+initialization alone, and to correlation $0.999$ with a no-fit early-time
+estimator. The one-line summary is that random feedback is not free: its
+cost is irreducible, but soft, and now exactly computable.
+\end{abstract}
+
+\section{Introduction}
+
+Backpropagation rests on a hidden symmetry: to send the error backward
+through a layer, it reuses the \emph{transpose of that layer's forward
+weights}. Nothing in a brain or in a piece of analog hardware ties the
+backward pathway to the forward one this way, which is why
+\citet{lillicrap2016} asked what happens if you simply cut the tie---if you
+replace each backward matrix $W^{\top}$ by a \emph{fixed random} matrix $B$
+drawn once and never touched again. The answer was unexpected: the network
+still learns, and the forward weights quietly rotate until they partly
+align with the random feedback. This is \emph{feedback alignment} (FA), and
+it and its variants train deep networks to low error~\cite{nokland2016}.
+
+A decade later, the field has thoroughly answered one question---\emph{can}
+FA learn?---and left the more useful one untouched: \emph{what does it
+cost?} ``FA eventually reaches low error'' is perfectly compatible with FA
+paying a steady, measurable price relative to BP at every step, a price one
+might hope a wide enough network simply absorbs. Whether it does is not a
+matter of opinion; it is a quantity, and in this paper we compute it.
+
+Our message is simple and, we will argue, complete: \textbf{random feedback
+imposes a cost that is irreducible but soft, and we give its exact ledger.}
+We mean three things by this, and the paper is organized around them.
+
+\begin{itemize}
+\item \textbf{The static cost is a clean law (Sec.~\ref{sec:cap}).} How
+badly does a random feedback matrix match the one BP wants? Exactly as
+badly as the geometry forces: their squared cosine is
+$\Beta(\tfrac12,\tfrac{D-1}{2})$-distributed, with mean $1/D$. Reading this
+as a ``log-volume cost'' makes it \emph{add} across layers to
+$\Theta(Ln^2)$.
+
+\item \textbf{You cannot cheat the cost (Sec.~\ref{sec:minimax}).} Maybe a
+cleverer fixed feedback would help? Not without knowing the task: the best
+worst-case alignment any initialization can guarantee is $1/D$, and plain
+isotropic random feedback already achieves it.
+
+\item \textbf{The optimization gap is positive, soft, and computable
+(Sec.~\ref{sec:gap}).} The interesting cost is the actual FA/BP training
+gap. We prove it is always nonzero; that at initialization its expected
+size is \emph{exactly} the fraction of BP's learning speed carried by the
+hidden layers; and that over training it grows as a smooth ramp---not a
+phase transition---which we write in closed form and confirm against dense
+simulation. A natural hypothesis (the gap appears suddenly when redundant
+capacity runs out) turns out to be wrong, and we can say precisely why.
+\end{itemize}
+
+We keep every model and theorem exact; what we work hard at is making them
+\emph{legible}, because the value here is not a single inequality but a
+coherent, quantitative picture of a phenomenon that has so far been
+described only qualitatively.
+
+\section{Setup: BP, FA, and ``Learning Speed''}\label{sec:setup}
+
+Fix an MLP with widths $n_0,\dots,n_L$, forward weights
+$W_l\in\R^{n_l\times n_{l-1}}$, and ReLU activations
+$h_l=\phi(W_l h_{l-1})$. Train on $N$ examples with squared loss
+$\mathcal{L}=\tfrac{1}{2N}\|r\|^2$, where $r=f_\theta(X)-Y$ stacks all
+residuals.
+
+The one thing that distinguishes FA from BP lives entirely in the backward
+pass. BP propagates the error using $A_l=W_{l+1}^{\top}$; FA uses a fixed
+random $B_l\in\R^{n_l\times n_{l+1}}$ instead. Write $g_l^{\mathrm{BP}}$ and
+$g_l^{\mathrm{FA}}$ for the resulting gradient of layer $l$. One fact does
+all the work later and is worth stating now: \emph{the output layer is
+identical under both rules}, because no backward matrix sits above it, so
+$g_L^{\mathrm{FA}}=g_L^{\mathrm{BP}}$. The flattened backward matrices live
+in dimension $D_l=n_l n_{l+1}$.
+
+To talk about ``cost'' we need a yardstick, and the natural one is how fast
+the loss goes down. A step $-\eta g$ decreases the loss, to first order, by
+$\eta\langle g^{\mathrm{BP}},g\rangle$---the true gradient dotted with the
+step you actually take. So define
+\begin{align}
+\mathrm{speed}_{\mathrm{BP}} &= \textstyle\sum_l \|g_l^{\mathrm{BP}}\|^2,\\
+\mathrm{speed}_{\mathrm{FA}} &= \textstyle\sum_l \langle g_l^{\mathrm{BP}},g_l^{\mathrm{FA}}\rangle .
+\end{align}
+The first is how fast BP descends; the second is how fast the FA step
+descends the \emph{true} loss. Their normalized difference,
+\begin{equation}\label{eq:e0}
+e_0 = 1-\frac{\mathrm{speed}_{\mathrm{FA}}}{\mathrm{speed}_{\mathrm{BP}}},
+\end{equation}
+is the fraction of BP's learning speed that FA throws away in one step.
+$e_0=0$ means FA is as good as BP right now; $e_0=1$ means FA makes no
+progress at all. Everything in Section~\ref{sec:gap} is about how large
+$e_0$ is, and what it compounds to over training.
+
+Finally, two operators. Let $J=\partial f_\theta(X)/\partial\theta$ be the
+Jacobian of all outputs. BP's tangent kernel (the NTK) is $\KBP=JJ^{\top}$,
+and the residual moves as $r_{t+1}\approx(I-\tfrac{\eta}{N}\KBP)r_t$---this
+is just gradient descent written in output space, and we use it as our
+lens on training. FA swaps the update Jacobian for a surrogate $\tilde J$,
+giving a \emph{non-symmetric} operator $\KFA=J\tilde J^{\top}$ whose
+symmetric part $\SFA=\tfrac12(\KFA+\KFA^{\top})$ is all that affects the
+loss to first order. Keep $\KBP$ (``what BP does'') and $\SFA$ (``what FA
+does'') in mind; the whole story is a comparison of these two.
+
+\section{The Static Cost Is a Clean Law}\label{sec:cap}
+
+Start with the most basic question: at initialization, how far is a random
+feedback matrix $B$ from the matrix $A=W^{\top}$ that BP would use? Measure
+``far'' by the squared cosine between them. The answer is not approximate
+or asymptotic---it is an exact classical distribution.
+
+\begin{theorem}[Alignment law]\label{thm:beta}
+Let $a=\vecop(A)/\|A\|_F$ and $b=\vecop(B)/\|B\|_F$ be independent isotropic
+unit directions in $\R^D$, and $Q=(a^{\top}b)^2$. Then
+\begin{equation}
+Q\sim\Beta\!\left(\tfrac12,\tfrac{D-1}{2}\right),\qquad \E[Q]=\tfrac1D .
+\end{equation}
+\end{theorem}
+\begin{proof}[Proof]
+Rotational invariance lets us fix $a=e_1$, so $Q=b_1^2$ with $b$ uniform on
+$\mathbb{S}^{D-1}$. The squared first coordinate of a uniform sphere vector
+is exactly $\Beta(\tfrac12,\tfrac{D-1}{2})$.
+\end{proof}
+
+Read off the consequence: the typical cosine is about $1/\sqrt D$, so in
+any realistic layer (with $D=n^2$ in the thousands) \emph{a random feedback
+matrix is essentially perpendicular to the one BP wants}. That near-orthogonality is
+the entire geometric source of FA's difficulty, and Theorem~\ref{thm:beta}
+says exactly how perpendicular, with no slack.
+
+It is convenient to price this. For a target alignment $q$, define the
+\emph{log-volume cost} $C(q)=-\log\Pr(Q\ge q)$---the number of nats by
+which reaching alignment $q$ is improbable by chance, equivalently the log
+of how small a sliver of feedback space clears the bar. For fixed $q$ in
+high dimension, $C(q)\approx\tfrac{D-1}{2}\log\tfrac1{1-q}=\Theta(D)$. The
+reason to bother defining it is that it behaves like a true cost---it adds.
+
+\begin{proposition}[Multilayer scaling]\label{prop:scaling}
+For independent layer-wise feedback blocks the alignment events are
+independent, so their probabilities multiply and the costs add:
+$C_{\mathrm{all}}=\sum_l C_l(q_l)$. For an equal-width MLP ($D_l\approx
+n^2$) at fixed $q$,
+\begin{equation}
+C_{\mathrm{all}}=\Theta(Ln^2),\qquad
+\Pr(\text{all }q_l)=e^{-\Theta(Ln^2)} .
+\end{equation}
+\end{proposition}
+
+So the ``bits of luck'' required to make random feedback align grow
+linearly in the number of weights, and the probability of getting that luck
+shrinks exponentially in it. Both depth and width make raw alignment
+exponentially harder---a clean, if unsurprising, fact that fixes the units
+for everything that follows.
+
+\section{You Cannot Cheat the Cost}\label{sec:minimax}
+
+A reader's first instinct is to fight the geometry: surely some carefully
+chosen, still-fixed feedback aligns better than dumb isotropic noise? The
+honest answer is that you can---but only if you already know which backward
+direction the task will need, which at initialization you do not.
+
+\begin{theorem}[Prior-free minimax]\label{thm:minimax}
+Let $b\in\mathbb{S}^{D-1}$ be drawn from any distribution $\mu$ and let
+$a\in\mathbb{S}^{D-1}$ be an unknown target. With $M_\mu=\E_\mu[bb^{\top}]$
+(so $\mathrm{tr}\,M_\mu=1$),
+\begin{equation}
+\sup_{\mu}\ \inf_{\|a\|=1}\ \E_\mu[(a^{\top}b)^2]
+=\sup_\mu \lambda_{\min}(M_\mu)=\tfrac1D,
+\end{equation}
+attained by isotropic feedback ($M_\mu=\tfrac1D I$).
+\end{theorem}
+\begin{proof}[Proof]
+$\E_\mu[(a^{\top}b)^2]=a^{\top}M_\mu a$; its worst case over $a$ is
+$\lambda_{\min}(M_\mu)\le\mathrm{tr}\,M_\mu/D=1/D$. Isotropic $\mu$ makes
+every direction equal to $1/D$.
+\end{proof}
+
+The point of stating it as a minimax is to forestall a class of ``fixes.''
+Any anisotropic feedback that aligns better with some directions does so by
+aligning \emph{worse} with others, and an adversarially chosen task lands
+on the bad ones. Sparsity, orthogonality, or low rank can change
+conditioning, signal norm, or hardware cost---but they cannot beat $1/D$ in
+the worst case. Isotropic random feedback is not a lazy default; it is
+\emph{provably the best prior-free choice}. The cost in
+Section~\ref{sec:cap} is therefore not an artifact of using noise; it is a
+floor.
+
+\section{The Optimization Gap: Positive, Soft, Computable}\label{sec:gap}
+
+Static geometry is the setup; the payoff is the actual FA/BP training gap.
+We make three claims, in increasing strength: the gap is never zero; at
+initialization its size is \emph{exactly} a quantity you can read off the
+forward pass; and over training it follows a closed-form \emph{soft ramp}.
+
+\subsection{The gap is never zero, and never sudden}
+
+There is a tempting story---``a wide network has spare directions, so it
+absorbs the random-feedback constraint for free until the spares run out,
+then the gap snaps open.'' It is wrong, and a one-line null model shows why.
+Suppose alignment simply deleted a uniformly random $k$-dimensional
+subspace $E$ of the $P$ parameters, leaving the projected kernel
+$JQ_E J^{\top}$ with $Q_E=I-P_E$. Writing $v=J^{\top}r$, the one-step
+erosion is $\|P_E v\|^2/\|v\|^2$, and for Haar-random $E$,
+\begin{equation}
+e(r)\sim\Beta\!\left(\tfrac k2,\tfrac{P-k}{2}\right),\quad
+\E[e(r)]=\tfrac kP,\quad \Pr(e=0)=0 .
+\end{equation}
+For \emph{any} positive burden $k$, the erosion is almost surely nonzero,
+with mean $k/P$. Over-parameterization (large $P$) shrinks the gap through
+the ratio $k/P$, but it never reaches zero, and nothing special happens at
+any threshold. There is no phase transition to wait for.
+
+\takeaway{Capacity dilutes the FA burden; it does not delete it. The gap is
+soft.}
+
+This null model is deliberately crude---it over-predicts the real gap
+(Sec.~\ref{sec:exp})---but it isolates the mechanism. The exact statement
+for genuine FA is next, and it is sharper than $k/P$.
+
+\subsection{At initialization, the gap is the hidden speed share}
+
+Here is the result we would put on the board first. It says the effective
+initial cost of FA is not some opaque function of the random matrices; it
+is a number already sitting in the BP gradients.
+
+\begin{theorem}[Initial erosion = hidden speed share]\label{thm:e0}
+Fix the forward weights $W$, data, and residual $r$. Assume the feedback
+matrices $B_l$ are independent of $W,r$ with zero mean, and that the output
+layer uses the true gradient. Then
+$\E_B[\mathrm{speed}_{\mathrm{FA}}\mid W,r]=\|g_L^{\mathrm{BP}}\|^2$, so
+\begin{equation}\label{eq:hidden-share}
+\E_B[e_0\mid W,r]
+= 1-\frac{\|g_L^{\mathrm{BP}}\|^2}{\sum_l \|g_l^{\mathrm{BP}}\|^2}
+\;=:\;1-\rho .
+\end{equation}
+\end{theorem}
+\begin{proof}[Proof]
+The output FA gradient equals the BP gradient, contributing
+$\|g_L^{\mathrm{BP}}\|^2$. Every hidden FA gradient is multilinear in
+zero-mean independent feedback matrices, so its conditional expectation
+vanishes; hence each hidden term
+$\langle g_l^{\mathrm{BP}},g_l^{\mathrm{FA}}\rangle$ has zero mean. Summing
+leaves only the output term.
+\end{proof}
+
+Unpack what this says, because it is more intuitive than it looks. On the
+first step, random feedback points the hidden updates in random directions,
+so \emph{on average they accomplish nothing}; FA gets only the output
+layer, which it computes correctly, ``for free.'' The expected cost is
+therefore precisely the share of the work the hidden layers were doing---a
+number, $1-\rho$, that you get from one BP backward pass. We will call the
+surviving fraction $\rho=\mathrm{output\ share}\in(0,1)$; it is the single
+parameter that drives the rest of the paper. Note also what this does to
+the kernel: in the $\KFA=G+H_{\mathrm{FA}}$ language of \citet{song2021},
+it says $\E_B[H_{\mathrm{FA}}]=0$, i.e.\ at initialization FA's expected
+operator is exactly the top-layer kernel.
+
+For the simplest case we get the whole distribution, not just its mean.
+
+\begin{proposition}[Exact Gaussian erosion]\label{prop:gauss}
+For a one-hidden-layer ReLU MLP with i.i.d.\ Gaussian feedback
+$B\sim\mathcal{N}(0,\sigma_B^2)$, the hidden mixed term is linear in $B$:
+$\langle g_1^{\mathrm{BP}},g_1^{\mathrm{FA}}(B)\rangle=\langle C(W,r),B\rangle$
+for an explicit $C(W,r)$. Hence
+$e_0\sim\mathcal{N}\!\big(1-\rho,\ \sigma_B^2\|C\|_F^2/\mathrm{speed}_{\mathrm{BP}}^2\big)$,
+conditionally on $(W,r)$.
+\end{proposition}
+
+\subsection{Over training, the gap is a closed-form soft ramp}
+
+The number $1-\rho$ is a single step's loss; what reaches the reviewer's
+plots is its cumulative effect over $T$ steps. To get there we make two
+approximations, and we state them plainly rather than hide them, because
+knowing exactly where the model is idealized is the point.
+
+\begin{assumption}[Lazy operators]\label{ass:lazy}
+The tangent operators are approximately frozen at initialization, so
+$r_t^{\mathrm{BP}}=(I-\tfrac{\eta}{N}\KBP)^t r_0$.
+\end{assumption}
+\begin{assumption}[Scalar erosion]\label{ass:scalar}
+$\SFA\approx\rho\,\KBP$. This is \emph{exact in the residual direction} by
+Theorem~\ref{thm:e0} ($r^{\top}\E_B[\SFA]r=\rho\,r^{\top}\KBP r$); the
+assumption is that the same slowdown $\rho$ applies in every eigendirection.
+\end{assumption}
+
+With these, FA is simply BP slowed by a factor $\rho$ in every mode, and the
+gap drops out in one line.
+
+\begin{theorem}[Closed-form gap]\label{thm:gap}
+Under Assumptions~\ref{ass:lazy}--\ref{ass:scalar}, in the eigenbasis
+$\KBP v_i=\lambda_i v_i$ with $c_i=v_i^{\top}r_0$,
+\begin{equation}\label{eq:gap}
+\mathrm{gap}_T=\frac{1}{2N}\sum_i c_i^2\Big[
+\big(1-\tfrac{\eta\rho\lambda_i}{N}\big)^{2T}
+-\big(1-\tfrac{\eta\lambda_i}{N}\big)^{2T}\Big].
+\end{equation}
+\end{theorem}
+\begin{proof}[Proof]
+Expand $r_0=\sum_i c_i v_i$. Assumption~\ref{ass:lazy} gives
+$\|r_T^{\mathrm{BP}}\|^2=\sum_i c_i^2(1-\tfrac{\eta\lambda_i}{N})^{2T}$;
+Assumption~\ref{ass:scalar} makes $\SFA$ share eigenvectors with eigenvalues
+$\rho\lambda_i$, giving the FA term. Subtract, divide by $2N$.
+\end{proof}
+
+Now read the formula, because it settles the phase-transition question on
+sight. Every bracket is nonnegative---$\rho<1$ makes each FA mode decay
+slower than its BP counterpart, so each contributes a little extra residual.
+And the whole sum is \emph{analytic} in $\rho$, the spectrum, and $T$. An
+analytic function has no kinks; deform the capacity smoothly and the gap
+moves smoothly. The only way to manufacture a sudden jump would be to send
+$\rho\to0$ (FA stalling completely), which never happens because
+$\rho\ge\mathrm{output\ share}>0$. The soft ramp is not an empirical
+accident---it is forced by the algebra.
+
+\begin{corollary}[Soft, not a phase transition]\label{cor:soft}
+$\mathrm{gap}_T$ is real-analytic in $(\rho,\{\lambda_i\},T)$ and strictly
+positive whenever $\rho<1$; it has no threshold. A discontinuity would
+require $\rho\to0$.
+\end{corollary}
+
+The same formula tells you \emph{where} the ramp comes from. Once BP has
+fit the data, the gap is just the error FA still has in the single slowest
+direction:
+
+\begin{corollary}[The ramp law]\label{cor:ramp}
+When $\|r_T^{\mathrm{BP}}\|\approx0$,
+\begin{equation}
+\mathrm{gap}_T\approx\frac{c_{\min}^2}{2N}\,e^{-2\eta\rho\lambda_{\min}T/N},
+\quad
+\log\mathrm{gap}_T\approx\text{const}-\tfrac{2\eta T}{N}\,\rho\,\lambda_{\min}.
+\end{equation}
+\end{corollary}
+
+In words: the FA/BP gap is the leftover error in the slowest-learning
+direction, where FA crawls at a fraction $\rho$ of BP's rate. As you add
+capacity, the smallest kernel eigenvalue $\lambda_{\min}$ grows smoothly, so
+$\log\mathrm{gap}$ slides down linearly. That is the ramp, and
+Section~\ref{sec:exp} shows the data falls exactly on this line.
+
+\paragraph{Mechanism vs.\ magnitude.} We are upfront about the one thing the
+closed form does not nail. Freezing the operators (Assumption~\ref{ass:lazy})
+captures the \emph{shape} of the ramp but underestimates its travel, because
+during training the forward weights start to align with $B$ and the
+operators drift---this drift is the very ``alignment'' phenomenon that
+makes FA work at all. We recover the missing magnitude with a no-fit
+\emph{early-velocity estimator}: read the kernels $K_0$ and $K_s$ at one
+early step $s$, extrapolate $\hat K_t=K_0+\tfrac ts(K_s-K_0)$, and roll out
+the residual recursion for both rules. No scalar is fit; the only inputs are
+two early snapshots. The closed form tells you \emph{why} the ramp is soft;
+the estimator tells you \emph{how big} it is. Together they are the
+complete finite-time answer.
+
+\section{Experiments}\label{sec:exp}
+
+Everything below uses full-batch training of MLPs on synthetic data, and we
+emphasize the headline up front: \textbf{no quantity in this paper is
+fitted.} Every prediction is computed from initialization or from two early
+kernel snapshots.
+
+\paragraph{The static law holds (Thm.~\ref{thm:beta}).} Sampling matrix
+pairs, the empirical squared cosine matches
+$\Beta(\tfrac12,\tfrac{D-1}{2})$ across dimensions (Kolmogorov--Smirnov
+$p>0.3$). The geometry is as predicted, exactly.
+
+\paragraph{The initial gap is the hidden share (Thm.~\ref{thm:e0}).}
+Running the real FA backward pass over many feedback draws on a
+two-hidden-layer network, the empirical mean erosion lands on the no-fit
+prediction \eqref{eq:hidden-share}: $\E_B[e_0]=0.4378\pm0.008$ versus the
+predicted hidden share $0.4403$. Proposition~\ref{prop:gauss}'s Gaussian
+shape passes KS tests across widths. We did not tune anything; the number
+comes straight from one BP backward pass.
+
+\paragraph{The ramp is soft, and the closed form explains it
+(Cor.~\ref{cor:soft}--\ref{cor:ramp}).} Sweeping width on a random-label
+task to $T=30{,}000$ steps, the FA/BP train gap slides smoothly from
+$0.147$ to $1.6\times10^{-4}$ as the (hard) capacity margin runs from $-190$
+to $+130$---\emph{with no kink at margin zero and a gap that stays nonzero
+even when the network is over-capacity} (Fig.~\ref{fig:ramp}, blue). The
+empirical $\log\mathrm{gap}$ is linear in the margin ($R^2=0.993$), exactly
+the form Corollary~\ref{cor:ramp} predicts. Evaluating the closed-form law
+\eqref{eq:gap} with $\rho,\lambda_i,c_i$ measured \emph{only at
+initialization} reproduces the ramp's shape at log-space correlation
+$0.977$ (Fig.~\ref{fig:ramp}, orange), and $\lambda_{\min}$ alone---which
+grows from $0.07$ to $0.99$ across the sweep---explains the gap at
+correlation $0.963$. The frozen closed form compresses the range, exactly
+the operator drift the estimator was built to recover.
+
+\paragraph{The estimator is essentially exact (no fit).} Across $256$
+trajectories and a stress grid over depth $\{1,2,3\}$, width
+$\{32,64,96\}$, and horizon $\{25,50,100\}$, the early-velocity estimator
+predicts the finite-time gap at mean absolute error
+$\approx1.9\times10^{-3}$ and correlation $0.994$--$0.99989$---about a
+ten-fold improvement over the frozen-$K_0$ baseline, again with no fitted
+parameter. For a phenomenon previously described only by adjectives, this
+is a quantitative predictor that simply works.
+
+\begin{figure*}[t]
+\centering
+\includegraphics[width=0.92\textwidth]{closed_form_vs_measured_ramp.png}
+\caption{\textbf{The FA/BP gap is a soft ramp, and our closed form explains
+it.} \emph{Left:} the empirical gap (blue) and the no-fit closed-form
+prediction \eqref{eq:gap} (orange) across capacity margin---both smooth,
+neither with a kink at margin $0$, and the gap stays positive even at
+positive margin (no phase transition). \emph{Right:} the ramp is driven by
+the smallest BP-kernel eigenvalue $\lambda_{\min}$ exactly as
+Corollary~\ref{cor:ramp} predicts ($\mathrm{corr}=0.963$).}
+\label{fig:ramp}
+\end{figure*}
+
+\section{Related Work}
+
+\paragraph{Feedback alignment.} \citet{lillicrap2016} introduced FA and
+documented that forward weights drift into alignment with the random
+feedback; \citet{nokland2016} carried this to direct feedback alignment in
+deep nets; \citet{refinetti2021} described the align-then-memorize
+dynamics. These works answer ``can FA learn?''. We answer ``what does it
+cost?''---a different and, we would argue, more basic question, and one
+their results leave entirely open.
+
+\paragraph{Convergence theory.} \citet{song2021} prove two-layer FA
+converges when over-parameterized, with alignment optional absent
+regularization. Far from conflicting with us, this is the perfect
+complement: their decomposition $\KFA=G+H_{\mathrm{FA}}$ is, at
+initialization, exactly our Theorem~\ref{thm:e0}, which forces
+$\E_B[H_{\mathrm{FA}}]=0$ and hence $\E_B[\KFA]=G$. Convergence to zero
+\emph{final} error and a nonzero \emph{operator} gap are simply different
+statements, and our paper is about the second.
+
+\paragraph{Tangent kernels and capacity.} The NTK and lazy-training
+line~\cite{jacot2018,lee2019} licenses the residual recursion we build on;
+our contribution there is to handle FA's non-symmetric operator. And while
+the fact that nets can fit random labels~\cite{zhang2017} invites the guess
+that capacity makes random feedback free, our soft-erosion result is exactly
+the rebuttal: capacity dilutes the cost through $k/P$ and never removes it.
+
+\section{Limitations}
+
+We are explicit about scope. The experiments are synthetic MLPs trained
+full-batch; convolutional and transformer architectures, stochastic
+optimization, and real data are future work. The closed-form law
+(Thm.~\ref{thm:gap}) uses a lazy-training approximation and a scalar erosion
+model that is exact only in the residual direction---a per-eigenmode
+refinement is the obvious next theorem. And the finite-time estimator,
+though unfitted, is \emph{conditional} on two early kernel snapshots;
+deriving the operator drift from the architecture alone---via infinite-width
+recursions for $\rho$ and $\lambda_{\min}$---would close the last gap
+between mechanism and magnitude. None of these qualify the three results we
+do prove.
+
+\section{Conclusion}
+
+Random feedback is not free, and after this paper it is no longer
+mysterious. Its static cost is an exact $\Beta$ law that adds to
+$\Theta(Ln^2)$; that cost is provably the best any prior-free initialization
+can do; and the optimization gap it induces is always positive, equals the
+hidden-layer speed share at initialization, and grows as a closed-form soft
+ramp that we predict without fitting anything. The reason over-parameterized
+FA can look almost free is now precise: the cost is real and irreducible, but
+soft, scaling as $k/P$ and as the leftover error in the slowest learning
+direction. We think this is the right way to understand feedback
+alignment---as a quantity, not an anecdote.
+
+\begin{thebibliography}{9}
+\bibitem[Lillicrap et al.(2016)]{lillicrap2016}
+T.~Lillicrap, D.~Cownden, D.~Tweed, and C.~Akerman.
+\newblock Random synaptic feedback weights support error backpropagation
+for deep learning.
+\newblock \emph{Nature Communications}, 2016.
+
+\bibitem[N{\o}kland(2016)]{nokland2016}
+A.~N{\o}kland.
+\newblock Direct feedback alignment provides learning in deep neural
+networks.
+\newblock In \emph{NeurIPS}, 2016.
+
+\bibitem[Refinetti et al.(2021)]{refinetti2021}
+M.~Refinetti, S.~d'Ascoli, R.~Ohana, and S.~Goldt.
+\newblock Align, then memorise: the dynamics of learning with feedback
+alignment.
+\newblock In \emph{ICML}, 2021.
+
+\bibitem[Song et al.(2021)]{song2021}
+G.~Song, R.~Xu, and J.~Lafferty.
+\newblock Convergence and alignment of gradient descent with random
+backpropagation weights.
+\newblock In \emph{NeurIPS}, 2021.
+
+\bibitem[Jacot et al.(2018)]{jacot2018}
+A.~Jacot, F.~Gabriel, and C.~Hongler.
+\newblock Neural tangent kernel: convergence and generalization in neural
+networks.
+\newblock In \emph{NeurIPS}, 2018.
+
+\bibitem[Lee et al.(2019)]{lee2019}
+J.~Lee et al.
+\newblock Wide neural networks of any depth evolve as linear models under
+gradient descent.
+\newblock In \emph{NeurIPS}, 2019.
+
+\bibitem[Zhang et al.(2017)]{zhang2017}
+C.~Zhang, S.~Bengio, M.~Hardt, B.~Recht, and O.~Vinyals.
+\newblock Understanding deep learning requires rethinking generalization.
+\newblock In \emph{ICLR}, 2017.
+\end{thebibliography}
+
diff --git a/paper/defs.tex b/paper/defs.tex
new file mode 100644
index 0000000..0f6d335
--- /dev/null
+++ b/paper/defs.tex
@@ -0,0 +1,22 @@
+% Shared macros and theorem environments for main.tex (AAAI) and preview.tex.
+\usepackage{amsmath,amssymb,amsthm}
+\usepackage{graphicx}
+\graphicspath{{../outputs/closed_form_soft_ramp/}}
+
+\theoremstyle{plain}
+\newtheorem{theorem}{Theorem}
+\newtheorem{proposition}{Proposition}
+\newtheorem{corollary}{Corollary}
+\theoremstyle{definition}
+\newtheorem{assumption}{Assumption}
+
+\newcommand{\takeaway}[1]{\par\smallskip\noindent\textbf{Takeaway.} \emph{#1}\par\smallskip}
+
+\newcommand{\R}{\mathbb{R}}
+\newcommand{\E}{\mathbb{E}}
+\renewcommand{\Pr}{\mathbb{P}}
+\newcommand{\vecop}{\mathrm{vec}}
+\newcommand{\Beta}{\mathrm{Beta}}
+\newcommand{\KBP}{K_{\mathrm{BP}}}
+\newcommand{\KFA}{K_{\mathrm{FA}}}
+\newcommand{\SFA}{S_{\mathrm{FA}}}
diff --git a/paper/main.tex b/paper/main.tex
new file mode 100644
index 0000000..8006ed0
--- /dev/null
+++ b/paper/main.tex
@@ -0,0 +1,45 @@
+% =====================================================================
+% AAAI AuthorKit 27 format.
+%
+% TO COMPILE: download the AuthorKit from https://aaai.org/authorkit27/ ,
+% place aaai27.sty (and aaai27.bst if you switch to a .bib) next to this
+% file, then run:   pdflatex main ; pdflatex main     (or: tectonic main)
+%
+% aaai27.sty is distributed only inside the AuthorKit zip (not on CTAN),
+% so it is not bundled here. The body lives in body.tex and is shared with
+% preview.tex (a generic wrapper that compiles WITHOUT the kit, for a quick
+% look: tectonic preview.tex).
+% Figure: python scripts/closed_form_soft_ramp.py
+% =====================================================================
+\documentclass[letterpaper]{article} % DO NOT CHANGE THIS
+\usepackage[submission]{aaai27}       % DO NOT CHANGE THIS
+\usepackage{times}                    % DO NOT CHANGE THIS
+\usepackage{helvet}                   % DO NOT CHANGE THIS
+\usepackage{courier}                  % DO NOT CHANGE THIS
+\usepackage[hyphens]{url}             % DO NOT CHANGE THIS
+\usepackage{graphicx}                 % DO NOT CHANGE THIS
+\urlstyle{rm}                         % DO NOT CHANGE THIS
+\def\UrlFont{\rm}                     % DO NOT CHANGE THIS
+\usepackage{natbib}                   % DO NOT CHANGE THIS
+\usepackage{caption}                  % DO NOT CHANGE THIS
+\frenchspacing                        % DO NOT CHANGE THIS
+\setlength{\pdfpagewidth}{8.5in}      % DO NOT CHANGE THIS
+\setlength{\pdfpageheight}{11in}      % DO NOT CHANGE THIS
+
+\input{defs}
+
+\setcounter{secnumdepth}{2} % number sections so \ref{sec:...} resolves
+
+\pdfinfo{
+/TemplateVersion (2027.1)
+}
+
+\title{What Does Random Feedback Cost?\\
+A Distributional Capacity and Operator-Gap Analysis of Feedback Alignment}
+\author{Anonymous Submission}
+\affiliations{Association for the Advancement of Artificial Intelligence}
+
+\begin{document}
+\maketitle
+\input{body}
+\end{document}
diff --git a/paper/preview.tex b/paper/preview.tex
new file mode 100644
index 0000000..03e00ee
--- /dev/null
+++ b/paper/preview.tex
@@ -0,0 +1,22 @@
+% =====================================================================
+% Generic two-column preview wrapper. Compiles WITHOUT the AAAI kit, so
+% you can read the paper now:   tectonic preview.tex
+% Shares the exact body (body.tex) and macros (defs.tex) with the official
+% AAAI version (main.tex). Layout differs from AAAI; content is identical.
+% =====================================================================
+\documentclass[twocolumn,letterpaper,10pt]{article}
+\usepackage[margin=0.7in]{geometry}
+\usepackage{natbib}
+\usepackage{microtype}
+
+\input{defs}
+
+\title{\bfseries What Does Random Feedback Cost?\\
+\large A Distributional Capacity and Operator-Gap Analysis of Feedback Alignment}
+\author{Anonymous submission}
+\date{}
+
+\begin{document}
+\maketitle
+\input{body}
+\end{document}