From 66e0d8b9fd4d0f7a2231d689c055e26fdf1cf04a Mon Sep 17 00:00:00 2001 From: YurenHao0426 Date: Sat, 13 Jun 2026 12:35:36 -0500 Subject: rrm workspace: TRM/HRM/SRM code, Maze dataset, dynamical-analysis pipeline Curated export for clone-and-run Maze training (2x A6000) + diagnostics. trm/hrm pretrain.py carry trajectory-augmentation code (backward-compatible). Heavy artifacts (checkpoints/wandb/npz) gitignored; see PROVENANCE.md. Co-Authored-By: Claude Fable 5 --- .../rainer_email_bundle_20260605/README.md | 17 ++++++++++++++++ .../rainer_email_bundle_20260605/email_draft.md | 18 +++++++++++++++++ .../figure_captions.md | 23 ++++++++++++++++++++++ 3 files changed, 58 insertions(+) create mode 100644 research/flossing/rainer_email_bundle_20260605/README.md create mode 100644 research/flossing/rainer_email_bundle_20260605/email_draft.md create mode 100644 research/flossing/rainer_email_bundle_20260605/figure_captions.md (limited to 'research/flossing/rainer_email_bundle_20260605') diff --git a/research/flossing/rainer_email_bundle_20260605/README.md b/research/flossing/rainer_email_bundle_20260605/README.md new file mode 100644 index 0000000..c02097b --- /dev/null +++ b/research/flossing/rainer_email_bundle_20260605/README.md @@ -0,0 +1,17 @@ +# Rainer Email Figure Bundle + +Prepared on 2026-06-05. + +Contents: + +- `email_draft.md`: short email draft with `(Fig. n)` references. +- `figure_captions.md`: figure captions and caveats. +- `figures/Fig1_lambda1_success_failure_HRM_TRM.png` +- `figures/Fig2_full_spectrum_success_failure_shift.png` +- `figures/Fig3_trajectory_perturbation_improves_peak_accuracy.png` +- `figures/Fig4_optional_PTRM_Q_head_vs_stability.png` + +Suggested attachment strategy: + +- Attach Fig. 1, Fig. 2, and Fig. 3 by default. +- Attach Fig. 4 only if you want to include the PTRM/Q-head side observation. diff --git a/research/flossing/rainer_email_bundle_20260605/email_draft.md b/research/flossing/rainer_email_bundle_20260605/email_draft.md new file mode 100644 index 0000000..cd92465 --- /dev/null +++ b/research/flossing/rainer_email_bundle_20260605/email_draft.md @@ -0,0 +1,18 @@ +Subject: Question on gradient flossing vs forward trajectory stability in recursive reasoning models + +Hi Rainer, + +I hope you are doing well. I have been studying recursive reasoning models: small recurrent/iterative models that solve a problem by repeatedly refining a latent reasoning state before emitting an answer, for example HRM/TRM-style models on Sudoku-like reasoning tasks. + +We found a strong dynamical signal during inference. If we measure finite-time Lyapunov exponents along the model's recurrent inference trajectory, failed examples are much more chaotic than successful examples. This is already visible in the first exponent (Fig. 1). After measuring more of the spectrum, the effect looks less like one isolated bad mode and more like a broad shift of the spectrum toward expansion on failing examples (Fig. 2). + +This made me revisit your gradient flossing work. My current interpretation is that gradient flossing is mainly about improving the stability/conditioning of Jacobian products for learning through recurrent dynamics, while our signal may be more about forward inference stability: whether the correct answer lies in a sufficiently stable attractor basin. We tried some Engelken-style pre/interflossing analogues; they reproduce the toy RNN effect, but in our RRM setting the effect is weak so far. In contrast, a simple forward perturbation training objective looks more promising: for the same supervised pair `(x, y)`, we run additional trajectories with small perturbations to the recurrent latent state and require them to reach the same target. This raises peak accuracy/ceiling in our current runs (Fig. 3). + +My question is whether this distinction between gradient-propagation stability and forward-inference attractor stability makes sense from your perspective. If so, would you expect spectrum flossing to need to be local, task-conditioned, or late-trajectory-only to help here? Or is there a more standard dynamical-systems/control framing, such as transverse stability, basin enlargement, shadowing, or stochastic stabilization, that you think is more appropriate? + +I attached a small figure pack. Fig. 4 is optional context: in a probabilistic TRM variant with multiple noisy rollouts and a learned Q-head selector, the learned Q score is correlated with a finite-difference stability proxy, suggesting that the selector may be learning a low-dimensional projection of trajectory stability. + +I would be grateful for any pointer or sanity check, especially if we are misusing the gradient flossing intuition. + +Best, +Yuren diff --git a/research/flossing/rainer_email_bundle_20260605/figure_captions.md b/research/flossing/rainer_email_bundle_20260605/figure_captions.md new file mode 100644 index 0000000..04872f6 --- /dev/null +++ b/research/flossing/rainer_email_bundle_20260605/figure_captions.md @@ -0,0 +1,23 @@ +# Figure Captions + +## Fig. 1: `figures/Fig1_lambda1_success_failure_HRM_TRM.png` + +First finite-time Lyapunov exponent distribution for successful versus failed inference trajectories in HRM and TRM checkpoints. Failures are shifted toward larger/more positive exponents, motivating chaos as a failure detector. + +## Fig. 2: `figures/Fig2_full_spectrum_success_failure_shift.png` + +Top Lyapunov spectrum for successful versus failed examples. The separation is not only a top-exponent effect; many leading modes shift toward expansion on failed examples. + +## Fig. 3: `figures/Fig3_trajectory_perturbation_improves_peak_accuracy.png` + +Peak exact accuracy for baseline training versus trajectory perturbation training. The perturbation method keeps the same supervised input/target pair but trains additional recurrent rollouts with small latent-state perturbations to reach the same answer. This is a ceiling/peak result; HRM later shows final-checkpoint collapse. + +## Fig. 4: `figures/Fig4_optional_PTRM_Q_head_vs_stability.png` + +Optional context. In PTRM-style stochastic multi-rollout inference, the learned Q-head score is correlated with a finite-difference stability proxy on mixed-success problems, suggesting that learned rollout selection may partly approximate a low-dimensional stability score. + +# Caveats + +- Lyapunov measurements are finite-time diagnostic estimates on sampled subsets, not full asymptotic exponents. +- Fig. 3 reports best-checkpoint/peak accuracy, not necessarily final-checkpoint accuracy. +- The gradient flossing analogue is still preliminary; the main question for Rainer is conceptual, not a claim that flossing does or does not work in RRMs. -- cgit v1.2.3