diff options
| author | YurenHao0426 <blackhao0426@gmail.com> | 2026-06-13 12:35:36 -0500 |
|---|---|---|
| committer | YurenHao0426 <blackhao0426@gmail.com> | 2026-06-13 12:35:36 -0500 |
| commit | 66e0d8b9fd4d0f7a2231d689c055e26fdf1cf04a (patch) | |
| tree | c29cba61124018755a19b02c9d33e3ad5f2e05cc /trm/config/arch/hrm.yaml | |
Curated export for clone-and-run Maze training (2x A6000) + diagnostics.
trm/hrm pretrain.py carry trajectory-augmentation code (backward-compatible).
Heavy artifacts (checkpoints/wandb/npz) gitignored; see PROVENANCE.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Diffstat (limited to 'trm/config/arch/hrm.yaml')
| -rw-r--r-- | trm/config/arch/hrm.yaml | 24 |
1 files changed, 24 insertions, 0 deletions
diff --git a/trm/config/arch/hrm.yaml b/trm/config/arch/hrm.yaml new file mode 100644 index 0000000..0d183be --- /dev/null +++ b/trm/config/arch/hrm.yaml @@ -0,0 +1,24 @@ +name: recursive_reasoning.hrm@HierarchicalReasoningModel_ACTV1 +loss: + name: losses@ACTLossHead + loss_type: stablemax_cross_entropy + +halt_exploration_prob: 0.1 +halt_max_steps: 16 + +H_cycles: 2 +L_cycles: 2 + +H_layers: 4 +L_layers: 4 + +hidden_size: 512 +num_heads: 8 # min(2, hidden_size // 64) +expansion: 4 + +puzzle_emb_ndim: ${.hidden_size} + +pos_encodings: rope +forward_dtype: bfloat16 + +mlp_t: False # use mlp on L instead of transformer
\ No newline at end of file |