diff options
| author | YurenHao0426 <blackhao0426@gmail.com> | 2026-06-13 12:35:36 -0500 |
|---|---|---|
| committer | YurenHao0426 <blackhao0426@gmail.com> | 2026-06-13 12:35:36 -0500 |
| commit | 66e0d8b9fd4d0f7a2231d689c055e26fdf1cf04a (patch) | |
| tree | c29cba61124018755a19b02c9d33e3ad5f2e05cc /trm/config/cfg_pretrain.yaml | |
Curated export for clone-and-run Maze training (2x A6000) + diagnostics.
trm/hrm pretrain.py carry trajectory-augmentation code (backward-compatible).
Heavy artifacts (checkpoints/wandb/npz) gitignored; see PROVENANCE.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Diffstat (limited to 'trm/config/cfg_pretrain.yaml')
| -rw-r--r-- | trm/config/cfg_pretrain.yaml | 42 |
1 files changed, 42 insertions, 0 deletions
diff --git a/trm/config/cfg_pretrain.yaml b/trm/config/cfg_pretrain.yaml new file mode 100644 index 0000000..ec45faf --- /dev/null +++ b/trm/config/cfg_pretrain.yaml @@ -0,0 +1,42 @@ +# ARC training config + +defaults: + - arch: trm + - _self_ + +hydra: + output_subdir: null + +# Data path +data_paths: ['data/arc-aug-1000'] +data_paths_test: [] + +evaluators: + - name: arc@ARC + +# Hyperparams - Training +global_batch_size: 768 + +epochs: 100000 +eval_interval: 10000 +checkpoint_every_eval: True + +lr: 1e-4 +lr_min_ratio: 1.0 +lr_warmup_steps: 2000 + +# Standard hyperparameter settings for LM, as used in Llama +beta1: 0.9 +beta2: 0.95 +weight_decay: 0.1 +puzzle_emb_weight_decay: 0.1 + +# Hyperparams - Puzzle embeddings training +puzzle_emb_lr: 1e-2 + +seed: 0 +min_eval_interval: 0 # when to start the eval + +ema: False # use Exponential-Moving-Average +ema_rate: 0.999 # EMA-rate +freeze_weights: False # If True, freeze weights and only learn the embeddings
\ No newline at end of file |