| Age | Commit message (Collapse) | Author | |
|---|---|---|---|
| 2026-02-11 | A12-A14 init_logit ablation: confirm frozen OLMo cannot benefit from sparse ↵HEADmain | YurenHao0426 | |
| topology - A12 (logit=3): NLL 2.76, A13 (logit=0): NLL 3.51, A14 (logit=1): NLL 3.26 - All worse than baseline (2.46). Lower init_logit = more deviation = worse NLL - Confirms: gradient flows (gates move), but A=1 is global optimum for frozen model - Added Dolma streaming retry logic (max 10 retries, 30s wait) - Phase 1 frozen approach has fundamental limitation; Phase 2 (unfreeze) needed Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> | |||
| 2026-02-09 | Initial implementation: DAGFormer Phase 1 | YurenHao0426 | |
| - olmo_graph.py: Modified OLMo2-1B forward with per-head routing via 256x256 adjacency matrix A - Proportional attribution for post-norm decomposition - All 6 GPU sanity checks pass (baseline diff = 0.000001) - predictor.py: Qwen3-Embedding encoder + MLP decoder + Gumbel-Sigmoid + cascading gate - pipeline.py: End-to-end glue (predictor -> A -> OLMo -> NLL) - trainer.py: Full training loop with DDP, gradient accumulation, eval, checkpointing - dolma.py: Streaming Dolma v1.7 with sequence packing - 43/43 unit tests pass Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> | |||
 |
index : dagformer.git | |
| Unnamed repository; edit this file 'description' to name the repository. | Ubuntu |
| summaryrefslogtreecommitdiff |