diff options
Diffstat (limited to 'collaborativeagents/slurm/logs/run_expts_a100_14355885.err')
| -rw-r--r-- | collaborativeagents/slurm/logs/run_expts_a100_14355885.err | 17 |
1 files changed, 17 insertions, 0 deletions
diff --git a/collaborativeagents/slurm/logs/run_expts_a100_14355885.err b/collaborativeagents/slurm/logs/run_expts_a100_14355885.err new file mode 100644 index 0000000..ffc2040 --- /dev/null +++ b/collaborativeagents/slurm/logs/run_expts_a100_14355885.err @@ -0,0 +1,17 @@ +2025-12-25 08:58:07,929 - INFO - Loaded dataset: math-500 +2025-12-25 08:58:08,121 - INFO - Loaded 100 profiles from ../data/complex_profiles_v2/profiles_100.jsonl +2025-12-25 08:58:08,122 - INFO - Running method: vanilla +2025-12-25 08:58:08,122 - INFO - Profile 1/20 +`torch_dtype` is deprecated! Use `dtype` instead! +
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:02<00:06, 2.33s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:06<00:06, 3.19s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:09<00:03, 3.35s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:09<00:00, 2.10s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:09<00:00, 2.46s/it] +
Loading checkpoint shards: 0%| | 0/5 [00:00<?, ?it/s]
Loading checkpoint shards: 20%|██ | 1/5 [00:03<00:14, 3.68s/it]
Loading checkpoint shards: 40%|████ | 2/5 [00:10<00:16, 5.46s/it]
Loading checkpoint shards: 60%|██████ | 3/5 [00:15<00:10, 5.08s/it]
Loading checkpoint shards: 80%|████████ | 4/5 [00:17<00:04, 4.22s/it]
Loading checkpoint shards: 100%|██████████| 5/5 [00:19<00:00, 3.30s/it]
Loading checkpoint shards: 100%|██████████| 5/5 [00:19<00:00, 3.91s/it] +2025-12-25 08:58:47,629 - ERROR - Error in session: 'NoneType' object is not subscriptable +
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:03<00:10, 3.44s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:08<00:08, 4.11s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:12<00:04, 4.27s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:12<00:00, 2.65s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:12<00:00, 3.16s/it] +
Loading checkpoint shards: 0%| | 0/5 [00:00<?, ?it/s]
Loading checkpoint shards: 20%|██ | 1/5 [00:03<00:13, 3.45s/it]
Loading checkpoint shards: 40%|████ | 2/5 [00:09<00:15, 5.12s/it]
Loading checkpoint shards: 60%|██████ | 3/5 [00:13<00:09, 4.65s/it]
Loading checkpoint shards: 80%|████████ | 4/5 [00:16<00:04, 4.00s/it]
Loading checkpoint shards: 100%|██████████| 5/5 [00:18<00:00, 3.10s/it]
Loading checkpoint shards: 100%|██████████| 5/5 [00:18<00:00, 3.67s/it] +2025-12-25 08:59:23,124 - ERROR - Error in session: 'NoneType' object is not subscriptable +
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:02<00:08, 2.95s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:07<00:07, 3.65s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:11<00:03, 3.88s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:11<00:00, 2.41s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:11<00:00, 2.85s/it] +
Loading checkpoint shards: 0%| | 0/5 [00:00<?, ?it/s]
Loading checkpoint shards: 20%|██ | 1/5 [00:02<00:09, 2.41s/it]
Loading checkpoint shards: 40%|████ | 2/5 [00:09<00:15, 5.30s/it]
Loading checkpoint shards: 60%|██████ | 3/5 [00:14<00:09, 4.86s/it]
Loading checkpoint shards: 80%|████████ | 4/5 [00:17<00:04, 4.12s/it]
Loading checkpoint shards: 100%|██████████| 5/5 [00:18<00:00, 3.11s/it]
Loading checkpoint shards: 100%|██████████| 5/5 [00:18<00:00, 3.68s/it] +2025-12-25 08:59:57,630 - ERROR - Error in session: 'NoneType' object is not subscriptable +
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:02<00:08, 2.88s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:06<00:07, 3.60s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:11<00:03, 3.84s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:11<00:00, 2.39s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:11<00:00, 2.82s/it] +
Loading checkpoint shards: 0%| | 0/5 [00:00<?, ?it/s]
Loading checkpoint shards: 20%|██ | 1/5 [00:01<00:07, 1.91s/it]
Loading checkpoint shards: 40%|████ | 2/5 [00:08<00:14, 4.87s/it][2025-12-25T09:00:23.259] error: *** JOB 14355885 ON gpua055 CANCELLED AT 2025-12-25T09:00:23 DUE to SIGNAL Terminated *** +
Loading checkpoint shards: 60%|██████ | 3/5 [00:13<00:09, 4.74s/it]
\ No newline at end of file |