1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
|
/u/yurenh2/miniforge3/envs/eval/lib/python3.11/site-packages/transformers/utils/hub.py:110: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead.
warnings.warn(
2025-12-27 02:31:09,224 - INFO - Loaded dataset: mmlu
2025-12-27 02:31:09,224 - INFO - Loaded dataset: aime
2025-12-27 02:31:09,224 - INFO - Loaded dataset: math-hard
2025-12-27 02:31:09,224 - INFO - Loaded dataset: humaneval
2025-12-27 02:31:09,299 - INFO - Loaded 100 profiles from ../data/complex_profiles_v2/profiles_100.jsonl
2025-12-27 02:31:09,299 - INFO - Running method: reflection_grpo
`torch_dtype` is deprecated! Use `dtype` instead!
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:06<00:19, 6.49s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:12<00:12, 6.14s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:18<00:06, 6.13s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:20<00:00, 4.41s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:20<00:00, 5.07s/it]
2025-12-27 02:31:33,591 - INFO - Profile 1/30
/u/yurenh2/miniforge3/envs/eval/lib/python3.11/site-packages/awq/__init__.py:21: DeprecationWarning:
I have left this message as the final dev message to help you transition.
Important Notice:
- AutoAWQ is officially deprecated and will no longer be maintained.
- The last tested configuration used Torch 2.6.0 and Transformers 4.51.3.
- If future versions of Transformers break AutoAWQ compatibility, please report the issue to the Transformers project.
Alternative:
- AutoAWQ has been adopted by the vLLM Project: https://github.com/vllm-project/llm-compressor
For further inquiries, feel free to reach out:
- X: https://x.com/casper_hansen_
- LinkedIn: https://www.linkedin.com/in/casper-hansen-804005170/
warnings.warn(_FINAL_DEV_MESSAGE, category=DeprecationWarning, stacklevel=1)
Loading checkpoint shards: 0%| | 0/9 [00:00<?, ?it/s]
Loading checkpoint shards: 11%|█ | 1/9 [00:03<00:31, 3.95s/it]
Loading checkpoint shards: 22%|██▏ | 2/9 [00:08<00:31, 4.48s/it]
Loading checkpoint shards: 33%|███▎ | 3/9 [00:13<00:27, 4.63s/it]
Loading checkpoint shards: 44%|████▍ | 4/9 [00:18<00:23, 4.75s/it]
Loading checkpoint shards: 56%|█████▌ | 5/9 [00:23<00:19, 4.93s/it]
Loading checkpoint shards: 67%|██████▋ | 6/9 [00:29<00:15, 5.04s/it]
Loading checkpoint shards: 78%|███████▊ | 7/9 [00:33<00:09, 4.96s/it]
Loading checkpoint shards: 89%|████████▉ | 8/9 [00:37<00:04, 4.46s/it]
Loading checkpoint shards: 100%|██████████| 9/9 [00:39<00:00, 3.63s/it]
Loading checkpoint shards: 100%|██████████| 9/9 [00:39<00:00, 4.34s/it]
2025-12-27 03:43:56,265 - WARNING - User agent failed to respond at turn 4
2025-12-27 03:44:19,363 - INFO - Profile 2/30
2025-12-27 04:41:18,679 - INFO - Profile 3/30
2025-12-27 04:50:08,015 - WARNING - User agent failed to respond at turn 6
2025-12-27 05:37:09,400 - WARNING - User agent failed to respond at turn 4
2025-12-27 05:39:22,155 - INFO - Profile 4/30
2025-12-27 05:51:40,082 - WARNING - User agent failed to respond at turn 4
2025-12-27 06:30:54,910 - WARNING - User agent failed to respond at turn 4
2025-12-27 06:55:12,778 - INFO - Profile 5/30
2025-12-27 07:48:39,008 - INFO - Profile 6/30
2025-12-27 08:01:42,219 - WARNING - User agent failed to respond at turn 3
2025-12-27 08:23:42,492 - WARNING - User agent failed to respond at turn 3
2025-12-27 08:54:04,212 - INFO - Profile 7/30
2025-12-27 08:58:13,539 - WARNING - User agent failed to respond at turn 3
2025-12-27 09:24:36,991 - WARNING - User agent failed to respond at turn 7
2025-12-27 10:01:43,345 - WARNING - User agent failed to respond at turn 3
2025-12-27 10:04:41,897 - WARNING - User agent failed to respond at turn 2
2025-12-27 10:20:11,751 - INFO - Profile 8/30
2025-12-27 11:08:02,876 - WARNING - User agent failed to respond at turn 4
2025-12-27 11:20:28,004 - WARNING - User agent failed to respond at turn 5
2025-12-27 11:46:14,996 - WARNING - User agent failed to respond at turn 4
2025-12-27 11:46:33,648 - INFO - Profile 9/30
2025-12-27 12:22:26,369 - WARNING - User agent failed to respond at turn 9
2025-12-27 12:56:13,166 - INFO - Profile 10/30
2025-12-27 13:02:01,791 - WARNING - User agent failed to respond at turn 2
2025-12-27 13:24:51,498 - WARNING - User agent failed to respond at turn 3
2025-12-27 14:16:50,083 - INFO - Profile 11/30
2025-12-27 14:27:09,697 - WARNING - User agent failed to respond at turn 3
2025-12-27 15:40:22,936 - INFO - Profile 12/30
2025-12-27 15:52:57,164 - WARNING - User agent failed to respond at turn 5
2025-12-27 16:28:17,345 - WARNING - User agent failed to respond at turn 4
2025-12-27 16:50:21,596 - INFO - Profile 13/30
2025-12-27 17:41:49,444 - WARNING - User agent failed to respond at turn 4
2025-12-27 17:50:43,295 - INFO - Profile 14/30
2025-12-27 18:08:38,210 - WARNING - User agent failed to respond at turn 2
2025-12-27 18:44:39,617 - WARNING - User agent failed to respond at turn 8
2025-12-27 18:47:39,503 - WARNING - User agent failed to respond at turn 4
2025-12-27 19:00:23,116 - INFO - Profile 15/30
2025-12-27 19:12:53,841 - WARNING - User agent failed to respond at turn 4
2025-12-27 20:03:36,023 - INFO - Profile 16/30
2025-12-27 20:50:11,725 - INFO - Profile 17/30
2025-12-27 20:54:36,277 - WARNING - User agent failed to respond at turn 3
2025-12-27 22:18:15,804 - INFO - Profile 18/30
2025-12-27 22:40:24,135 - WARNING - User agent failed to respond at turn 3
2025-12-27 23:04:20,252 - WARNING - User agent failed to respond at turn 4
2025-12-27 23:23:13,204 - INFO - Profile 19/30
2025-12-28 00:30:41,183 - INFO - Profile 20/30
2025-12-28 01:13:04,372 - WARNING - User agent failed to respond at turn 6
2025-12-28 01:21:59,883 - WARNING - User agent failed to respond at turn 3
2025-12-28 01:47:59,918 - WARNING - User agent failed to respond at turn 7
2025-12-28 01:53:24,077 - WARNING - User agent failed to respond at turn 5
2025-12-28 02:13:56,170 - WARNING - User agent failed to respond at turn 3
2025-12-28 02:14:14,770 - INFO - Profile 21/30
2025-12-28 03:20:09,605 - WARNING - User agent failed to respond at turn 3
2025-12-28 04:10:58,912 - INFO - Profile 22/30
2025-12-28 05:16:04,670 - WARNING - User agent failed to respond at turn 4
2025-12-28 05:29:32,044 - INFO - Profile 23/30
2025-12-28 05:46:53,577 - WARNING - User agent failed to respond at turn 6
2025-12-28 05:57:05,360 - WARNING - User agent failed to respond at turn 6
2025-12-28 06:14:11,895 - WARNING - User agent failed to respond at turn 5
2025-12-28 06:21:21,665 - WARNING - User agent failed to respond at turn 3
2025-12-28 06:43:49,754 - INFO - Profile 24/30
2025-12-28 06:56:35,737 - WARNING - User agent failed to respond at turn 3
2025-12-28 07:54:52,613 - INFO - Profile 25/30
2025-12-28 08:24:24,212 - WARNING - User agent failed to respond at turn 2
2025-12-28 09:01:32,435 - INFO - Profile 26/30
2025-12-28 09:24:20,607 - WARNING - User agent failed to respond at turn 4
2025-12-28 09:28:45,402 - WARNING - User agent failed to respond at turn 3
2025-12-28 09:31:07,307 - WARNING - User agent failed to respond at turn 2
2025-12-28 09:37:47,214 - WARNING - User agent failed to respond at turn 2
2025-12-28 09:49:55,833 - WARNING - User agent failed to respond at turn 3
2025-12-28 09:54:03,278 - WARNING - User agent failed to respond at turn 2
2025-12-28 10:13:11,944 - INFO - Profile 27/30
2025-12-28 10:34:58,991 - WARNING - User agent failed to respond at turn 5
2025-12-28 10:42:04,222 - WARNING - User agent failed to respond at turn 4
2025-12-28 10:49:59,056 - WARNING - User agent failed to respond at turn 3
2025-12-28 11:22:55,596 - WARNING - User agent failed to respond at turn 2
2025-12-28 11:36:59,263 - INFO - Profile 28/30
2025-12-28 11:49:22,297 - WARNING - User agent failed to respond at turn 6
2025-12-28 11:52:54,358 - WARNING - User agent failed to respond at turn 3
2025-12-28 12:04:04,093 - WARNING - User agent failed to respond at turn 4
2025-12-28 12:11:08,251 - WARNING - User agent failed to respond at turn 3
2025-12-28 12:17:19,369 - WARNING - User agent failed to respond at turn 6
2025-12-28 12:20:37,784 - WARNING - User agent failed to respond at turn 4
2025-12-28 12:33:13,220 - WARNING - User agent failed to respond at turn 4
2025-12-28 12:41:36,621 - WARNING - User agent failed to respond at turn 5
2025-12-28 12:44:34,822 - WARNING - User agent failed to respond at turn 3
2025-12-28 12:50:56,698 - WARNING - User agent failed to respond at turn 6
2025-12-28 12:56:12,269 - INFO - Profile 29/30
2025-12-28 14:22:05,011 - INFO - Profile 30/30
2025-12-28 15:13:10,367 - INFO - Report saved to ../results/reflection_grpo_20251227_023047/20251227_023109/report.md
|
