1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
|
/u/yurenh2/miniforge3/envs/eval/lib/python3.11/site-packages/transformers/utils/hub.py:110: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead.
warnings.warn(
2025-12-27 02:02:06,308 - INFO - Loaded dataset: mmlu
2025-12-27 02:02:06,308 - INFO - Loaded dataset: aime
2025-12-27 02:02:06,308 - INFO - Loaded dataset: math-hard
2025-12-27 02:02:06,308 - INFO - Loaded dataset: humaneval
2025-12-27 02:02:06,329 - INFO - Loaded 100 profiles from ../data/complex_profiles_v2/profiles_100.jsonl
2025-12-27 02:02:06,329 - INFO - Running method: contextual
`torch_dtype` is deprecated! Use `dtype` instead!
Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards: 25%|██▌ | 1/4 [00:07<00:21, 7.14s/it]
Loading checkpoint shards: 50%|█████ | 2/4 [00:13<00:12, 6.50s/it]
Loading checkpoint shards: 75%|███████▌ | 3/4 [00:19<00:06, 6.61s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:21<00:00, 4.66s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:21<00:00, 5.40s/it]
2025-12-27 02:02:32,180 - INFO - Profile 1/30
/u/yurenh2/miniforge3/envs/eval/lib/python3.11/site-packages/awq/__init__.py:21: DeprecationWarning:
I have left this message as the final dev message to help you transition.
Important Notice:
- AutoAWQ is officially deprecated and will no longer be maintained.
- The last tested configuration used Torch 2.6.0 and Transformers 4.51.3.
- If future versions of Transformers break AutoAWQ compatibility, please report the issue to the Transformers project.
Alternative:
- AutoAWQ has been adopted by the vLLM Project: https://github.com/vllm-project/llm-compressor
For further inquiries, feel free to reach out:
- X: https://x.com/casper_hansen_
- LinkedIn: https://www.linkedin.com/in/casper-hansen-804005170/
warnings.warn(_FINAL_DEV_MESSAGE, category=DeprecationWarning, stacklevel=1)
Loading checkpoint shards: 0%| | 0/9 [00:00<?, ?it/s]
Loading checkpoint shards: 11%|█ | 1/9 [00:04<00:33, 4.15s/it]
Loading checkpoint shards: 22%|██▏ | 2/9 [00:08<00:31, 4.52s/it]
Loading checkpoint shards: 33%|███▎ | 3/9 [00:13<00:28, 4.68s/it]
Loading checkpoint shards: 44%|████▍ | 4/9 [00:18<00:24, 4.81s/it]
Loading checkpoint shards: 56%|█████▌ | 5/9 [00:23<00:19, 4.88s/it]
Loading checkpoint shards: 67%|██████▋ | 6/9 [00:28<00:14, 4.91s/it]
Loading checkpoint shards: 78%|███████▊ | 7/9 [00:33<00:09, 4.97s/it]
Loading checkpoint shards: 89%|████████▉ | 8/9 [00:37<00:04, 4.44s/it]
Loading checkpoint shards: 100%|██████████| 9/9 [00:39<00:00, 3.63s/it]
Loading checkpoint shards: 100%|██████████| 9/9 [00:39<00:00, 4.34s/it]
2025-12-27 02:19:52,764 - WARNING - User agent failed to respond at turn 8
2025-12-27 02:34:35,957 - WARNING - User agent failed to respond at turn 9
2025-12-27 02:42:19,862 - WARNING - User agent failed to respond at turn 6
2025-12-27 02:50:50,883 - WARNING - User agent failed to respond at turn 4
2025-12-27 03:04:43,668 - WARNING - User agent failed to respond at turn 8
2025-12-27 03:18:17,893 - WARNING - User agent failed to respond at turn 8
2025-12-27 03:31:55,124 - WARNING - User agent failed to respond at turn 9
2025-12-27 03:47:00,955 - WARNING - User agent failed to respond at turn 9
2025-12-27 04:14:22,548 - WARNING - User agent failed to respond at turn 9
2025-12-27 04:37:23,679 - WARNING - User agent failed to respond at turn 8
2025-12-27 04:40:56,780 - WARNING - User agent failed to respond at turn 2
2025-12-27 05:05:21,340 - WARNING - User agent failed to respond at turn 8
2025-12-27 05:19:26,915 - WARNING - User agent failed to respond at turn 8
2025-12-27 05:34:10,737 - WARNING - User agent failed to respond at turn 10
2025-12-27 05:46:47,804 - WARNING - User agent failed to respond at turn 5
'(ReadTimeoutError("HTTPSConnectionPool(host='huggingface.co', port=443): Read timed out. (read timeout=10)"), '(Request ID: 65a54fd4-dc47-449f-99f1-8b72cd64f258)')' thrown while requesting HEAD https://huggingface.co/datasets/lighteval/MATH-Hard/resolve/cf0716b8bafa192bcb6b455b0679538787dc43f0/.huggingface.yaml
2025-12-27 05:47:08,075 - WARNING - '(ReadTimeoutError("HTTPSConnectionPool(host='huggingface.co', port=443): Read timed out. (read timeout=10)"), '(Request ID: 65a54fd4-dc47-449f-99f1-8b72cd64f258)')' thrown while requesting HEAD https://huggingface.co/datasets/lighteval/MATH-Hard/resolve/cf0716b8bafa192bcb6b455b0679538787dc43f0/.huggingface.yaml
Retrying in 1s [Retry 1/5].
2025-12-27 05:47:08,075 - WARNING - Retrying in 1s [Retry 1/5].
2025-12-27 05:47:31,159 - INFO - Profile 2/30
2025-12-27 06:06:02,584 - WARNING - User agent failed to respond at turn 8
2025-12-27 06:28:56,672 - WARNING - User agent failed to respond at turn 9
2025-12-27 06:33:43,493 - WARNING - User agent failed to respond at turn 3
2025-12-27 06:55:51,361 - WARNING - User agent failed to respond at turn 8
2025-12-27 07:04:21,768 - WARNING - User agent failed to respond at turn 9
2025-12-27 07:36:32,573 - WARNING - User agent failed to respond at turn 6
2025-12-27 08:03:58,605 - WARNING - User agent failed to respond at turn 10
2025-12-27 08:31:36,913 - WARNING - User agent failed to respond at turn 8
2025-12-27 08:45:06,689 - WARNING - User agent failed to respond at turn 10
2025-12-27 08:57:40,880 - WARNING - User agent failed to respond at turn 8
2025-12-27 09:09:30,300 - WARNING - User agent failed to respond at turn 10
2025-12-27 09:09:30,302 - INFO - Profile 3/30
2025-12-27 09:23:09,110 - WARNING - User agent failed to respond at turn 9
2025-12-27 09:38:33,087 - WARNING - User agent failed to respond at turn 10
2025-12-27 09:55:31,104 - WARNING - User agent failed to respond at turn 10
2025-12-27 10:08:01,546 - WARNING - User agent failed to respond at turn 10
2025-12-27 10:11:50,915 - WARNING - User agent failed to respond at turn 3
2025-12-27 10:20:33,908 - WARNING - User agent failed to respond at turn 8
2025-12-27 10:36:06,815 - WARNING - User agent failed to respond at turn 13
2025-12-27 11:04:44,557 - WARNING - User agent failed to respond at turn 9
2025-12-27 11:19:08,586 - WARNING - User agent failed to respond at turn 10
2025-12-27 11:32:30,613 - WARNING - User agent failed to respond at turn 8
2025-12-27 11:45:28,346 - WARNING - User agent failed to respond at turn 8
2025-12-27 11:59:10,443 - WARNING - User agent failed to respond at turn 8
2025-12-27 12:02:44,417 - WARNING - User agent failed to respond at turn 2
2025-12-27 12:24:05,983 - WARNING - User agent failed to respond at turn 9
2025-12-27 12:36:47,157 - WARNING - User agent failed to respond at turn 8
2025-12-27 12:56:50,763 - WARNING - User agent failed to respond at turn 8
2025-12-27 13:14:23,987 - INFO - Profile 4/30
2025-12-27 13:41:52,436 - WARNING - User agent failed to respond at turn 8
2025-12-27 14:02:43,583 - WARNING - User agent failed to respond at turn 8
2025-12-27 14:06:32,824 - WARNING - User agent failed to respond at turn 2
2025-12-27 14:23:22,832 - WARNING - User agent failed to respond at turn 14
2025-12-27 14:37:30,153 - WARNING - User agent failed to respond at turn 8
2025-12-27 14:51:10,701 - WARNING - User agent failed to respond at turn 8
2025-12-27 15:04:43,741 - WARNING - User agent failed to respond at turn 8
2025-12-27 15:19:48,665 - WARNING - User agent failed to respond at turn 8
2025-12-27 15:33:01,051 - WARNING - User agent failed to respond at turn 8
2025-12-27 15:49:22,254 - WARNING - User agent failed to respond at turn 8
2025-12-27 16:03:56,686 - WARNING - User agent failed to respond at turn 8
2025-12-27 16:20:53,945 - WARNING - User agent failed to respond at turn 7
2025-12-27 16:49:33,698 - WARNING - User agent failed to respond at turn 8
2025-12-27 17:04:50,896 - WARNING - User agent failed to respond at turn 9
2025-12-27 17:31:00,796 - WARNING - User agent failed to respond at turn 8
2025-12-27 17:31:00,799 - INFO - Profile 5/30
2025-12-27 18:00:49,422 - WARNING - User agent failed to respond at turn 8
2025-12-27 18:15:01,789 - WARNING - User agent failed to respond at turn 10
2025-12-27 18:27:47,576 - WARNING - User agent failed to respond at turn 8
2025-12-27 18:40:13,645 - WARNING - User agent failed to respond at turn 8
2025-12-27 18:53:15,915 - WARNING - User agent failed to respond at turn 8
2025-12-27 19:19:08,201 - WARNING - User agent failed to respond at turn 8
2025-12-27 19:37:35,846 - WARNING - User agent failed to respond at turn 8
2025-12-27 19:51:06,794 - WARNING - User agent failed to respond at turn 10
2025-12-27 20:05:06,821 - WARNING - User agent failed to respond at turn 10
2025-12-27 20:17:39,488 - WARNING - User agent failed to respond at turn 8
2025-12-27 20:31:13,429 - WARNING - User agent failed to respond at turn 7
2025-12-27 20:53:38,048 - WARNING - User agent failed to respond at turn 9
2025-12-27 21:06:30,110 - WARNING - User agent failed to respond at turn 9
2025-12-27 21:18:58,273 - WARNING - User agent failed to respond at turn 8
2025-12-27 21:31:52,874 - WARNING - User agent failed to respond at turn 8
2025-12-27 21:45:02,984 - WARNING - User agent failed to respond at turn 8
2025-12-27 21:45:02,987 - INFO - Profile 6/30
2025-12-27 21:59:20,567 - WARNING - User agent failed to respond at turn 12
2025-12-27 22:12:05,436 - WARNING - User agent failed to respond at turn 9
2025-12-27 22:24:26,612 - WARNING - User agent failed to respond at turn 8
2025-12-27 22:37:34,213 - WARNING - User agent failed to respond at turn 8
2025-12-27 22:41:11,469 - WARNING - User agent failed to respond at turn 2
2025-12-27 22:54:18,480 - WARNING - User agent failed to respond at turn 8
2025-12-27 22:57:33,805 - WARNING - User agent failed to respond at turn 4
2025-12-27 23:09:44,313 - WARNING - User agent failed to respond at turn 9
2025-12-27 23:22:24,512 - WARNING - User agent failed to respond at turn 8
2025-12-27 23:35:10,690 - WARNING - User agent failed to respond at turn 8
2025-12-27 23:51:25,271 - WARNING - User agent failed to respond at turn 8
2025-12-28 00:04:32,550 - WARNING - User agent failed to respond at turn 8
2025-12-28 00:17:05,022 - WARNING - User agent failed to respond at turn 7
2025-12-28 00:29:45,695 - WARNING - User agent failed to respond at turn 8
2025-12-28 00:42:31,155 - WARNING - User agent failed to respond at turn 8
2025-12-28 00:55:35,641 - WARNING - User agent failed to respond at turn 8
2025-12-28 01:09:42,819 - WARNING - User agent failed to respond at turn 8
2025-12-28 01:22:54,235 - WARNING - User agent failed to respond at turn 8
2025-12-28 01:35:57,818 - WARNING - User agent failed to respond at turn 8
2025-12-28 01:35:57,820 - INFO - Profile 7/30
2025-12-28 02:29:13,033 - WARNING - User agent failed to respond at turn 8
2025-12-28 02:34:18,064 - WARNING - User agent failed to respond at turn 2
2025-12-28 02:47:50,033 - WARNING - User agent failed to respond at turn 8
2025-12-28 03:01:20,372 - WARNING - User agent failed to respond at turn 8
2025-12-28 03:17:29,349 - WARNING - User agent failed to respond at turn 8
2025-12-28 03:31:22,192 - WARNING - User agent failed to respond at turn 8
2025-12-28 03:44:57,553 - WARNING - User agent failed to respond at turn 8
2025-12-28 03:59:52,603 - WARNING - User agent failed to respond at turn 8
[2025-12-28T04:23:53.546] error: *** JOB 14360423 ON gpua091 CANCELLED AT 2025-12-28T04:23:53 DUE to SIGNAL Terminated ***
|
