`torch_dtype` is deprecated! Use `dtype` instead!

Fetching 9 files:   0%|          | 0/9 [00:00<?, ?it/s]
Fetching 9 files:  11%|█         | 1/9 [01:04<08:38, 64.82s/it]
Fetching 9 files:  22%|██▏       | 2/9 [01:05<03:09, 27.05s/it]
Fetching 9 files:  44%|████▍     | 4/9 [01:06<00:52, 10.45s/it]
Fetching 9 files:  78%|███████▊  | 7/9 [01:33<00:19,  9.67s/it]
Fetching 9 files: 100%|██████████| 9/9 [01:33<00:00, 10.43s/it]
/u/yurenh2/miniforge3/envs/eval/lib/python3.11/site-packages/awq/__init__.py:21: DeprecationWarning: 
I have left this message as the final dev message to help you transition.

Important Notice:
- AutoAWQ is officially deprecated and will no longer be maintained.
- The last tested configuration used Torch 2.6.0 and Transformers 4.51.3.
- If future versions of Transformers break AutoAWQ compatibility, please report the issue to the Transformers project.

Alternative:
- AutoAWQ has been adopted by the vLLM Project: https://github.com/vllm-project/llm-compressor

For further inquiries, feel free to reach out:
- X: https://x.com/casper_hansen_
- LinkedIn: https://www.linkedin.com/in/casper-hansen-804005170/

  warnings.warn(_FINAL_DEV_MESSAGE, category=DeprecationWarning, stacklevel=1)

Loading checkpoint shards:   0%|          | 0/9 [00:00<?, ?it/s]
Loading checkpoint shards:  11%|█         | 1/9 [00:03<00:26,  3.34s/it]
Loading checkpoint shards:  22%|██▏       | 2/9 [00:09<00:33,  4.81s/it]
Loading checkpoint shards:  33%|███▎      | 3/9 [00:11<00:22,  3.73s/it]
Loading checkpoint shards:  44%|████▍     | 4/9 [00:17<00:22,  4.51s/it]
Loading checkpoint shards:  56%|█████▌    | 5/9 [00:23<00:20,  5.10s/it]
Loading checkpoint shards:  67%|██████▋   | 6/9 [00:28<00:15,  5.08s/it]
Loading checkpoint shards:  78%|███████▊  | 7/9 [01:23<00:43, 21.52s/it]
Loading checkpoint shards:  89%|████████▉ | 8/9 [01:26<00:15, 15.40s/it]
Loading checkpoint shards: 100%|██████████| 9/9 [01:27<00:00, 10.90s/it]
Loading checkpoint shards: 100%|██████████| 9/9 [01:27<00:00,  9.69s/it]

Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:05<00:17,  5.90s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:11<00:11,  5.91s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:19<00:06,  6.56s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:20<00:00,  4.39s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:20<00:00,  5.05s/it]

Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s]
Loading checkpoint shards:  20%|██        | 1/5 [00:15<01:02, 15.59s/it]
Loading checkpoint shards:  40%|████      | 2/5 [00:21<00:30, 10.08s/it]
Loading checkpoint shards:  60%|██████    | 3/5 [00:27<00:16,  8.23s/it]
Loading checkpoint shards:  80%|████████  | 4/5 [00:32<00:06,  6.79s/it]
Loading checkpoint shards: 100%|██████████| 5/5 [00:34<00:00,  5.01s/it]
Loading checkpoint shards: 100%|██████████| 5/5 [00:34<00:00,  6.85s/it]

Loading checkpoint shards:   0%|          | 0/9 [00:00<?, ?it/s]
Loading checkpoint shards:  11%|█         | 1/9 [00:02<00:19,  2.47s/it]
Loading checkpoint shards:  22%|██▏       | 2/9 [01:00<04:05, 35.01s/it]
Loading checkpoint shards:  33%|███▎      | 3/9 [01:05<02:07, 21.33s/it]
Loading checkpoint shards:  44%|████▍     | 4/9 [01:10<01:14, 14.81s/it]
Loading checkpoint shards:  56%|█████▌    | 5/9 [01:15<00:45, 11.36s/it]
Loading checkpoint shards:  67%|██████▋   | 6/9 [01:20<00:27,  9.25s/it]
Loading checkpoint shards:  78%|███████▊  | 7/9 [01:30<00:19,  9.50s/it]
Loading checkpoint shards:  89%|████████▉ | 8/9 [01:32<00:07,  7.02s/it]
Loading checkpoint shards: 100%|██████████| 9/9 [01:33<00:00,  5.34s/it]
Loading checkpoint shards: 100%|██████████| 9/9 [01:33<00:00, 10.43s/it]

Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:04<00:13,  4.44s/it]
Loading checkpoint shards:  50%|█████     | 2/4 [00:09<00:09,  4.72s/it]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:15<00:05,  5.15s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:15<00:00,  3.30s/it]
Loading checkpoint shards: 100%|██████████| 4/4 [00:15<00:00,  3.87s/it]

Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s]
Loading checkpoint shards:  20%|██        | 1/5 [00:05<00:21,  5.25s/it]
Loading checkpoint shards:  40%|████      | 2/5 [00:09<00:13,  4.63s/it]
Loading checkpoint shards:  60%|██████    | 3/5 [00:13<00:09,  4.53s/it]
Loading checkpoint shards:  80%|████████  | 4/5 [00:16<00:03,  3.87s/it]
Loading checkpoint shards: 100%|██████████| 5/5 [00:18<00:00,  3.05s/it]
Loading checkpoint shards: 100%|██████████| 5/5 [00:18<00:00,  3.66s/it]