From 4cf7e11cbf4dd60a358f8dd452488b3158f32ad7 Mon Sep 17 00:00:00 2001
From: Yuren Hao <yurenh2@timan108.cs.illinois.edu>
Date: Thu, 4 Sep 2025 23:46:09 -0500
Subject: some check scripts

---
 README.md | 87 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 87 insertions(+)

(limited to 'README.md')

diff --git a/README.md b/README.md
index 1d58ed4..1878be5 100644
--- a/README.md
+++ b/README.md
@@ -58,6 +58,93 @@ bash sh/eval_all_math.sh
 
 ---
 
+### Caching (Hugging Face)
+
+To avoid repeated downloads across runs, we persist Hugging Face caches in the user cache directory. When activating the `one-shot-em` conda environment, the following environment variables are set:
+
+```bash
+HF_HOME="$HOME/.cache/huggingface"
+HF_DATASETS_CACHE="$HF_HOME/datasets"
+HF_HUB_CACHE="$HF_HOME/hub"
+TRANSFORMERS_CACHE="$HF_HUB_CACHE"
+```
+
+You can change these by editing the conda env activation hook under:
+
+```
+$CONDA_PREFIX/etc/conda/activate.d/98-hf-cache.sh
+```
+
+Models and tokenizers are cached under `~/.cache/huggingface/hub` and will be reused automatically.
+
+---
+
+### Weights & Tokenizer Prefetch (Qwen2.5-7B-Instruct)
+
+To pre-download the text-only Instruct variant (not long-context/multimodal) and its tokenizer into the cache:
+
+```bash
+conda activate one-shot-em
+python - <<'PY'
+from huggingface_hub import snapshot_download
+repo = "Qwen/Qwen2.5-7B-Instruct"
+# First grab tokenizer-related small files (fast verification)
+snapshot_download(repo_id=repo, allow_patterns=[
+    "tokenizer*","vocab*","merges*",
+    "special_tokens_map.json","tokenizer.json",
+    "tokenizer_config.json","tokenizer.model",
+], resume_download=True)
+# Then optionally grab the full snapshot (large download; resumes automatically)
+snapshot_download(repo_id=repo, resume_download=True)
+PY
+```
+
+---
+
+### Accelerate Configuration
+
+We keep a default Accelerate config at:
+
+```
+configs/accelerate/default_config.yaml
+```
+
+This is a placeholder you can modify with `accelerate config` for multi-GPU runs later.
+
+---
+
+### Weights & Biases (W&B)
+
+By default, W&B is disabled in the `one-shot-em` environment. To enable it, unset `WANDB_DISABLED` (or set it to `false`) and ensure your API key is set, for example:
+
+```bash
+export WANDB_DISABLED=false
+export WANDB_API_KEY=...  # your key
+```
+
+If you wish to keep it off (default), no action is required.
+
+---
+
+### One-click Self-check
+
+Run a comprehensive environment check (cache, model/tokenizer, W&B, Accelerate, GPU) and write a hardware snapshot to `docs/hardware.md`:
+
+```bash
+conda activate one-shot-em
+python scripts/self_check.py
+```
+
+To avoid writing the hardware snapshot (stdout only):
+
+```bash
+python scripts/self_check.py --no-write
+```
+
+The script will also verify that `Qwen/Qwen2.5-7B-Instruct` (text-only Instruct) is cached and loadable locally.
+
+---
+
 ### Acknowledgements
 
 Our dataset references and builds upon the following open-source contributions:
-- 
cgit v1.2.3