Add collaborativeagents module and update gitignore

- Add collaborativeagents subproject with adapters, agents, and evaluation modules
- Update .gitignore to exclude large binary files (.whl, .tar), wandb logs, and results

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

1 files changed, 83 insertions, 0 deletions

diff --git a/logs/grpo_reflection_15498033.out b/logs/grpo_reflection_15498033.out
new file mode 100644
index 0000000..92e7b2b
--- /dev/null
+++ b/logs/grpo_reflection_15498033.out
@@ -0,0 +1,83 @@
+=== Starting vLLM judge server ===
+vLLM server PID: 1914169
+Waiting for vLLM server to start...
+(APIServer pid=1914169) INFO 01-11 15:24:10 [api_server.py:1351] vLLM API server version 0.13.0
+(APIServer pid=1914169) INFO 01-11 15:24:10 [utils.py:253] non-default args: {'model': 'hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4', 'trust_remote_code': True, 'max_model_len': 8192, 'tensor_parallel_size': 2}
+(APIServer pid=1914169) INFO 01-11 15:24:13 [model.py:514] Resolved architecture: LlamaForCausalLM
+(APIServer pid=1914169) INFO 01-11 15:24:13 [model.py:1661] Using max model len 8192
+(APIServer pid=1914169) INFO 01-11 15:24:15 [awq_marlin.py:162] The model is convertible to awq_marlin during runtime. Using awq_marlin kernel.
+(APIServer pid=1914169) INFO 01-11 15:24:15 [scheduler.py:230] Chunked prefill is enabled with max_num_batched_tokens=2048.
+(EngineCore_DP0 pid=1914581) INFO 01-11 15:24:30 [core.py:93] Initializing a V1 LLM engine (v0.13.0) with config: model='hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4', speculative_config=None, tokenizer='hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=8192, download_dir=None, load_format=auto, tensor_parallel_size=2, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=awq_marlin, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False), seed=0, served_model_name=hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer'], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [2048], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': True, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False}, 'local_cache_dir': None}
+(EngineCore_DP0 pid=1914581) WARNING 01-11 15:24:30 [multiproc_executor.py:882] Reducing Torch parallelism from 16 threads to 1 to avoid unnecessary CPU contention. Set OMP_NUM_THREADS in the external environment to tune this value as needed.
+INFO 01-11 15:24:42 [parallel_state.py:1203] world_size=2 rank=1 local_rank=1 distributed_init_method=tcp://127.0.0.1:49495 backend=nccl
+INFO 01-11 15:24:42 [parallel_state.py:1203] world_size=2 rank=0 local_rank=0 distributed_init_method=tcp://127.0.0.1:49495 backend=nccl
+INFO 01-11 15:24:42 [pynccl.py:111] vLLM is using nccl==2.27.5
+WARNING 01-11 15:24:42 [symm_mem.py:67] SymmMemCommunicator: Device capability 8.0 not supported, communicator is not available.
+WARNING 01-11 15:24:42 [symm_mem.py:67] SymmMemCommunicator: Device capability 8.0 not supported, communicator is not available.
+INFO 01-11 15:24:43 [parallel_state.py:1411] rank 1 in world size 2 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 1, EP rank 1
+INFO 01-11 15:24:43 [parallel_state.py:1411] rank 0 in world size 2 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0
+(Worker_TP0 pid=1914626) INFO 01-11 15:24:44 [gpu_model_runner.py:3562] Starting to load model hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4...
+(Worker_TP0 pid=1914626) INFO 01-11 15:24:45 [cuda.py:351] Using FLASH_ATTN attention backend out of potential backends: ('FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION')
+(Worker_TP0 pid=1914626) INFO 01-11 15:25:34 [weight_utils.py:487] Time spent downloading weights for hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4: 48.183306 seconds
+(Worker_TP0 pid=1914626) INFO 01-11 15:25:47 [default_loader.py:308] Loading weights took 13.80 seconds
+(Worker_TP0 pid=1914626) INFO 01-11 15:25:49 [gpu_model_runner.py:3659] Model loading took 18.5766 GiB memory and 64.704730 seconds
+(Worker_TP0 pid=1914626) INFO 01-11 15:26:11 [backends.py:643] Using cache directory: /u/yurenh2/.cache/vllm/torch_compile_cache/6437ac94ed/rank_0_0/backbone for vLLM's torch.compile
+(Worker_TP0 pid=1914626) INFO 01-11 15:26:11 [backends.py:703] Dynamo bytecode transform time: 20.38 s
+(Worker_TP1 pid=1914627) INFO 01-11 15:26:25 [backends.py:261] Cache the graph of compile range (1, 2048) for later use
+(Worker_TP0 pid=1914626) INFO 01-11 15:26:25 [backends.py:261] Cache the graph of compile range (1, 2048) for later use
+(Worker_TP0 pid=1914626) INFO 01-11 15:26:34 [backends.py:278] Compiling a graph for compile range (1, 2048) takes 12.21 s
+(Worker_TP0 pid=1914626) INFO 01-11 15:26:34 [monitor.py:34] torch.compile takes 32.59 s in total
+(Worker_TP0 pid=1914626) INFO 01-11 15:26:35 [gpu_worker.py:375] Available KV cache memory: 15.70 GiB
+(EngineCore_DP0 pid=1914581) INFO 01-11 15:26:35 [kv_cache_utils.py:1291] GPU KV cache size: 102,896 tokens
+(EngineCore_DP0 pid=1914581) INFO 01-11 15:26:35 [kv_cache_utils.py:1296] Maximum concurrency for 8,192 tokens per request: 12.56x
+(Worker_TP1 pid=1914627) INFO 01-11 15:26:47 [custom_all_reduce.py:216] Registering 13685 cuda graph addresses
+(Worker_TP0 pid=1914626) INFO 01-11 15:26:47 [custom_all_reduce.py:216] Registering 13685 cuda graph addresses
+(Worker_TP0 pid=1914626) INFO 01-11 15:26:48 [gpu_model_runner.py:4587] Graph capturing finished in 12 secs, took 1.40 GiB
+(EngineCore_DP0 pid=1914581) INFO 01-11 15:26:48 [core.py:259] init engine (profile, create kv cache, warmup model) took 57.57 seconds
+(APIServer pid=1914169) INFO 01-11 15:26:49 [api_server.py:1099] Supported tasks: ['generate']
+(APIServer pid=1914169) WARNING 01-11 15:26:49 [model.py:1487] Default sampling parameters have been overridden by the model's Hugging Face generation config recommended from the model creator. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`.
+(APIServer pid=1914169) INFO 01-11 15:26:49 [serving_responses.py:201] Using default chat sampling params from model: {'temperature': 0.6, 'top_p': 0.9}
+(APIServer pid=1914169) INFO 01-11 15:26:49 [serving_chat.py:137] Using default chat sampling params from model: {'temperature': 0.6, 'top_p': 0.9}
+(APIServer pid=1914169) INFO 01-11 15:26:49 [serving_completion.py:77] Using default completion sampling params from model: {'temperature': 0.6, 'top_p': 0.9}
+(APIServer pid=1914169) INFO 01-11 15:26:50 [serving_chat.py:137] Using default chat sampling params from model: {'temperature': 0.6, 'top_p': 0.9}
+(APIServer pid=1914169) INFO 01-11 15:26:50 [api_server.py:1425] Starting vLLM API server 0 on http://0.0.0.0:8000
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:38] Available routes are:
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /openapi.json, Methods: HEAD, GET
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /docs, Methods: HEAD, GET
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /docs/oauth2-redirect, Methods: HEAD, GET
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /redoc, Methods: HEAD, GET
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /scale_elastic_ep, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /is_scaling_elastic_ep, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /tokenize, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /detokenize, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /inference/v1/generate, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /pause, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /resume, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /is_paused, Methods: GET
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /metrics, Methods: GET
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /health, Methods: GET
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /load, Methods: GET
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /v1/models, Methods: GET
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /version, Methods: GET
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /v1/responses, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /v1/responses/{response_id}, Methods: GET
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /v1/responses/{response_id}/cancel, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /v1/messages, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /v1/chat/completions, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /v1/completions, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /v1/audio/transcriptions, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /v1/audio/translations, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /ping, Methods: GET
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /ping, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /invocations, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /classify, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /v1/embeddings, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /score, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /v1/score, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /rerank, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /v1/rerank, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /v2/rerank, Methods: POST
+(APIServer pid=1914169) INFO 01-11 15:26:50 [launcher.py:46] Route: /pooling, Methods: POST
+(APIServer pid=1914169) INFO:     127.0.0.1:57116 - "GET /health HTTP/1.1" 200 OK
+vLLM server is ready!
+=== Starting GRPO training ===