From 08a08befe8ac0202ce952b88aa144cbffe2a73f9 Mon Sep 17 00:00:00 2001 From: YurenHao0426 Date: Wed, 11 Feb 2026 00:21:34 +0000 Subject: Add dual-vector ablation experiment (z_short/z_long necessity) - Add rag_vector_no_short (eta_short=0) and rag_vector_no_long (eta_long=0) configs - Add ablation run script for 60p60s experiment - Document ablation design and expected results in notes Co-Authored-By: Claude Opus 4.6 --- notes.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) (limited to 'notes.md') diff --git a/notes.md b/notes.md index 5864092..919633c 100644 --- a/notes.md +++ b/notes.md @@ -97,6 +97,32 @@ z_short 不和长期 preference Jaccard 相关是**符合预期的**： **可视化**: `collaborativeagents/results/fig_main_results.png` panel (c) +### z_short Ablation Experiment (02/10 启动) + +**目的**: 通过消融实验证明dual-vector中z_short的必要性 + +**实验设计**: +| 条件 | eta_long | eta_short | beta_long | beta_short | 说明 | +|------|----------|-----------|-----------|------------|------| +| Full (baseline) | 0.01 | 0.05 | 2.0 | 5.0 | 完整dual-vector (已完成的60s实验) | +| No z_short | 0.01 | **0.0** | 2.0 | 5.0 | 禁用session内适应 | +| No z_long | **0.0** | 0.05 | 2.0 | 5.0 | 禁用跨session学习 | + +**配置**: 60 profiles × 60 sessions, max 10 turns, same as main experiment +**方法名**: `rag_vector_no_short`, `rag_vector_no_long` + +**预期结果**: +1. **No z_short** (z_long only): 跨session学习正常，但session内第1-2 turn后如果agent犯错无法快速修正偏好权重 → 预计E/T会更高（更多enforcement） +2. **No z_long** (z_short only): 每个session从零开始（z_short在session开始时reset），完全没有跨session记忆 → 预计后期session表现差，没有学习曲线 +3. **Full** (both): 最佳表现，z_long提供跨session基础，z_short提供session内微调 + +**关键对比指标**: +- **E/T per-session曲线**: No z_short应该在early turns有更高E/T +- **Success rate over sessions**: No z_long应该flat，Full应该upward +- **Late-session performance**: Full应该最好，No z_long应该和early一样 + +**状态**: 🔄 Running (02/10 21:48 启动) + --- ## RAG vs Reflection 分析 -- cgit v1.2.3