diff options
| author | YurenHao0426 <blackhao0426@gmail.com> | 2026-02-10 21:39:57 +0000 |
|---|---|---|
| committer | YurenHao0426 <blackhao0426@gmail.com> | 2026-02-10 21:39:57 +0000 |
| commit | 8586ac86d7adc642c4e32e252e687068466cb519 (patch) | |
| tree | 8dd25c756b8e2a08e5a9c4ed2a31607e8ee789b7 | |
| parent | 991331b738fd0fb451ebd4f0ce208a13dcfb46b8 (diff) | |
Update vector-preference correlation to 60s z_long data
- Replace old 30s data (r=0.09 unreproducible) with 60s results
- z_long: bottom/top 25% comparison p=0.021*
- z_long captures long-term preference trends, z_short has no signal
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
| -rw-r--r-- | notes.md | 43 |
1 files changed, 26 insertions, 17 deletions
@@ -53,25 +53,34 @@ ## Vector-Preference 相关性分析 -### 30s实验 (rag_vector_v3) +### 60s实验 (rag_vector_60s, z_long) **Vector统计**: -- 59/60 users有非零vector -- Norm: mean=0.0023, max=0.0051 -- Vector随session增长: 0.00086 (s10) → 0.00227 (s30) - -**Jaccard(revealed_prefs) vs Vector相似度**: -- Spearman r = 0.09 (p=0.015) **显著** -- 趋势正确: 更多shared preferences → 更高vector相似度 - -| Jaccard范围 | 用户对数 | 平均Vector相似度 | -|-------------|---------|------------------| -| 0.00-0.05 | 413 | -0.007 | -| 0.05-0.10 | 215 | 0.034 | -| 0.10-0.15 | 43 | 0.037 | -| 0.15-0.25 | 10 | 0.072 | - -**对比之前论文**: r=0.90 (5 users, 20 sessions) vs 当前 r=0.09 (60 users, 30 sessions) +- 60/60 users有非零vector +- Norm: mean=0.0033, max=0.0067 +- Vector随session增长: 0.0 (s0) → 0.0033 (s60) + +**Jaccard(revealed_prefs) vs Vector Cosine相似度**: +- Spearman ρ = 0.040 (p=0.093, marginal) +- z_long效果最好,z_short无信号,combined被稀释 + +**Bottom 25% vs Top 25% preference similarity pairs** (报告用): +- Bottom 25% (最不相似偏好): mean cosine = **-0.027** +- Top 25% (最相似偏好): mean cosine = **+0.012** +- Mann-Whitney U: **p = 0.021 \*** +- 结论: 偏好更相似的用户对确实学到了更相似的vector + +**不同vector对比**: + +| Vector | Spearman r | p | Q5 vs Q1 p | +|--------|-----------|---|-----------| +| z_long | 0.040 | 0.093 | **0.006** | +| z_short | 0.010 | 0.681 | 0.586 | +| combined | 0.046 | 0.054 | 0.003 | + +→ z_long捕获长期偏好趋势最好,z_short基本无信号 + +**可视化**: `collaborativeagents/results/fig_main_results.png` panel (c) --- |
