Merge master into main

1 files changed, 207 insertions, 0 deletions

diff --git a/runs/slurm_logs/15112872_target1.out b/runs/slurm_logs/15112872_target1.out
new file mode 100644
index 0000000..b4babc0
--- /dev/null
+++ b/runs/slurm_logs/15112872_target1.out
@@ -0,0 +1,207 @@
+============================================================
+SMARTER TARGET Experiment (lambda_target=1.0)
+Job ID: 15112872 | Node: gpub074
+Start: Thu Jan  1 12:26:50 CST 2026
+============================================================
+NVIDIA A40, 46068 MiB
+============================================================
+================================================================================
+DEPTH SCALING BENCHMARK
+================================================================================
+Dataset: cifar100
+Depths: [4, 8, 12, 16]
+Timesteps: 4
+Epochs: 150
+λ_reg: 0.1, λ_target: 1.0
+Reg type: squared, Warmup epochs: 20
+Device: cuda
+================================================================================
+
+Loading cifar100...
+Classes: 100, Input: (3, 32, 32)
+Train: 50000, Test: 10000
+
+Depth configurations: [(4, '4×1'), (8, '4×2'), (12, '4×3'), (16, '4×4')]
+Regularization type: squared
+Warmup epochs: 20
+Stable init: False
+
+============================================================
+Depth = 4 conv layers (4 stages × 1 blocks)
+============================================================
+    Vanilla: depth=4, params=1,756,836
+      Epoch  10: train=0.498 test=0.436  σ=9.42e-01/3.51e-08
+      Epoch  20: train=0.629 test=0.499  σ=5.87e-01/2.46e-08
+      Epoch  30: train=0.701 test=0.550  σ=4.77e-01/2.01e-08
+      Epoch  40: train=0.753 test=0.545  σ=4.17e-01/1.75e-08
+      Epoch  50: train=0.798 test=0.577  σ=3.69e-01/1.53e-08
+      Epoch  60: train=0.830 test=0.582  σ=3.33e-01/1.39e-08
+      Epoch  70: train=0.862 test=0.580  σ=3.29e-01/1.26e-08
+      Epoch  80: train=0.884 test=0.578  σ=3.07e-01/1.21e-08
+      Epoch  90: train=0.907 test=0.598  σ=2.79e-01/1.08e-08
+      Epoch 100: train=0.921 test=0.608  σ=2.72e-01/1.06e-08
+      Epoch 110: train=0.935 test=0.608  σ=2.54e-01/9.36e-09
+      Epoch 120: train=0.943 test=0.608  σ=2.46e-01/9.12e-09
+      Epoch 130: train=0.949 test=0.615  σ=2.34e-01/8.83e-09
+      Epoch 140: train=0.951 test=0.610  σ=2.32e-01/8.64e-09
+      Epoch 150: train=0.954 test=0.614  σ=2.32e-01/8.63e-09
+      Best test acc: 0.615
+    Lyapunov: depth=4, params=1,756,836
+      Epoch  10: train=0.285 test=0.024 λ=1.483 σ=7.62e-01/2.92e-08
+      Epoch  20: train=0.344 test=0.014 λ=1.560 σ=4.80e-01/2.01e-08
+      Epoch  30: train=0.348 test=0.012 λ=1.679 σ=3.85e-01/1.69e-08
+      Epoch  40: train=0.389 test=0.013 λ=1.635 σ=3.33e-01/1.54e-08
+      Epoch  50: train=0.431 test=0.011 λ=1.635 σ=3.11e-01/1.45e-08
+      Epoch  60: train=0.461 test=0.016 λ=1.622 σ=2.95e-01/1.44e-08
+      Epoch  70: train=0.478 test=0.014 λ=1.660 σ=2.90e-01/1.40e-08
+      Epoch  80: train=0.499 test=0.013 λ=1.657 σ=2.82e-01/1.40e-08
+      Epoch  90: train=0.522 test=0.013 λ=1.663 σ=2.78e-01/1.36e-08
+      Epoch 100: train=0.537 test=0.015 λ=1.678 σ=2.78e-01/1.36e-08
+      Epoch 110: train=0.550 test=0.014 λ=1.684 σ=2.92e-01/1.38e-08
+      Epoch 120: train=0.559 test=0.016 λ=1.704 σ=2.90e-01/1.42e-08
+      Epoch 130: train=0.570 test=0.018 λ=1.709 σ=2.79e-01/1.36e-08
+      Epoch 140: train=0.571 test=0.017 λ=1.865 σ=2.83e-01/1.37e-08
+      Epoch 150: train=0.576 test=0.017 λ=1.816 σ=2.82e-01/1.37e-08
+      Best test acc: 0.212
+
+============================================================
+Depth = 8 conv layers (4 stages × 2 blocks)
+============================================================
+    Vanilla: depth=8, params=4,892,196
+      Epoch  10: train=0.388 test=0.351  σ=8.72e-01/3.16e-08
+      Epoch  20: train=0.544 test=0.427  σ=4.73e-01/2.16e-08
+      Epoch  30: train=0.630 test=0.466  σ=3.81e-01/1.79e-08
+      Epoch  40: train=0.698 test=0.502  σ=3.22e-01/1.53e-08
+      Epoch  50: train=0.747 test=0.519  σ=3.08e-01/1.42e-08
+      Epoch  60: train=0.799 test=0.515  σ=2.87e-01/1.29e-08
+      Epoch  70: train=0.836 test=0.524  σ=2.76e-01/1.18e-08
+      Epoch  80: train=0.869 test=0.534  σ=2.44e-01/1.05e-08
+      Epoch  90: train=0.898 test=0.528  σ=2.39e-01/9.52e-09
+      Epoch 100: train=0.918 test=0.527  σ=2.29e-01/8.96e-09
+      Epoch 110: train=0.933 test=0.542  σ=2.26e-01/8.58e-09
+      Epoch 120: train=0.943 test=0.542  σ=2.09e-01/7.88e-09
+      Epoch 130: train=0.951 test=0.545  σ=1.97e-01/7.80e-09
+      Epoch 140: train=0.955 test=0.542  σ=2.06e-01/7.61e-09
+      Epoch 150: train=0.954 test=0.535  σ=1.94e-01/7.46e-09
+      Best test acc: 0.550
+    Lyapunov: depth=8, params=4,892,196
+      Epoch  10: train=0.035 test=0.010 λ=1.583 σ=3.25e-01/9.17e-09
+      Epoch  20: train=0.049 test=0.010 λ=1.574 σ=2.46e-01/6.77e-09
+      Epoch  30: train=0.061 test=0.010 λ=1.571 σ=2.03e-01/5.87e-09
+      Epoch  40: train=0.033 test=0.010 λ=1.544 σ=1.80e-01/2.89e-09
+      Epoch  50: train=0.030 test=0.010 λ=1.550 σ=1.59e-01/9.89e-10
+      Epoch  60: train=0.030 test=0.010 λ=1.567 σ=1.39e-01/5.26e-10
+      Epoch  70: train=0.029 test=0.010 λ=1.571 σ=1.16e-01/1.53e-10
+      Epoch  80: train=0.041 test=0.010 λ=1.646 σ=1.41e-01/3.16e-09
+      Epoch  90: train=0.036 test=0.010 λ=1.808 σ=1.37e-01/2.76e-09
+      Epoch 100: train=0.031 test=0.010 λ=1.940 σ=1.71e-01/2.97e-09
+      Epoch 110: train=0.047 test=0.010 λ=1.976 σ=1.42e-01/3.22e-09
+      Epoch 120: train=0.047 test=0.008 λ=1.993 σ=1.26e-01/3.43e-09
+      Epoch 130: train=0.046 test=0.010 λ=2.057 σ=1.50e-01/3.50e-09
+      Epoch 140: train=0.026 test=0.010 λ=2.014 σ=2.43e-01/3.40e-09
+      Epoch 150: train=0.031 test=0.010 λ=2.334 σ=1.30e-01/4.11e-09
+      Best test acc: 0.024
+
+============================================================
+Depth = 12 conv layers (4 stages × 3 blocks)
+============================================================
+    Vanilla: depth=12, params=8,027,556
+      Epoch  10: train=0.214 test=0.070  σ=5.69e-01/2.19e-08
+      Epoch  20: train=0.289 test=0.057  σ=3.32e-01/1.63e-08
+      Epoch  30: train=0.340 test=0.107  σ=2.62e-01/1.36e-08
+      Epoch  40: train=0.374 test=0.083  σ=2.35e-01/1.28e-08
+      Epoch  50: train=0.410 test=0.073  σ=2.28e-01/1.25e-08
+      Epoch  60: train=0.436 test=0.101  σ=2.23e-01/1.23e-08
+      Epoch  70: train=0.473 test=0.087  σ=2.33e-01/1.22e-08
+      Epoch  80: train=0.505 test=0.083  σ=2.21e-01/1.22e-08
+      Epoch  90: train=0.534 test=0.090  σ=2.24e-01/1.21e-08
+      Epoch 100: train=0.561 test=0.096  σ=2.29e-01/1.23e-08
+      Epoch 110: train=0.584 test=0.074  σ=2.30e-01/1.21e-08
+      Epoch 120: train=0.602 test=0.088  σ=2.35e-01/1.22e-08
+      Epoch 130: train=0.609 test=0.093  σ=2.30e-01/1.20e-08
+      Epoch 140: train=0.620 test=0.094  σ=2.27e-01/1.19e-08
+      Epoch 150: train=0.624 test=0.086  σ=2.31e-01/1.22e-08
+      Best test acc: 0.109
+    Lyapunov: depth=12, params=8,027,556
+      Epoch  10: train=0.013 test=0.010 λ=1.639 σ=3.64e-01/1.06e-12
+      Epoch  20: train=0.015 test=0.010 λ=1.598 σ=2.98e-01/1.14e-12
+      Epoch  30: train=0.019 test=0.010 λ=1.630 σ=3.30e-01/3.22e-12
+      Epoch  40: train=0.021 test=0.010 λ=1.592 σ=1.82e-01/2.19e-12
+      Epoch  50: train=0.020 test=0.010 λ=1.658 σ=1.51e-01/2.96e-12
+      Epoch  60: train=0.015 test=0.010 λ=1.616 σ=1.03e-01/2.55e-13
+      Epoch  70: train=0.018 test=0.010 λ=1.617 σ=1.18e-01/4.53e-13
+      Epoch  80: train=0.020 test=0.010 λ=1.636 σ=1.22e-01/4.69e-12
+      Epoch  90: train=0.021 test=0.010 λ=1.593 σ=1.05e-01/7.58e-12
+      Epoch 100: train=0.026 test=0.010 λ=1.593 σ=1.16e-01/7.00e-10
+      Epoch 110: train=0.021 test=0.010 λ=1.590 σ=9.46e-02/4.97e-12
+      Epoch 120: train=0.024 test=0.010 λ=1.740 σ=9.83e-02/3.89e-11
+      Epoch 130: train=0.020 test=0.010 λ=1.901 σ=1.09e-01/9.81e-11
+      Epoch 140: train=0.027 test=0.010 λ=1.972 σ=1.21e-01/1.96e-09
+      Epoch 150: train=0.019 test=0.010 λ=2.112 σ=6.82e-02/1.40e-11
+      Best test acc: 0.019
+
+============================================================
+Depth = 16 conv layers (4 stages × 4 blocks)
+============================================================
+    Vanilla: depth=16, params=11,162,916
+      Epoch  10: train=0.091 test=0.011  σ=4.40e-01/1.32e-08
+      Epoch  20: train=0.135 test=0.014  σ=2.84e-01/1.06e-08
+      Epoch  30: train=0.157 test=0.017  σ=2.21e-01/9.39e-09
+      Epoch  40: train=0.177 test=0.019  σ=1.93e-01/8.97e-09
+      Epoch  50: train=0.191 test=0.024  σ=1.81e-01/9.00e-09
+      Epoch  60: train=0.202 test=0.024  σ=1.74e-01/8.89e-09
+      Epoch  70: train=0.215 test=0.026  σ=1.66e-01/8.97e-09
+      Epoch  80: train=0.227 test=0.030  σ=1.65e-01/8.89e-09
+      Epoch  90: train=0.240 test=0.026  σ=1.55e-01/8.93e-09
+      Epoch 100: train=0.248 test=0.028  σ=1.61e-01/9.09e-09
+      Epoch 110: train=0.254 test=0.031  σ=1.59e-01/9.18e-09
+      Epoch 120: train=0.262 test=0.033  σ=1.64e-01/9.30e-09
+      Epoch 130: train=0.267 test=0.033  σ=1.58e-01/9.22e-09
+      Epoch 140: train=0.269 test=0.030  σ=1.61e-01/9.32e-09
+      Epoch 150: train=0.269 test=0.030  σ=1.64e-01/9.32e-09
+      Best test acc: 0.036
+    Lyapunov: depth=16, params=11,162,916
+      Epoch  10: train=0.014 test=0.010 λ=1.688 σ=3.64e-01/6.52e-13
+      Epoch  20: train=0.010 test=0.010 λ=1.803 σ=3.56e-01/1.53e-13
+      Epoch  30: train=0.012 test=0.002 λ=1.664 σ=4.44e-01/4.25e-14
+      Epoch  40: train=0.018 test=0.010 λ=1.733 σ=2.04e-01/9.43e-13
+      Epoch  50: train=0.010 test=0.010 λ=1.598 σ=1.55e-01/0.00e+00
+      Epoch  60: train=0.010 test=0.010 λ=1.594 σ=4.41e-02/0.00e+00
+      Epoch  70: train=0.009 test=0.010 λ=1.601 σ=7.50e-02/1.52e-14
+      Epoch  80: train=0.012 test=0.010 λ=2.219 σ=5.78e-02/1.54e-42
+      Epoch  90: train=0.016 test=0.010 λ=2.122 σ=9.91e-02/9.49e-15
+      Epoch 100: train=0.017 test=0.010 λ=2.163 σ=1.05e-01/7.08e-13
+      Epoch 110: train=0.020 test=0.010 λ=2.124 σ=1.10e-01/1.17e-12
+      Epoch 120: train=0.010 test=0.010 λ=2.181 σ=6.50e-02/6.09e-15
+      Epoch 130: train=0.010 test=0.010 λ=2.755 σ=6.77e-09/1.45e-20
+      Epoch 140: train=0.016 test=0.010 λ=2.217 σ=1.13e-01/5.35e-14
+      Epoch 150: train=0.018 test=0.010 λ=2.219 σ=1.21e-01/7.28e-14
+      Best test acc: 0.012
+
+====================================================================================================
+DEPTH SCALING RESULTS: CIFAR100
+====================================================================================================
+Depth    Vanilla Acc  Lyapunov Acc Δ Acc    Lyap λ     Van ∇norm    Lyap ∇norm   Van κ     
+----------------------------------------------------------------------------------------------------
+4        0.614        0.017        -0.597   1.816      4.52e-01     7.31e-01     1.1e+09   
+8        0.535        0.010        -0.525   2.334      3.87e-01     3.44e-01     8.0e+08   
+12       0.086        0.010        -0.076   2.112      6.47e-01     2.28e+00     4.9e+07   
+16       0.030        0.010        -0.020   2.219      5.07e-01     9.41e-01     2.1e+07   
+====================================================================================================
+
+GRADIENT HEALTH ANALYSIS:
+  Depth 4: ⚠️ Vanilla has ill-conditioned gradients (κ > 1e6)
+  Depth 8: ⚠️ Vanilla has ill-conditioned gradients (κ > 1e6)
+  Depth 12: ⚠️ Vanilla has ill-conditioned gradients (κ > 1e6)
+  Depth 16: ⚠️ Vanilla has ill-conditioned gradients (κ > 1e6)
+
+
+KEY OBSERVATIONS:
+  Vanilla  4→16 layers: -0.584 accuracy change
+  Lyapunov 4→16 layers: -0.007 accuracy change
+  ✓ Lyapunov regularization enables better depth scaling!
+
+Results saved to runs/depth_scaling_target1/cifar100_20260102-133339
+============================================================
+Finished: Fri Jan  2 13:33:43 CST 2026
+============================================================