Merge master into main

1 files changed, 206 insertions, 0 deletions

diff --git a/runs/slurm_logs/14632852_asymm.out b/runs/slurm_logs/14632852_asymm.out
new file mode 100644
index 0000000..bc2bb1e
--- /dev/null
+++ b/runs/slurm_logs/14632852_asymm.out
@@ -0,0 +1,206 @@
+============================================================
+ASYMMETRIC Lyapunov Regularization
+Job ID: 14632852 | Node: gpub079
+Start: Wed Dec 31 10:16:37 CST 2025
+============================================================
+NVIDIA A40, 46068 MiB
+============================================================
+================================================================================
+DEPTH SCALING BENCHMARK
+================================================================================
+Dataset: cifar100
+Depths: [4, 8, 12, 16]
+Timesteps: 4
+Epochs: 150
+λ_reg: 0.3, λ_target: -0.1
+Reg type: asymmetric, Warmup epochs: 20
+Device: cuda
+================================================================================
+
+Loading cifar100...
+Classes: 100, Input: (3, 32, 32)
+Train: 50000, Test: 10000
+
+Depth configurations: [(4, '4×1'), (8, '4×2'), (12, '4×3'), (16, '4×4')]
+Regularization type: asymmetric
+Warmup epochs: 20
+
+============================================================
+Depth = 4 conv layers (4 stages × 1 blocks)
+============================================================
+    Vanilla: depth=4, params=1,756,836
+      Epoch  10: train=0.498 test=0.438  σ=9.52e-01/3.57e-08
+      Epoch  20: train=0.631 test=0.506  σ=5.87e-01/2.45e-08
+      Epoch  30: train=0.706 test=0.562  σ=4.78e-01/1.99e-08
+      Epoch  40: train=0.756 test=0.554  σ=4.24e-01/1.76e-08
+      Epoch  50: train=0.800 test=0.583  σ=3.77e-01/1.55e-08
+      Epoch  60: train=0.833 test=0.593  σ=3.51e-01/1.39e-08
+      Epoch  70: train=0.863 test=0.591  σ=3.18e-01/1.26e-08
+      Epoch  80: train=0.883 test=0.596  σ=3.04e-01/1.21e-08
+      Epoch  90: train=0.907 test=0.601  σ=2.91e-01/1.12e-08
+      Epoch 100: train=0.922 test=0.615  σ=2.69e-01/1.04e-08
+      Epoch 110: train=0.933 test=0.614  σ=2.62e-01/9.69e-09
+      Epoch 120: train=0.942 test=0.615  σ=2.47e-01/9.10e-09
+      Epoch 130: train=0.950 test=0.620  σ=2.44e-01/8.88e-09
+      Epoch 140: train=0.951 test=0.623  σ=2.46e-01/8.88e-09
+      Epoch 150: train=0.952 test=0.618  σ=2.22e-01/8.47e-09
+      Best test acc: 0.624
+    Lyapunov: depth=4, params=1,756,836
+      Epoch  10: train=0.036 test=0.010 λ=1.488 σ=6.35e-01/1.45e-08
+      Epoch  20: train=0.009 test=0.010 λ=1.425 σ=1.08e-01/3.34e-15
+      Epoch  30: train=0.009 test=0.010 λ=1.451 σ=4.78e-02/9.92e-21
+      Epoch  40: train=0.010 test=0.010 λ=1.455 σ=1.57e-02/0.00e+00
+      Epoch  50: train=0.009 test=0.010 λ=1.476 
+      Epoch  60: train=0.009 test=0.010 λ=1.473 
+      Epoch  70: train=0.010 test=0.010 λ=1.476 σ=1.34e-03/0.00e+00
+      Epoch  80: train=0.009 test=0.010 λ=1.479 
+      Epoch  90: train=0.009 test=0.010 λ=1.480 
+      Epoch 100: train=0.009 test=0.010 λ=1.480 
+      Epoch 110: train=0.009 test=0.010 λ=1.482 
+      Epoch 120: train=0.009 test=0.010 λ=1.481 
+      Epoch 130: train=0.009 test=0.010 λ=1.481 
+      Epoch 140: train=0.009 test=0.010 λ=1.480 
+      Epoch 150: train=0.010 test=0.010 λ=1.481 
+      Best test acc: 0.089
+
+============================================================
+Depth = 8 conv layers (4 stages × 2 blocks)
+============================================================
+    Vanilla: depth=8, params=4,892,196
+      Epoch  10: train=0.393 test=0.335  σ=7.83e-01/3.02e-08
+      Epoch  20: train=0.547 test=0.453  σ=4.71e-01/2.14e-08
+      Epoch  30: train=0.629 test=0.462  σ=3.73e-01/1.78e-08
+      Epoch  40: train=0.701 test=0.500  σ=3.33e-01/1.55e-08
+      Epoch  50: train=0.752 test=0.517  σ=3.06e-01/1.40e-08
+      Epoch  60: train=0.798 test=0.528  σ=2.76e-01/1.25e-08
+      Epoch  70: train=0.835 test=0.530  σ=2.70e-01/1.18e-08
+      Epoch  80: train=0.872 test=0.534  σ=2.51e-01/1.07e-08
+      Epoch  90: train=0.897 test=0.538  σ=2.38e-01/9.81e-09
+      Epoch 100: train=0.920 test=0.536  σ=2.26e-01/9.08e-09
+      Epoch 110: train=0.935 test=0.544  σ=2.06e-01/8.19e-09
+      Epoch 120: train=0.947 test=0.551  σ=2.11e-01/7.87e-09
+      Epoch 130: train=0.954 test=0.551  σ=2.02e-01/7.42e-09
+      Epoch 140: train=0.956 test=0.553  σ=1.92e-01/7.38e-09
+      Epoch 150: train=0.956 test=0.546  σ=2.05e-01/7.55e-09
+      Best test acc: 0.554
+    Lyapunov: depth=8, params=4,892,196
+      Epoch  10: train=0.031 test=0.010 λ=1.573 σ=4.25e-01/1.02e-08
+      Epoch  20: train=0.019 test=0.010 λ=1.580 σ=2.80e-01/1.31e-09
+      Epoch  30: train=0.010 test=0.010 λ=1.519 σ=1.87e-01/8.73e-13
+      Epoch  40: train=0.009 test=0.010 λ=1.532 σ=7.89e-02/1.22e-26
+      Epoch  50: train=0.010 test=0.010 λ=1.536 σ=6.60e-02/4.34e-19
+      Epoch  60: train=0.009 test=0.010 λ=1.542 σ=5.19e-02/0.00e+00
+      Epoch  70: train=0.010 test=0.010 λ=1.540 σ=2.67e-02/0.00e+00
+      Epoch  80: train=0.009 test=0.010 λ=1.544 σ=6.28e-03/0.00e+00
+      Epoch  90: train=0.009 test=0.010 λ=1.542 σ=4.08e-02/0.00e+00
+      Epoch 100: train=0.010 test=0.010 λ=1.543 
+      Epoch 110: train=0.009 test=0.010 λ=1.543 
+      Epoch 120: train=0.010 test=0.010 λ=1.543 σ=3.14e-03/0.00e+00
+      Epoch 130: train=0.009 test=0.010 λ=1.543 
+      Epoch 140: train=0.010 test=0.010 λ=1.545 
+      Epoch 150: train=0.010 test=0.010 λ=1.542 
+      Best test acc: 0.020
+
+============================================================
+Depth = 12 conv layers (4 stages × 3 blocks)
+============================================================
+    Vanilla: depth=12, params=8,027,556
+      Epoch  10: train=0.214 test=0.062  σ=6.26e-01/2.24e-08
+      Epoch  20: train=0.293 test=0.061  σ=3.27e-01/1.57e-08
+      Epoch  30: train=0.335 test=0.075  σ=2.66e-01/1.35e-08
+      Epoch  40: train=0.374 test=0.066  σ=2.32e-01/1.23e-08
+      Epoch  50: train=0.407 test=0.063  σ=2.29e-01/1.23e-08
+      Epoch  60: train=0.442 test=0.078  σ=2.18e-01/1.20e-08
+      Epoch  70: train=0.472 test=0.098  σ=2.30e-01/1.20e-08
+      Epoch  80: train=0.501 test=0.108  σ=2.21e-01/1.21e-08
+      Epoch  90: train=0.532 test=0.110  σ=2.24e-01/1.18e-08
+      Epoch 100: train=0.557 test=0.098  σ=2.25e-01/1.20e-08
+      Epoch 110: train=0.579 test=0.105  σ=2.26e-01/1.18e-08
+      Epoch 120: train=0.599 test=0.104  σ=2.31e-01/1.20e-08
+      Epoch 130: train=0.609 test=0.117  σ=2.28e-01/1.19e-08
+      Epoch 140: train=0.619 test=0.118  σ=2.23e-01/1.17e-08
+      Epoch 150: train=0.620 test=0.112  σ=2.30e-01/1.19e-08
+      Best test acc: 0.122
+    Lyapunov: depth=12, params=8,027,556
+      Epoch  10: train=0.017 test=0.010 λ=1.619 σ=3.10e-01/2.44e-12
+      Epoch  20: train=0.014 test=0.010 λ=1.620 σ=3.48e-01/1.80e-12
+      Epoch  30: train=0.010 test=0.010 λ=1.551 σ=1.95e-02/1.07e-16
+      Epoch  40: train=0.010 test=0.010 λ=1.556 σ=2.86e-02/0.00e+00
+      Epoch  50: train=0.010 test=0.010 λ=1.552 σ=1.02e-01/3.07e-15
+      Epoch  60: train=0.009 test=0.010 λ=1.560 σ=5.22e-02/0.00e+00
+      Epoch  70: train=0.009 test=0.010 λ=1.567 σ=3.97e-02/0.00e+00
+      Epoch  80: train=0.009 test=0.010 λ=1.564 
+      Epoch  90: train=0.009 test=0.010 λ=1.570 σ=1.20e-02/0.00e+00
+      Epoch 100: train=0.009 test=0.010 λ=1.566 
+      Epoch 110: train=0.009 test=0.010 λ=1.568 σ=1.35e-02/0.00e+00
+      Epoch 120: train=0.009 test=0.010 λ=1.566 
+      Epoch 130: train=0.009 test=0.010 λ=1.568 
+      Epoch 140: train=0.009 test=0.010 λ=1.566 
+      Epoch 150: train=0.010 test=0.010 λ=1.565 
+      Best test acc: 0.016
+
+============================================================
+Depth = 16 conv layers (4 stages × 4 blocks)
+============================================================
+    Vanilla: depth=16, params=11,162,916
+      Epoch  10: train=0.091 test=0.011  σ=4.40e-01/1.32e-08
+      Epoch  20: train=0.133 test=0.015  σ=2.83e-01/1.07e-08
+      Epoch  30: train=0.156 test=0.018  σ=2.23e-01/9.48e-09
+      Epoch  40: train=0.176 test=0.018  σ=1.95e-01/9.15e-09
+      Epoch  50: train=0.191 test=0.021  σ=1.80e-01/8.93e-09
+      Epoch  60: train=0.204 test=0.022  σ=1.71e-01/8.85e-09
+      Epoch  70: train=0.218 test=0.028  σ=1.62e-01/9.03e-09
+      Epoch  80: train=0.227 test=0.025  σ=1.64e-01/8.92e-09
+      Epoch  90: train=0.238 test=0.028  σ=1.60e-01/9.11e-09
+      Epoch 100: train=0.249 test=0.028  σ=1.63e-01/9.28e-09
+      Epoch 110: train=0.257 test=0.031  σ=1.60e-01/9.29e-09
+      Epoch 120: train=0.265 test=0.027  σ=1.65e-01/9.32e-09
+      Epoch 130: train=0.270 test=0.028  σ=1.62e-01/9.29e-09
+      Epoch 140: train=0.273 test=0.026  σ=1.66e-01/9.44e-09
+      Epoch 150: train=0.274 test=0.026  σ=1.70e-01/9.40e-09
+      Best test acc: 0.033
+    Lyapunov: depth=16, params=11,162,916
+      Epoch  10: train=0.011 test=0.010 λ=1.693 σ=2.40e-01/9.58e-25
+      Epoch  20: train=0.009 test=0.010 λ=1.575 σ=1.05e-01/7.85e-12
+      Epoch  30: train=0.010 test=0.010 λ=1.590 σ=2.47e-01/2.27e-12
+      Epoch  40: train=0.009 test=0.010 λ=1.600 σ=4.94e-02/6.93e-15
+      Epoch  50: train=0.009 test=0.010 λ=1.594 σ=7.66e-02/2.70e-17
+      Epoch  60: train=0.010 test=0.010 λ=1.592 σ=5.57e-02/0.00e+00
+      Epoch  70: train=0.010 test=0.010 λ=1.595 σ=5.96e-02/0.00e+00
+      Epoch  80: train=0.010 test=0.010 λ=1.592 σ=4.24e-02/0.00e+00
+      Epoch  90: train=0.010 test=0.010 λ=1.594 σ=3.31e-02/0.00e+00
+      Epoch 100: train=0.011 test=0.010 λ=1.597 σ=3.36e-02/0.00e+00
+      Epoch 110: train=0.010 test=0.010 λ=1.599 σ=2.33e-02/0.00e+00
+      Epoch 120: train=0.010 test=0.010 λ=1.593 
+      Epoch 130: train=0.010 test=0.010 λ=1.595 σ=1.07e-02/0.00e+00
+      Epoch 140: train=0.010 test=0.010 λ=1.593 σ=6.22e-03/0.00e+00
+      Epoch 150: train=0.010 test=0.010 λ=1.590 
+      Best test acc: 0.011
+
+====================================================================================================
+DEPTH SCALING RESULTS: CIFAR100
+====================================================================================================
+Depth    Vanilla Acc  Lyapunov Acc Δ Acc    Lyap λ     Van ∇norm    Lyap ∇norm   Van κ     
+----------------------------------------------------------------------------------------------------
+4        0.618        0.010        -0.608   1.481      4.60e-01     8.81e-02     1.2e+09   
+8        0.546        0.010        -0.536   1.542      3.79e-01     1.58e-01     4.7e+09   
+12       0.112        0.010        -0.102   1.565      6.41e-01     9.04e-02     3.8e+07   
+16       0.026        0.010        -0.016   1.590      5.09e-01     2.32e+00     3.0e+07   
+====================================================================================================
+
+GRADIENT HEALTH ANALYSIS:
+  Depth 4: ⚠️ Vanilla has ill-conditioned gradients (κ > 1e6)
+  Depth 8: ⚠️ Vanilla has ill-conditioned gradients (κ > 1e6)
+  Depth 12: ⚠️ Vanilla has ill-conditioned gradients (κ > 1e6)
+  Depth 16: ⚠️ Vanilla has ill-conditioned gradients (κ > 1e6)
+
+
+KEY OBSERVATIONS:
+  Vanilla  4→16 layers: -0.592 accuracy change
+  Lyapunov 4→16 layers: +0.000 accuracy change
+  ✓ Lyapunov regularization enables better depth scaling!
+
+Results saved to runs/depth_scaling_asymm/cifar100_20260101-112330
+============================================================
+Finished: Thu Jan  1 11:23:32 CST 2026
+============================================================