============================================================ SMARTER TARGET Experiment (lambda_target=1.0) Job ID: 15112872 | Node: gpub074 Start: Thu Jan 1 12:26:50 CST 2026 ============================================================ NVIDIA A40, 46068 MiB ============================================================ ================================================================================ DEPTH SCALING BENCHMARK ================================================================================ Dataset: cifar100 Depths: [4, 8, 12, 16] Timesteps: 4 Epochs: 150 λ_reg: 0.1, λ_target: 1.0 Reg type: squared, Warmup epochs: 20 Device: cuda ================================================================================ Loading cifar100... Classes: 100, Input: (3, 32, 32) Train: 50000, Test: 10000 Depth configurations: [(4, '4×1'), (8, '4×2'), (12, '4×3'), (16, '4×4')] Regularization type: squared Warmup epochs: 20 Stable init: False ============================================================ Depth = 4 conv layers (4 stages × 1 blocks) ============================================================ Vanilla: depth=4, params=1,756,836 Epoch 10: train=0.498 test=0.436 σ=9.42e-01/3.51e-08 Epoch 20: train=0.629 test=0.499 σ=5.87e-01/2.46e-08 Epoch 30: train=0.701 test=0.550 σ=4.77e-01/2.01e-08 Epoch 40: train=0.753 test=0.545 σ=4.17e-01/1.75e-08 Epoch 50: train=0.798 test=0.577 σ=3.69e-01/1.53e-08 Epoch 60: train=0.830 test=0.582 σ=3.33e-01/1.39e-08 Epoch 70: train=0.862 test=0.580 σ=3.29e-01/1.26e-08 Epoch 80: train=0.884 test=0.578 σ=3.07e-01/1.21e-08 Epoch 90: train=0.907 test=0.598 σ=2.79e-01/1.08e-08 Epoch 100: train=0.921 test=0.608 σ=2.72e-01/1.06e-08 Epoch 110: train=0.935 test=0.608 σ=2.54e-01/9.36e-09 Epoch 120: train=0.943 test=0.608 σ=2.46e-01/9.12e-09 Epoch 130: train=0.949 test=0.615 σ=2.34e-01/8.83e-09 Epoch 140: train=0.951 test=0.610 σ=2.32e-01/8.64e-09 Epoch 150: train=0.954 test=0.614 σ=2.32e-01/8.63e-09 Best test acc: 0.615 Lyapunov: depth=4, params=1,756,836 Epoch 10: train=0.285 test=0.024 λ=1.483 σ=7.62e-01/2.92e-08 Epoch 20: train=0.344 test=0.014 λ=1.560 σ=4.80e-01/2.01e-08 Epoch 30: train=0.348 test=0.012 λ=1.679 σ=3.85e-01/1.69e-08 Epoch 40: train=0.389 test=0.013 λ=1.635 σ=3.33e-01/1.54e-08 Epoch 50: train=0.431 test=0.011 λ=1.635 σ=3.11e-01/1.45e-08 Epoch 60: train=0.461 test=0.016 λ=1.622 σ=2.95e-01/1.44e-08 Epoch 70: train=0.478 test=0.014 λ=1.660 σ=2.90e-01/1.40e-08 Epoch 80: train=0.499 test=0.013 λ=1.657 σ=2.82e-01/1.40e-08 Epoch 90: train=0.522 test=0.013 λ=1.663 σ=2.78e-01/1.36e-08 Epoch 100: train=0.537 test=0.015 λ=1.678 σ=2.78e-01/1.36e-08 Epoch 110: train=0.550 test=0.014 λ=1.684 σ=2.92e-01/1.38e-08 Epoch 120: train=0.559 test=0.016 λ=1.704 σ=2.90e-01/1.42e-08 Epoch 130: train=0.570 test=0.018 λ=1.709 σ=2.79e-01/1.36e-08 Epoch 140: train=0.571 test=0.017 λ=1.865 σ=2.83e-01/1.37e-08 Epoch 150: train=0.576 test=0.017 λ=1.816 σ=2.82e-01/1.37e-08 Best test acc: 0.212 ============================================================ Depth = 8 conv layers (4 stages × 2 blocks) ============================================================ Vanilla: depth=8, params=4,892,196 Epoch 10: train=0.388 test=0.351 σ=8.72e-01/3.16e-08 Epoch 20: train=0.544 test=0.427 σ=4.73e-01/2.16e-08 Epoch 30: train=0.630 test=0.466 σ=3.81e-01/1.79e-08 Epoch 40: train=0.698 test=0.502 σ=3.22e-01/1.53e-08 Epoch 50: train=0.747 test=0.519 σ=3.08e-01/1.42e-08 Epoch 60: train=0.799 test=0.515 σ=2.87e-01/1.29e-08 Epoch 70: train=0.836 test=0.524 σ=2.76e-01/1.18e-08 Epoch 80: train=0.869 test=0.534 σ=2.44e-01/1.05e-08 Epoch 90: train=0.898 test=0.528 σ=2.39e-01/9.52e-09 Epoch 100: train=0.918 test=0.527 σ=2.29e-01/8.96e-09 Epoch 110: train=0.933 test=0.542 σ=2.26e-01/8.58e-09 Epoch 120: train=0.943 test=0.542 σ=2.09e-01/7.88e-09 Epoch 130: train=0.951 test=0.545 σ=1.97e-01/7.80e-09 Epoch 140: train=0.955 test=0.542 σ=2.06e-01/7.61e-09 Epoch 150: train=0.954 test=0.535 σ=1.94e-01/7.46e-09 Best test acc: 0.550 Lyapunov: depth=8, params=4,892,196 Epoch 10: train=0.035 test=0.010 λ=1.583 σ=3.25e-01/9.17e-09 Epoch 20: train=0.049 test=0.010 λ=1.574 σ=2.46e-01/6.77e-09 Epoch 30: train=0.061 test=0.010 λ=1.571 σ=2.03e-01/5.87e-09 Epoch 40: train=0.033 test=0.010 λ=1.544 σ=1.80e-01/2.89e-09 Epoch 50: train=0.030 test=0.010 λ=1.550 σ=1.59e-01/9.89e-10 Epoch 60: train=0.030 test=0.010 λ=1.567 σ=1.39e-01/5.26e-10 Epoch 70: train=0.029 test=0.010 λ=1.571 σ=1.16e-01/1.53e-10 Epoch 80: train=0.041 test=0.010 λ=1.646 σ=1.41e-01/3.16e-09 Epoch 90: train=0.036 test=0.010 λ=1.808 σ=1.37e-01/2.76e-09 Epoch 100: train=0.031 test=0.010 λ=1.940 σ=1.71e-01/2.97e-09 Epoch 110: train=0.047 test=0.010 λ=1.976 σ=1.42e-01/3.22e-09 Epoch 120: train=0.047 test=0.008 λ=1.993 σ=1.26e-01/3.43e-09 Epoch 130: train=0.046 test=0.010 λ=2.057 σ=1.50e-01/3.50e-09 Epoch 140: train=0.026 test=0.010 λ=2.014 σ=2.43e-01/3.40e-09 Epoch 150: train=0.031 test=0.010 λ=2.334 σ=1.30e-01/4.11e-09 Best test acc: 0.024 ============================================================ Depth = 12 conv layers (4 stages × 3 blocks) ============================================================ Vanilla: depth=12, params=8,027,556 Epoch 10: train=0.214 test=0.070 σ=5.69e-01/2.19e-08 Epoch 20: train=0.289 test=0.057 σ=3.32e-01/1.63e-08 Epoch 30: train=0.340 test=0.107 σ=2.62e-01/1.36e-08 Epoch 40: train=0.374 test=0.083 σ=2.35e-01/1.28e-08 Epoch 50: train=0.410 test=0.073 σ=2.28e-01/1.25e-08 Epoch 60: train=0.436 test=0.101 σ=2.23e-01/1.23e-08 Epoch 70: train=0.473 test=0.087 σ=2.33e-01/1.22e-08 Epoch 80: train=0.505 test=0.083 σ=2.21e-01/1.22e-08 Epoch 90: train=0.534 test=0.090 σ=2.24e-01/1.21e-08 Epoch 100: train=0.561 test=0.096 σ=2.29e-01/1.23e-08 Epoch 110: train=0.584 test=0.074 σ=2.30e-01/1.21e-08 Epoch 120: train=0.602 test=0.088 σ=2.35e-01/1.22e-08 Epoch 130: train=0.609 test=0.093 σ=2.30e-01/1.20e-08 Epoch 140: train=0.620 test=0.094 σ=2.27e-01/1.19e-08 Epoch 150: train=0.624 test=0.086 σ=2.31e-01/1.22e-08 Best test acc: 0.109 Lyapunov: depth=12, params=8,027,556 Epoch 10: train=0.013 test=0.010 λ=1.639 σ=3.64e-01/1.06e-12 Epoch 20: train=0.015 test=0.010 λ=1.598 σ=2.98e-01/1.14e-12 Epoch 30: train=0.019 test=0.010 λ=1.630 σ=3.30e-01/3.22e-12 Epoch 40: train=0.021 test=0.010 λ=1.592 σ=1.82e-01/2.19e-12 Epoch 50: train=0.020 test=0.010 λ=1.658 σ=1.51e-01/2.96e-12 Epoch 60: train=0.015 test=0.010 λ=1.616 σ=1.03e-01/2.55e-13 Epoch 70: train=0.018 test=0.010 λ=1.617 σ=1.18e-01/4.53e-13 Epoch 80: train=0.020 test=0.010 λ=1.636 σ=1.22e-01/4.69e-12 Epoch 90: train=0.021 test=0.010 λ=1.593 σ=1.05e-01/7.58e-12 Epoch 100: train=0.026 test=0.010 λ=1.593 σ=1.16e-01/7.00e-10 Epoch 110: train=0.021 test=0.010 λ=1.590 σ=9.46e-02/4.97e-12 Epoch 120: train=0.024 test=0.010 λ=1.740 σ=9.83e-02/3.89e-11 Epoch 130: train=0.020 test=0.010 λ=1.901 σ=1.09e-01/9.81e-11 Epoch 140: train=0.027 test=0.010 λ=1.972 σ=1.21e-01/1.96e-09 Epoch 150: train=0.019 test=0.010 λ=2.112 σ=6.82e-02/1.40e-11 Best test acc: 0.019 ============================================================ Depth = 16 conv layers (4 stages × 4 blocks) ============================================================ Vanilla: depth=16, params=11,162,916 Epoch 10: train=0.091 test=0.011 σ=4.40e-01/1.32e-08 Epoch 20: train=0.135 test=0.014 σ=2.84e-01/1.06e-08 Epoch 30: train=0.157 test=0.017 σ=2.21e-01/9.39e-09 Epoch 40: train=0.177 test=0.019 σ=1.93e-01/8.97e-09 Epoch 50: train=0.191 test=0.024 σ=1.81e-01/9.00e-09 Epoch 60: train=0.202 test=0.024 σ=1.74e-01/8.89e-09 Epoch 70: train=0.215 test=0.026 σ=1.66e-01/8.97e-09 Epoch 80: train=0.227 test=0.030 σ=1.65e-01/8.89e-09 Epoch 90: train=0.240 test=0.026 σ=1.55e-01/8.93e-09 Epoch 100: train=0.248 test=0.028 σ=1.61e-01/9.09e-09 Epoch 110: train=0.254 test=0.031 σ=1.59e-01/9.18e-09 Epoch 120: train=0.262 test=0.033 σ=1.64e-01/9.30e-09 Epoch 130: train=0.267 test=0.033 σ=1.58e-01/9.22e-09 Epoch 140: train=0.269 test=0.030 σ=1.61e-01/9.32e-09 Epoch 150: train=0.269 test=0.030 σ=1.64e-01/9.32e-09 Best test acc: 0.036 Lyapunov: depth=16, params=11,162,916 Epoch 10: train=0.014 test=0.010 λ=1.688 σ=3.64e-01/6.52e-13 Epoch 20: train=0.010 test=0.010 λ=1.803 σ=3.56e-01/1.53e-13 Epoch 30: train=0.012 test=0.002 λ=1.664 σ=4.44e-01/4.25e-14 Epoch 40: train=0.018 test=0.010 λ=1.733 σ=2.04e-01/9.43e-13 Epoch 50: train=0.010 test=0.010 λ=1.598 σ=1.55e-01/0.00e+00 Epoch 60: train=0.010 test=0.010 λ=1.594 σ=4.41e-02/0.00e+00 Epoch 70: train=0.009 test=0.010 λ=1.601 σ=7.50e-02/1.52e-14 Epoch 80: train=0.012 test=0.010 λ=2.219 σ=5.78e-02/1.54e-42 Epoch 90: train=0.016 test=0.010 λ=2.122 σ=9.91e-02/9.49e-15 Epoch 100: train=0.017 test=0.010 λ=2.163 σ=1.05e-01/7.08e-13 Epoch 110: train=0.020 test=0.010 λ=2.124 σ=1.10e-01/1.17e-12 Epoch 120: train=0.010 test=0.010 λ=2.181 σ=6.50e-02/6.09e-15 Epoch 130: train=0.010 test=0.010 λ=2.755 σ=6.77e-09/1.45e-20 Epoch 140: train=0.016 test=0.010 λ=2.217 σ=1.13e-01/5.35e-14 Epoch 150: train=0.018 test=0.010 λ=2.219 σ=1.21e-01/7.28e-14 Best test acc: 0.012 ==================================================================================================== DEPTH SCALING RESULTS: CIFAR100 ==================================================================================================== Depth Vanilla Acc Lyapunov Acc Δ Acc Lyap λ Van ∇norm Lyap ∇norm Van κ ---------------------------------------------------------------------------------------------------- 4 0.614 0.017 -0.597 1.816 4.52e-01 7.31e-01 1.1e+09 8 0.535 0.010 -0.525 2.334 3.87e-01 3.44e-01 8.0e+08 12 0.086 0.010 -0.076 2.112 6.47e-01 2.28e+00 4.9e+07 16 0.030 0.010 -0.020 2.219 5.07e-01 9.41e-01 2.1e+07 ==================================================================================================== GRADIENT HEALTH ANALYSIS: Depth 4: ⚠️ Vanilla has ill-conditioned gradients (κ > 1e6) Depth 8: ⚠️ Vanilla has ill-conditioned gradients (κ > 1e6) Depth 12: ⚠️ Vanilla has ill-conditioned gradients (κ > 1e6) Depth 16: ⚠️ Vanilla has ill-conditioned gradients (κ > 1e6) KEY OBSERVATIONS: Vanilla 4→16 layers: -0.584 accuracy change Lyapunov 4→16 layers: -0.007 accuracy change ✓ Lyapunov regularization enables better depth scaling! Results saved to runs/depth_scaling_target1/cifar100_20260102-133339 ============================================================ Finished: Fri Jan 2 13:33:43 CST 2026 ============================================================