1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
|
============================================================
WEAK REG + EXTREME Experiment
Job ID: 15261461 | Node: gpub005
Start: Sat Jan 3 09:41:01 CST 2026
============================================================
NVIDIA A40, 46068 MiB
============================================================
================================================================================
DEPTH SCALING BENCHMARK
================================================================================
Dataset: cifar100
Depths: [4, 8, 12, 16]
Timesteps: 4
Epochs: 150
λ_reg: 0.01, λ_target: -0.1
Reg type: extreme, Warmup epochs: 20
Device: cuda
================================================================================
Loading cifar100...
Classes: 100, Input: (3, 32, 32)
Train: 50000, Test: 10000
Depth configurations: [(4, '4×1'), (8, '4×2'), (12, '4×3'), (16, '4×4')]
Regularization type: extreme
Warmup epochs: 20
Stable init: False
Lyapunov threshold: 3.0
============================================================
Depth = 4 conv layers (4 stages × 1 blocks)
============================================================
Vanilla: depth=4, params=1,756,836
Epoch 10: train=0.494 test=0.428 σ=9.51e-01/3.55e-08
Epoch 20: train=0.623 test=0.490 σ=5.85e-01/2.42e-08
Epoch 30: train=0.700 test=0.549 σ=4.90e-01/2.03e-08
Epoch 40: train=0.754 test=0.566 σ=4.23e-01/1.75e-08
Epoch 50: train=0.795 test=0.561 σ=3.72e-01/1.53e-08
Epoch 60: train=0.830 test=0.575 σ=3.37e-01/1.39e-08
Epoch 70: train=0.859 test=0.584 σ=3.17e-01/1.28e-08
Epoch 80: train=0.880 test=0.581 σ=3.00e-01/1.18e-08
Epoch 90: train=0.904 test=0.589 σ=2.71e-01/1.07e-08
Epoch 100: train=0.920 test=0.601 σ=2.69e-01/1.03e-08
Epoch 110: train=0.930 test=0.605 σ=2.62e-01/9.69e-09
Epoch 120: train=0.941 test=0.606 σ=2.49e-01/9.34e-09
Epoch 130: train=0.948 test=0.607 σ=2.34e-01/8.60e-09
Epoch 140: train=0.951 test=0.610 σ=2.36e-01/8.72e-09
Epoch 150: train=0.951 test=0.608 σ=2.33e-01/8.71e-09
Best test acc: 0.617
Lyapunov: depth=4, params=1,756,836
Epoch 10: train=0.493 test=0.409 λ=2.047 σ=9.58e-01/3.58e-08
Epoch 20: train=0.624 test=0.456 λ=1.989 σ=5.87e-01/2.42e-08
Epoch 30: train=0.700 test=0.509 λ=1.967 σ=4.94e-01/2.04e-08
Epoch 40: train=0.752 test=0.526 λ=1.964 σ=4.18e-01/1.71e-08
Epoch 50: train=0.799 test=0.526 λ=1.952 σ=3.65e-01/1.54e-08
Epoch 60: train=0.828 test=0.565 λ=1.958 σ=3.47e-01/1.46e-08
Epoch 70: train=0.860 test=0.570 λ=1.953 σ=3.16e-01/1.28e-08
Epoch 80: train=0.882 test=0.580 λ=1.952 σ=3.02e-01/1.18e-08
Epoch 90: train=0.906 test=0.583 λ=1.960 σ=2.77e-01/1.09e-08
Epoch 100: train=0.921 test=0.592 λ=1.964 σ=2.67e-01/1.01e-08
Epoch 110: train=0.932 test=0.590 λ=1.972 σ=2.55e-01/9.60e-09
Epoch 120: train=0.943 test=0.591 λ=1.968 σ=2.54e-01/9.22e-09
Epoch 130: train=0.949 test=0.596 λ=1.972 σ=2.51e-01/9.12e-09
Epoch 140: train=0.950 test=0.597 λ=1.973 σ=2.42e-01/8.72e-09
Epoch 150: train=0.952 test=0.595 λ=1.972 σ=2.43e-01/8.92e-09
Best test acc: 0.602
============================================================
Depth = 8 conv layers (4 stages × 2 blocks)
============================================================
Vanilla: depth=8, params=4,892,196
Epoch 10: train=0.389 test=0.341 σ=8.87e-01/3.14e-08
Epoch 20: train=0.545 test=0.432 σ=4.81e-01/2.15e-08
Epoch 30: train=0.634 test=0.462 σ=3.76e-01/1.76e-08
Epoch 40: train=0.700 test=0.476 σ=3.21e-01/1.52e-08
Epoch 50: train=0.754 test=0.498 σ=3.14e-01/1.42e-08
Epoch 60: train=0.797 test=0.510 σ=2.85e-01/1.31e-08
Epoch 70: train=0.833 test=0.503 σ=2.73e-01/1.18e-08
Epoch 80: train=0.862 test=0.512 σ=2.52e-01/1.06e-08
Epoch 90: train=0.895 test=0.503 σ=2.32e-01/9.57e-09
Epoch 100: train=0.916 test=0.509 σ=2.23e-01/9.32e-09
Epoch 110: train=0.933 test=0.508 σ=2.13e-01/8.31e-09
Epoch 120: train=0.945 test=0.522 σ=2.06e-01/7.74e-09
Epoch 130: train=0.952 test=0.517 σ=2.08e-01/7.74e-09
Epoch 140: train=0.955 test=0.521 σ=1.92e-01/7.10e-09
Epoch 150: train=0.956 test=0.506 σ=1.95e-01/7.33e-09
Best test acc: 0.529
Lyapunov: depth=8, params=4,892,196
Epoch 10: train=0.308 test=0.232 λ=2.862 σ=7.62e-01/2.96e-08
Epoch 20: train=0.454 test=0.249 λ=2.882 σ=4.79e-01/2.22e-08
Epoch 30: train=0.468 test=0.267 λ=2.892 σ=3.75e-01/1.85e-08
|
