1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
|
============================================================
EXTREME THRESHOLD 3.0 (Extended)
Job ID: 15334042 | Node: gpub074
Start: Sun Jan 4 03:07:44 CST 2026
============================================================
NVIDIA A40, 46068 MiB
============================================================
================================================================================
DEPTH SCALING BENCHMARK
================================================================================
Dataset: cifar100
Depths: [4, 8, 12, 16]
Timesteps: 4
Epochs: 150
λ_reg: 0.3, λ_target: -0.1
Reg type: extreme, Warmup epochs: 10
Device: cuda
================================================================================
Loading cifar100...
Classes: 100, Input: (3, 32, 32)
Train: 50000, Test: 10000
Depth configurations: [(4, '4×1'), (8, '4×2'), (12, '4×3'), (16, '4×4')]
Regularization type: extreme
Warmup epochs: 10
Stable init: False
Lyapunov threshold: 3.0
============================================================
Depth = 4 conv layers (4 stages × 1 blocks)
============================================================
Vanilla: depth=4, params=1,756,836
Epoch 10: train=0.496 test=0.398 σ=9.36e-01/3.51e-08
Epoch 20: train=0.627 test=0.457 σ=5.78e-01/2.40e-08
Epoch 30: train=0.702 test=0.528 σ=4.75e-01/1.99e-08
Epoch 40: train=0.750 test=0.561 σ=4.20e-01/1.75e-08
Epoch 50: train=0.797 test=0.540 σ=3.68e-01/1.52e-08
Epoch 60: train=0.829 test=0.570 σ=3.43e-01/1.42e-08
Epoch 70: train=0.863 test=0.581 σ=3.01e-01/1.20e-08
Epoch 80: train=0.884 test=0.591 σ=2.91e-01/1.15e-08
Epoch 90: train=0.904 test=0.589 σ=2.78e-01/1.10e-08
Epoch 100: train=0.920 test=0.595 σ=2.62e-01/1.00e-08
Epoch 110: train=0.930 test=0.600 σ=2.59e-01/9.52e-09
Epoch 120: train=0.943 test=0.600 σ=2.42e-01/8.96e-09
Epoch 130: train=0.948 test=0.602 σ=2.36e-01/8.69e-09
Epoch 140: train=0.949 test=0.601 σ=2.25e-01/8.32e-09
Epoch 150: train=0.951 test=0.604 σ=2.47e-01/9.14e-09
Best test acc: 0.613
Lyapunov: depth=4, params=1,756,836
Epoch 10: train=0.493 test=0.399 λ=2.042 σ=9.34e-01/3.52e-08
Epoch 20: train=0.626 test=0.496 λ=1.985 σ=5.77e-01/2.42e-08
Epoch 30: train=0.702 test=0.553 λ=1.966 σ=4.78e-01/2.00e-08
Epoch 40: train=0.753 test=0.565 λ=1.951 σ=4.12e-01/1.74e-08
Epoch 50: train=0.798 test=0.581 λ=1.950 σ=3.70e-01/1.55e-08
Epoch 60: train=0.830 test=0.557 λ=1.956 σ=3.36e-01/1.41e-08
Epoch 70: train=0.859 test=0.581 λ=1.950 σ=3.22e-01/1.30e-08
Epoch 80: train=0.885 test=0.589 λ=1.956 σ=2.98e-01/1.17e-08
Epoch 90: train=0.904 test=0.594 λ=1.956 σ=2.94e-01/1.11e-08
Epoch 100: train=0.921 test=0.599 λ=1.957 σ=2.70e-01/1.04e-08
Epoch 110: train=0.931 test=0.601 λ=1.958 σ=2.64e-01/9.99e-09
Epoch 120: train=0.940 test=0.598 λ=1.965 σ=2.46e-01/9.24e-09
Epoch 130: train=0.948 test=0.604 λ=1.966 σ=2.31e-01/8.63e-09
Epoch 140: train=0.948 test=0.607 λ=1.967 σ=2.31e-01/8.78e-09
Epoch 150: train=0.950 test=0.604 λ=1.965 σ=2.43e-01/9.00e-09
Best test acc: 0.610
============================================================
Depth = 8 conv layers (4 stages × 2 blocks)
============================================================
Vanilla: depth=8, params=4,892,196
Epoch 10: train=0.391 test=0.354 σ=8.06e-01/3.09e-08
Epoch 20: train=0.544 test=0.429 σ=4.70e-01/2.14e-08
Epoch 30: train=0.631 test=0.474 σ=3.81e-01/1.78e-08
Epoch 40: train=0.699 test=0.501 σ=3.41e-01/1.57e-08
Epoch 50: train=0.754 test=0.492 σ=3.13e-01/1.42e-08
Epoch 60: train=0.794 test=0.519 σ=2.92e-01/1.30e-08
Epoch 70: train=0.839 test=0.513 σ=2.78e-01/1.18e-08
Epoch 80: train=0.868 test=0.531 σ=2.52e-01/1.07e-08
Epoch 90: train=0.897 test=0.534 σ=2.33e-01/9.43e-09
Epoch 100: train=0.917 test=0.529 σ=2.23e-01/9.14e-09
Epoch 110: train=0.934 test=0.534 σ=2.26e-01/8.39e-09
Epoch 120: train=0.946 test=0.546 σ=2.10e-01/7.95e-09
Epoch 130: train=0.953 test=0.538 σ=2.03e-01/7.87e-09
Epoch 140: train=0.955 test=0.550 σ=1.97e-01/7.37e-09
Epoch 150: train=0.957 test=0.533 σ=1.85e-01/7.16e-09
Best test acc: 0.550
Lyapunov: depth=8, params=4,892,196
Epoch 10: train=0.208 test=0.141 λ=2.533 σ=6.81e-01/2.65e-08
Epoch 20: train=0.339 test=0.223 λ=2.611 σ=4.24e-01/2.00e-08
Epoch 30: train=0.439 test=0.298 λ=2.633 σ=3.64e-01/1.78e-08
Epoch 40: train=0.523 test=0.372 λ=2.627 σ=3.10e-01/1.61e-08
Epoch 50: train=0.592 test=0.347 λ=2.609 σ=3.03e-01/1.49e-08
Epoch 60: train=0.648 test=0.354 λ=2.621 σ=2.91e-01/1.43e-08
Epoch 70: train=0.694 test=0.347 λ=2.610 σ=2.85e-01/1.37e-08
Epoch 80: train=0.739 test=0.385 λ=2.617 σ=2.68e-01/1.30e-08
Epoch 90: train=0.716 test=0.417 λ=2.580 σ=2.57e-01/1.26e-08
Epoch 100: train=0.783 test=0.429 λ=2.584 σ=2.54e-01/1.21e-08
Epoch 110: train=0.813 test=0.434 λ=2.588 σ=2.46e-01/1.15e-08
Epoch 120: train=0.803 test=0.436 λ=2.614 σ=2.54e-01/1.17e-08
Epoch 130: train=0.845 test=0.459 λ=2.595 σ=2.42e-01/1.06e-08
Epoch 140: train=0.855 test=0.461 λ=2.590 σ=2.27e-01/1.05e-08
Epoch 150: train=0.855 test=0.455 λ=2.595 σ=2.34e-01/1.06e-08
Best test acc: 0.472
============================================================
Depth = 12 conv layers (4 stages × 3 blocks)
============================================================
Vanilla: depth=12, params=8,027,556
Epoch 10: train=0.214 test=0.034 σ=5.57e-01/2.12e-08
Epoch 20: train=0.284 test=0.036 σ=3.28e-01/1.58e-08
Epoch 30: train=0.326 test=0.025 σ=2.68e-01/1.36e-08
Epoch 40: train=0.365 test=0.029 σ=2.39e-01/1.26e-08
Epoch 50: train=0.397 test=0.031 σ=2.31e-01/1.24e-08
Epoch 60: train=0.429 test=0.034 σ=2.24e-01/1.21e-08
Epoch 70: train=0.461 test=0.052 σ=2.31e-01/1.21e-08
Epoch 80: train=0.490 test=0.038 σ=2.21e-01/1.21e-08
Epoch 90: train=0.514 test=0.032 σ=2.25e-01/1.20e-08
Epoch 100: train=0.544 test=0.051 σ=2.29e-01/1.20e-08
Epoch 110: train=0.564 test=0.047 σ=2.26e-01/1.20e-08
Epoch 120: train=0.524 test=0.126 σ=2.52e-01/1.30e-08
Epoch 130: train=0.557 test=0.087 σ=2.22e-01/1.19e-08
Epoch 140: train=0.570 test=0.080 σ=2.25e-01/1.19e-08
Epoch 150: train=0.571 test=0.073 σ=2.30e-01/1.22e-08
Best test acc: 0.142
Lyapunov: depth=12, params=8,027,556
Epoch 10: train=0.064 test=0.026 λ=2.449 σ=4.92e-01/1.62e-08
Epoch 20: train=0.111 test=0.048 λ=2.432 σ=3.00e-01/1.25e-08
Epoch 30: train=0.149 test=0.039 λ=2.425 σ=2.42e-01/1.18e-08
Epoch 40: train=0.184 test=0.026 λ=2.415 σ=2.16e-01/1.16e-08
Epoch 50: train=0.212 test=0.022 λ=2.409 σ=2.04e-01/1.18e-08
Epoch 60: train=0.239 test=0.028 λ=2.408 σ=1.93e-01/1.15e-08
|
