1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
|
Device: cuda:0
Depths: [4], Widths: [512]
Seeds: [123, 456], BP seeds: [99999]
============================================================
Config: d512_L4_s123, methods: ['dfa', 'credit_bridge']
============================================================
d=512, L=4, seed=123, method=dfa
[DFA] Ep 1: loss=2.0624 train=0.2399 test=0.2780
[DFA] Ep 10: loss=1.9937 train=0.2692 test=0.2896
[DFA] Ep 20: loss=1.9914 train=0.2763 test=0.2907
[DFA] Ep 30: loss=1.9863 train=0.2793 test=0.3103
[DFA] Ep 40: loss=1.9821 train=0.2795 test=0.2838
[DFA] Ep 50: loss=1.9797 train=0.2815 test=0.3044
[DFA] Ep 60: loss=1.9779 train=0.2836 test=0.3078
[DFA] Ep 70: loss=1.9746 train=0.2853 test=0.3045
[DFA] Ep 80: loss=1.9749 train=0.2850 test=0.3127
[DFA] Ep 90: loss=1.9713 train=0.2850 test=0.3100
[DFA] Ep 100: loss=1.9721 train=0.2881 test=0.3094
Done in 934s: acc=0.3094 Gamma=0.1070 rho=0.0024
d=512, L=4, seed=123, method=credit_bridge
[CB] Ep 1 (warmup): loss=2.0474 train=0.2498 test=0.2708 vloss=0.570750
[CB] Ep 10 (warmup): loss=1.9885 train=0.2744 test=0.2904 vloss=0.072501
[CB] Ep 20 (warmup): loss=1.9826 train=0.2787 test=0.3163 vloss=0.044637
[CB] Ep 30 (blend=0.50): loss=1.9813 train=0.2803 test=0.2957 vloss=0.034919
[CB] Ep 40 (blend=1.00): loss=1.9706 train=0.2851 test=0.2988 vloss=0.024587
[CB] Ep 50 (blend=1.00): loss=1.9926 train=0.2667 test=0.2809 vloss=0.023065
[CB] Ep 60 (blend=1.00): loss=1.9741 train=0.2725 test=0.2772 vloss=0.019018
[CB] Ep 70 (blend=1.00): loss=1.9646 train=0.2812 test=0.2917 vloss=0.011482
[CB] Ep 80 (blend=1.00): loss=1.9559 train=0.2854 test=0.3062 vloss=0.005063
[CB] Ep 90 (blend=1.00): loss=1.9517 train=0.2901 test=0.3151 vloss=0.002326
[CB] Ep 100 (blend=1.00): loss=1.9479 train=0.2907 test=0.3091 vloss=0.001450
Done in 1572s: acc=0.3091 Gamma=0.1319 rho=0.0010
============================================================
Config: d512_L4_s456, methods: ['dfa', 'credit_bridge']
============================================================
d=512, L=4, seed=456, method=dfa
[DFA] Ep 1: loss=2.0361 train=0.2552 test=0.2764
[DFA] Ep 10: loss=1.9941 train=0.2756 test=0.2900
[DFA] Ep 20: loss=1.9883 train=0.2767 test=0.2958
[DFA] Ep 30: loss=1.9867 train=0.2788 test=0.3011
[DFA] Ep 40: loss=1.9829 train=0.2834 test=0.3067
[DFA] Ep 50: loss=1.9822 train=0.2800 test=0.2991
[DFA] Ep 60: loss=1.9784 train=0.2827 test=0.2979
[DFA] Ep 70: loss=1.9806 train=0.2869 test=0.3041
[DFA] Ep 80: loss=1.9778 train=0.2845 test=0.3078
[DFA] Ep 90: loss=1.9771 train=0.2875 test=0.3110
[DFA] Ep 100: loss=1.9766 train=0.2863 test=0.3092
Done in 1075s: acc=0.3092 Gamma=0.1012 rho=0.0045
d=512, L=4, seed=456, method=credit_bridge
[CB] Ep 1 (warmup): loss=2.0472 train=0.2499 test=0.2680 vloss=0.491338
[CB] Ep 10 (warmup): loss=1.9976 train=0.2715 test=0.2905 vloss=0.067081
[CB] Ep 20 (warmup): loss=1.9967 train=0.2732 test=0.2852 vloss=0.044847
[CB] Ep 30 (blend=0.50): loss=1.9887 train=0.2796 test=0.2918 vloss=0.025167
[CB] Ep 40 (blend=1.00): loss=1.9825 train=0.2789 test=0.3019 vloss=0.023611
[CB] Ep 50 (blend=1.00): loss=1.9632 train=0.2860 test=0.3026 vloss=0.028265
[CB] Ep 60 (blend=1.00): loss=1.9739 train=0.2757 test=0.2977 vloss=0.021634
[CB] Ep 70 (blend=1.00): loss=1.9594 train=0.2854 test=0.2976 vloss=0.011156
[CB] Ep 80 (blend=1.00): loss=1.9557 train=0.2895 test=0.3066 vloss=0.004731
[CB] Ep 90 (blend=1.00): loss=1.9549 train=0.2912 test=0.3087 vloss=0.001627
[CB] Ep 100 (blend=1.00): loss=1.9500 train=0.2922 test=0.3082 vloss=0.000972
Done in 8663s: acc=0.3082 Gamma=0.1447 rho=0.0076
Summary saved to results/cifar_depth_scan_multiseed/summary.json
==========================================================================================
CIFAR-10 DEPTH SCAN SUMMARY
==========================================================================================
Config Method Acc Gamma rho nudge
------------------------------------------------------------------------------------------
d512_L4_s123 credit_bridge 0.3091 0.1319 0.0010 -0.000002
d512_L4_s123 dfa 0.3094 0.1070 0.0024 -0.000001
d512_L4_s456 credit_bridge 0.3082 0.1447 0.0076 -0.000002
d512_L4_s456 dfa 0.3092 0.1012 0.0045 -0.000001
|
