blob: 083176b783c15c930a2a6e58f4c489dc17140f9c (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
|
========================================================================================
DIAGNOSTIC (a) per-block growth: sensitivity over threshold
========================================================================================
method seed value >5× >10× >20× >50× >100× >500× >1000× >5000×
bp 42 1.00e+00 ok ok ok ok ok ok ok ok
dfa 42 2.04e+03 FIRE FIRE FIRE FIRE FIRE FIRE FIRE ok
state_bridge 42 1.28e+04 FIRE FIRE FIRE FIRE FIRE FIRE FIRE FIRE
credit_bridge 42 1.82e+03 FIRE FIRE FIRE FIRE FIRE FIRE FIRE ok
ep 42 2.87e+00 ok ok ok ok ok ok ok ok
bp 123 9.59e-01 ok ok ok ok ok ok ok ok
dfa 123 9.78e+02 FIRE FIRE FIRE FIRE FIRE FIRE ok ok
state_bridge 123 2.41e+04 FIRE FIRE FIRE FIRE FIRE FIRE FIRE FIRE
credit_bridge 123 6.94e+02 FIRE FIRE FIRE FIRE FIRE FIRE ok ok
ep 123 1.10e+01 FIRE FIRE ok ok ok ok ok ok
bp 456 9.63e-01 ok ok ok ok ok ok ok ok
dfa 456 2.55e+03 FIRE FIRE FIRE FIRE FIRE FIRE FIRE ok
state_bridge 456 1.05e+04 FIRE FIRE FIRE FIRE FIRE FIRE FIRE FIRE
credit_bridge 456 1.03e+03 FIRE FIRE FIRE FIRE FIRE FIRE FIRE ok
ep 456 6.10e+00 FIRE ok ok ok ok ok ok ok
Reading: BP/EP rows should be 'ok' across the entire row (the whole
threshold range is healthy for them). DFA/SB/CB rows should be 'FIRE'
at the chosen threshold and have a comfortable margin on either side.
========================================================================================
DIAGNOSTIC (b) g-norm floor: sensitivity over threshold
========================================================================================
method seed value <1e-09 <1e-08 <1e-07 <1e-06 <1e-05
bp 42 3.70e-04 ok ok ok ok ok
dfa 42 4.17e-09 ok FIRE FIRE FIRE FIRE
state_bridge 42 1.84e-09 ok FIRE FIRE FIRE FIRE
credit_bridge 42 9.01e-10 FIRE FIRE FIRE FIRE FIRE
ep 42 1.64e-04 ok ok ok ok ok
bp 123 3.09e-04 ok ok ok ok ok
dfa 123 2.84e-09 ok FIRE FIRE FIRE FIRE
state_bridge 123 2.24e-09 ok FIRE FIRE FIRE FIRE
credit_bridge 123 4.18e-09 ok FIRE FIRE FIRE FIRE
ep 123 1.02e-04 ok ok ok ok ok
bp 456 4.02e-04 ok ok ok ok ok
dfa 456 1.90e-09 ok FIRE FIRE FIRE FIRE
state_bridge 456 2.40e-09 ok FIRE FIRE FIRE FIRE
credit_bridge 456 2.38e-09 ok FIRE FIRE FIRE FIRE
ep 456 1.16e-04 ok ok ok ok ok
========================================================================================
DIAGNOSTIC (c) stability ceiling: sensitivity over threshold
========================================================================================
method seed value >0.10 >0.20 >0.30 >0.50 >0.70 >0.90
bp 42 0.099 ok ok ok ok ok ok
dfa 42 0.047 ok ok ok ok ok ok
state_bridge 42 0.992 FIRE FIRE FIRE FIRE FIRE FIRE
credit_bridge 42 0.352 FIRE FIRE FIRE ok ok ok
ep 42 -0.036 ok ok ok ok ok ok
bp 123 0.087 ok ok ok ok ok ok
dfa 123 0.436 FIRE FIRE FIRE ok ok ok
state_bridge 123 0.561 FIRE FIRE FIRE FIRE ok ok
credit_bridge 123 0.250 FIRE FIRE ok ok ok ok
ep 123 0.120 FIRE ok ok ok ok ok
bp 456 0.114 FIRE ok ok ok ok ok
dfa 456 -0.005 ok ok ok ok ok ok
state_bridge 456 0.035 ok ok ok ok ok ok
credit_bridge 456 0.518 FIRE FIRE FIRE FIRE ok ok
ep 456 -0.024 ok ok ok ok ok ok
========================================================================================
VERDICT ROBUSTNESS: at what threshold does each verdict CHANGE?
========================================================================================
method seed (a) value (a) flip near (b) value (b) flip near
bp 42 1.00e+00 1.00e+00 3.70e-04 3.70e-04
dfa 42 2.04e+03 2.04e+03 4.17e-09 4.17e-09
state_bridge 42 1.28e+04 1.28e+04 1.84e-09 1.84e-09
credit_bridge 42 1.82e+03 1.82e+03 9.01e-10 9.01e-10
ep 42 2.87e+00 2.87e+00 1.64e-04 1.64e-04
bp 123 9.59e-01 9.59e-01 3.09e-04 3.09e-04
dfa 123 9.78e+02 9.78e+02 2.84e-09 2.84e-09
state_bridge 123 2.41e+04 2.41e+04 2.24e-09 2.24e-09
credit_bridge 123 6.94e+02 6.94e+02 4.18e-09 4.18e-09
ep 123 1.10e+01 1.10e+01 1.02e-04 1.02e-04
bp 456 9.63e-01 9.63e-01 4.02e-04 4.02e-04
dfa 456 2.55e+03 2.55e+03 1.90e-09 1.90e-09
state_bridge 456 1.05e+04 1.05e+04 2.40e-09 2.40e-09
credit_bridge 456 1.03e+03 1.03e+03 2.38e-09 2.38e-09
ep 456 6.10e+00 6.10e+00 1.16e-04 1.16e-04
Interpretation:
- For DFA/SB/CB, the 'flip near' values are the diagnostic raw values.
Default (a) threshold 50 catches all if raw values > 50; default (b)
threshold 1e-7 catches all if raw values < 1e-7. Compare:
(a) max BP per-block growth across 3 seeds: 1.00e+00
(a) max EP per-block growth across 3 seeds: 1.10e+01
(a) min DFA per-block growth across 3 seeds: 9.78e+02
(a) min SB per-block growth across 3 seeds: 1.05e+04
(a) min CB per-block growth across 3 seeds: 6.94e+02
-> separation gap: healthy max = 1.10e+01,
degenerate min = 6.94e+02,
gap factor = 63×
(b) min BP ‖g_L‖ across 3 seeds: 3.09e-04
(b) min EP ‖g_L‖ across 3 seeds: 1.02e-04
(b) max DFA ‖g_L‖ across 3 seeds: 4.17e-09
(b) max SB ‖g_L‖ across 3 seeds: 2.40e-09
(b) max CB ‖g_L‖ across 3 seeds: 4.18e-09
-> separation gap: healthy min = 1.02e-04,
degenerate max = 4.18e-09,
gap factor = 24338×
|
