summaryrefslogtreecommitdiff
path: root/ep_run/alert.sh
diff options
context:
space:
mode:
authorYuren Hao <yurenh2@illinois.edu>2026-07-03 05:56:50 -0500
committerYuren Hao <yurenh2@illinois.edu>2026-07-03 05:56:50 -0500
commitb83947778e2c776f757a07d4719b7ce961d7ed55 (patch)
treeb9cc01d7adda691d9156d9d04f4fb2f644674e96 /ep_run/alert.sh
Initial commit: ept — backprop-free equilibrium transformer (EP)
Code (ep_run/), organized docs (docs/{method,campaign,hardware,outreach,paper}), analysis scripts (scripts/), ONBOARDING.md entry point. Large data/checkpoints git-ignored (share separately). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
Diffstat (limited to 'ep_run/alert.sh')
-rwxr-xr-xep_run/alert.sh13
1 files changed, 13 insertions, 0 deletions
diff --git a/ep_run/alert.sh b/ep_run/alert.sh
new file mode 100755
index 0000000..6fe5929
--- /dev/null
+++ b/ep_run/alert.sh
@@ -0,0 +1,13 @@
+#!/bin/bash
+LOG=ep_run/runs/ep_resreg_warm.log
+cd /home/yurenh2/ept
+while true; do
+ sleep 900
+ if [ -n "$(find "$LOG" -mmin +45 2>/dev/null)" ]; then echo "LOG STALE >45min (resreg_warm dead/stuck)"; break; fi
+ LAST=$(grep -E "val CE" "$LOG" | tail -1)
+ BEST=$(echo "$LAST" | grep -oE "best [0-9.]+" | grep -oE "[0-9.]+$")
+ EMA=$(echo "$LAST" | grep -oE "ema=[0-9.]+" | grep -oE "[0-9.]+$")
+ awk "BEGIN{exit !($BEST < 2.02)}" 2>/dev/null && { echo "NEW BEST <2.02 (full recovery + improvement): $LAST"; break; }
+ awk "BEGIN{exit !($EMA > 4.0)}" 2>/dev/null && { echo "RE-COLLAPSE ema>4: $LAST"; break; }
+done
+echo "FIRED: $LAST"