Legend / contract
Status is judged from each unit's heartbeat stats, not just "process alive":
π’ up β heartbeating, accounts logged in, follows in place, polls succeeding
βͺοΈ starting β just (re)started: logging in + re-following (β20 min for a cold shard)
π‘ degraded β polling, but with a listed problem (accounts out, blind callers, error rate, thin coverage)
π‘ stalled β process alive, no successful poll for >2 min
π΄ blind β NOT detecting: 0 follows / every poll failing / 0 logged in (the 09-17 403-wall shape)
π΄ down β no heartbeat >90s
Each card lists why. Click a card to filter the log console to that unit; the raw link
shows the full heartbeat JSON. Telegram gets one message per status change (30s of evidence
before a worse verdict sticks) and a 30-min reminder while anything is not up.
Logs come from fleet-log-shipper on every box (journald β Redis stream per unit,
last 10k lines each). Levels are derived from the app's β οΈ/β prefixes + systemd priority.
sniper-guard on each box auto-restarts a unit that is enabled but down; a unit stopped with
disable --now is left alone. box1's sniper.service is never auto-restarted.