Sniper Fleet

πŸ›° Sniper Fleet

loading…

Logs β€” all boxes, live

❌ error ⚠️ warn ℹ️ info Β· debug Β· traces

Guard events (why things went down)

Legend / contract

Status is judged from each unit's heartbeat stats, not just "process alive":
🟒 up β€” heartbeating, accounts logged in, follows in place, polls succeeding
βšͺ️ starting β€” just (re)started: logging in + re-following (β‰ˆ20 min for a cold shard)
🟑 degraded β€” polling, but with a listed problem (accounts out, blind callers, error rate, thin coverage)
🟑 stalled β€” process alive, no successful poll for >2 min
πŸ”΄ blind β€” NOT detecting: 0 follows / every poll failing / 0 logged in (the 09-17 403-wall shape)
πŸ”΄ down β€” no heartbeat >90s

Each card lists why. Click a card to filter the log console to that unit; the raw link shows the full heartbeat JSON. Telegram gets one message per status change (30s of evidence before a worse verdict sticks) and a 30-min reminder while anything is not up.

Logs come from fleet-log-shipper on every box (journald β†’ Redis stream per unit, last 10k lines each). Levels are derived from the app's ⚠️/❌ prefixes + systemd priority. sniper-guard on each box auto-restarts a unit that is enabled but down; a unit stopped with disable --now is left alone. box1's sniper.service is never auto-restarted.