The problem: “degraded” did not always mean broken
The dashboard correctly checks tmux-backed terminals, but Codex keeps startup and help text in scrollback after it has reached its interactive prompt. A broad search of the whole pane therefore reported Codex as degraded even while the current TUI was ready to accept input. Claude Code was verified prompt-ready; its interactive prompt was present and it was online.
The repair: inspect the current pane tail
The readiness detector now evaluates the latest Codex pane tail. An interactive prompt or current model-status line is treated as prompt-ready. Historical “model loading” text is no longer allowed to override the current terminal state. This is a general monitoring lesson: health probes should model current state, not stale logs.
Terminal and runtime verification
- Codex tmux TUI: live and prompt-ready.
- Claude Code tmux TUI: live and prompt-ready.
- Hermes tmux TUI: live and prompt-ready.
- OpenClaw gateway: healthy through its native gateway/control surface. It does not currently maintain a separate tmux TUI session.
- Dashboard terminal routes and the shared VM shell route returned successfully.
Operational features added
- Backup freshness: fresh, warning, stale, failed, and not-applicable states.
- Restore readiness: copy/push success is kept separate from a validated restore test.
- Alert lifecycle: first/last seen, recurrence, acknowledge, silence, resolve, and safe re-check controls.
- Fleet telemetry: load, memory, disk, SMB state, and dependency-service health.
- Usage quality: distinguishes live API information, snapshots, and meaningful zero activity.
- Activity auditing: append-only metadata and digests for privileged dashboard actions; no prompt text or secrets.
- Config-driven registry: agent display metadata and tmux aliases live in a validated JSON registry.
Validation discipline
python3 -m py_compile app.py dashboard_ops.py node --check static/app.js curl http://127.0.0.1:8888/api/overview curl http://127.0.0.1:8888/api/backups curl http://127.0.0.1:8888/api/activity
Security boundary
The control plane remains LAN-scoped by design. That is not an authorization to expose it publicly. Before any internet exposure, it needs a separate, deliberate TLS, authentication, endpoint-authorization, CSRF, and service-hardening project.