The AI Agent Reliability Retainer — the audit that catches the lie your dashboard can't

The problem, in one story

Agent fleets fail quietly. I watched a scheduled job report success five runs in a row — exit code 0, dashboard green — while the process it was supposed to run had been dead for four days. The lie wasn't in the bot; it was in the wrapper: a log handle inherited by a background server bricked every future run in under a second, and the success ledger got written anyway. No exit code caught it. No heartbeat caught it. In one week on my own fleet I hit three flavors of this: that four-day green-but-dead wrapper, a trading bot that crash-looped 187 times in a single day because its restarter faithfully relaunched a process an exhausted API quota kept killing, and a pipeline that ran "healthy" for five weeks on default inputs because its real data sources had silently died — and its own graceful degradation was designed to hide exactly that. A green dashboard proves the process returned. It never proves the work happened.

What I built

The Reliability Retainer's Week-1 Audit is a read-only sweep of a client's scheduled tasks and wrappers against a hazard catalog — every finding gets a file:line, a severity, and a hazard class. The engineering that makes that sellable standalone, not just a plausible pitch:

• 6 universal failure principles, not one platform's dialect — lying-green (PR-1), split-brain logging (PR-2), authority-vs-need (PR-3), fail-open-vs-fail-closed (PR-4), missing freshness asserts (PR-5), unbounded retry/restart (PR-6). Each principle cross-references its tooled check ID and its per-platform mechanism appendix (Windows, Linux/cron/systemd, cloud/IaC, CI/CD). tools/hazard_catalog.md
• A portable static-check layer — tools/portable_checks.py: six language-aware heuristics (UC-1..UC-6) over bash/python/node/yaml/hcl, plus tools/discovery.py enumerating cron, systemd timers, GitHub Actions, and Windows Task Scheduler CSV exports. Every finding is honestly labelled confidence: "tool-flagged" — the tool can never emit "expert-confirmed" itself; that promotion is a structural, code-enforced honesty boundary (spec §5), not a documentation promise.
• A zero-effector rail, test-enforced — the audit layer only reads a list of text lines and returns findings. It never writes to, deletes from, executes, or networks against a scanned target. That's not a claim in a pitch deck; it's asserted by tests/test_portable_checks.py::test_portable_checks_module_has_no_effector_shaped_symbol.
• An 8-section client-facing report generator — tools/report.py and tools/audit_report_gen.py turn the findings into the actual A1-A4 deliverables (findings table, live-evidence appendix, ranked fix-lane plan, hazard-catalog cross-reference), not a hand-written summary.
• The Month-1 status surface, built and wired — fleet-health-mcp/: a config-driven MCP (fleet_status / bot_status / recent_alerts / dump_markdown_report) that distinguishes "process alive" from "actually producing" from "inputs still fresh" — the exact distinction a lying-green wrapper erases. It's wired out of the box to this repo's own toy fleet, not a diagram.

The self-dogfood proof: I found this failure mode on my own fleet first

This offer didn't start as a market hypothesis — it started as an incident. The fleet-reliability-day case study walks through the same day a monitoring dashboard reported one of my own trading bots as dead, and a diagnosis lane wouldn't take the dashboard's word for it: it cross-referenced the bot's beacon file, its session logs, and Windows' own boot-time record, and found two independent bugs layered on top of each other — a disabled scheduled task and a monitor watching the wrong task name. Three earlier manual restarts had each produced a silent 0-byte log because nobody was watching the right thing. The bot came back after eight days dark, closed the same day it was found — not restarted on hope, root-caused from evidence. That's the Week-1 Audit's whole thesis, applied to my own stack before it was ever pitched to anyone else's.

What it shows about how I work

Anyone can promise a reliability audit. The honest version of that promise is a tool with a code-enforced boundary on what it's allowed to touch, a sample report you can regenerate yourself and get the identical two findings at the identical two lines, and an incident history on my own infrastructure that predates the pitch. That's the same standard behind a client engagement: no finding without a file:line, no severity without live evidence, no "trust the dashboard."

Book a scoping call  ← The incident that motivated this