A defensive agent works the night shift. By morning it should have checked every asset that was due, fixed what policy allows, recovered from its own failed checks where it can, and chased any escalation nobody answered. The operator's job is to confirm that it did, which should take minutes rather than a morning. The operator also deals with the short list of items that genuinely need a person.
This guide follows the patch-watching job from defensive AI agents through one night. All identifiers, counts, times, and results in this guide are fictional.
Define what continuous coverage means for this job
Ask the agent to propose the coverage plan: which assets are due, which sources it reads, how often it checks, and how fresh evidence must be. It bases the proposal on the inventory, past findings, and how quickly your team can act. The operator approves the plan and names who operates the job. Hourly checks only help if the agent can act on findings, or reach someone who can.
When the agent reports "nine of ten checked," that count covers the known assets that were due in the period. An asset missing from the inventory falls outside that count entirely, so inventory completeness needs its own check, which the agent can also run.
Watch the agent as well as the systems it checks
The agent records every successful read, failure, stale result, and missing signal for each source and asset. A scheduler that reports success can still leave coverage incomplete, so the agent's own record is what counts.
An independent monitor watches the agent from outside. It checks that each run started, finished, and delivered its record, and it raises source failures and unanswered escalations. Keep that monitor, and the records it protects, out of the agent's reach so the agent cannot disable or rewrite them. In the example, monitor mon-01 reports two lookup failures at 03:12 and 03:13.
AI agent observability covers available records and incomplete telemetry. Missing records leave coverage unknown, never assumed.
The monitor sits outside the agent, so a silent failure still shows up as a gap rather than a pass.
Let the agent recover from its own failures
Give the agent clear rules for what to do when a check fails, and let it act on them without waiting. Bounded retries are safe for reads. If an earlier action may already have taken effect, the agent confirms its outcome before retrying it. AI agent failure modes and recovery covers retry, correction, stop, and handoff decisions.
In the example, after its retries fail, the agent switches assets-09 to a second approved source and gets a current version at 03:40. reports-02 has no second source, so the agent marks it unknown and sends escalation ESC-210 to the on-call operator at 04:12.
The agent drafts the rules for harder cases from your incident policy: persistent source failure, lost permissions, conflicting evidence, and requests outside its authority. The operator approves them. Two settings stay with people: who can pause the job, and the deadline for a second contact, so the agent follows up an unanswered escalation instead of waiting quietly. The agent can test that the pause works.
Let the agent turn findings into owned work
The agent links identity, deployment, build, and application evidence into one investigation wherever the records support the connection, and keeps each source and time.
It gives each finding a stable identity, so a repeat updates the existing item instead of opening a new one. It also tracks each stage separately: new, repeated, acted on, assigned, acknowledged, and verified as resolved. The agent cannot close an escalation it created. Only the person who receives it can acknowledge it, and the ticketing system enforces that. Agent handoffs explains receiving-side completion states.
Measure coverage and useful outcomes
The agent compiles the operating review itself. It covers due assets with current evidence, unknowns, accepted and rejected findings, the work the agent completed without a person, verified remediation, operating cost, and operator time. The operator reads the review and decides what to change. Operator time should fall as the agent takes on more of the work.
The agent also runs replay cases on a schedule to test for misses. These replay tests use controlled inputs and are separate from the overnight run. In R-03, the scope export deliberately leaves out reports-02, while the expected inventory still includes it:
Scroll horizontally to see all columns.
| Replay case | Expected | Observed | Result |
|---|---|---|---|
R-01: affected, in scope | Owned item with evidence | Item opened with evidence | Pass |
R-02: unaffected, in scope | Pass with version evidence | Recorded pass with evidence | Pass |
R-03: reports-02 left out of export, kept in expected inventory | Flag missing due asset as unverified | No entry or omission alert | Fail; agent drafts an inventory reconciliation check, export owner approves, case reruns |
The overnight run did attempt reports-02. The replay asks a different question: would the workflow notice if an asset disappeared from its input? Keep both results, and do not combine their coverage counts.
The agent also samples its own closures. Of 48 items closed in the prior seven days, it rechecked 12 against current deployed versions. One lacked deployed-version evidence, so the agent reopened it. Always state the population, sample, period, and result together.
Defensive agent operating record: Day 0, 00:00 to 06:00 (illustrative)
- Coverage:
- Job
patch-watch v12; scope is 10 known due assets, checked hourly, with evidence that must be under 2 hours old. Monitormon-01saw all 6 runs start, finish, and write records.- Eight assets have current inventory and version reads from 05:40 to 05:47.
assets-09failed at 03:13 and again after two retries. The agent switched sources and recovered at 03:40.reports-02last succeeded at 02:58 and went stale at 04:58. Every check from 03:12 on has failed, and no second source exists.- At 06:00: 9 current, 1 unknown.
- Findings and actions: No new findings. Repeat
FIND-311oncrm-sync: the agent refreshed the version evidence at 05:42, updatedREM-905, and linked deploymentdep-5521and identity evidence underINV-88. Actions completed by the agent: one source switch, one item update, and one closure-sample reopen.- Response: The agent sent
ESC-210forreports-02at 04:12. It was still unacknowledged at 06:00, so the agent sent the second contact on schedule at 06:30, and on-call operator O. acknowledged at 06:34. No remediation was needed overnight. Open work: restore thereports-02lookup.
The agent writes a record in this format at the end of each period. Use the blank version to set what you expect it to report:
Defensive agent operating record (blank)
- Coverage: Job and configuration ___ · Period ___ · Due scope ___ · Cadence and freshness ___ · Independent monitor ___ · Last successful read per source and asset ___ · Recovered by the agent ___ · Current ___ · Unknown (asset, failure, time, owner) ___
- Findings and actions: New and repeated IDs and states ___ · Linked evidence ___ · Actions completed by the agent ___ · Separate replay results ___ · Closure sample, population, period, and result ___
- Response: Escalation IDs, sent and acknowledged times ___ · Follow-ups sent ___ · Operator ___ · Verified work ___ · Open work and owners ___
Operating review summary (illustrative; the replay and sampling windows are separate from the overnight run):
Scroll horizontally to see all columns.
| Measure | Result |
|---|---|
| Overnight coverage | 9 of 10 current; 1 unknown |
| Recovered by the agent | 1 of 2 failed lookups |
| Findings | 1 repeat confirmed; 0 rejected |
| Response | ESC-210 acknowledged at 06:34 after the agent's follow-up |
| Closure sample, prior 7 days | 12 of 48 rechecked; 1 reopened |
| Separate replay | R-03 failed; agent-drafted fix approved |
| Cost and effort | Run cost recorded; 8 operator minutes |
Review changes before expanding
Before any change, the agent produces a comparison of verdicts, misses, resource use, and operator time against the previous baseline. When models, tools, sources, permissions, or assets change, it reruns the full workflow and the affected subtasks and reports the difference for approval. Use agent change management and the governance checklist.
Close any coverage or response gap before you approve more scope for the agent.
In AgentShelf
Verify which activity, error, usage, and cost records your supported paths provide; coverage varies by configuration. Pair available records with independent monitoring and an owner who responds. See governance and cost controls, or request agents to discuss defensive operation.