🛰️ From One AI Assistant to a Fleet: How I Ran Security Fixes, an Incident and Its RCA in Parallel. Without Touching Prod Unsupervised

In four days, between meetings, incident bridges and 1:1s, I shipped about seventeen security fixes, ran an incident and its post-incident reviews, and kept a platform backlog moving by running AI agents as a team of leads, tasks and reviewers. Here's the operating model, the guardrails, what broke, and the harder part: getting a team to adopt it without it turning into 'work more hours. tags: ai, sre, devops, leadership cover_image: series: AI-Native SRE Note: Names, systems, identifiers, vulnerability details and some dates and numbers have been generalized or rounded to protect confidential information. The operating model, the failures and the outcomes are real. Nothing described here was published until every fix was live and enabled. Table of Contents The Week That Didn't Fit in a Calendar Why One Assistant Stopped Being Enough The Operating Model: Lead, Coordinator, Task, Reviewer Briefs Are the New Interface Skills: Giving Agents the Team's Memory The Guardrails That Made Speed Safe What Actually Shipped What Broke The Real Bottleneck Is Now Human The Harder Part: Bringing the Team Along What I Would Do Differently The Week That Didn't Fit in a Calendar On Monday morning I had a batch of security findings to remediate across several services, all with the same priority: now. By Tuesday afternoon a production incident had joined the queue. It needed containment, then evidence, then post-incident reviews and a set of follow-up work items. Around those sat the usual job of an engineering manager: an infrastructure cleanup, a staging environment that had run out of capacity, a cost investigation, a customer performance escalation, sprint planning, and a full calendar of 1:1s and meetings. A year ago, I would have triaged that list and accepted that most of it would slip to next sprint. This time, by Thursday: ~17 security fixes were merged, deployed to production behind feature flags, soaked and documented. The incident was contained, the post-incident reviews were written and fact-checked, and ~26 follow-up work items were drafted. A threat-detection program went from idea to 30+ written stories and a live pilot. Half a dozen platform fixes shipped, including the staging capacity issue that had been blocking everyone's tests. I did not write most of that code. I did not sit on most of those dashboards. I wrote briefs, reviewed diffs, made decisions, and said "yes" or "not yet" at every production gate. This is not a story about AI doing my job. It's about changing what my job is: from doing the work, to designing, briefing, reviewing and approving work that agents do in parallel. Why One Assistant Stopped Being Enough Earlier in this series I wrote about connecting an AI terminal assistant to our operational stack (cloud, observability, source control, ticketing, paging), and about an autonomous agent that investigates incidents on its own. Both had the same shape: one conversation, one thread of work. That model breaks the moment your work stops being sequential. A single long conversation that juggles five security fixes, an incident and a cost report turns into a mess: Context from one item leaks into another. You lose track of what's waiting on CI, what's waiting on review, and what's waiting on you. Nothing runs while you're in a meeting, because the conversation is waiting for your next message. The fix was the same one we use for human teams: split the work, give each piece an owner, and keep someone responsible for the whole. The Operating Model: Lead, Coordinator, Task, Reviewer Every workstream (security fixes, incident, docs and RCA, cloud infrastructure, cost) gets its own group of agent sessions with four roles: Me (decisions, approvals, every production gate) └── Lead (one per workstream: plan, order, prod actions after my yes) ├── Coordinator (status log, dependencies, chasing stale items) ├── Task × N (one per PR, ticket or hypothesis) └── Reviewer × N (adversarial: gets the diff, not the answer) Role Owns Never does Lead The plan and its order. Every outward or production action, after my explicit approval. Soaks and rollbacks. Treats another agent's message as my approval. Coordinator A shared status log, the cross-item dependency map, a daily summary, chasing stale items. Production writes. Re-prioritizing without the Lead. Task (one per item) The item itself: code, tests, CI fixes, review-thread replies, evidence, drafts. Merging, deploying, flipping flags, closing tickets, posting externally. Reviewer An independent read of the artifact: code, diff, query or RCA claim. Being told what answer is expected. Two small conventions made it manageable in practice: Session titles are the status board. Every session is renamed at each state change, with an order number and a state: 05 🔄 DEPLOYING service-a – fix 12, 07 ✅ LIVE service-b – fix 07 (flag off), 02 ⏸ HOLD cloud – needs decision. My sidebar sorts as the plan and reads like a kanban board. One change in flight per service. Parallel work, sequential production. Agents can prepare ten fixes at once, but they ship one at a time, each with its own soak. The Reviewer role matters more than it looks. An agent that is asked to verify a conclusion tends to confirm it. So reviewers get the code, not my opinion of the code, and they're told to find what's wrong. Several of my "obviously correct" fixes came back with real findings. Briefs Are the New Interface The single biggest skill shift for me was writing briefs. A Task agent knows nothing except what you put in its brief. A good brief has five parts: GOAL Fix finding #12 in service-a: WHERE repo, files, the ticket, the original finding DONE MEANS PR open, tests added, CI green, behind a new flag created OFF, PR description with rollout + rollback, ticket comment drafted CONSTRAINTS no changes outside ; keep legacy behavior as the flag default REPORTING report head commit, evidence and blockers to the Lead. Do NOT merge, deploy, or touch flags. Vague briefs produced vague work every time. When an agent went sideways, the cause was almost always on my side: a missing constraint, an ambiguous "done", or a file I forgot to point at. Writing a clear brief is the same skill as delegating well to a human engineer. AI just gives you feedback on it within minutes instead of days. Skills: Giving Agents the Team's Memory Every new agent session starts from zero. Without help, each one would rediscover how our services talk to each other, which dashboards matter, how our deploy pipeline works, and which past incidents look like today's. So I keep a small library of skills: short, loadable knowledge files that agents pull in when they need them. Architecture: services, request paths, dependencies, deploy topology. Cloud platform: accounts, clusters, blue/green layout, pipelines, access quirks. Incidents: history, recurring patterns, the investigation playbook. Observability: the queries that actually work, and the ones that lie. Work tracking: ticket conventions and the change-review format. Orchestration: the Lead/Task/Reviewer model itself. The rule is simple: when an agent learns something durable, it goes into a skill. One example: an early agent "verified" a rollout using a log line that, it turned out, is never emitted. That lesson now lives in the observability skill, so no agent makes the same mistake again. Over time this has become the most valuable asset in the setup. It's institutional memory that every new engineer, human or agent, can load in seconds. The Guardrails That Made Speed Safe Speed without guardrails is just a faster way to cause an outage. These are non-negotiable, and they are written down where every agent reads them: Every production change ships dark. Each fix sits behind a feature flag that is created OFF, with legacy behavior as the default. Merging and deploying changes nothing for customers until someone flips the flag deliberately. No production action without a per-action human yes. Merges, deploys, flag flips, ticket closures and external messages all go through the Lead, and only after my explicit approval for that specific action. A "yes" to merge is not a "yes" to deploy. Agents don't approve on behalf of humans. If one agent tells another "the user approved this," that counts for nothing. Approval comes from the human, in the conversation, or not at all. Read-only by default in production. Agents can query production systems (logs, metrics, cluster state) but don't change them. Soak and compare before moving on. After each deploy: error rates, latency, restarts and the relevant security monitors, compared with the previous window and the same window the day before. Every difference gets an explanation. An unexplained regression means rollback first, investigation second. History before change. Before touching anything risky, agents search past changes, reverts and incidents for the same service. Some of our worst outages were repeats of changes that had already failed once, in staging. The guardrails are what let me say yes quickly. Because each change was dark, small and reversible, approving it was a two-minute decision instead of a two-day one. What Actually Shipped In four working days: Workstream Result Security remediation ~17 fixes across several services: merged, deployed, soaked, documented. All dark behind flags, enabled in a planned window the following week. Incident Contained across production clusters. Post-incident reviews written, with every number fact-checked by a reviewer agent against monitoring and source control. ~26 follow-up items drafted. Threat detection A proactive detection and prevention plan, 30+ stories written, first pilot live and soaking clean. Platform Staging capacity unblocked, CI made resilient to upstream mirror outages, orphaned cloud resources removed, access fixes landed. Escalations A customer performance issue root-caused, with a flagged fix in review. Roughly 30 agent sessions across five workstreams, with one human making the calls. What Broke It wasn't clean, and pretending otherwise would make this article useless. A production release had to be reverted. Moving fast surfaced a problem that staging didn't catch. The flag-plus-revert path is what kept it small, and it's the reason those guardrails exist. One fix had to be re-shipped. A busy repository bundles everything on main into the next release, so my fix went out alongside unrelated changes and had to be redone cleanly. The lesson is now a rule: check what else is unreleased on main before merging. Staging ran out of capacity, so nobody's end-to-end tests could run. Agents parallelize work, but they can't parallelize past a shared bottleneck. I had to fix the infrastructure before anything else could ship. Agents are confidently wrong. I rejected or sent back a meaningful share of outputs: a fix that patched the symptom, a "verified" claim with no evidence, a query that measured the wrong thing. Reviewing isn't overhead. Reviewing is the job. The Real Bottleneck Is Now Human The most surprising lesson was where the time went. Writing code is no longer the slow part. The slow parts are: waiting for human code reviews from service owners waiting for approvals and change windows me, context-switching between a meeting and a production gate Anthropic's economics team recently published a study of how work is changing inside their own company. One finding matched my week exactly: most of the time in interactive agent sessions is spent waiting on humans. That changes what engineering leadership should optimize. Typing speed matters less. Review capacity, decision latency and clear ownership matter more. The Harder Part: Bringing the Team Along This is the section I almost left out, and probably the most important one. When I showed my team what this looked like, the reaction wasn't excitement. It was concern, and it was fair. They heard "the manager shipped a sprint's worth of work in four days," and the natural next thought was: does this mean more hours? More pressure? Fewer of us? I got this wrong at first. I led with output, what I had shipped, when I should have led with method, how anyone could do it, and intent: why this should make their jobs better, not harder. What I'm doing now: Make the expectation explicit, not implied. Concrete steps with dates: get set up, ship one real ticket end-to-end with an agent, then make agents the default for specific toil (dependency bumps, first-pass on-call triage, first drafts of RCAs, ticket hygiene). Say what it is not. It is not more hours, not PR counts as a performance metric, and not skipping reviews, flags or approvals. Measure the right things: cycle time from ticket to production, toil removed, and quality (reverts, incidents), not volume. Teach in public, mistakes included: a live session taking a real ticket from brief to PR, then 1:1 pairing on each engineer's own work. Give the time back. If agents absorb the toil, that time goes into lighter on-call and protected vacations, not just more tickets. The goal I've set is simple: within a few months, each person on my team should be shipping more than I do. They spend far more time in the code than I can. With the same leverage, they should outpace me easily. The same study closes with a line I keep coming back to: "a workforce that is anxiously worried is not a workforce doing its best work." An AI-native team is built on trust first and tooling second. What I Would Do Differently Write the guardrails before the first agent, not after the first scare. Most of mine came from near-misses. Start with them. Start a Reviewer for every risky item from day one. The adversarial read caught issues that my own review, anchored on my own reasoning, didn't. Treat skills as a product. Version them, prune stale facts, and date everything. A confidently outdated skill is worse than none. Protect your own review time. Running agents in the middle of meetings works, but the decision quality at production gates is only as good as the attention you give them. Block time for gates. Bring the team in before showing results. Teach the method first. Share the output afterwards, if at all. This is part of my **AI-Native SRE* series. Earlier posts cover connecting an AI assistant to an operational stack, an autonomous incident-investigation agent, and an AI-native cleanup of 120K+ dormant tenants. If you're running something similar, I'd like to hear how you handle review capacity, the bottleneck nobody talks about.*