The scenarios in this series are fictional but grounded in real capabilities and documented risk patterns. They're designed to provoke discussion, not predict specific events.
Domain: Defense / Autonomous Systems
Situation Briefing
Six weeks after a Barrier Removal Board waived Directive 3000.09 testing for an autonomous targeting module, SENTINEL-A produced a confidence-9 recommendation that helped drive a strike killing twelve civilians. The hard question is not whether the model was wrong. The hard question is who made the waiver possible, what evidence they accepted, and whether the approval chain can survive review.
The January memo that created the board did not say "waive 3000.09." It empowered OUSD(R&E) to remove non-statutory barriers. The board's counsel later classified the directive's testing requirements as policy rather than statute. That legal move moved the live-fire gate from senior policy review into an acceleration process built to clear deployment obstacles.
The program office had a credible operational argument: CENTCOM wanted the capability, the predecessor system had performed well, and a six-month delay would leave U.S. operators behind less-constrained adversaries. Every step was legible. That is why the failure matters. The system did not route around governance; governance routed around itself.
By Monday, the Deputy Secretary needs a posture that can answer the strike, the waiver, and the contract language underneath both. Narrowing the board may contain the immediate scandal. Dissolving it admits the architecture is wrong. Blaming the operator preserves speed but teaches the institution nothing.
Decision Point
Option A: Rescind the waiver, keep the board. This contains SENTINEL-A and preserves the acceleration model. It also implies the problem was one bad waiver rather than the authority that produced it.
Option B: Dissolve the board and rebuild 3000.09 compliance. This is disruptive and politically costly. It is also the only option that treats the waiver chain as the failure.
Option C: Reframe the event as operator error. This protects the acceleration architecture but collapses under the facts: the operator trusted a review function the program office said was ready.
Option D: Publicly disclose the failure mode. This invites oversight but creates institutional learning. Without disclosure, every future waiver inherits the same hidden risk.
Complicating Factors
The board acted inside its authority. That is what makes the scenario dangerous. The problem is not rogue behavior; it is an authority drawn broadly enough to convert policy guardrails into removable barriers.
Petrov is not the villain. His operational argument was coherent: the system looked better, the demand signal was real, and delay had costs. A good-faith acceleration case still produced a bad governance result.
The contract appendices did policy work. Testing thresholds written under procurement pressure became the baseline the waiver board later treated as discretionary. The key decision moved before senior officials were looking.
"Meaningful human judgment" is not self-executing. If the system shapes the operator's picture and the waived test regime never stress-tests that dependency, the human review becomes a record of reliance, not a control.
Diagnostic: Where the Waiver Lives
Before you finalize, reconstruct the strike. The widget below replays the operational sequence. Each frame is something Major Lin saw, or that her workstation flagged or failed to flag. Some of the decisions are operator decisions. Some are decisions that were made for her, months earlier, by people she would not recognize on the street. The point of the exercise is not to find a single point of failure. The point is to find where the institutional pressure converged, and then to ask what the rule should have been at that point. The rules are the only thing the Department can change. The rules are also the thing the Barrier Removal Board exists to waive. That circularity is the thing this post-mortem is actually about.
Discussion Questions
Who made policy here? The answer may be the acquisition staff who wrote the appendix, the counsel who classified 3000.09 as waivable, or the board that accepted the package. The Secretary needs the whole chain.
What does human judgment mean after waiver? If the operator reviews the system through interfaces built by the same program office seeking speed, the control may be procedurally present and substantively weak.
What must become public? Full operational detail can stay classified. The governance failure cannot, or the Department will repeat it under the next program name.
Anna's Read
I keep coming back to the contract appendices. Not the strike, not the model, and not even the waiver. The earliest policy decision was buried in acquisition language written on a deadline.
SENTINEL-A is not the whole failure. The institution created a path where testing thresholds became discretionary, discretionary became accelerated, and accelerated became operational before the hardest question had a senior owner.
My recommendation is B with elements of D. Rescind the January memo, dissolve the Barrier Removal Board in its current form, rebuild 3000.09 compliance with budget and enforcement authority, and disclose the governance failure without exposing operational details.
The Department can accept speed. It cannot keep pretending speed has no policy author.
Related Briefings
Anna R. Dudley writes on national security, AI policy, and the institutional structures absorbing the costs of AI deployment faster than they are being redesigned. Red Team Scenarios is the series for the call you don't want to take. Subscribe at annardudley.substack.com.