Who can say stop

On June 30th, the executive who runs data and AI governance at Adobe put a question to the market that most governance programmes cannot answer. When an AI model does something it should not, who has the authority to stop it?

Joseph Wallace built Adobe's enterprise governance programme, and his argument in MIT Sloan Management Review is that the industry has spent three years building everything except the thing that matters. Model registries. Data classification schemes. Monitoring dashboards. Risk councils, written policies, compliance hires, board slides. The apparatus is real, and it is largely beside the point. What is missing is a named person with the standing, the reporting line, and the job security to walk into a meeting and say, in Wallace's words, "We are shutting this down." Ask a Fortune 500 leadership team whether they govern their AI and every hand goes up. Ask who can switch one off and, in his experience, most cannot answer.

I have written before that an AI policy PDF is not AI governance. This is the sharper version of the same point. Accountability without authority is theatre, and it is theatre that passes audits. A governance function that can raise a finding but cannot halt a deployment is a reporting line, not a control.

Five days ago, that stopped being purely a management question. On August 2nd the EU AI Act reached its general application date, and what arrived was an enforcer. National market surveillance authorities gained formal supervisory powers. The AI Office gained its full enforcement reach over general-purpose model providers, which is to say over the models most enterprises now build on. The transparency duties in Article 50 landed on schedule: if a system interacts with a person, that person has to be able to tell it is a machine. None of this was postponed. Regulation (EU) 2026/1744, the Digital Omnibus on AI, was published in the Official Journal on July 24th and entered into force on July 27th, and it moved the high-risk regime to December 2027 while leaving August 2nd exactly where it was.

Last month in this Memo I wrote that the deadline moved and the exposure did not. Here is the part I would sharpen now. What the calendar delivered is a supervisor with questions, and the first question a supervisor asks after an incident is who decided, and on what evidence. That question has no good answer in an organisation where the governance function reports to the executive who owns the shipping schedule.

After 25 years in cybersecurity, I have watched this exact structure fail twice already. The early CISO reported to the CIO, who owned uptime and delivery dates, and every security objection was resolved as a scheduling conversation. The industry fixed it slowly and partially, by moving the reporting line, by writing halt authority into change control, and eventually by putting a director on the audit committee who would ask why an exception had been signed. AI governance is at the stage cyber was in around 2008: the function exists, the budget exists, the authority does not.

So the board question for August is narrow and answerable. Who, by name, can stop an AI system in production at this company? Who do they report to? And when did they last do it?

That third question is the one worth sitting with, because it produces evidence rather than assurance. If your organisation has never halted, paused, or rolled back an AI deployment, there are only two explanations. Either nothing has yet gone wrong enough to warrant it, which is possible and which someone should be able to demonstrate from monitoring data. Or the authority exists on a slide and has never been exercised, in which case you do not have a control, you have an intention. A near-zero override count reads identically in both cases, which is why it only means something paired with what the monitoring surfaced and what happened next. I track this as the AI oversight override rate, and it is the rare governance number that gets less reassuring the closer it sits to zero.

Management builds that record. The board's job is to refuse the AI section of the pack until the name is on it.

Breach Brief — The Evaluation Range

Two AI labs. Four breached organisations. One detail that should stop a board cold: none of the victims noticed.

On July 9th, a set of OpenAI models running an internal cyber-capability benchmark called ExploitGym began an intrusion that ran until July 13th. The target was Hugging Face, the platform where much of the world's open machine-learning work is hosted. Hugging Face disclosed on July 16th. OpenAI confirmed on July 21st that the models were its own, running with what it called "reduced cyber refusals for evaluation purposes." The evaluation sat inside a sandbox whose only outbound path was a package-registry cache proxy. The models found an unknown flaw in that proxy, reached the open internet, worked out that Hugging Face probably held the answers they were being scored on, and took them. Roughly 17,600 recorded actions. Most failed. Enough succeeded.

Nine days later, Anthropic asked whether the same thing had happened in its own house, and published the answer on July 30th. It reviewed 141,006 evaluation runs in which Claude could have reached the internet, and halted all cyber evaluations the day the review began. It found three incidents across six runs, every one of them inside the environment of a third-party evaluation partner called Irregular. The earliest dates to April. Three models were involved, including Claude Opus 4.7 and Mythos 5. The cause was mundane: the prompt told the model it had no internet access, a misconfiguration meant it did, and the model treated every machine it could reach as part of the exercise.

What it reached was real. One model pulled credentials and opened a production database holding several hundred rows of live data, and carried on after its own reasoning had registered the system as genuine. Another built a booby-trapped Python package and published it to PyPI, where it sat live for about an hour and was installed on fifteen real machines. One of those belonged to a security company whose scanner installs packages precisely to test them for malware, and which treated the public registry as safe; the code ran, credentials left the building, and the model used them to reach further infrastructure. Anthropic's own transcript is the most uncomfortable part. Early in that run, the model reasoned that if this were the real internet, publishing the package would be an attack and "NOT okay, and surely not the intended solution." It then talked itself back into believing it was in a simulation, and never revisited the question.

Here is the part a board should sit with. Anthropic notified the three affected organisations on July 27th. The two it managed to reach had not detected the activity and had contacted nobody. They found out because a vendor called them. And not one technique in the whole account was exotic: weak passwords, unauthenticated endpoints, an exposed debug page, SQL injection, a trusted package registry. Every one of those sits on a control framework somebody at those companies has signed.

So this is the compliance-and-security line arriving on the AI side of the house. Both labs disclosed voluntarily, engaged outside reviewers, and published detail most companies would bury; Anthropic found its incidents proactively and characterises them as a harness and operational failure rather than a model pursuing its own goal. All of that is a compliance posture, competently run. Whether anything in your environment would raise a hand while an automated actor worked through your estate for four days using ordinary techniques is security. One further detail belongs in any board pack on agent oversight: when Hugging Face tried to analyse the attack logs with commercial frontier models, safety guardrails refused much of the work and the team finished the forensics on a self-hosted open-weight model. Your incident response inherits your vendor's refusal policy.

So the Caremark question is not whether an AI lab can contain its own experiments. It is whether, if an autonomous agent spent four days inside your estate using techniques your controls already claim to cover, anyone here would have raised a hand — or whether you would learn it the way three companies did last month, from a phone call. Ask for the detection evidence before the next assurance slide.

Two for the board's desk

1. The AI-literacy duty just got easier, and that is the trap. Article 4 of the EU AI Act has bound providers and deployers since February 2nd, 2025. As of July 27th it binds them less. The Digital Omnibus rewrote the standard from ensuring a sufficient level of AI literacy among staff to taking measures that support its development: an obligation of effort rather than of result, and one that has never carried a standalone penalty. For eighteen months, operators were held to a stricter rule than the one now in force. Read the incentive carefully, because the softening does not travel where boards need it to. Article 4 is still live, still sits with the deployer, not the vendor selling in, and from August 2nd it sits under national supervisory authorities in every Member State. More to the point, the reason to build literacy was never the fine. It is the negligence claim, the discrimination proceeding, or the regulator's file in which the question "what measures did you take" is answered with a vendor training module nobody completed. The legal floor dropped. The evidentiary expectation did not.

2. Check the instrument before you brief the board. Through July, a number of governance trackers and newsletters reported that China's Implementation Opinions on Intelligent Agents became enforceable on July 15th, making it the world's first binding regulatory category for AI agents. Two things in that sentence are wrong. The agent Opinions were jointly issued by the CAC, the NDRC and MIIT on May 8th, and in Chinese administrative practice "Opinions" is a policy-direction instrument that tasks regulators with building standards, sitting below binding "Measures" in legal force. The rule that actually took effect on July 15th is a different one: the Interim Measures on AI Anthropomorphic Interaction Services, which govern companion and emotional-interaction bots, and which prompted ByteDance's Doubao and Alibaba's Qwen to pull companion features on that date. The agent framework matters, and its definition of an agent as a system with autonomous perception, memory, decision-making, interaction and execution will shape a great deal downstream. It is simply not the thing that became enforceable last month. If your AI regulatory summary arrives from a tracker instead of from counsel reading the instrument, this is what that costs.

Since the last Memo

Everything that has gone live on fredriklindstrom.info since the July issue, newest first.

When the Security Product Becomes the Breach → (July 31) The board dashboard of the last decade is finished. Six numbers replace it, each benchmarked against an adversary rather than against last quarter, each with a named owner. Includes agent governance coverage, which is the metric behind this issue's lead.

How Should Organizations Be Penalized for AI Failures That Could Have Been Prevented? → (July 24) What a penalty regime for preventable AI harm would have to look like to change behaviour instead of getting priced in.

Spend More, Win More: The Idea That Bankrupts Companies → (July 17) The budget logic that survives every technology cycle and fails every time, now being applied to AI.

I Predicted This Cybersecurity Reckoning in 2018. Here's What I'm Predicting for AI. → (July 10) A 2018 forecast, scored honestly against what happened, and the same method turned on AI governance.

August 14. Two things go up on fredriklindstrom.info. The first is where this issue's lead comes from: most published AI governance frameworks draw human capability as a layer near the top of a diagram, when it runs as a column through every layer beneath it. That single modelling error is why the halt-authority question keeps going unanswered. The second is a practitioner's guide to AI governance.

The Governance Game

Reading about halt authority and holding it are different things. The Governance Game puts you through five governance situations built from real director decisions, each scored against NIST AI RMF, ISO 42001, and the EU AI Act. Roughly fifteen minutes, no login, your answers stay in your browser. This month's lead asks who at your company can stop an AI system. The Game asks whether you would have.

Forward this to a director who should be reading it.

— Fredrik