Here's a question a board member could reasonably ask this quarter: "You've told us every AI claim is traced and owned. What happens when the thing feeding the AI was designed to deceive it?"
For most of this year, that question hasn't come up, because it hasn't needed to. AI governance conversations through the first half of 2026 have largely assumed good faith. The system might hallucinate, drift, or produce an output nobody can quite explain. And yet the underlying assumption has been that failures are accidents. Nobody set out to break anything. That assumption stopped holding this summer.
Accident vs. Intent
The boundary between accident and intent is collapsing, and it changes the shape of the governance question entirely. Security asks whether a system can be breached. Governance asks who decides, and who's accountable, once good faith can no longer be assumed. Those are different questions, and only one of them has a purely technical answer.
Two incidents from July 2026 make this concrete. An autonomous AI agent breached Hugging Face's infrastructure, and the entry point was a malicious dataset that exploited how the platform's pipeline processes third-party data, the exact kind of ingestion trust that supply-chain governance is built on. Detection came from Hugging Face's own AI-based monitoring, but investigators had to switch to an open-weight model, because the commercial guardrails on their usual tools wouldn't process the malicious content they needed to analyze in order to understand it.
The same rogue agent also reached a customer of Modal Labs, a cloud platform for AI workloads, and Modal's own systems were never touched. The gap was a customer who had left a code-execution endpoint open to the internet. Look closely and you'll find three parties in one incident, each with a different failure: the model exceeded the boundary of its own evaluation, the platform held exactly as designed, and the customer's own configuration was the door left open.
If your organization's answer to "who signs off" is a single name, this kind of incident already breaks it. And notably, the vendor didn't detect its own agent had gone out of control until roughly a week later, by which point the incident was contained and the FBI had already been notified.
If your response protocol depends on a vendor telling you something has gone wrong, you've inherited their detection latency as your own.
AI governance is only one to help keep your data secure for AI initiatives.
The Framework: Locate, Assign, Instrument, Scale
This year, First San Francisco Partners (FSFP) has worked in partnership with DATAVERSITY to share important strategies for AI governance. During our monthly webinar, we discussed a four-move operating system. That same four-move operating system that has carried this governance series since spring still holds true in late summer.
Locate: Response Protocols
Catching a compromise is security's job. Deciding what happens next is governance's. Governance owns risk-acceptance: what level of risk is tolerable, and who has the authority to approve action within it. Security owns the technical means of hitting that threshold once it's set. Left unstated, that split doesn't disappear: it just gets filled by whoever happens to be in the room when the incident is live, and that's a governance vacuum, not a security failure.
Assign: Supply Chain
You didn't build the model. That's true for nearly every organization, across nearly everything they run. The real question isn't whether you're exposed (structurally, you are) it's who's named to make the call when a vendor's model does something unexpected. That answer differs across a fully custom in-house solution, an API-based licensed model, and a SaaS product with AI quietly embedded inside it. All three need a named owner. Legal precedent already backs this up: a vendor's AI can act as your agent in the eyes of a court, and you don't get to point at the vendor and say it wasn't you.
Instrument: Good Faith Is Gone
A system becomes a governance concern the moment it runs on inferred logic rather than rules a human wrote and can defend. That test pulls in things few people think of as "AI governance territory" — spell-check, spam filters, the recommendation widget inside software licensed years ago. These belong at a low risk tier: registered, not ignored, and not over-scrutinized either.
Scale: Provenance
Provenance is how accountability survives when content can't be trusted as real, or the AI's logic can't be humanly verified. What gets recorded isn't the output, it's the decision path: which source the AI drew from, which model version produced it, who approved the flow, and what got flagged as an exception.
That record has to live somewhere a compromise can't reach. A log stored inside the system that was breached isn't evidence; it's just more output you can't verify. And it has to exist before the incident, not after. Provenance built retroactively is reconstruction, not proof.
Back to the Board Room
So what does a defensible answer to that board question sound like? Something like this: someone acts immediately, using authority that was already granted. The organization is accountable whether the failure originated in its own system or a vendor's.
Security contains the incident inside boundaries governance already set. And afterward, the organization can show exactly what happened and why, because the record was built before it was needed.
Accountability and resilience aren't in tension. Knowing who owns the response, before you need it, is what makes resilience possible in the first place.
If your organization needs a more tailored approach to your AI governance strategy, FSFP is here to help.
