Trust Is Not a Feature: How We Work Safely With AI

The AI industry wants you to trust its agents. We think that is exactly the wrong instinct.

Every week brings a new model that writes better code, reasons more carefully and sounds more confident than the last. The pitch is always the same: hand it more of the work, give it more access, let it run. Capability is presented as if it were trustworthiness.

It isn’t. A more capable agent with too much access is not a safer one. It is a faster route to a bigger mistake.

At Outcome Works we use AI agents every day. They help us engineer software, assess systems and produce bid-grade documentation. We are not sceptics. But we have spent long enough in defence, government and regulated industry to know that trust is something a system earns through constraint and evidence. It is never something you buy with a subscription.

The governance gap is already here

The numbers are stark. A Cloud Security Alliance survey published in April found that 82% of enterprises have AI agents running that they did not know about (opens in a new tab), and 65% had suffered an agent-related incident in the past year. Only 21% had a formal process for retiring agents.

Gravitee’s 2026 research found that fewer than one in five organisations fully secure and govern their agents before they go live (opens in a new tab). In July, DigiCert reported that 78% of enterprises had experienced AI-related security incidents or vulnerabilities (opens in a new tab), driven largely by unauthorised or misconfigured agents rather than bad code.

The pattern is clear. Organisations are deploying agents at the speed of the hype and governing them at the speed of a committee.

Untrusted by default

Our starting position is simple. Every AI agent is powerful, useful and untrusted.

That sounds harsh, but it is how good security has always worked. You don’t give a new contractor domain admin on their first morning because they interviewed well. You give them what the task needs, you watch what they do, and you widen access as trust is earned.

So our agents run inside boundaries we control. Each has its own identity, never a borrowed human login. Tools are granted per task. Credentials are scoped, short-lived and kept out of the agent’s reach. Nothing has standing access to production systems or client environments.

We have even chosen to build our own agent harness rather than embed someone else’s open-source one. Every dependency inside your security boundary is a dependency you must defend. We would rather own a smaller supply chain than inherit a large one.

Humans own the decisions

AI drafts. AI analyses. AI recommends. People decide.

Anything irreversible, client-facing, contractual or security-relevant stops at a human gate. A named, competent person approves the deployment, the message, the change or the commitment. The agent does not get a vote.

We apply separation of duties to agents just as we do to people. The role that designs a control does not sign it off. Our multi-agent engineering teams always run with assurance roles switched on, and work is reviewed in mandatory pairings. No agent ever marks its own homework.

We also treat everything an agent reads as data, not instruction. Prompt injection is the phishing of the agentic era. An agent that obeys whatever it finds in a web page or an email is not autonomous. It is hijackable.

Provable, not plausible

This is the principle we care about most, and the one the market talks about least.

AI output is extraordinarily plausible. It reads well, it cites confidently and it is often right. But “often right” is not a standard you can take to an auditor, a regulator or a court.

So we capture the evidence. For every material AI interaction we record the question, the context the model saw, the tools it used, its reasoning where available, the response and the human who approved it. Those records are vaulted, tamper-evident and linked to the requirements and deliverables they shaped.

The test we set ourselves is demanding. A year from now, could we reconstruct exactly how a piece of AI-assisted work was produced and prove who stood behind it? If the answer is no, the work is not finished.

Standards, not slogans

We align our practice to ISO/IEC 42001, the international standard for AI management systems, alongside NCSC’s Cyber Assessment Framework and Secure by Design principles.

The regulatory ground is moving too. The EU’s AI Omnibus pushed the main high-risk deadline to December 2027 (opens in a new tab), but the obligations themselves did not shrink. Meanwhile EN 18286, the first harmonised standard supporting the AI Act (opens in a new tab), is now cited in the Official Journal. A delay is not a reprieve. It is time to build properly.

The advantage goes to the provable

The next phase of AI won’t be won by whoever adopts agents fastest. It will be won by whoever can prove what their agents did.

Boards will ask for it. Insurers will price it. Regulators will demand it, and clients in defence, government and critical infrastructure already do. The organisations that treated trust as a feature they could switch on will spend the next three years retrofitting controls they should have designed in.

We would rather do the hard part first. Bound the agent. Keep a human on every decision that matters. Capture the evidence. Then let the AI do what it is genuinely brilliant at.

That is not caution for its own sake. It is how you get outcomes you can stand behind.

About the author

Mark is co-founder and CTO of Outcome Works, a UK outcome engineering firm working across commercial, defence and government sectors. A Fellow of the British Computer Society and the Royal Society of Arts, he has spent more than 25 years leading technology at AWS, HPE, PA Consulting, General Motors and Serco, and now focuses on building AI-native systems that are secure, accountable and provable by design.