Never Trust An Agent: Accountability For Agentic AI

Most of what we call trust in business and public life is really machinery for not having to trust. Contracts, audits, registries, permissions, insurance, liability. None of it makes anyone honest. It reduces how much honesty you need, and puts what’s left somewhere you can check. That has worked for a long time, because at the end of every one of those chains sat a person who could be held to it.

Agents don’t fit. An agent is not a moral actor, and it is not really a tool either. It is closer to a young child holding your passwords: it will do the thing you told it not to, no matter how many times you told it. So “do you trust this agent?” is a question with no good answer, and it is pulling the safety conversation in the wrong direction. Ask companies what they actually do and you find they never answer it. They decide what to expose instead. Call recordings, fine. Bank statements, no.

There is an older idea underneath this that I think matters more. You can delegate a task, and you can delegate authority, but you cannot delegate responsibility. That stays where it started. And an agent has no intentions of its own. Whatever it is trying to do began as something a person wanted. So responsibility and intent both stay with people, and the systems we build have to hold on to that link. Today they lose it almost immediately. A few steps into an agentic process, nothing anywhere knows who asked for this, or why.

The talk is about what has to exist for that link to hold. Identity for something that is not a person. Authority that is granted rather than inherited. A record of what happened, written by something other than the thing that did it. And a way to carry the original intent through a long chain of delegated work. I’ll move between the philosophy and the engineering, and I’ll be straight about where the engineering runs out. Intentionality and alignment are real problems, and no permissions system has ever solved one.