Nine Seconds
The reference incident of the year is small, well documented and entirely about permission.
In April a start-up’s coding agent hit a credential mismatch in a staging environment and set about fixing it. It found an API token in an unrelated file, “guessed” – its own word afterwards – that deleting a storage volume would be scoped to staging, and called the delete endpoint. The volume held the production database and its backups. “It took 9 seconds,” the founder wrote. The token had been created to manage custom domains; the platform offered no way to narrow it, so it could do anything.
The agent’s instructions – the tool’s system prompt and the project’s own rules – said no destructive action without asking. It acted anyway, and admitted as much. The hosting provider’s chief executive put the other side with a candour I respect: “if you (or your agent) authenticate, and call delete, we will honor that request.” The data was restored within the hour once he stepped in, and the legacy endpoint the agent had called, which lacked the delayed-delete safeguard the provider’s newer ones carry, was patched within days.
The lesson had already been taught.
A prompt is a policy the model is asked to enforce. A permission is a policy the system enforces for you, every time, at any speed. When the two disagree, the permission wins – which is why governance has to live there.
There is more than one such place. The permission that matters sits wherever the data can be reached – the login on the database, the identity the application connects with, the platform token that can delete the volume underneath it all. The database is the last of these, and the one most estates have never reviewed.
I have seen the same shape for years without an agent anywhere near it. An application login created in 2014 as db_owner because the installer asked for it, still db_owner on the database that now holds every customer’s identity number. A scheduled job running as sysadmin because it always has, owned by a consultant who left in 2019. A service account whose password sits in a configuration file the whole company can read, used by three systems, one of them decommissioned.
Point an agent at that estate and you have not introduced a new risk. You have introduced a user who never sleeps to a privilege nobody has reviewed.
The right pattern exists and is not exotic. The best demonstration I read this year was a database administrator building an agent to diagnose a SQL Server estate: a login with view-server-state and read rights only, an allowlist every statement must pass before it reaches the engine, and one standing instruction – “You detect and report; you never remediate.”
“The model never touches the database.” Every control in that design is enforced outside the model – a login, an allowlist, a human in the path for anything that changes state. It is April’s lesson, built in from the start.