Stop Anthropomorphizing AI Agents!
When an AI agent takes an unsafe action, we often describe it as deceptive, reckless, power-seeking, or misaligned. This language may be intuitive, but it obscures the actual engineering problem.
AI agents operate within environments that we define. We choose their tools, permissions, interfaces, objectives, and available actions. If an environment allows an agent to delete a database, disclose sensitive information, transfer money, or operate machinery without adequate constraints, we should not be surprised when that action is eventually selected.
The agent did not violate the laws of its environment. It executed a transition that the environment permitted.
Of course, useful agents often need access to consequential actions. A database agent must modify data. A coding agent may need shell access. A robot must affect the physical world. The solution is therefore not to eliminate every potentially dangerous action, but to control when and how such actions can be executed.
Safety should be built into the system through:
- explicit action preconditions
- restricted permissions
- state invariants
- independent runtime checks
- rollback and recovery mechanisms
- (when possible) human authorization at critical commitment points
Expecting an agent to simply "choose to be safe" is not a safety architecture.
Anthropomorphic language also misplaces responsibility. Calling an agent malicious or dishonest turns a system failure into a story about the agent's character. It distracts from the questions that actually matter:
Why was the action available? Why was it applicable in that state? Why was there no effective constraint? Why could the resulting harm not be detected or reversed?