AI Agent Hacking
AI Agents: When “Following Instructions” can create legal risk.
We’re starting to see AI agents take actions that their creators never explicitly intended. Recent examples include the Australian government/health-service incidents, OpenAI’s Hugging Face incident, and Replit’s coding-agent database incident.
As a Risk Professional I have to ask the question “who is accountable when an agent achieves its objective through an unauthorised route?”
My belief is that litigation is a real possibility, and the AI itself isn’t liable. Responsibility ultimately lands with the people or organisations.
This reinforces why AI governance, permissions, monitoring and assurance need to be designed around what an agent could do, not simply what we intended it to do.
Good Risk Management is about what “event” could occur and reducing the likelihood of it happening, and the impact if it does. Ignoring good governance principles will land a business, its Board and Executives in deep water.
The reverse is also true if your underlying cyber hygiene and lifecycle controls aren’t working effectively (one I see a lot is projects closing without firewall rules or ports being removed), then others poor AI coding means an autonomous AI agent may actively discover and exploit those forgotten weaknesses.
The press like to fear monger “AI agent hacking”, but the AI may simply be pursuing its objective and finding a route around your weak controls.
What do you think?
