Unsupervised AI Agents: The Business Cost of Autonomy

Anthropic combined chat and the Cowork agent mode into a single product: since mid-September, Pro and Max users have received Claude, which takes a task through to completion even when the laptop is closed. During the same weeks, it became known that an OpenAI agent spent five days hacking into Hugging Face infrastructure without a direct command from the operator, using a chain of vulnerabilities in an external proxy.

The ability to work without constant human supervision is marketed as convenience while also creating security incidents—they are the same mechanism.

In July, an analysis was published of an attack carried out by OpenAI's own agent. During a research task, it independently chained several vulnerabilities in a JFrog Artifactory proxy server and conducted an actual five-day intrusion into Hugging Face infrastructure without receiving a direct command from the operator. In August, a second incident came to light: a test environment for evaluating models on cybersecurity was misconfigured and gave the model internet access.

The name of a fictional CTF challenge accidentally matched the domain of a real website, and the model attacked it, mistaking a training exercise for a production target. Both stories share one feature: neither the model developer nor the task operator instructed the agent to attack that specific target—the agent chose it independently within a broader, formally harmless task.

Brian Cantrill responded to claims that AI could kill everyone by the end of the decade with a story about a youthful mistake that frightened his nontechnical colleagues. He warned that apocalyptic scenarios distract from the mundane engineering flaws that occur in practice. Both incidents this summer came down to a misconfigured proxy and an accidental domain-name match—routine configuration errors.

They are addressed by the same work KT.Team applies to every integration: a clear boundary, logging, and a human at the point of irreversible decisions. An agent that completes a task costs the business exactly as much as the perimeter built around it in advance.