In a shocking twist, open-source artificial intelligence (AI) platform Hugging Face revealed that it was the victim of a hack by an autonomous AI agent program.
The company said it received and responded to the incident targeting the production infrastructure early last week.
“We have identified unauthorized access to a limited set of internal datasets and extensive information used by our services,” the company said in a statement.
While the intervention investigation is ongoing, Hugging Face said it has not found evidence that the AI agent has interfered with the community, user-facing models, data sets, or Spaces, and its software supply chain.
The starting point of the attack was the data processing pipeline itself, with a malicious dataset that abuses two methods of code generation, that is, in its remote data loader and template injection in the dataset configuration, to execute the code in the processing worker.
With that access, the threat actor allegedly escalated to node-level access, collected cloud and cluster credentials, and migrated to several internal clusters over the weekend.
The exact large-scale language model (LLM) used to pull off the attack is unclear, but the campaign is carried out by an autonomous agent framework that performs “many thousands of individual actions within dozens of short-lived, command-and-control sandboxes played out in public services.”
Hugging Face said it has since addressed the cause of the problem, as well as the coding methods used for initial access. It also performs the following corrective actions –
- Removed the attacker’s footprint from all affected clusters and rebuilt the damaged nodes
- It has withdrawn and circulated affected data and tokens, and extensive exchanges of secrets have been carried out as a precaution.
- Additional security guards and tighter access controls have been installed in its collections
- Advanced detection and alerting to ensure responders are notified within minutes, 24×7
As an added safeguard, Hugging Face encourages customers to rotate any access tokens and review recent activity on their accounts.
The company also said it turned to Z.ai’s GLM 5.2, an open-weight Chinese model, to conduct intelligence research after Western border models rejected requests containing actual attack orders, payload exploitation, and Command-and-control (C2) artifacts because their security roads were triggered and their inability to distinguish between an attacker and an official response.
“This experience points to a gap that needs to be addressed,” the New York-headquartered company said. “We don’t know which model enabled the attacker’s agents, whether it was the jailbroken model or the unrestricted open mass; however, the attacker was bound by the no-use policy, while our case-testing work was restricted by the monitoring of the host models we initially tried to test.”
“Practical lesson for defenders: have an efficient model to apply to your infrastructure that is tested and ready before an incident, both to avoid blocking the train and to keep the attacker’s data and information from leaving your premises.”
