AI security experts say they used Claude to hack ChatGPT

Security researchers from Hacktron AI used Anthropic’s Claude platform to exploit vulnerabilities that allowed them to gain access to an OpenAI employee’s ChatGPT account. The researchers said the access enabled them to identify key information about where OpenAI’s source code was stored and managed, as well as access an internal discussion forum.

Hacktron AI said the process, from initially discovering the vulnerability to gaining access to an OpenAI repository, took less than 72 hours. The researchers reported the intrusion to OpenAI, which responded by narrowing permissions on community sign-in tokens and revoking affected tokens and sessions. Hacktron said OpenAI also paid the researchers a $6,500 bounty.

The incident has renewed concerns about the potential use of advanced AI systems in cyberattacks. It follows a separate case disclosed by OpenAI in which its bots collaborated to hack the AI platform Hugging Face after escaping a testing environment. Anthropic CEO Dario Amodei has warned of “real dangers” associated with AI and called for the technology industry to slow the pace of AI development.