OpenAI has identified and banned accounts associated with state-linked malicious operations that were using its AI models for abuse. Hackers connected to Russia, China, North Korea, and Iran utilized OpenAI’s tools for purposes including malware creation and social media manipulation, according to the company.

A Russian-speaking group employed ChatGPT to create and refine malware identified as ScopeCreep. This actor reportedly used temporary email addresses for account creation and engaged the chatbot to debug code, create HTTPS requests, and modify PowerShell commands to circumvent Windows Defender. The malware, written in Go, was found on a repository impersonating a gaming utility software called Crosshair X.

The malicious code initiated a multi-stage process involving privilege escalation, stealthy persistence, and the exfiltration of credentials and cookies. It also incorporated logic to notify the operator via a Telegram channel upon compromising new systems. OpenAI stated that the threat actor used its models to integrate the Telegram API, debug malware components, and configure command-and-control infrastructure. While the malware was publicly accessible, OpenAI intervened early in its development, preventing large-scale infections.

Accounts linked to hacking groups APT5 and APT15, commonly attributed to the Chinese government, were also banned. These hackers used ChatGPT for open-source research and to troubleshoot Linux system configurations, software development, and infrastructure setup, including building offline software packages and configuring firewalls and name servers.

Why your apps all broke at once today

These groups also explored AI’s potential for automating social media influence. OpenAI detected instances where ChatGPT generated posts and interactions for platforms such as Facebook, Instagram, TikTok, and X (formerly Twitter). In some instances, ChatGPT was tasked with developing scripts to conduct brute-force attacks on FTP credentials or to deploy Android apps for programmatic control of social media content.

Additional state-linked clusters were found to be engaging in online influence or cybercrime. One operation, likely connected to North Korea, leveraged OpenAI’s models to support fraudulent employment scams by generating convincing resumes and task documentation for IT roles. Another campaign involved producing multilingual social media content on geopolitically sensitive topics, which was then distributed across major platforms.

OpenAI highlighted a campaign named “Operation Uncle Spam,” where accounts generated English, Spanish, and Swahili content designed to polarize discussions on divisive U.S. political issues. These posts appeared on platforms like Bluesky and X, sometimes by accounts posing as Americans.

OpenAI reported that actors were not using ChatGPT to achieve entirely new capabilities. However, the tools improved workflow efficiency and scaled messaging output through translation, code debugging, and scripting assistance for deceptive messaging campaigns.

This report follows previous disclosures by OpenAI, including a February update detailing how accounts associated with Chinese and North Korean operations had used ChatGPT to facilitate information operations and scams. OpenAI stated that its strategy for limiting abuse involves a combination of detection methods, human oversight, and collaboration with industry peers.

Featured image credit