OpenAI Bans Cambodian AI-Enabled Scam Network

According to Chinese media reports on August 1, OpenAI recently banned a group of coordinated ChatGPT accounts. The accounts were believed to originate from Cambodia and may have operated primarily in the Poipet area of Banteay Meanchey Province. The criminal network behind them used ChatGPT to facilitate scams involving fraudulent investments, online romance, online gambling, and identity impersonation.

The incident occurred earlier this year, with the initial investigative lead coming from WhatsApp. OpenAI subsequently investigated the accounts based on coordinated behavior, conversation content, and usage patterns, and shared the threat indicators it identified with industry partners and relevant organizations.

No comprehensive data is currently available on how many victims were linked to these accounts or how much money the scam network obtained. In conversations disclosed by OpenAI, some scammers mentioned that victims had lost thousands of dollars. This figure reflects only individual cases and does not represent the overall scale of the network.

ChatGPT Was Integrated Into the Scam Production Line

These accounts were not simply asking, "How do I scam people?" Instead, they embedded ChatGPT into the criminal organization's everyday operations.

According to the information disclosed, the scammers primarily used ChatGPT for the following tasks:

• Creating or refining fake identities, including names, occupations, personal histories, and social backgrounds;
• Translating messages sent to victims, lowering the barrier to cross-language scams;
• Producing promotional copy, recruitment images, and marketing materials for fraudulent schemes;
• Drafting internal notices and work schedules for the scam organization;
• Translating communications between employees;
• Developing different narratives involving cryptocurrency, gold trading, and online gambling;
• Maintaining various personas, such as romantic partners, investment mentors, or law enforcement officers.

This does not entirely match the common perception of "AI-automated scams." At this stage, the model functions more like a cheap, on-demand content hub: it does not identify every victim or independently complete money transfers, but it can compress work that previously required copywriters, translators, and trainers into a single chat window.

For legitimate companies, this is called improving operational efficiency; in scam compounds, the same capability becomes criminal production capacity.

Cross-language capabilities are especially concerning. Traditional cross-border scams require personnel familiar with the target country's language and culture, and awkward translations can easily expose the fraud. Although large language models still make mistakes, they are capable of helping ordinary operators quickly generate grammatically correct, natural-sounding messages and adjust the wording based on a victim's age, occupation, and response style.

This means scam groups no longer need a complete localization team for every market. A single operator can simultaneously maintain personas for multiple countries and languages, using the model for translation and polishing before manually advancing the relationship and extracting money.

From Romance to Gold Investments, the Script Changes but the Transfer Path Remains the Same

OpenAI observed that the network flexibly switched between types of scams rather than relying on a single template over the long term.

Some accounts used romance and friendship as an entry point, building trust through fake identities and sustained conversations. Once the relationship reached a certain stage, they steered the discussion toward cryptocurrency, gold trading, or supposed insider investment opportunities. Other accounts directly posed as investment advisers, successful traders, or gaming platform employees and showed victims fabricated returns.

Different scripts typically converged on a similar path:

1. Contact the target using a carefully crafted identity;
2. Build trust through frequent communication;
3. Show fake profit screenshots or success stories;
4. Direct the victim to a designated website, app, or chat group;
5. Initially allow a small investment to generate apparent returns;
6. Demand additional transfers under the pretext of adding margin, paying taxes, or lifting risk-control restrictions;
7. Cut off contact once the victim can no longer pay.

What large language models truly improve is not the creativity of the scam itself, but the speed at which scripts can be changed. When a "gold analyst" persona stops working, criminals can quickly switch to a "cryptocurrency researcher." If the investment story becomes unconvincing, they can pivot to romance, gambling, or impersonating law enforcement. In the past, replacing a mature script required rewriting materials and retraining operators. Now, a model can generate the first draft, which the team then adjusts based on experience.

It is therefore inaccurate to reduce the issue to "ChatGPT wrote scam copy." The more serious change is that generative AI is reducing operational friction for organized crime, allowing the same personnel to test more identities, cover more languages, and communicate with victims in a more personalized way at lower cost.

The Conversations Also Revealed Signs of Human Trafficking and Forced Criminal Activity

OpenAI said the relevant accounts also generated content involving human trafficking and forced criminal activity, consistent with public reporting on scam compounds in parts of Cambodia.

Poipet lies on the border between Cambodia and Thailand. It is a hub for casinos and cross-border commerce and has long drawn attention for risks related to online fraud and human trafficking. Some scam organizations are not staffed entirely by willing participants. Some individuals may be lured into compounds through fraudulent job offers, then have their freedom restricted and be forced to commit fraud.

This means the incident cannot be treated merely as a case of "accounts violating platform rules." There may be two layers of victims: people deceived into transferring money online, and workers inside the scam organization who are subjected to confinement, threats of violence, or debt bondage.

The use of ChatGPT to produce recruitment materials and internal notices also shows that the model's role has expanded beyond victim-facing conversations into the internal management of criminal organizations. Scam groups are not only using AI to "acquire customers" externally, but also to handle training, translation, publicity, and employee communications.

In other words, generative AI is not an isolated tool for committing crimes in this context; it has been incorporated into an already mature offline criminal system.

Account Bans Help, but They Are Far From Dismantling the Network

OpenAI's decision to ban coordinated accounts and share threat intelligence was necessary and addressed the core of the problem more directly than relying solely on filtering individual prompts.

Scammers generally do not describe their entire criminal plan in a single request. They may divide the work into seemingly ordinary steps: creating a character background, translating a conversation, polishing an investment pitch, or writing an employee notice. Viewed in isolation, each request may not be clearly malicious. Organized activity becomes easier to identify only when account relationships, usage frequency, repeated identity templates, and content shared across accounts are considered together.

This is also why the lead provided by WhatsApp was important. Scam operations are inherently cross-platform: social platforms are used to contact targets, messaging tools to cultivate relationships, AI services to generate content, fake trading platforms to display returns, and cryptocurrency wallets or bank accounts to receive funds. Each platform sees only a fragment of the operation.

If platforms cannot exchange processed threat indicators, criminals can exploit these information gaps and remain active for long periods. This operation at least demonstrates that cross-platform intelligence sharing can help model providers trace a single suspicious request to a coordinated group of accounts.

However, the effectiveness of account bans should not be overstated. ChatGPT accounts are replaceable resources, not the scam organization itself. After existing accounts are banned, criminals may register new ones, purchase stolen accounts, switch to other models, or use open-source models to generate content locally. As long as recruitment, coercion, traffic acquisition, and payment channels remain intact, banning a group of accounts merely raises the cost of committing crimes rather than eradicating the operation.

OpenAI's action is more like disabling a set of office software accounts inside a scam factory than shutting down the entire factory.

AI Safety Cannot Focus Only on Individual Responses

For model platforms and API providers, this incident raises a practical question: if content safety reviews only check whether a particular output contains explicitly illegal instructions, they will struggle to identify industrial-scale abuse.

Effective risk controls must simultaneously address multiple layers:

• Content layer: Is the user repeatedly generating materials for fraudulent investments, identity impersonation, and inducing money transfers?
• Behavioral layer: Is the user creating similar personas in bulk or translating large volumes of repetitive scripts within a short period?
• Relationship layer: Do multiple accounts share templates, operational rhythms, devices, or payment characteristics?
• Context layer: Do ordinary marketing materials and scam-oriented lead generation form a continuous workflow?
• Enforcement layer: After accounts are banned, do related actors quickly migrate to new accounts and resume operations?

This type of detection also has clear limitations. More extensive account-linkage analysis may create privacy concerns and false positives, while excessive reliance on keywords allows criminals to evade detection easily by splitting tasks and changing their wording. Platforms must make trade-offs among data minimization, user privacy, and abuse tracking, while providing human review and appeal mechanisms for high-risk enforcement actions.

Developers should take this issue seriously as well. Any product that provides bulk content generation, automated translation, direct-message assistance, or marketing automation can be integrated into a scam workflow. A checkbox stating that "illegal uses are prohibited" is essentially ineffective. More practical measures include anomalous usage monitoring, account rate limits, detection of bulk identity generation, payment risk identification, and human review for high-risk use cases.

Agent products that can simultaneously generate content, automatically contact people, read replies, and continue conversations pose greater risks than simple chatbots. Once scammers connect language models to social media accounts, contact databases, and automation scripts, much of the account cultivation and follow-up work previously performed by humans could be further automated.

AI Did Not Invent Scams; It Gave Them an Industrial Upgrade

Based on the available information, ChatGPT did not create an unprecedented type of scam. Romance scams, fraudulent investments, gambling-related lead generation, and law enforcement impersonation have existed for years, and Cambodian scam compounds did not emerge only after the arrival of generative AI.

What has changed is the marginal cost.

Models can cheaply generate identities, standardize messaging, translate languages, and rewrite scripts at scale. Scam organizations can use fewer people to reach more victims while rapidly changing narratives for different targets. The content may not reach professional standards, but for scam operations driven by probability and scale, even a slight increase in response rates can significantly amplify profits.

OpenAI's bans are therefore commendable, but they amount only to localized damage control. Effective governance requires model providers, social platforms, communication services, payment institutions, and law enforcement agencies to jointly trace the full chain: who is creating identities at scale, who is directing traffic, where the money ultimately goes, whether workers are being coerced, and whether the same organization is reestablishing accounts on other model platforms.

Banning only the accounts that generate content cannot eliminate offline compounds or financial networks. But if model platforms take no action at all, scam organizations gain free access to a global content infrastructure.

The conclusion this case leaves the industry is straightforward: the general-purpose capabilities of large language models do not inherently distinguish customer service from fraud, marketing from manipulation, or translation from criminal collaboration. What platforms truly need to demonstrate is not merely that their models can reject an obviously malicious prompt, but that they can identify criminal workflows that have been fragmented, disguised, and executed at scale.

References

• ITHome: ChatGPT Exploited by Cambodian Scam Network, OpenAI Takes Action to Ban Accounts—An overview of the investigative leads, account uses, operating locations, and enforcement measures related to OpenAI's investigation into the Cambodian scam network.