AI scam factories force trafficked workers to defraud global victims

Young Indonesians are increasingly falling for fraudulent job ads on Telegram, Facebook, and other social media and getting trafficked to scam farms in Southeast Asia, former scammers and cybersecurity experts told Rest of World. There, they learn to use AI-generated deepfakes, voice clones, large language models, and other technologies to con people into parting with their life savings. 

“Thousands of jobs, usually IT-related ones, are circulated on social media, but they are fake,” Anis Hidayah, commissioner of the National Commission on Human Rights in Jakarta, told Rest of World.  “They generally target those who already have passports. They are recruited very quickly without predeparture training. Later, they can be trained within only two days and start working.”

Many of the scammers are young men and women from Southeast Asia, China, Ethiopia, India, and other developing countries. Hundreds of thousands of them sign up for what they believe are staid office jobs doing digital marketing and online sales that they find on social media. But they are instead taken to fortified scam compounds along the border of Myanmar, Cambodia, and Laos and in the Philippines, run by Chinese criminal syndicates. These farms earn about $40 billion in profits every year. Americans alone lost $12.5 billion in 2024, mostly to investment scams, according to the U.S. Federal Trade Commission.

Rest of World interviewed seven former scammers in Indonesia to learn how they were lured into scam farms and about the technologies they used to defraud victims. The workers requested anonymity to protect their reputations in their communities, as there is stigma attached to scam work. 

They recounted having their passports and cell phones confiscated at the scam centers. They said they were paid poorly and could not leave. Under the close supervision of their bosses, they were forced to lurk on social media sites and dating apps to find victims. They said they spoke to victims on Telegram and WhatsApp using AI-enabled tools that generate deepfake videos in real time. They had “investment” targets to meet, failing which, they were sold to other scam centers. 

“What makes human trafficking for the purpose of online scams different from other kinds of human trafficking is the abuse of technology,” Hidayah said.

The United Nations Office on Drugs and Crime said that there was a 600% increase last year in mentions of deepfake-related cybercrime tools in Telegram marketplaces. The channels sold technologies and services, including malware, web hosting, audio deepfake apps, jailbroken LLMs that force the AI to respond even if the prompts are deemed malicious, and advanced victim profiling software.

“These developments have not only expanded the scope and efficiency of cyber-enabled fraud and cybercrime, they have also lowered the barriers to entry for criminal networks that previously lacked the technical skills to exploit more sophisticated and profitable methods,” John Wojcik, UNDOC’s regional analyst in Bangkok said in the report.

The integration of generative AI by transnational criminal groups represents “a powerful force multiplier for criminal activities,” he said. 

Telegram spokesperson Remi Vaughn told Rest of World that its automated anti-spam systems stop millions of scams, and moderators monitor and remove millions of pieces of harmful content each day, including job scams. 

Meta, Facebook’s parent company, told Rest of World that it works closely with law enforcement authorities across countries to protect people from scams.

The Indonesian government has set up a special division to combat illegal online recruitment and stopped the trafficking of more than 7,000 job seekers, Abdul Kadir Karding, minister of Indonesian migrant workers protection, told Parliament in April.

Every day, for eight months, I deceived people. If God really punishes me, it’s my fault.

The job ads posted on Telegram groups promote high salaries and employee benefits. Telegram

A 26-year-old IT graduate from West Sumatra ended up in a scam compound in March 2024 after a string of bad luck. He once worked as a freelance front-end developer but found opportunities drying up in IT. Frustrated, he tried his hand at a fruit distribution business, which failed. 

One day, while browsing Facebook, he saw an opening for a search engine optimization specialist at a Singapore-based stock trading company. Following a job interview with a recruiter on Telegram, he was placed at the company’s satellite office in Cambodia and promised a salary of $800 a month.

He did not realize he was trafficked until his passport was confiscated in Phnom Penh, Cambodia, and he was driven to a remote compound protected by armed guards. He worked 15-hour shifts in a call center and had to defraud victims of $40,000 every month, he told Rest of World . He was paid less than half his promised salary.

One of his victims was an Indonesian woman, a fitness enthusiast, whom he groomed into a romantic relationship. He persuaded her to bet $10,000 in a casino in Macau, he recalled.

More than 6,700 Indonesians have been tricked into scam jobs via social media sites since 2020, according to the Ministry of Foreign Affairs. The scam syndicates target young workers, Wahyu Susilo, executive director of migrant worker advocacy group Migrant CARE, told Rest of World. “[Young people] aspire to have a remote and flexible job, operating laptops, and adopting technology,” he said.

Dicky Wahyudin, 25, a university student from West Java, saw a Telegram post last December advertising a marketing job at Lazada, one of Southeast Asia’s largest e-commerce companies. He was promised $800 and a chance to live in Bangkok. Wahyudin was a social media influencer, and he was thrilled at the chance to create content from abroad, he told Rest of World. 

“I brought all my costumes and shoes as preparation to produce content [in Thailand], but then, I got trapped,” he said.

At Bangkok airport, he was abducted and driven to Myanmar, where he ended up in a fortified compound trawling for victims on a Chinese dating app. He had to persuade them to invest at least $10,000 every month in a bogus e-commerce platform. He escaped in January and now works as a content creator in Bandung, West Java’s provincial capital.

The former scammers described how they got around the defenses of social media sites:

The Scammers Playbook

Meta said that it removes content that enables human trafficking, and ads that facilitate trafficking violate its policies. It has shut down over 7 million scam accounts since 2024, but scammers evolve their tactics constantly to evade detection, it said. The most effective way to stop this abuse is for local authorities to find and prosecute these criminals, it said.

Meta has deprioritized scam enforcement in recent years and treats scams as a “low severity” user experience issue, according to the Wall Street Journal.

State-of-the-art scam centers are easy to set up these days, Alfons Tanujaya, a cybersecurity expert in Jakarta, told Rest of World. “It only takes one or two people to set up the technology in a scam operation,” he said. 

Even unskilled workers can learn the workflow in just a day, a 32-year-old communications graduate who was rescued from a Myanmar scam compound in March, told Rest of World. 

The toughest part of the job is not the tech but building a relationship with the victims enough to be able to exploit them, he said. 

“Building trust is to make the clients willing to invest and do business with us,” he said. He was so good at it, he earned bonuses, he recalled. 

“Every day, for eight months, I deceived people. If God really punishes me, it’s my fault,” he said. “I’m, like, waiting for karma to come to me.”  

Large language models (LLMs) — the AI models that power chatbots like ChatGPT — are useful for trust-building across cultures and languages, Judah Tana, director of nonprofit Global Advance Projects, who hears firsthand accounts from trafficking victims, told Rest of World.  

“AI is used to impersonate characters, to bring fake personas to life. Meet a person online that appears to be a beautiful Russian woman living in America, when, in fact, it’s an Indian man,” he said.

Voices, too, are easily cloned using just 20 seconds of an audio clip and can be used for phone scams, Tanujaya said. 

During video calls, the scammers use AI tools that replace their face with that of the person on the Facebook profile, the communications graduate recalled. The deepfakes are undetectable, he said. 

“It was enough to deceive the victims. There was no watermark. It was completely flawless,” he said.

Deepfakes have become prevalent due to advancements in audio and video compression, which allows files to be streamed and viewed even when the internet is poor, Ruby Alamsyah, CEO and lead analyst at digital forensic services firm Digital Forensic Indonesia, told Rest of World. 

“Visually, it’s hard to detect deepfake images, even for people who understand technology,” he said.

What makes human trafficking for the purpose of online scams different… is the abuse of technology.

After months of failing to meet targets, the IT graduate was sold to other scam farms, he said. He eventually escaped while being transported to a new scam operation in Cambodia. 

Now back in Indonesia, he works as a mechanic on an oil palm plantation, earning daily wages of 70,000 rupiah ($4.15) to 150,000 rupiah ($9.10) — barely enough to support his infant daughter and wife. In his free time, he learns the latest programming languages, driven by his dream of returning to his former career.

“I have a big hope that one day, I’ll find my way back into the IT sector,” he said. “As a human being, we can only plan.”