In yet another example of deepfakes and other artificial intelligence tools being exploited, a tech company owner in the southeastern city of Fuzhou was defrauded of 4.3 million yuan ($610,000) last month. Using AI to mimic the face and voice of the owner’s friend, scammers convinced him to transfer money to their accounts. 

The owner, surnamed Guo, said he received a video call on WeChat from his friend on April 20. The friend requested that Guo transfer the money using his company’s bank account and vowed to return the amount immediately. 

Guo sent the money and was even sent a fake screenshot of a transaction record showing that the money had been returned. “I let my guard down after confirming his face and voice in a video chat,” the businessman told local media, adding that his real friend later denied ever asking for money. 

Amid an ongoing investigation, local police said Monday that nearly 3.36 million yuan had already been recovered. Police also advised the public to remain cautious about giving out biometric information and refrain from sharing videos and other images of themselves widely on the internet.

Soon after, several hashtags about AI-related scams trended on the microblogging platform Weibo, with users expressing concern over the surge in such incidents and the difficulty in identifying cases of fraud. 

Incidentally, another Fuzhou-based employee surnamed Li, along with other colleagues, was defrauded of 48,000 yuan. Using deepfake software, scammers pretended to be Li’s boss at the company where he works and ordered him to transfer the money. 

The uptick in scams has added to mounting public concerns over the inappropriate use of such technology amid a sweeping craze for AI following the release of ChatGPT last year. 

In the e-commerce industry, livestreamers have since mimicked famous celebrities using face-swapping software to sell their products, drawing widespread criticism.  

In March, a nude photo of a woman sparked outrage after a netizen downloaded her photo online and used AI tools to create a fake image. That same month, prosecutors in Xiaoshan District in the eastern city of Hangzhou filed a criminal lawsuit against a young man who exploited deepfake programs to create pornographic images and videos, which he sold in a group chat.

To curb the growing threat, China has sought to ratchet up its regulatory framework in the past few months. In January, China’s internet watchdog introduced a slew of policies targeting deepfake technologies, ranging from banning illegal activities to making it mandatory to set up a real-name verification system.

Editor: Apurva. 

(Header image: VCG)