Chinese AI-Generated Fake News: A Tool of Cyber Warfare and Suppression

In recent months, there have been two notable publications shedding light on the role of artificial intelligence (AI) in China's censorship and influence operations. One paper from Stanford University researchers Jennifer Pan and Xu Xu explores how government regulation shapes output from Chinese companies' LLM chatbots. The other is a report by OpenAI, which offers broad insight into influence operations both at home and overseas, including efforts to discredit Japanese Prime Minister Sanae Takaichi, and to deplatform Chinese dissidents on Western social media platforms.

According to Pan and Xu's paper, China's AI regulations are an extension of its censorship regime, building on and reinforcing existing government censorship efforts. Their findings confirm and expand on the intuitive expectation that Chinese models are more likely to refuse to answer questions on sensitive political topics, or to give brief, selective, or otherwise misleading answers.

The authors note that the difference appears to be partly a result of training models on material already shaped by PRC information controls, rather than direct manipulation. However, this seems to be a relatively minor factor. The paper's abstract states: "A growing body of research on large language models (LLMs) has identified various biases, primarily in contexts where biases reflect societal patterns. This article focuses on a different source of bias in LLMs—government censorship. By comparing foundation models developed in China and those from outside China, we find substantially higher rates of refusal to respond, shorter responses, and inaccurate responses to a battery of 145 political questions in China-originating models."

OpenAI's report highlights Chinese authorities' "systematic use of AI for monitoring, profiling, translation, content creation, and internal documentation." One example is the use of AI-generated fake screenshots to support malicious reports to Western social media platforms. ChatGPT was also asked to help plan an influence campaign targeting Japanese Prime Minister Sanae Takaichi, who has been a primary propaganda focus since late last year.

The impact of these tactics appears to have varied greatly. Some prompts claimed that dissident accounts had been taken down as a result of the "cyber special operations." However, manual investigation showed only a handful of instances of the operation's hashtags occurring across social media, and many posts did not receive engagement from authentic audiences.

Targets named in the report include the China-focused rights organization Safeguard Defenders and "Teacher Li," who runs the prominent X account @whyyoutouzhele. He posted a lengthy response to the report, including the following: “We hope that X, YouTube, Bluesky, and other social media platforms recognize that your automated content-moderation systems are being weaponized by the CCP. We urge these platforms to build mechanisms capable of detecting state-level coordinated attacks, rather than forcing victims to bear the consequences of being silenced again and again."

Another notable point from the banned user's prompts was "the importance of combining online and offline operations, especially when it related to government critics within China." This highlights the complexity and scale of Chinese cyber warfare efforts, which often involve a combination of online and offline tactics to suppress dissent.

The Role of AI in Chinese Censorship and Influence Operations

AI has become a key tool for the Chinese government to suppress dissent and silence critics both online and offline. The use of AI-generated fake news, propaganda, and disinformation is a hallmark of these efforts, which often involve a combination of online and offline tactics.

The impact of these tactics can be severe. Dissidents have reported being targeted by harassment campaigns, including flooding their social media accounts with irrelevant content, creating fake social media accounts to spread and amplify pro-CCP messages, and even targeting their mental health. In some cases, dissident accounts have been taken down as a result of the "cyber special operations."

However, not all targets have been silenced. Some, like Teacher Li, who runs the prominent X account @whyyoutouzhele, have spoken out against the use of AI-generated fake news and propaganda in Chinese cyber warfare efforts.

The Need for Greater Transparency and Regulation

The use of AI in Chinese censorship and influence operations is a pressing concern that requires greater transparency and regulation. Social media platforms, governments, and civil society organizations must work together to build mechanisms capable of detecting state-level coordinated attacks, rather than forcing victims to bear the consequences of being silenced again and again.

OpenAI's report highlights the need for greater transparency and accountability in AI development and deployment. The company has taken steps to identify and disclose this threat, but more needs to be done to ensure that AI is not used to systematically suppress human rights.

Conclusion

The use of AI-generated fake news, propaganda, and disinformation is a key tool in Chinese cyber warfare efforts. These efforts often involve a combination of online and offline tactics to suppress dissent and silence critics both within China and abroad.

The impact of these tactics can be severe, with dissidents reporting being targeted by harassment campaigns and having their accounts taken down as a result of the "cyber special operations." However, not all targets have been silenced, and some, like Teacher Li, have spoken out against the use of AI-generated fake news and propaganda in Chinese cyber warfare efforts.

Greater transparency and regulation are needed to ensure that AI is not used to systematically suppress human rights. Social media platforms, governments, and civil society organizations must work together to build mechanisms capable of detecting state-level coordinated attacks, rather than forcing victims to bear the consequences of being silenced again and again.