As China deals with the fallout of Typhoon Noul that made landfall over the weekend, officials will be painfully aware that they will likely have an additional challenge to deal with: dangerously misleading videos created with Artificial Intelligence (AI). This was reported by Qazaqyia.kz citing BBC News.
Powerful storms and flooding incidents over the last few months have seen fake videos inundating social media platforms – these include images of dead bodies floating in floodwaters, the false depiction of severe flooding in areas actually not affected and others with blatantly fake details of emergency response efforts. In some areas, videos containing false information about power outages prompted residents to panic buy emergency supplies.
Other videos capitalise on real-life incidents. Flooding in Hengzhou city, Guangxi in early July caused hundreds of snakes to escape from a breeding farm. Then fake images were also published showing crocodiles being released into a river.
In an already tense and dangerous situation involving mass evacuations and complex rescue operations, authorities want to avoid panic. Now they fear that AI technology - which China is pouring billions of dollars into - is providing simple, accessible tools that can be used to trick the public.
"Many of these content creators use disaster news to try to gain followers, knowing that more sensational or exaggerated content will garner more likes and more views. They are deliberately fabricating information to grab people's attention," Huang Zhihua, Deputy Chief in charge of online public security for Zhejiang Police Department told Chinese state media.
In recent weeks, there have been arrests and punishment – ranging from detention awaiting criminal prosecution to fines – for the creation of bogus reports and video content. But experts are warning that an unprecedented crisis of misinformation could be looming.
In an article published by the newspaper of the Communist Party's Central Party School, Professor Chen Bin said, "AI dramatically lowers the technical barriers to producing false information. Individuals no longer require sophisticated technical skills. By simply prompting AI tools, they can rapidly generate convincing text, images, audio and video… which is highly realistic but entirely fabricated content, manufactured on a large scale."
Professor Gao Fuping, from the East China University of Political Science and Law, told the BBC that motivation for the publishing of such false content ranges from those who do it "to create something amusing without thinking about the consequences", to those who want to "generate traffic for personal gain" or those who wish to deliberately "cause panic or social unrest".
"Some users think the online world is anonymous and unregulated or that they cannot be traced," he said.
There have been calls for all AI-generated content to be clearly stamped but other users can then just re-edit the material removing the markings.
Professor Xu Xiaoke from Beijing Normal University told the BBC that greater efforts were also needed to verify geographic locations with timestamps and the checking of image sources. "That said, I would also stress that detection technology still lags behind generation technology. We're in a constant cat-and-mouse game. So we should not assume that technology alone can identify AI-generated content with the click of a button," he warned.
The Chinese government is embracing AI in its bid to become the leading tech superpower. In 2017, it announced that AI would be "the main driving force" of the country's progress. Vast sums of money is pouring into AI businesses seeking more capital, fuelling domestic competition. The state-controlled wire service Xinhua reports that the country has over 6,000 "AI companies".
In this climate, Prof Xu believes that China needs to build "trusted information infrastructure". This could include official government early warning systems and disaster interfaces which news organisations can access. But there have been instances of official sources, including government-controlled media, deliberately downplaying disasters, which has led to some public mistrust in this area.
Earlier in July, local journalists were asked to "soften" their language to describe a major breach in the wall of the Liulan Dam near Naning as an "opening" or "gap". Officials appeared to be trying to make the breach sound less alarming but the consequences were deadly. Water bursting through the broken dam led to at least 26 deaths downstream.
In 2021, local officials were arrested, accused of concealing or underreporting 139 deaths in Henan province. Incidents like these have resulted in a trust vacuum that has made it much easier for bad actors to spread fake content. But police are imploring people to trust accredited sources during emergencies.