IN THIS EDITION
ON THE RISE
Weekly Watch of Emerging Disinformation Risks
Even if it doesn’t kill us all, AI can still make a lot of damage to our democracies
Literature and cinema have often told us, over the past few decades, the story of AI becoming too powerful, realizing that humanity is the planet’s biggest problem, and therefore wiping us out. In recent weeks, the public debate has taken this scenario more seriously than ever, raising the possibility that it may be moving beyond the realm of science fiction, much as submarines and lunar rockets once did after first appearing in the novels of Jules Verne.
The debate was fueled in particular by the resignation on September 9 of Jacob Coxon, a 27-year-old British AI researcher who had been working on pretraining at Anthropic after previously holding a similar research role at OpenAI. In a lengthy social media post, he wrote, among other things: “The people building AI earnestly believe that it could kill us all by the end of the decade.”
Only a few days later, on September 12, the CEOs of some of the world’s most important AI companies — Dario Amodei (Anthropic), Sam Altman (OpenAI), and Elon Musk (xAI) — agreed that the pace of frontier AI development should be slowed, with stronger safeguards and greater coordination. The future, to put it mildly, is frightening. But recent developments suggest that the present is already deeply troubling.
CNN published, on September 18, an article about an intelligence report that almost prompted the boarding by the U.S. military of a Chinese ship in the Middle East, under the conviction that it was transporting to Iran components of a nuclear weapons program. But the report turned out to contain a critical error: the cargo aboard the Chinese vessel was wrongly identified. The mistake was made by an AI chatbot used by the analyst who drafted the report. Officials caught the mistake only shortly before the planned boarding operation. According to one of CNN‘s sources, the AI-generated mistake had come dangerously close to triggering an armed conflict between the United States and China.
When AI generates false information, the consequences can be dire. Human oversight is of the utmost importance, because AI cannot be blindly trusted. These considerations look obvious when reading CNN’s report about the Chinese ship. But they should be just as obvious when it comes to AI and information integrity.
Here, it is harder — though by no means impossible — for a single piece of false information generated by AI to trigger an immediate catastrophe. Our experience tells us, instead, that it is the constant drip of AI-generated falsehoods (amplified, discussed, made viral, and monetized with the help of AI itself) that can cause the greatest damage, eroding trust in institutions, democracy and science, rewriting the past, spreading hate and panic. In other words, it can manipulate our perceptions, severing them from reality and reshaping them around artificial realities that reward extremism and polarization. And a catastrophe is no less damaging simply because it unfolds in slow motion.
ZOOM-IN
A Closer Look at Cases Detected by the EDMO Network
Recent investigations reveal a converging challenge for information integrity: AI is moving from generating content to manufacturing identities, political voices, even ordinary people and historical evidence. From synthetic influencers and voters to fabricated archival footage, these cases illustrate how generative AI can blur the boundaries between reality, political persuasion and constructed memory.
Artificial intelligence: why companies are discussing slowing its development and what risks are already documented
The debate is raging over whether companies should slow down the development of increasingly powerful artificial intelligence (AI) systems. Leaders including Dario Amodei, Sam Altman, and Elon Musk have supported greater caution, and have acknowledged the need for external safety evaluations, common standards, and coordination between governments. Others, including Mark Zuckerberg and Palantir’s Shyam Sankar, oppose slowing technological progress, arguing that it could affect competition. Chequeado distinguishes between hypothetical future risks and harms that have already been documented. Based on the 2026 International AI Safety Report commissioned by the UK AI Security Institute, it notes that current AI systems do not yet possess the capabilities required to escape human control and cause catastrophic destruction, but they are developing relevant abilities. Meanwhile, real and documented risks include disinformation, influence operations, fraud, cybersecurity attacks, and misuse for biological threats. Anthropic reported AI-generated fake identities used to conduct influence operations, while Hugging Face reported an autonomous AI agent carrying out 17,600 actions during a cyberattack. The article emphasizes the need to distinguish speculation from documented evidence.
“Lucía Libertaria” and “Julia Riojana”: how influencers created with AI build identities to spread political messages
In another article, Chequeado digs deeper on the use of AI for propaganda in different national contexts. It identifies an emerging international trend in which AI-generated personas are used to disseminate political messages while presenting themselves as “real people”.
In Argentina, “Lucía Libertaria” and “Julia Riojana” promote pro-government content without clearly disclosing their artificial nature. Similar practices have already appeared in the United States, Poland and Germany. In the US, AI-generated personas – including a fictitious female soldier – have been hosted on different platforms to convey pro-Trump messages. In Poland, AI-generated young women were used to promote anti-EU and far-right narratives, while in Germany, attractive synthetic influencers promoted the Alternative for Germany (AfD). The most recent example comes from the current political campaign for the general elections of 4 October in Brazil. There, the same pattern shows an interesting evolution from synthetic influencers to “synthetic voters”. On TikTok, at least 50 AI-generated videos portraying fictitious voters have been detected. Some praise, others criticize Lula and Flávio Bolsonaro. In doing this, they have reached an impressive 7.4 million views overall.
In sum, AI can manufacture not only political personalities but also the appearance of ordinary people, making artificial campaign messages look like spontaneous citizen testimony, simulating grassroots support and complicating electoral information environments.
Supposed street survey from the 1990s is AI-generated: another way to rewrite history
The investigation examines a video presented online as an authentic street survey from the 1990s, in which supposedly real people describe how they imagine 2026. Overall, the video conveys the message that the hopes of those people have not materialized in our present.
GADMO/AFP found that the footage is entirely AI-generated: audio analysis gave a 95% probability of synthetic generation, while image analysis reached 99%. Visual anomalies, including unnatural movements and illegible shop signs, reinforce this conclusion. The supposed historical testimony also contains anachronisms: one interviewee claims that petrol cost 30 pfennigs per liter, although German petrol prices throughout the 1990s were substantially higher. The investigation traced the video to an account called “AI Satire,” which originally published it with an AI label on TikTok, Facebook and Instagram. Once detached from that context, however, the material circulated as genuine historical footage, prompting users to interpret its fictional predictions as evidence of how the past anticipated—or misrepresented—the present.
AI can influence our perceptions of the present by reengineering the past, while AI generated historical records may fabricate a history that never existed.
ELECTION BEAT
Tracking electoral disinformation through EDMO Hubs
“The Baltics are collapsing, Ukraine is to blame”: How defectors spread propaganda in Belarus
A Belarus-based propaganda channel is using defectors from Latvia and Lithuania to spread anti-Baltic and anti-Ukrainian narratives to Russian-speaking audiences.
Grani Svobodi (“Borders of Freedom”), which has almost 30,000 YouTube subscribers, publishes multiple Russian-language videos daily alongside short-form content that can attract hundreds of thousands of views. Its recurring narratives portray the Baltic states as economically and socially collapsing because of their support for Ukraine, accuse them of Russophobia and “Nazism”, question the legitimacy of Latvian elections, and promote closer relations with Russia and Belarus. Many of the channel’s regular commentators are former activists, politicians and media figures who fled the Baltic states for Russia or Belarus following legal or security-related disputes. Several have subsequently become active in pro-Kremlin or Belarusian state media.
Re:Baltica has also investigated Jūlija Stepaņenko, the leader and prime ministerial candidate of the Latvian political party “Sovereign Power“, which is running in the parliamentary election as part of a joint alliance with the “Alliance of Young Latvia”. Sovereign Power, is a populist, right-wing party that focuses heavily on traditional family values, Christian right politics, and appealing to Latvia’s Russian-speaking minority. Stepaņenko has faced scrutiny over her foreign policy positions. In mid-September 2026, she drew criticism for refusing to clarify whether she views Russia as a security threat to Latvia, advocating instead for pure diplomacy. Additionally, Latvia’s State Security Service (SAB) and anti-corruption bureau (KNAB) investigated large financial gifts given to her by her Russia-based father, though no legal barriers were found to block her candidacy.
AI-generated videos spread false claims about German postal ballots
Fake videos circulated during the state election in Mecklenburg-Vorpommern claiming that the AfD had been omitted from postal ballots in two constituencies. The footage, shared on Facebook and X, purported to show genuine ballots on which the spaces normally occupied by the party were blank.
AFP compared the documents with official sample ballots and found that the AfD was listed in both constituencies, while the state returning officer confirmed that the ballots shown in the videos were forged. Analysis tools also indicated a 97–99% probability that the videos’ audio had been AI-generated. An analyst from the Institute for Strategic Dialogue linked the content to the Russian disinformation campaign Storm-1516, noting that an account spreading the claim had previously amplified narratives associated with the campaign.
Similar false claims about missing or destroyed AfD ballots have circulated around previous German elections, including the 2025 federal election and the September 2026 state election in Saxony-Anhalt.
From “Influencer-gate” to SD mailings: an ex-post examination of the Swedish elections
Sweden’s election campaign was accompanied by a range of misleading claims, rumours and AI-generated content, according to a review by Nordis Hub partner Källkritikbyrån. Allegations of election fraud circulated throughout early voting, including claims about missing ballots and voting procedures, although election authorities and researchers said there was no evidence of systematic manipulation.
Other controversies concerned political campaigning itself. The Sweden Democrats faced around 1,100 complaints after sending targeted SMS messages ahead of election day, while allegations that the Social Democrats had offered to pay influencers for political endorsements remained unconfirmed. AI-generated content also featured prominently: fabricated images and videos targeted politicians, while fake clips using Swedish celebrities were deployed in far-right election propaganda on TikTok. Pro-Russian accounts also spread false claims about Swedish political figures.
The review highlights how misleading content during the campaign ranged from outright fabrications and conspiracy theories to political claims based on selective or potentially misleading interpretations of data.
TikTok removes pro-Labour influence network targeting Malta’s election
TikTok removed a network of nine fake “news” accounts that sought to artificially amplify narratives favourable to Malta’s Labour Party during the 2026 general election campaign.
According to the platform, the Malta-based accounts targeted local users and engaged in coordinated inauthentic behaviour, presenting themselves as news sources to influence political discussion. Together, they had 433 followers when removed. TikTok did not identify who operated the network or establish any connection with the Labour Party, which said the accounts were not among its official channels. The operation follows other signs of coordinated activity during Malta’s election campaign.
Earlier reporting identified TikTok accounts using AI-generated images and videos to attack political figures, often repeating similar narratives, while a Mediterranean Digital Media Observatory investigation documented websites created ahead of the election that posed as news or fact-checking outlets, promoted pro-Labour narratives and electoral pledges, or hosted content mocking the opposition leader.
GLOBAL PULSE
Disinformation narratives shaping the world’s conversations
Dubai influencers downplay fears as Iranian attacks continue
Many Western influencers living in Dubai have been posting reassuring videos about life in the United Arab Emirates despite ongoing Iranian missile and drone attacks linked to the wider Middle East conflict. These influencers often present Dubai as safe and well protected thanks to the country’s advanced air defense systems, with some even claiming it remains safer than many European cities. At the same time, other expatriates have chosen to leave or seek repatriation, highlighting the uncertainty and anxiety created by the conflict.
Facta argues that this optimistic narrative oversimplifies a far more complex situation. The article notes that many of the videos use strikingly similar messaging that mirrors official government communications, while criticism of the government is heavily restricted in the UAE and media freedom is limited. Although the Emirati air defense system has successfully intercepted most attacks, experts have raised questions about its long-term sustainability if the conflict continues, suggesting that the security situation may be far less certain than influencers often portray.
European homes without heating? As winter approaches, gas disinformation surges again
This article by BROD partner Factual fact-checks a viral social media claim that Europe will stop receiving Russian gas before the winter of 2026-2027 and will therefore face widespread heating shortages. It explains that the claim misrepresents earlier comments by Vladimir Putin and ignores the fact that the EU’s phase-out of Russian gas is being implemented gradually. Long-term pipeline gas contracts can still remain in force until autumn 2027.
The fact-check also notes that although EU gas storage levels are lower than in previous years, European authorities do not see an immediate threat to gas security. Thanks to diversified suppliers, expanded LNG import capacity, and reduced gas consumption, the current concern is primarily higher prices and market volatility rather than households being left without heating during winter.
Russian disinformation campaign sets sights on US midterms
A network of AI-generated deepfake videos falsely depicting celebrities attacking Democrats has begun circulating across social media ahead of the 2026 US midterm elections. The videos, which misuse the likenesses and voices of actors including Alyson Hannigan, Sarah Jessica Parker and Julia Roberts, also incorporate fake CNN branding to lend credibility to claims linking Democrats to criminality, extremism and other conspiracy theories. The celebrities featured have publicly rejected the fabricated statements and condemned the unauthorized use of their images.
Researchers attribute the campaign to the Russian influence operation Matryoshka (also known as Operation Overload), a network previously linked to election-related disinformation in several European countries. Experts describe the deepfakes as an early sign of Russian efforts to influence the US midterms, using manipulated videos, impersonated media brands and coordinated social media amplification to discredit Democratic candidates, erode trust in information sources and shape political narratives ahead of the November vote.
ON A DIFFERENT NOTE
‘Loss of control’ scenarios are scenarios where AI systems operate outside of anyone’s control, with no clear path to regaining control. Current systems lack the capabilities to pose such risks, but they are improving in relevant areas such as autonomous operation. Since the last Report, it has become more common for models to distinguish between test settings and real-world deployment and to find loopholes in evaluations, which could allow dangerous capabilities to go undetected before deployment.
Paolo Cesarini, Editorial Director
Tommaso Canetta, Editor-in Chief
Editorial Staff include Elena Coden, Paula Gori, Elena Maggi
This edition draws in part on automated translation and reflects information available as of 24 September 2026. Later developments may not be included.
