London Mayor Sadiq Khan has voiced alarm following the circulation of a deepfake audio clip copying his voice, which nearly incited ‘serious disorder’ within the city. The fake audio, generated by AI, falsely depicted Khan making inflammatory remarks before Armistice Day, suggesting a preference for pro-Palestinian marches over commemorations for Armistice Day.

According to Mr. Khan on the BBC’s Why Do You Hate Me? Podcast, the AI-generated fake content had the potential to exacerbate the tense situation around Armistice Day, nearly leading to significant unrest. He cautioned that the proliferation of deep fake audio could pose serious concerns in various scenarios, including tight electoral contests, referendums, or instances of societal tension.

Why Does It Matter?

Khan criticised the apparent inaction of social media platforms in promptly notifying authorities or removing the misleading content, as he did with the Ultra Low Emission Zone (ULEZ) in London.

Mayor Khan’s condemnation of the incident highlights grave concerns regarding the inadequacy of current legal frameworks and approaches in addressing the challenges posed by AI-generated deepfakes. Notably, the lack of accountability for the creator of the deceptive audio underscores the potential peril such fabrications pose to public order and the integrity of democratic processes, including the 2024 UK general election.