Barely a month after Pakistan Tehreek-i-Insaf (PTI) employed artificial intelligence (AI) to generate an audio speech of incarcerated former Prime Minister Imran Khan for an online rally, social media platforms have become awash with AI-produced voiceovers of political figures.

From AI-created songs to promotional images, the rise of free AI voice-cloning and image-editing tools has turned electioneering into a steady stream of campaign materials created by the technology, introducing a new playbook for digital politics.

Previous national elections have showcased the increasing sophistication of digital political campaigns, featuring tactics such as creating fake accounts, running coordinated hashtag trends, developing customized mobile apps, as well as staging grand social media “conventions”. With Pakistan heading into its largest digital election in February, political parties are tempted to push the envelope and explore different ways to wage the technology’s recent intrusion into politics.

“We used AI to create content for Nawaz Sharif’s arrival campaign ahead of his Minar-i-Pakistan jalsa in October,” Muzakir Ijaz, a digital media consultant leading PML-N’s digital strategy, told Dawn.

There is a difference between reality and AI-generated content. We had to go over 36 iterations in Urdu to reach 65pc diction accuracy. People can tell it’s not real —
Jibran Ilyas, PTI’s social media lead

“AI is useful for generating image content but it is not good at integrating Urdu yet so our use is limited,” said Ijaz, adding that he onboarded a team of 30 digital media professionals to support PML-N’s election campaigning.

While the use of AI may be cost-effective for social media teams, who would otherwise require expensive experts to analyse data and create promotional campaigns, the dissemination implications may be troubling.

Last year, an altered video of Imran Khan purportedly showed him with his eyes closed, raising alarm among his supporters about his treatment in prison. The 26-second clip was posted on X, formerly known as Twitter, where it received more than 500 shares. An AFP fact-check found that the altered video appeared to use a “closed-eyes filter”, which gave the illusion that Khan’s eyes were shut.

“The implications of AI use are concerning,” said Nighat Dad, Executive Director of the Digital Rights Foundation (DRF). “Political parties are using AI to influence the mindset of voters who are not digitally literate and politically charged. How will they tell apart what is genuine media and what is synthetic media?”

Ms Dad, who is also a member of the UN’s Advisory Body on AI, warned questions about ethics on the use of AI were yet unanswered. “In this context, the use of AI can lead to erosion of the public’s trust in the integrity of the information they consume,” she said.

“The good thing is there is a difference between reality and AI-generated content,” Jibran Ilyas, PTI’s social media lead told Dawn.

“We had to go over 36 iterations in Urdu to reach 65pc diction accuracy. People can tell it’s not real,” he said.

Ilyas said PTI was mindful of AI’s use and added a clear disclaimer to Khan’s online rally audio flagging it as the “AI voice of Imran Khan based on his notes”.

“We are testing a lot of ways to use AI but also considering ethics. We might consider another Khan speech right before the polling day,” he said, adding that the party would be focusing on TikTok for the days leading up to the elections given its popularity among the youth and rural areas.

Digital literacy at a glance

The number of internet users in Pakistan has increased significantly since the last national elections in 2018. According to PTA, in 2018, there were 55 million mobile broadband subscribers and 58 million broadband internet users, but by November 2023, there were 126 million mobile broadband users and 129 million broadband internet users (54 per cent of the population).

Given the significant growth in digital access, the Election Commission of Pakistan (ECP) has for the first time extended its code of conduct for national elections to digital media and social media influencers. The code prohibits the dissemination of content that is prejudicial to Pakistan’s ideology, sovereignty, dignity, or security, or that harms national solidarity or creates law and order problems.

Despite working with tech platforms on digital literacy initiatives in the lead-up to the polls, the ECP does not offer any guidelines or accountability on the funding and responsible use of technology by political parties for campaigning.

According to the latest study on how university students understand disinformation by Freedom Network for the Coalition Against Disinformation (CAD), most young people (63pc of the respondents) believe they come across disinformation on the Internet every day and that disinformation poses a threat to democracy and elections (62pc).

“The value of truth has eroded,” said Amber Rahim Shamsi, Director of the Center of Excellence in Journalism (CEJ) at the Institute of Business Administration (IBA). “We conducted digital literacy workshops in 13 universities for 800 students around Pakistan. Across the board, we found young people are still not able to recognize the difference between fact and propaganda,” she said.

Shamsi and her team recently launched a non-partisan fact-checking initiative, iVerify, to advance the scope of independent and unbiased reporting in the country’s journalistic landscape. “Since 2018, media organisations have become more conscious of fact-checking. However, none of the students we spoke to are aware of these [fact-checking] efforts. There is still a lot that needs to be done.”

Published in Dawn, January 16th, 2024