LONDON: Conversational artificial intelligence (AI) tools may soon “covertly influence” users’ decision-making in a new commercial frontier called the “intention economy”, University of Cambridge researchers warned in a paper published on Monday.

The research argues the potentially “lucrative yet troubling” marketplace emerging for “digital signals of intent” could, in the near future, influence everything from buying movie tickets to voting for political candidates. Our increasing familiarity with chatbots, digital tutors and other so-called “anthropomorphic” AI agents is helping enable this new array of “persuasive technologies”, it added.

It will see AI combine knowledge of our online habits with a growing ability to know the user and anticipate his or her desires and build “new levels of trust and understanding”, the paper’s two co-authors noted.

“Tremendous resources are being expended to position AI assistants in every area of life, which should raise the question of whose interests and purposes these so-called assistants are designed to serve”, said Leverhulme Centre for the Future of Intelligence Visiting Scholar and co-author Dr Yaqub Chaudhary.

Increasing familiarity with AI emerging tools is helping enable this new array of ‘persuasive technologies’

Left unchecked, that could allow for “social manipulation on an industrial scale”, argued in the paper published in the Harvard Data Science Review. It characterises how this emergent sector — dubbed the “intention economy” — will profile users’ attention and communicative styles and connect them to patterns of behaviour and choices they make.

“AI tools are already being developed to elicit, infer, collect, record, understand, forecast, and ultimately manipulate and commodify human plans and purposes,” co-author Yaqub Chaudhary said.

Dr Jonnie Penn, another co-author, and Chaudhary write that the intention economy will be the attention economy “plotted in time”: profiling how user attention and communicative style connect to patterns of behaviour and the choices we end up making.

“While some intentions are fleeting, classifying and targeting the intentions that persist will be extremely profitable for advertisers,” said Chaudhary, according to University of Cambridge.

The new AI will rely on so-called Large Language Models — or LLMs — to target a user’s cadence, politics, vocabulary, age, gender, online history, and even preferences for flattery and ingratiation, according to the research.

That would be linked with other emerging AI tech that bids to achieve a given aim, such as selling a cinema trip, or steer conversations towards particular platforms, advertisers, businesses and even political organisations.

Penn said: “For decades, attention has been the currency of the internet. Sharing your attention with social media platforms such as Facebook and Instagram drove the online economy”, reported by the University of Cambridge. Penn warned: “Unless regulated, the intention economy will treat your motivations as the new currency.” “It will be a gold rush for those who target, steer, and sell human intentions,” he added.

Published in Dawn, December 31st, 2024