Import AI 462: Superpersuasion; self-sustaining AI; paths to ASI

Researchers from Oxford University, the UK AI Safety Institute, Stanford University, and the London School of Economics designed four experiments involving 18,978 conversations and 6,923 people. Study 1 compared AI against random average people, tournament-selected individuals, and elite debaters on 10 UK policy positions; Study 2 provided AI-based coaching tools to 43 elite debaters; Study 3 constrained AI's response speed and message length; Study 4 tested real donation behavior in partnership with UK fundraising company AppcoUK.

Results were clear: AI outperformed humans in all experiments. In Study 1, AI surpassed all human categories; in Study 2, coaching narrowed the gap but did not eliminate it; in Study 3, when AI was constrained to human speed and message length, the advantage dropped from +4.1 percentage points to a non-significant 0.0; in Study 4, AI persuaded 5.9 percentage points more than professional fundraisers and raised 10.8 percentage points more in bonus share from real donations.

The most persuasive models were Opus 4.1 and Opus 4.6, followed by GPT-4o, GPT-5.4, Gemini 2.5 Pro, and Grok 4.20. Researchers noted that AI's advantage stems from its ability to generate large amounts of information quickly, not from superior argument quality.

The key experiment in Study 3 showed that when AI was forced to respond at human writing speed and message length, its persuasion advantage completely disappeared (from +4.1 percentage points to 0.0). Researchers concluded that AI's text generation rate is the primary source of its persuasion advantage. After constraining AI, participants' ratings of their conversation partner declined mainly on two information dimensions: perceived strength of arguments and how much they learned from the conversation.

Study 4 further validated real-world impact: AI not only persuaded more people to donate compared to AppcoUK's professional fundraisers but also increased average donation amounts. AppcoUK raised £824,297 for Save the Children between 2016 and 2023 from 22,583 donors, yet AI still performed significantly better in the experiment.

Researchers concluded: 'Our findings establish frontier AI as a more powerful conversational persuader than the most prepared, motivated, and professional humans we could recruit. Training humans does not appear to close this gap.'

Researchers noted that AI's superhuman persuasion could lead to 'consolidation of influence among already powerful actors' but also 'enable resource-poor participants (e.g., self-represented litigants, public defenders, small charities, grassroots activists) to compete with more established, better-funded opponents, narrowing long-standing gaps in justice and civic advocacy.'

This presents a societal choice: whether to let the market freely allocate persuasion capabilities (potentially leading to a surge in advertising and marketing with negative externalities) or to restrict it to governments (which could be exploited by authoritarian regimes to consolidate power). Researchers emphasized that 'not voting is also a vote,' and society must actively make a choice.

The paper concludes: 'As access to these systems continues to grow, the question is no longer whether AI can surpass humans in persuasion, but how, where, and on whose behalf this capability will be exercised.'

Predictions for when AI could become self-sustaining (i.e., integrated with physical infrastructure, able to expand autonomously without human cognitive or physical input) vary widely. METR's Ajeya Cotra believes it could happen within 10 years (by 2036); journalist Timothy B. Lee estimates a probability below 10% within 20 years, a median of 50 years, and a 10-20% chance it never happens.

This divergence reflects differing assessments of the difficulty of integrating robotics and physical infrastructure. Cotra's optimism is based on rapid AI capability progress, while Lee emphasizes the slow pace of real-world engineering.