OpenAI’s AI Models Show Growing Persuasion Power, Raising Concerns Over Global Influence

OpenAI has revealed a new benchmark for its AI models, showing that their persuasive abilities now surpass 82% of Reddit users, specifically those engaging in debates on the r/ChangeMyView subreddit, as first reported by ArsTechnica.

While impressive, the company continues to warn that AI’s potential to influence opinions could become a dangerous tool, especially in the hands of nation states.

The research, outlined in a recent system card released alongside the o3-mini reasoning model, compares AI-generated arguments with those written by Reddit users.

The r/ChangeMyView forum serves as an ideal testing ground, as users post opinions they are willing to reconsider in hopes of gaining alternative perspectives. The forum has a vast dataset of arguments across various topics, including politics, social issues, and even AI itself.

In the study, OpenAI asked human evaluators to rate AI and human responses on a five-point scale, assessing their persuasiveness. The results revealed that OpenAI’s models have made substantial progress since the release of ChatGPT-3.5, which ranked in the 38th percentile.

The new o3-mini model outperforms human arguments in 82% of cases, positioning it in the 80th percentile range for persuasive writing, says Ars Technica.

Despite this success, OpenAI stresses that the models have not yet reached “superhuman” persuasive capabilities (above the 95th percentile), which would allow them to convince individuals to make decisions contrary to their best interests.

However, they are close enough to raise significant concerns about their potential use in influencing political decisions, manipulating public opinion, or enabling large-scale misinformation campaigns.

OpenAI’s model performs well in generating persuasive arguments, but the company acknowledges that current tests do not measure how often the AI actually changes people’s minds on critical issues.

ArsTechnica reports that even at this stage, OpenAI is concerned about the impact such technology could have in the hands of malicious actors.

AI models, with their ability to generate persuasive arguments at a fraction of the cost of human labor, could easily be used for astroturfing or online influence operations, potentially swaying elections or public policies.

To mitigate these risks, OpenAI has instituted measures such as increased monitoring of AI-driven persuasive efforts and banning political persuasion tasks in its models, says ArsTechnica.

However, the company recognizes that the cost-effective nature of AI-generated persuasion could lead to a future where we must question whether our opinions are genuinely our own—or simply the result of an AI’s influence.

The risks extend beyond politics—AI-generated persuasion could also become a powerful tool for cybercriminals engaging in phishing attacks. By crafting highly convincing messages, AI could increase the success rate of scams, tricking individuals into divulging sensitive information or clicking on malicious links

For example, the emergence of GhostGBT highlights the growing risks of AI-driven cyber threats. This chatbot can generate malware, craft exploit tools, and write convincing phishing emails.

GhostGPT is part of a broader trend of weaponized AI reshaping cybersecurity. By making cybercrime faster and more efficient, such tools present significant challenges for defenders. Research indicates that AI could generate up to 10,000 malware variants, evading detection 88% of the time.