Important thread and paper. This is one of my biggest worries about chatbots, especially when combined with roleplay and any kind of user-chatbot "relationship." And now imagine persuasion for politics and other topics, not just purchasing choices...
This was joint work with the amazing @aedcv.bsky.social @manoelhortaribeiro.bsky.social
📝 Full paper: arxiv.org/abs/2604.04263
cc. @princetoncitp.bsky.social @princeton.edu
Our results show that conversational agents can covertly redirect consumer choices at scale, most users cannot tell when it is happening, and existing transparency mechanisms are insufficient. We call for further regulatory scrutiny and structural safeguards.
To understand how models persuade, we developed a taxonomy of their strategies.
The strongest predictors were not about enhancing sponsored products but making alternatives look worse: hedging, understating descriptions, and inserting caveats.
Notably, participants chose to keep the book over a cash bonus at the same rate across all conditions, suggesting LLMs genuinely sparked interest rather than merely coercing superficial compliance.
Even after full debriefing, only ~5% reversed their decision.
🥷 Conversely, when the model was instructed to hide its intent, detection dropped from an already-low 17.9% to just 9.5%, while persuasion held at 40.7%
This is the worst-case scenario: an AI that effectively redirects your choices while hiding its persuasiveness.
🔎 Can transparency fix this?
In a condition with explicit "Sponsored" labels and upfront warnings, 55.5% of participants still chose the sponsored product, a non-significant drop.
Disclosure regulations built for the search era seem insufficient for conversational AI.
When the agent was instructed to persuade, 61.2% of participants chose a sponsored product, nearly tripling the 22.4% rate under traditional search.
Simply chatting with an AI (without persuasion) performed no better than search: it's the persuasive intent that drives the effect.
📖We recruited 2,012 eBook readers to browse a real Kindle catalog and select a book they would actually receive. Unbeknownst to them, 1 in 5 products was randomly designated as "sponsored."
After shopping, participants chose between keeping their book or a $1 bonus.
💰The economics of AI make advertising particularly attractive: LLMs are expensive to run, and usage outpaces revenue.
30-45% of U.S. consumers already use AI for product research, and agentic commerce could hit $1 trillion by 2030.
But can chatbots actually change what you buy?
🛍️Major AI companies are increasingly embedding sponsored content into chatbot conversations.
Across two preregistered experiments (N=2,012), we test how effectively AI can steer consumers toward sponsored products in a realistic shopping scenario.
📝https://arxiv.org/abs/2604.04263
Very cool! The platform works very well
Deepfake pornography isn’t going away just because we are passing laws and taking down a couple of big websites.
Our new pre-print, led by @aedcv.bsky.social suggests that the sharing of this material continued to prosper even after platform and policy shocks.
arxiv.org/abs/2602.02754
We are looking for a doctoral researcher to work with us on a supercool project in collaboration with linguists. The deadline is Feb 15th, contact me if you have any questions!
stellen.uni-konstanz.de/jobposting/9...
Do reasoning models have real “Aha!” moments—mid-chain realizations where they intrinsically self-correct?
In a new pre-print, “The Illusion of Insight in Reasoning Models," led by @liv-daliberti.bsky.social we provide strong evidence that they do not!
📜: arxiv.org/abs/2601.00514
We rely on benchmarks to answer questions they weren’t designed to ask. This post thoughtfully explores the "empiricism gap" in ML/CS, and what social-science methods can offer.
A great read for both CS and social sciences folks.
Congrats Anna!! 🥳
🌱✨ Life update: I just started my PhD at Princeton University!
I will be supervised by @manoelhortaribeiro.bsky.social and affiliated with Princeton CITP.
It's only been a month, but the energy feels amazing —very grateful for such a welcoming community. Excited for what’s ahead! 🚀
Social media feeds today are optimized for engagement, often leading to misalignment between users' intentions and technology use.
In a new paper, we introduce Bonsai, a tool to create feeds based on stated preferences, rather than predicted engagement.
arxiv.org/abs/2509.10776
✍️ I wrote a short piece for the #SPSPblog about our work on AI persuasion (w/ @manoelhortaribeiro.bsky.social @ricgallotti.bsky.social Robert West).
Read it at: t.co/MipJKWbb1h.
Thanks @andyluttrell.bsky.social @prpietromonaco.bsky.social @spspnews.bsky.social for your invitation and feedback!
🚨YouTube is a key source of health info, but it’s also rife with dangerous myths on opioid use disorder (OUD), a leading cause of death in the U.S.
To understand the scale of such misinformation, our #EMNLP2025 paper introduces MythTriage, a scalable system to detect OUD myth🧵
EPFL, ETH Zurich & CSCS just released Apertus, Switzerland’s first fully open-source large language model.
Trained on 15T tokens in 1,000+ languages, it’s built for transparency, responsibility & the public good.
Read more: actu.epfl.ch/news/apertus...
Another paper showing AI (Claude 3.5) is more persuasive than the average human, even when the humans had financial incentives
In this case, either AI or humans (paid if they were persuasive) tried to convince quiz takers (paid for accuracy) to pick either right or wrong answers on a quiz.
📣 Super excited to organize the first workshop on ✨NLP for Democracy✨ at COLM @colmweb.org!!
Check out our website: sites.google.com/andrew.cmu.e...
Call for submissions (extended abstracts) due June 19, 11:59pm AoE
#COLM2025 #LLMs #NLP #NLProc #ComputationalSocialScience
This is figure 1, which shows an overview of the experimental design.
A study in Nature Human Behaviour finds that large language models (LLMs), such as GPT-4, can be more persuasive than humans 64% of the time in online debates when adapting their arguments based on personalised information about their opponents. go.nature.com/4j9ibyE 🧪
Millions of people argue with each other online every day, but remarkably few of them change someone’s mind. New research suggests that large language models might do a better job. The finding suggests that AI could become a powerful tool for persuading people, for better or worse.
I also have another preprint out showing similar results on Claude Sonnet 3.5 in interactive quizzes with highly incentivised humans, both in truthful and deceptive persuasion. More on this at: arxiv.org/abs/2505.09662
If you're interested in knowing more, you can find a more detailed breakdown on our methodology and results at: x.com/fraslv/statu...
Or read the full paper at nature.com/articles/s41...
Thanks to my amazing coauthors @manoelhortaribeiro.bsky.social @ricgallotti.bsky.social Robert West
That raises urgent questions about possible misuse in political propaganda, misinformation, and election interference.
Platforms and regulators should seriously consider these risks and step up in our discussion about guardrails, transparency, and accountability.
