"measuring [LLM] agents’ relative issue saliences using party manifestos as a source of data is a harder problem than measuring their issue positions"
New article from @kenbenoit.bsky.social and Michael Laver on the challenges with LLMs measuring issue salience
Read here: doi.org/10.1080/0140...
Thrilled that my paper (w/@sarahobolt.bsky.social,@catherinedevries.bsky.social,@simonecremaschi.bsky.social) was accepted at the American Political Science Review!
We find that declining public services fuel support for the populist right — and show why the right benefits more than other parties 🧵
8/8 “Love them or hate them, LLMs are poised to drive a new measurement paradigm in political science, making it essential to establish clear and robust protocols to ensure their valid and replicable use”
Read here: onlinelibrary.wiley.com/doi/full/10....
7/8 The implications are profound and potentially revolutionary for social science research. Tasks that once required millions of dollars and decades of work can now be replicated quickly, cheaply, and in practically any language.
6/8 To test the reproducibility, the authors re-ran the entire analysis three months later and got virtually identical results. They then replicated with three open-weight models (DeepSeek, LLaMA, Gemma) and got the same findings again.
5/8 LLM estimates of party positions correlated with expert benchmarks at .87–.92 across most policy dimensions and match the upper bound of what you'd expect from two independent groups of human experts.
4/8 First, they ask LLMs (GPT-4o, Claude, and Gemini) to summarize what a manifesto says about a given policy issue. Then they ask them to score that summary on a structured 7-point scale. Average across models for robustness.
3/8 “Text-as-data” tools count words without understanding meaning and struggle to capture the nuances, including the intensity of positions expressed.
To address this puzzle, the authors developed methods for using LLMs as a scientific tool and found them to be reliable and valid.
2/8 However, the scale, language, and financial barriers make time-consuming qualitative studies inefficient and rare (eg “The Manifesto Project”). Another quantitative alternative is the “text-as-data” method, which was cheaper, more efficient but came with significant shortcomings.
AJPS article from @kenbenoit.bsky.social, Scott de Marchi, Conor Laver, Michael Laver, and Jinshuai Ma on using LLMs to analyse political texts.
🧵 1/8 Social science research requires reliable extraction of information from texts to identify authors' preferences on various policies and issues.
This analysis draws heavily on polling by @ipsosintheuk.bsky.social for @news.sky.com - there is a clear difference between the 2024 Reform vote and those who have defected from the Conservatives since (not to mention with the Tory loyalists who are still sticking with the party).
"Reform and the Greens are the electoral expression of that [Brexit] cleavage finally finding its natural home."
New @politicallyinsane.bsky.social write-up about @sarahobolt.bsky.social and James Tilley's new book on Brexit's continuing political impact
politicsinsane.substack.com/p/tribal-pol...
🚨 New POAL brief from @victoraraujo.bsky.social & @miriamsorace.bsky.social out now: how to properly evaluate the causal impact of shocks, debates, and political crises on public opinion.
Diagnostics, pitfalls, examples … all in three pages!
Read here: www.poal.co.uk/POAL_brief_U...
May be of interest: @opiniumresearch.bsky.social @veriangroup.com @deltapoll.bsky.social @anthonyjwells.bsky.social @adamdrummond.bsky.social @joetwyman.bsky.social
New POAL methods brief out from @miriamsorace.bsky.social 🚨
Outdated policy‑insight frameworks = bad strategy, as exemplified by the government's current woes.
Here’s how to fix how we track what voters think about policy: www.poal.co.uk/miriam_brief...
"The strategic use of negative campaigning in political fundraising is likely to continue through this next election cycle"
Check out @politicallyinsane.bsky.social's write-up on Substack about recent research from POAL members on negative campaigning
politicsinsane.substack.com/p/defeat-the...
📣 Call to all passionate political Behaviour scholars !
Are you an academic in Political Behaviour ?
Do you know people who you think would be great to discuss political behavior on our podcast?
Then reply to this by (i)tagging them here and (ii)repost !
Looking forward to your suggestions!
The latest episode, hosted by Sarina Sharma-Welsh in conversation with @miguelpereira.bsky.social, is about how politicians learn what voters want, why elite blind spots persist, and how representation can get distorted.
Listen here: pod.link/1851725697/e...
The third episode features @sarahobolt.bsky.social in a discussion of political entrepreneurs such as Farage and Trump and how they are disrupting traditional party system.
Listen here: pod.link/1851725697/e...
The second episode is with Dr Elisa Wirsching, discussing power struggles between politicians and public servants and explore how everyday decisions — and even elections — can be shaped by the clash between them.
Listen here: pod.link/1851725697/e...
The first episode is a discussion on the effect of campaigns with @florianfoos.bsky.social. They discuss how the relationship between political elites and citizens is evolving, whether persuasion still happens, and how it might happen.
Listen here: pod.link/1851725697/e...
We’re delighted to announce that the Politically In/Sane podcast is now part of POAL!
Hosted by Marc Bokobza, the podcast dives into Political Behaviour, exploring how people think, feel, and act politically with insights from prominent experts.
Find it here: www.poal.co.uk/events/
Delighted that our @jwfurlong.bsky.social @academic.oup.com book The Changing Electoral Map of England and Wales has been shortlisted for the @polstudiesassoc.bsky.social Mackenzie Book Prize (along with five other excellent books!). www.psa.ac.uk/news/mackenz...
Read the full report here -> www.southampton.ac.uk/publicpolicy...
📢 JOB ALERT! Postdoc opportunity in Political Behaviour & Political Economy (UKRI‑funded) - Please do share with anyone who might be a great fit. If you’re interested or would like to know more, please feel free to get in touch!! jobs.reading.ac.uk/Job/JobDetai...
For the historical polling nerds out there, an article about our project with @ropercenter.bsky.social that digitised ~800 surveys by Gallup poll in Britain between 1955 and 1991 has been published in JEPOP. The merged dataset contains over three-quarters of a million respondents.
"Safeguarding data quality is no longer a single checkpoint but an end-to-end pipeline - one that reallocates substantial resources from sampling to continuous fraud monitoring"
Read our new POAL Methods Brief by @laurenleek.eu on dealing with AI bots in surveys!
Link: www.poal.co.uk/briefai.pdf
"While testing one dimension at a time can yield simple results, those effects may not generalise to richer, real-world contexts."
Read our new POAL Methods Briefs on Conjoint Experiments from Thomas Robinson!
Link: www.poal.co.uk/research/met...
Tariffs might be the word of 2025.
Given the state of *looks around* we asked: is there demand for tariffs in Europe?
@grahn.bsky.social @katharinalawall.bsky.social @sophiemainz.bsky.social Maria Nordbrandt & I show the answer is a resolute no
@jeppjournal.bsky.social
doi.org/10.1080/1350...
Nano-targeting is not a persuasive weapon: it is a dissuasion and polarization weapon. New paper out osf.io/preprints/so... with the brilliant Thomas Robinson, @simonhix.bsky.social and @fresejoris.bsky.social – the first to quantify the perverse incentives/inherent democratic dangers of this tech.