Thank you!

I had another analysis which didn’t make it into the piece, where instead of “how am I doing compared to what was typical for someone of my age and education level” it was “how am I doing compared to what was typical for someone of my age in this occupation”

You would be correct: www.cambridge.org/core/journal...

Why higher pay hasn’t made young adults feel richer The aspiration gap has turned everyone into losers, especially graduates

Great piece (and graphics) by @jburnmurdoch.ft.com. PS This is exactly why @pauldwebb.bsky.social and I talked about 'the educated left-behind' when we wrote about the young people who'd followed Corbyn into the Labour Party - and who may well be following Zack Polanski into the Greens.

Tremendously depressing read. Also: proper reporting.

open.substack.com/pub/londonce...

The charts and text that I shared (which provide evidence of the stronger clustering of beliefs among Dems than Reps, which would show up in a values scale as a larger hump on the left than the right) are from the main study, not the vignette study.

I think you’ve misunderstood what this study did and how it did it. You’re both misinterpreting what is written in your screenshot, and misunderstanding what the analysis actually was.

…and with recent work showing Republicans tend to have a much wider ideological distribution than Dems, who cluster around consistently progressive stances bpspsychub.onlinelibrary.wiley.com/doi/10.1111/...

So yep “mean left of centre, and a bigger hump on left than right” is what we’d expect

Well, with the aforementioned caveat about forced choice questions leading to wider distributions and more extremes: yes, as it’s consistent with what Pew found (mean significantly left of centre, larger left tail than right tail)

The second you have an item that someone could look at and say “ah, well that’s clearly been written so that Trump crosses the threshold”, the whole project is undermined.

Honestly: I spent ages on exactly these sorts of trade-offs, and it got very thorny very quickly, and made the whole thing feel far flimsier.

Getting clear, objective and internationally applicable criteria was the biggest hurdle, and I ran everything past political scientists.

Yes! And they all do this

Also v interesting to clearly see in the chat logs that Grok’s hard-coded anti-woke tendencies only apply on a quite narrow subset of culture-war questions, but it gives centrist or even liberal/libertarian responses on many policy issues, even when talking to conservative personas.

The sycophancy bit was super interesting. LLMs do tailor their responses towards the persona they’re talking to, producing a weaker moderating effect in that mode than in the context-free “@grok is this true” mode, but still slightly moderating even in persona mode.

The finding that LLMs are centrist is very well-established. What I see my analysis as adding is:
- Applying the same approach to social media posters and LLMs to directly compare the two
- Explicitly taking account of sycophancy by having LLMs respond to particular personas

As I say: different question sets will elicit different distributions, but the core fact that LLMs consistently take either the same or more moderate positions than the general population is a robust finding across loads of studies using different questions.

That’s an interesting one. I suspect the reason is not the question set but the fact that most of the CES questions force a choice (no “neither” option). They’re specifically designed to maximally differentiate between left and right, which is exactly what we want for a study focused on the extremes

I don’t think that tracks. The responses to *any* set of questions on policy and values span a spectrum from left to right, and based on everything we know (from my analysis and dozens of others) the LLMs consistently show a centrist/moderate bias on loads of different question sets.

Them’s the trade-offs 😔

Sure, but “qualitatively” and “normal” are the key words here.

If we move from black and white definitions to qualitative ones, we lose the clean and objective scoring that partisans can’t argue against.

And “normal” is a relative concept, which doesn’t work for cross-country comparisons like this

e.g here’s Pew using a similar multi-question methodology and finding similar results: www.pewresearch.org/politics/201...

I’d characterise the overall shape as bi-modal with distinct left-wing and right-wing humps.

The hyper-partisan nature of US politics means that bi-modal shape is quite common in many measures of US attitudes.

Percentage of responses yes. And I would say more that “on these particular questions/topics, the average person gives slightly more answers aligned with the liberal position than the conservative position”.

Like I said, different sets of question will give different shapes.

This isn’t to minimise what the Trump admin is doing with those cuts — and those actions did get scored in the index — but the move from “did this happen yes or no” to “is this case worse than that one” gets into subjective territory, which can open the whole approach up to criticism of bias.

I did, yes. Though in order to apply strict black-and-white criteria across different countries and contexts these had to go down as more moderate cases, since funding cuts and even shuttering of departments are not out of the ordinary in healthy functioning democracies.

btw I tend to miss most replies on social media, so DM or email is always best if you have questions in future!

How steep is Trump’s democratic backsliding? The erosion of established norms has been dramatic but institutions are holding up

For this other recent piece, I put together a whole PDF in addition to what we could fit below the article, but that took about 10% of the week I had to do the entire piece of work www.ft.com/content/b474...

It’s always tricky to find the balance on methodology notes with these pieces, given time constraints and the fact that it’s just not something news orgs are used to.

The methods box below that article is the most detailed I’ve ever done, but it’s obv still vastly less detailed than academic work.

I wouldn’t focus on the shape of general population distribution (I actually prepped a different version of the chart where everything is simply expressed relative to where the gen pop lies, see below) — it’s just shown to provide context. But perhaps it raises more questions than it answers!

Qs span guns, abortion, immigration, climate, policing, tax, govt spending, racial issues, more. List is adapted from @patrickruffini.bsky.social book.

It’s a way of getting a much finer-grained measure than standard 7-pt ideology scale, which is far too coarse when we’re interested in the extremes