🚨 Content alert 🚨
New recording and workshop materials published!
➡️ Large Language Models for Social Research: Potentials and Challenges
👤 @indiiigo.bsky.social (University of Mannheim)
📺 youtu.be/p5wPJHK-74M
🗒️ github.com/SocialScienc...
👋 #ACL2025NLP 🇦🇹 @marlutz.bsky.social and I are presenting our poster on demographic representativeness of LLMs today!
🕦 10:30-12:00
📍 Hall X5 (board 1 or 14 according to different sources 🧐)
Here’s the paper on ACL anthology: aclanthology.org/2025.finding...
Drop by!
Chair for Data Science in the Economic and Social Sciences at University of Mannheim having lots of fun at #ic2s2 @janajung.bsky.social @wanlo.bsky.social @indiiigo.bsky.social @jrupprec.bsky.social @maximiliankreutner.bsky.social and Stefano Balietti
Really excited to also present this work at #IC2S2 next week in Norrköping! 🎉 I'd love to discuss how to produce LLM survey responses at my poster on Wed at 13:30 (Poster Session 2, Poster ID 68) 📊
A research setup for the evaluation of Answer Production Methods for closed-ended survey responses from LLMs. An LLM is prompted with a survey and an optional instruction, before a Answer Production Method is applied. These methods range from token-probabilities to open-ended text generation + classification. I then evaluated them against human survey answers and calculate individual-level accuracy as well as distribution alignment for sub-populations.
LLMs are trained to produce open-ended responses 📝, but most survey items require closed-ended responses instead 📊
This Wed 11:00–12:30 at #ESRA25, I'll discuss the large impact that Answer Production Methods have on prediction results + share recommendations for methods and parameters. 👈
Very excited to head to #IC2S2 next week! 🎉
In our project, we tested whether a psychological assessment can measure sexism in LLMs, and found that applying such tools to LLMs is not as straightforward as it seems.
Find me and my poster at Poster Session 1 (Tue 12:30-14:30) — hope to see you there
LLMs can generate synthetic survey responses, e.g. for imputation, but how reliable are they? 📋
At #IC2S2, I'll be sharing our research on the robustness of AI-generated responses to perturbations and if they mirror human survey biases. 🤖
Come by my poster on Tuesday between 1:30 and 3:30 p.m.