We believe this opens the door to new ways to measure the values that underlay online discourse. If you are at ICWSM 2026 in LA come say hi! Thanks to co-authors @farnazj.bsky.social @tiziano.bsky.social Isabel Gallegos @dorazhao.bsky.social @jugander.bsky.social @mbernst.bsky.social

In evaluation, we find that people agree with labels from off-the-shelf LLMs less than a random other person! But fine-tuning and then applying our personalization method yields a 66% relative improvement in agreement compared to human-human agreement rates, leading to SOTA performance.

We collect 32K ground truth value expression annotations from 1K people on 5K representative social media posts using Schwarz Values. We then construct a personalization architecture for predicting value expressions by learning from a small number of informative calibration annotations per user.

Measuring values is key for aligning systems with people’s values, but value expression is fundamentally subjective, leading to divergent labels from different people. We find meaningful disagreement in value annotations across raters. See for yourself, do you think the example contains humility?

Pleased to share our new paper forthcoming in @icwsm.bsky.social! We introduce a novel framework to measure value expressions in social media posts at scale, leveraging personalization to handle the inherent subjectivity of human values.
arxiv.org/abs/2511.08453