📖 So fun to collaborate with amazing folks at USC (Xiang Ren, @swabhs.bsky.social ) and Apple (Rik Koncel-Kedziorski, Tim Paek) on this work! 🤝💡📣
Excited to finally share this—let us know what you think! 🙌🏽✨🎉
More details in our draft here: arxiv.org/abs/2504.17993
(8/n)
🌎 PB&J improvements do not hinge on any particular group—we see gains across education, race, income, and gender. 🌈📊🤝
(7/n)
💡 It’s not just about having more reasoning tokens—it’s about what those tokens convey. 🧐🔍✨
We found that *human-written rationales* improved personas the most, but LLM-generated ones came close, showing that good scaffolds are very useful for personas. 🧑💻📈🔬
(6/n)
📊 We tested PB&J on two tasks: predicting opinions and movie ratings. PB&J consistently outperformed methods based on demographics and/or judgments. Using scaffolds boosted performance even more! 🎥🗳️🏆
(5/n)
🔧 We propose psychological scaffolds—frameworks based on life experiences, personality traits, or core beliefs—to further guide the structure of LLM rationales. 🛠️🧬🎨
(4/n)
🧠 We introduce PB&J (Psychology of Behaviour and Judgments) that draws from folk psychology—the way people naturally explain each other’s actions. PB&J adds LLM-generated rationales to personas, helping models better represent user behavior. 📚🧩💬
(3/n)
🤖 LLMs can mimic human behavior with personas, but most current approaches focus only on demographics or past responses as user context, often overlooking the reasons why users think or act a certain way. 🧐💭🔄
(2/n)
Reasoning about the "why" behind user behavior can improve LLM personas! ✨🧠📈
📝Excited to share our new work: Improving LLM Personas via Rationalization with Psychological Scaffolds
🔗 arxiv.org/abs/2504.17993
🧵 (1/n)
omggg it’s amazing 😭 combined together gave me queen vibes!!
🙋🏻♀️🙋🏻♀️
UGH ADORABLE
omg this can be 3D printed at the makerspace!!
These days, I’ve been obsessed with reading about how people use LLMs and this blog by Nicholas Carlini has been my TOP read since August - nicholas.carlini.com/writing/2024...
Not a lot of us who work with LLMs “use” them, and this is such a great starter to all of that!
@justachetan.bsky.social!!!
Netflix needs to stop asking if I’m still watching and start asking if I moved the laundry to the dryer
#EMNLP2024 has been a treat so far! Of course it doesn't hurt to win an outstanding paper award with my incredible PhD students Jaspreet Ranjit and @brihi.bsky.social and our wonderful collaborators at USC 😜
Paper: dill-lab.github.io/oath-frames/