Virtual information session for Georgetown’s 2-year Master’s in Computational Linguistics! Learn about our courses in NLP, psycholinguistics, low-resource languages, digital humanities, and LLMs, plus phonology, syntax, & semantics. DM for registration link. Friday Nov. 21 | 10–11 AM #linguistics
Screenshot of a figure with two panels, labeled (a) and (b). The caption reads: "Figure 1: (a) Illustration of messages (left) and strings (right) in toy domain. Blue = grammatical strings. Red = ungrammatical strings. (b) Surprisal (negative log probability) assigned to toy strings by GPT-2."
New work to appear @ TACL!
Language models (LMs) are remarkably good at generating novel well-formed sentences, leading to claims that they have mastered grammar.
Yet they often assign higher probability to ungrammatical strings than to grammatical strings.
How can both things be true? 🧵👇
I did not! Yikes! Another reason to include "pickle" and/or pickle-related emoji in any lab communication!
Georgetown Linguistics has a dedicated Computational Linguistics PhD track, and a lively CL community on campus (gucl.georgetown.edu), including my faculty colleagues @complingy.bsky.social and Amir Zeldes.
PICoL stands for “Psycholinguistics, Information, and Computational Linguistics,” and I encourage applications from anyone whose research interests connect with these topics!
I will be recruiting PhD students via Georgetown Linguistics this application cycle! Come join us in the PICoL (pronounced “pickle”) lab. We focus on psycholinguistics and cognitive modeling using LLMs. See the linked flyer for more details: bit.ly/3L3vcyA
🌟🌟This paper will appear at ACL 2025 (@aclmeeting.bsky.social)! New updated version is on arXiv: arxiv.org/pdf/2505.07659 🌟🌟
A key hypothesis in the history of linguistics is that different constructions share underlying structure. We take advantage of recent advances in mechanistic interpretability to test this hypothesis in Language Models.
New work with @kmahowald.bsky.social and @cgpotts.bsky.social!
🧵👇!
We see this project as in line with some other recent papers seeking to cast typological variation in information-theoretic terms, with shout-outs to Michaela Socolof, @postylem.bsky.social @futrell.bsky.social (aclanthology.org/2022.coling-...) and Julius Steuer (aclanthology.org/2023.sigtyp-...)
⭐ ⭐This paper also makes several technical contributions to the mixed-pair mutual information estimation pipeline of Wolf et al., (aclanthology.org/2023.emnlp-m...). Shout out to @cuiding.bsky.social for all her hard work on this aspect of the paper! ⭐⭐
✅In line with our prediction, we find that mutual information is higher in tonal languages than in non-tonal languages. BUT, the way one represents context is important. When full sentential context is taken into account (mBERT and mGPT), the distinction collapses.
🌏🌍We test this prediction by estimating mutual information in an audio dataset of 10 different languages across 6 language families. 🌏🌍
We propose a way to do so using …📡information theory.📡 In tonal languages, pitch reduces uncertainty about lexical identity, therefore, the mutual information between pitch and words should be higher.
🌐But there are intermediate languages, which have lexically contrastive tone, but only sporadically, making some linguists doubt the tonal/non-tonal dichotomy. So, how can we measure how “tonal” a language is? 🧐🧐
🌏 Different languages use pitch in different ways. 🌏 “Tonal” languages, like Cantonese, use it to make lexical distinctions. 📖 While others, like English, use it for other functions, like marking whether or not a sentence is a question. ❓
⭐🗣️New preprint out: 🗣️⭐ “Using Information Theory to Characterize Prosodic Typology: The Case of Tone, Pitch-Accent and Stress-Accent” with @cuiding.bsky.social , Giovanni Acampa, @tpimentel.bsky.social , @alexwarstadt.bsky.social ,Tamar Regev: arxiv.org/abs/2505.07659
I’ll also use this as a way to plug human-scale language modeling in the wild: This year’s BabyLM eval pipeline was just released last week at github.com/babylm/evalu.... For more info on BabyLM head to babylm.github.io
Couldn’t be happier to have co-authored this will a stellar team, including: Michael Hu, @amuuueller.bsky.social, @alexwarstadt.bsky.social, @lchoshen.bsky.social, Chengxu Zhuang, @adinawilliams.bsky.social, Ryan Cotterell, @tallinzen.bsky.social
This version includes 😱New analyses 😱new arguments 😱 and a whole new “Looking Forward” section! If you’re interested in what a team of (psycho) computational linguists thinks the future will hold, check out our brand new Section 8!
📣Paper Update 📣It’s bigger! It’s better! Even if the language models aren’t. 🤖New version of “Bigger is not always Better: The importance of human-scale language modeling for psycholinguistics” osf.io/preprints/ps...
Excited to share our preprint "Using MoTR to probe agreement errors in Russian"! w/ Metehan Oğuz, @wegotlieb.bsky.social, Zuzanna Fuchs Link: osf.io/preprints/ps...
1- We provide moderate evidence that processing of agreement errors is modulated by agreement type (internal vs external agr.)
Me and @wegotlieb.bsky.social were recently invited to write a wide-ranging reflection on the current state of linguistic theory and methodology.
A draft is up here. For anyone interested in thinking big about linguistics, we'd be happy to hear your thoughts!
arxiv.org/abs/2502.18313
#linguistics
⚖️📣This paper was a big departure from my typical cognitive science fare, and so much fun to write! 📣⚖️ Thank you to @bwal.bsky.social and especially to @kevintobia.bsky.social for their legal expertise on this project!
On the positive side, we suggest that LLMs can serve a role as “dialectic” partners 🗣️❔🗣️ helping judges and clerks strengthen their arguments, as long as judicial sovereignty is maintained 👩⚖️👑👩⚖️
⚖️ We also show, through demonstration, that it’s very easy to engineer prompts that steer models toward one’s desired interpretation of a word or phrase. 📖Prompting is the new “dictionary shopping” 😬 📖 😬
🏛️We identify five “myths” about LLMs which, when dispelled, reveal their limitations as legal tools for textual interpretation. To take one example, during instruction tuning, LLMs are trained on highly structured, non-natural inputs.
We argue no! 🙅♂️ While LLMs appear to possess excellent language capabilities, they should not be used as references for “ordinary language use,” at least in the legal setting. ⚖️ The reasons are manifold.
🏛️Last year a U.S. judge queried Chat GPT to help with their interpretation of “ordinary meaning,” in the same way one might use a dictionary to look up the ordinary definition of a word 📖 … But is it the same?