In-Mind: Psychology for You! (@in-mindmagazine.bsky.social) 🔔 𝗡𝗲𝘄 𝗮𝗿𝘁𝗶𝗰𝗹𝗲: 𝗔 𝗻𝗲𝘄 𝗽𝗲𝗿𝘀𝗽𝗲𝗰𝘁𝗶𝘃𝗲 𝗼𝗻 𝗹𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗮𝗻𝗱 𝗰𝗼𝗴𝗻𝗶𝘁𝗶𝗼𝗻 🔔 How can computers help us understand language learning? 🤖 From readability scores to…

📊 𝗛𝗶𝗴𝗵𝗹𝗶𝗴𝗵𝘁𝘀 𝗳𝗿𝗼𝗺 𝟮𝟬𝟮𝟱:

📑 33 articles were published on In-Mind.

📌 The most interacted article on Bluesky was “Language models: A new perspective on language and cognition” by @boevesam.bsky.social.
👉 Read the post here again:
bsky.app/profile/in-m...

📑 𝗪𝗵𝗮𝘁 𝘄𝗲 𝗽𝘂𝗯𝗹𝗶𝘀𝗵𝗲𝗱
In 2025, we published 33 articles and 12 blog posts.
📌 The most interacted article on Bluesky was “Language models: A new perspective on language and cognition” by @boevesam.bsky.social
👉 Read it again here: 
bsky.app/profile/in-m...

👁️ 📖 👁️
@boevesam.bsky.social
made this interactive visualisation to get a feeling for word predictability:
🔗 wordpredictabilityvisualized.vercel.app

Curious how these predictability indices were obtained? Find out in our new paper!
🔗 doi.org/10.3758/s134...

#Reading #LargeLanguageModels #MECO

Want to explore word predictability yourself on a sample of each corpus used in this work, check out this app:

wordpredictabilityvisualized.vercel.app

GroNLP/gpt2-small-dutch · Hugging Face We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Modelling reading times in Dutch?:

gpt2-small-dutch (huggingface.co/GroNLP/gpt2-...) or gpt2-medium-dutch-embeddings (huggingface.co/GroNLP/gpt2-...) are great options.

3. Predictability effects are also logarithmic in Dutch, corroborating effects found in English (= linear effect of surprisal):

For very unpredictable words, a decrease in predictability has a much larger slowing-down effect on reading times than the same decrease for highly predictable words.

2. Language-specific models are generally better than multilingual ones (multilingual models are shown in blue in the figure below).

Key findings 📝

1. Smaller Dutch models often predict reading times better (= inverse scaling trend) ~ in line with evidence of English models.

But, with more context (in a book reading corpus), larger models catch up.

Large language models are powerful tools for psycholinguistic research.

But, most evidence so far is limited to English.

How well do Dutch open-source language models fit reading times using their word predictability estimates?

A systematic evaluation of Dutch large language models’ surprisal estimates in sentence, paragraph and book reading - Behavior Research Methods Studies using computational estimates of word predictability from neural language models have garnered strong evidence in favour of surprisal theory. Upon encountering a word, readers experience a pro...

🚨 B&B is open for business 🚨

Not a new career move, the next Boeve & Bogaerts paper is out in Behavior Research Methods!
doi.org/10.3758/s134...
@bogaertslab.bsky.social

✨Playback at #teap2025 (Part 1)
Thanks to the amazing speakers and the audience for the successful symposium on #languagemodels in #psycholinguistics!

Katharina Menn, @hannawoloszyn.bsky.social, @boevesam.bsky.social, Marco Marelli, Fritz Günther, @benjamingagl.bsky.social

Proud PI! 👏 On #TeaP2025 two lab members presented their work:

Haoyu Zhou in the symposium #StatisticalLearning and its Role in #Language and #Reading acquisition.

@boevesam.bsky.social in the symposium From Babies to Semantics: Leveraging #LanguageModels for #Psycholinguistic Research.

A Systematic Evaluation of Dutch Large Language Models’ Surprisal Estimates in Sentence, Paragraph, and Book Reading A psychometric evaluation of Dutch large language models. Hosted on the Open Science Framework

Overall, our results provide a psychometric leaderboard of Dutch large language models, ideal for researchers interested in effects of predictability in Dutch.

Check out our full dataset and code here:
osf.io/wr4qf/

Finally, we found a linear link between surprisal and reading times except for the GECO corpus where a non-linear link between surprisal and reading times fitted the data best.

A challenge to the notion of an universal linear effect of surprisal.

Second, smaller Dutch models showed a better fit to reading times than the largest models, replicating the inverse scaling trend seen in English.
However, this effect varied depending on the corpus used.

First, across three eye-tracking corpora, we found that in each case, a Dutch LLMs' surprisal estimates outperformed the multilingual model (mGPT) and the N-gram model in predicting reading times.

3.

Does surprisal still show linear link with reading times when estimated with a Dutch-specific language model as opposed to a multilingual model?

2.

Do these Dutch-specific LLMs show a similar inverse scaling trend as English models?

That is, do the smaller transformer models' surprisal estimates account better for reading times than those of the very large models?

1.

What is the best computational method for estimating word predictability in Dutch?

We compare 14 Dutch large language models (LLMs), a multilingual model (mGPT) and an N-gram model in their ability of explaining reading times.

The effect of word predictability on reading times is well established for English but not so much for Dutch.

We adressed this and asked three questions:

A Systematic Evaluation of Dutch Large Language Models’ Surprisal Estimates in Sentence, Paragraph, and Book Reading A psychometric evaluation of Dutch large language models. Hosted on the Open Science Framework

Ending the year on a high note with the submission of a new preprint:

A Systematic Evaluation of Dutch Large Language Models’ Surprisal Estimates in Sentence, Paragraph, and Book Reading

Preprint: dx.doi.org/10.13140/RG....
OSF: osf.io/wr4qf/