title section of the paper: “Cross-Modal Taxonomic Generalization in (Vision) Language Models” by Tianyang Xu, Marcelo Sandoval-Castañeda, Karen Livescu, Greg Shakhnarovich, Kanishka Misra.

What is the interplay between representations learned from (language) surface forms alone, and those learned from more grounded evidence (e.g.,vision)?

Excited to share new work understanding “Cross-modal taxonomic generalization” in (V)LMs

arxiv.org/abs/2603.07474

1/

How to start the year off right 😊

Our commentary in Nature Mental Health proposes using AI and VR to simulate experiences similar to those produced by traditional psychedelics, offering a novel pathway to advance psychedelic-assisted therapy research🤩

Link 👉 www.nature.com/articles/s44...

The real problem isn’t p-values, it’s the epistemology behind them.

Science shouldn’t depend on arbitrary thresholds that change with context.

Knowledge should accumulate, not collapse into yes/no verdicts.

Turning continuous evidence into discrete “significance” decisions is information loss

a woman in front of a white board with the words take your time written on it ALT: a woman in front of a white board with the words take your time written on it

📢 Deadline extended! 📢

The registration deadline for #SNS2025 has been extended to Sunday, September 28th!

Register here 👉 meg.medizin.uni-tuebingen.de/sns_2025/reg...

PS: Students of the GTC (Graduate Training Center for Neuroscience) in Tübingen can earn 1 CP for presenting a poster! 👀

Are top-down feedback connections enough for robust vision?

We found ConvRNN with top-down feedback exhibiting OOD robustness only when trained with dropout, revealing a dual mechanism for robust sensory coding

with @marco-d.bsky.social, Karl Friston, Giovanni Pezzulo & @siegellab.bsky.social

🧵👇

I still get chills

Meet Mike
*30+ years severe depression
*first hospitalized @ 13y
*20 meds
*3 rounds of ECT
*2 near-fatal suicide attempts

Mike felt joy for the first time in decades after we turned on his new brain pacemaker or PACE

see videos, read paper, follow thread
doi.org/10.31234/osf...

🔵 Proud and honoured to share a project I had the chance to contribute to.
It’s been a long journey of research and collaboration, exploring how humans and deep neural networks perform in sound localization 👂📍
Check it out: www.biorxiv.org/content/10.1...

🧠⏱️ New preprint!
We found that temporal prediction errors are more strongly encoded when an overt response is required, and this encoding occurs in motor rather than sensory space.

www.biorxiv.org/content/10.1...
🧵👇

Honored to receive the ✨HIH Paper of the Year Award✨ for our work on predictive learning published in @natcomms.nature.com last year!

Huge thanks to @hih-tuebingen.bsky.social and all collaborators — this recognition means a lot! 🙏

Check out the paper here 👇
www.nature.com/articles/s41...

Word learning is usually about what a word does refer to. But can toddlers learn from what it doesn’t?

Our new Cognition paper shows 20-month-olds use negative evidence to infer novel word meanings, reshaping theories of language development.

www.sciencedirect.com/science/arti...

🤩New preprint out! 🎉 We explore the neural signatures of predictive coding during the acquisition of incidental sensory associations.
Huge thanks to @agreco.bsky.social for leading this work!

The European Workshop on Reinforcement Learning is coming to Tübingen! 🌸

🙌 Hot off the presses @natcomms.nature.com! We created a custom #Minecraft environment to study a long-standing puzzle in cognitive science:
How do humans flexibly adapt their individual and social learning strategies in dynamic, realistic situations? Check it out 👉 www.nature.com/articles/s41... 🧵👇

Neural dynamics of semantic control underlying generative storytelling - Communications Biology This study reveals the neural dynamics underlying the regulation of semantic control during generative storytelling, highlighting high-level and visual areas as cortical hubs for generating novel and ...

✨ Proud of this collaboration with
@agreco.bsky.social , Chiara Finocchiaro, Gabriele Penazzi, Christoph Braun & Nicola De Pisapia
📄 Read the full paper:
www.nature.com/articles/s42...

💡In sum, our results highlight the neural mechanisms underlying the regulation of semantic exploration during naturalistic narrative production and contribute to a deeper understanding of the neural mechanisms
underlying the role of semantic control in generative storytelling.

#creativityscience

⏳ Remarkably, we also found greater temporal irreversibility during creative ideation between visual and high-level areas, suggesting the enhanced hierarchical structure of causal interactions as a neural signature of creative storytelling.

🧠 Moreover, creative narratives showed unique (between and within) functional interactions across the default mode, salience, and control networks.

🧠 Next, we also applied Connectome Harmonic Decomposition (CHD) to neural dynamics into connectome harmonic modes and found specific spatial frequency patterns underlying the modulation of semantic control during story generation.

🧠 Using decoding analysis, we showed that creative stories were neurally distinct, differentiated from ordinary stories in frontal regions, and from random ones in fronto-temporo-parietal areas.
Crucially, the same brain regions encoded the semantic features that behaviorally defined creativity.

🧠 At the neural level, linguistic and visual areas exhibited neural synchrony across participants regardless of the semantic control level, with parietal and frontal regions being more synchronized during random ideation.

🗣️To quantify creativity and task adherence, we used word2vec to compute semantic distance (novelty) and BERT to estimate surprise (appropriateness). Creative stories fell between ordinary and random ones on both dimensions.

🧠 We isolated the ideation phase from the reporting phase, letting us track brain dynamics during silent story generation. No motion, no speech artifacts.

🎯 We recorded fMRI activity while participants generated 3 types of stories:
✔️ Ordinary (appropriate)
✔️ Random (novel)
✔️ Creative (balanced in novelty & appropriateness)

📖 Storytelling is central to human experience, but how do we regulate semantic exploration during narrative generation?
We used fMRI and deep language models to find out.

🧠 What happens in the brain when we turn ideas into stories, and what makes a story truly creative?

Our latest study explores the neural mechanisms supporting semantic control during narrative ideation.

#fMRI + #LLMs
Out in Communications Biology @commsbio.bsky.social

A thread 👇

<em>Psychophysiology</em> | SPR Journal | Wiley Online Library Verbal creativity in generating novel metaphors is widely recognized, but its electrophysiological basis has not been investigated. By applying brain state analysis to EEG data, we identify oscillato...

My latest installment of brain state modeling in creativity is out @ Psychophysiology! We found generating novel metaphors is associated with widespread alpha-band synchronization early on in ideation and, paradoxically, alpha-desynch right before response. onlinelibrary.wiley.com/doi/full/10....

Andrew Barto and Richard Sutton have won the A.M. Turing Award for developing the theoretical foundations of reinforcement learning, a key method behind many major breakthroughs in artificial intelligence. 🧵

🚨 New Preprint!!

LLMs trained on next-word prediction (NWP) show high alignment with brain recordings. But what drives this alignment—linguistic structure or world knowledge? And how does this alignment evolve during training? Our new paper explores these questions. 👇🧵

Language-like efficiency in whale communication Whale vocalizations follow efficiency rules seen in human language, revealing striking similarities in communication systems.

Do whales optimize their vocalizations for efficiency, just like human language? 🐋🎶 My latest study in
Science Advances (@science.org) suggests they do—following linguistic laws seen in human speech. 🧵 www.science.org/doi/10.1126/...

Cognitive psych is often traced back to the 1950s.
Psychology is often traced to Wundt's lab in 1879.
Textbooks speak of "proto-psychologists" like Fechner & Helmholtz in the 1800s... Westerners all.

Have you heard about Ibn al-Haytham, whose work predated them by 100s of years?

1/4