As I shared in the NYT, models often see the data but fail to weigh it like a physician, drifting toward generic "average patient" responses. Context window ≠ Clinical reasoning.
www.nytimes.com/2025/12/03/w...
Check out our editorial on Zazzetti et al (2025)'s paper on synthetic data generation for breast cancer, in JCO CCI! Synthetic data could help with many gaps in clinical AI research, but challenges remain especially (IMO) issues with out-of-domain generalization @shan23chen.bsky.social
🤔💭What even is reasoning? It's time to answer the hard questions!
We built the first unified taxonomy of 28 cognitive elements underlying reasoning
Spoiler—LLMs commonly employ sequential reasoning, rarely self-awareness, and often fail to use correct reasoning structures🧠
Super proud of @shan23chen.bsky.social for his podium presentation on his research into LLM sycophancy in the face of illogical medical queries at #AMIA25!
Full paper: www.nature.com/articles/s41...
Also cited yesterday in the NYT! www.nytimes.com/2025/11/16/w...
LLMs tend to prioritize helpfulness > reason. We show that safety-aware, compute-efficient fine-tuning helps models reason more critically in healthcare domain, and generalizes to improved safety alignment across other domains.
www.nature.com/articles/s41... @shan23chen.bsky.social
An overemphasis on helpfulness makes LLMs vulnerable.
Research shows models will comply with illogical medical requests, generating false information. This sycophantic tendency can be corrected with specific prompting and fine-tuning. #MedSky #MedAI #MLSky
[1/]💡New Paper
Large reasoning models (LRMs) are strong in English — but how well do they reason in your language?
Our latest work uncovers their limitation and a clear trade-off:
Controlling Thinking Trace Language Comes at the Cost of Accuracy
📄Link: arxiv.org/abs/2505.22888
Agents are all the rage and we need to track their abilities in the medical domain. Enter MedBrowseComp, the 1st benchmark to assess agents' abilities to reason, navigate the web, and search for verifiable med info!
Preprint: arxiv.org/abs/2505.14963
Site: moreirap12.github.io/mbc-browse-a...
✨ What if your face could tell something about how old your body really is?
Excited to share our latest paper just published in The Lancet Digital Health (open access!)
👉 www.thelancet.com/journals/lan...
congrats！
CALL FOR REMOTE SPEAKERS: Science in the News Seminar Series, hosted by Harvard x Beacon Hill Seminars
scientists, engineers & doctors, from academic researchers to industry professionals! 🧑🔬🧑💻
Email the organizers at scienceinthenews.bhs@gmail.com to sign up for a date! (First-come-first-served)
We have a NEW PAPER in @naturemedicine.bsky.social on reporting recommendations for addressing the unique challenges of #largelanguagemodels (LLMs) in biomedical applications
www.nature.com/articles/s41...
#MLSky #StatsSky #medSky #AISky #artificialintelligence #generativeAI #transparency
Yea… he does have problems portraying female in stereotypical ways, big critics in China too
During the QA session, one stood up to her regarding this issue really respectfully and her response was: “That was not based on my judgment. That was based on the student's quote saying that the school was not teaching it, which meant that it applied to a lot of people from there."
Most of the talk discussed about bad practices. But only one slide mentioned specific group of people.
Haha which one has more nowadays?
Haha transformers really transformed both.
However, I feel like the division is even further… currently, seems like RL is taking over LM post training and many NLProc are dealing with language model enabled new applications
I am always worrying about Benzene (my cat)! www.nytimes.com/2024/12/05/w...
But please don't stop wearing sunscreen! Sun exposure is a known cancer risk, benzene risks unknown. This article has good tips if you want to minimize benzene exposure.
Obligatory Benzene (cat) pic ⬇️
Thanks!
Imagine a world where these will be positively correlated
Quite possible!
Here, we found some early evidence that SAE features trained on language models are still meaningful to LLaVA.
More details will be provided in the post, and more details will be provided soon!
@JackGallifant
@oldbayes.bsky.social
@daniellebitterman.bsky.social
Team @AnthropicAI & @thesubhashk @joshengels.bsky.social shows SAE features can be good for classifications.
Good evidence by @arthurconmy.bsky.social & @neelnanda.bsky.social on SAE features are transferable across base and IT models.
🧐 How about LLaVA?
tiny.cc/sae1
More on future potential reliance on LLM agent doing reviews and audits
I’m terrified by the massive openreview data. Potentially gonna bite back on us 🥲😥
END/🧵 Thanks to all our awesome co-authors:
@jannahastings.bsky.social
@daniellebitterman.bsky.social
And all our awesome collaborators who are not on the right platform yet! 🦋
Happy Thanksgiving! 🍂
5/🧵 Dive deeper into our methods, findings, and the implications of our research by checking out the full 📜 paper here: arxiv.org/abs/2405.05506
All our data can be downloaded from our website: crosscare.net
4.5/🧵 For the arxiv pretraining dataset, we also have an overall trend based on entity mentions! Guess which two terms are the big bump there back in 2019
4/🧵 We've also developed a new data visualization tool, available at [http://crosscare.net] to allow researchers and practitioners to explore these biases from different pretraining corpus and understand their implications better. Tools in progress! 🛠️📊
3.5/🧵 Moreover, alignment methods don’t resolve inconsistencies in disease prevalence across languages (EN 🇺🇸, ES 🇪🇸, FR 🇫🇷, ZH 🇨🇳). And tuning on English usually only affects English prompt output