🚨 New paper alert 🚨
Ever asked an LLM-as-Marilyn Monroe who the US president was in 2000? 🤔 Should the LLM answer at all? We call these clashes Concept Incongruence. Read on! ⬇️
1/n 🧵
Do LLMs Think Like Humans?
They find that,
While LLMs achieve broad categorical alignment with human judgment, they falter in capturing fine-grained semantic nuances such as typicality and, critically, exhibit vastly different representational efficiency profiles.
Kudos to @nytimes.com for covering ARC-AGI in such an exquisite example of interactive data journalism. Amazing spot for @fchollet.bsky.social as well.
www.nytimes.com/interactive/...
The funny thing about multimodal image generation as released in the last week by Google and OpenAI is that now LLM image generation works like how most people using LLMs for the past two years always thought LLM image generation works.
It ought to be a property of a world model, but a language model models language. We use language capacities to describe counterfactuals, argue about facts, and imagine all the time. What assistants are lacking is any reasonable kind of world model. We need to marry the stochastic to the symbolic.
If you have used LLM image generators, you know they are hard to control: LLM had to send a prompt to a separate image generation tool, it did not not make the image.
Gemini is the first public release of a full multimodal LLM that can directly make images. This allows the systems to do detail work
A histogram showing the distribution of numeracy scores for students in the U.S. vs. other benchmark countries. The histogram uses an icon array made up of little figures of people, using the `weepeople` font. The chart has the title "U.S. Numeracy Education has room for improvement"
📊 #dataviz Putting the people 👨👨👧👦back into charts that talk about them.
An interesting histogram of numeracy scores for U.S. vs. some other countries,
using Alberto Cairo's [weepeople font](github.com/propublica/w...) to show the people involved in these distributions.
Src: bit.ly/3FrDq0v
👇
As part of the Science Homecoming project I wrote an opinion piece for the Albuquerque Journal on the dangers of cuts to federal science funding:
www.abqjournal.com/opinion/arti...
@sciencehomecoming.bsky.social
@cantlonlab.bsky.social
@spiantado.bsky.social
The public really needs to understand this. Every university system in the world rests on public funding, there has never been an alternative model at any time in history.
We have universities for literally the same reason that we have roads and armies.
1/13 LLM circuits tell us where the computation happens inside the model—but the computation varies by token position, a key detail often ignored!
We propose a method to automatically find position-aware circuits, improving faithfulness while keeping circuits compact. 🧵👇
This is a crazy paper. Fine-tuning a big GPT-4o on a small amount of insecure code or even "bad numbers" (like 666) makes them misaligned in almost everything else. They are more likely to start offering misinformation, spouting anti-human values, and talk about admiring dictators. Why is unclear.
Removing the gears part results in better performance, and that's surprising because it feels different from how humans learn. Perhaps relatedly, though anecdotally, telling e.g. an image generator what you didn't like about the previous response results in more, not less, of what you didn't like.
Screenshot of a twitter post showing that the latest openAI commercial model is better than previous models at doing arithmetic but still cannot reliably produce the correct answer of multiplication problems with values greater than 11 x 11. It's supposed to be impressive I think
you fucked up a perfectly good computer is what you did. look at it. it's got innumeracy
«Kim pointed to newer introductory offerings such as “Python for Humanities and Social Sciences,” “AI for Future Presidents” and “C Programming Language and Linux.”» and it's still available free online www.edx.org/cs50
Love the homage to Richard Muller, too!
Are linguists paying a lot of attention to LLMs? Because this seems like a fascinating finding with large implications: LLMs share highly abstract grammatical concept representations, even across unrelated languages, so even models trained mostly on English do well in other languages.
Tech oligarchs made their fortunes thanks in large part to government funded research done by scientists based in universities. The tech industry’s complicity in dismantling these govt agencies and higher ed is not only immoral, it’s also shortsighted. Where will new science breakthroughs come from?
We launched a bunch of Gemini 2.0 models today. Compared to the 1.5 series models, each of the 2.0 models is generally better than the "one size up" model in the 1.5 series.
2.0 Flash & Flash-Lite set new standards in the quality/cost Pareto frontier.
More details:
blog.google/technology/g...
open-Deep-Research by huggingface
as posted by @aymeric-roucher.bsky.social
An entirely open agent that can: navigate the web autonomously, scroll and search through pages, download and manipulate files, run calculation on data...
man stepping on rake labelled 4o man skatedboarding down stairs on a rake before getting hit in face labelled o3-mini-high
Exa & Deepseek R1 Chat App
Exa & Deepseek Chat App is a free and open-source chat app that uses Exa's API for web search and Deepseek R1 LLM for reasoning.
github.com/exa-labs/exa...
The Internet Archive has to date downloaded 500 terabytes of US government websites, which it crawls at the end of every presidential term. The whole archive is fully searchable. This effort's housed by a donation-funded nonprofit, not a branch of the US government. blog.archive.org/2024/05/08/e...
Text Shot: Professor Karsten emphasized the potential global impact of this development, noting that if major tech companies like Amazon, Google, and Meta choose to implement this method in their data centers, it could lead to savings of gigawatt-hours of energy worldwide.
Researchers claim Linux kernel tweak could reduce data center energy use by 30% www.techspot.com/news/106501-linux-kernel... #AI #climate
As someone who has reported on AI for 7 years and covered China tech as well, I think the biggest lesson to be drawn from DeepSeek is the huge cracks it illustrates with the current dominant paradigm of AI development. A long thread. 1/
The image depicts a monumental statue of Buddha, emphasizing serenity and grandeur. The statue's intricate design captures traditional Buddhist features, including a meditative posture with hands placed in a symbolic gesture, flowing robes, and a calm facial expression exuding peace. The perspective highlights the statue's immense size against a minimalistic white sky background, underscoring its significance as a spiritual and cultural landmark.
Explainer: What's R1 and Everything Else
This is an attempt to consolidate the dizzying rate of AI developments since Christmas. If you're into AI but not deep enough, this should get you oriented again.
timkellogg.me/blog/2025/01...
I'm not sure if people realize how quickly the Trumpzis can do enormous damage to US science, from basic research to translation. Really fast. REALLY fast. Labs with decades of irreplaceable domain and technique knowledge can break apart with a surprisingly short funding gap. When they're gone...1/
Next big thing for brands: knowing what sites agents prefer.
If you ask for stock prices, Claude with Computer Use goes to Yahoo Finance while Operator does a Bing search
Operator loves buying from the top search result on Bing. Claude has direct preferences like 1-800-Flowers
We don't know why
Worth also pointing out that there are many "tests so easy no AI system can pass them".
Moravec's paradox remains.
E.g., arxiv.org/abs/2404.12390
The new ability of AI video creators to add real people and products to scenes with just an image is likely to increase the utility (& more worryingly, misuse) of AI video.
Here I made Shakespeare at a cafe and the Girl with the Pearl Earring piloting a mech (just as Vermeer intended)
In December, I posted about our new paper on mastering board games using internal + external planning. 👇
Here's a talk now on Youtube about it given by my awesome colleague John Schultz!
www.youtube.com/watch?v=JyxE...
