This was one of those posts where you toil for a very long time around a point, in this case how open models are keeping up, in order to test every possible way your worldview could be wrong.
When done writing, it becomes obviously clear.
A TLDR is that unless the training dynamics of leading LLMs change or open model builders run out of money, this ~6 month performance gap from closed to open models is here to stay.
www.interconnects.ai/p/reading-to...
A good thread about this from @juliekallini.bsky.social
x.com/JulieKallini...
New video! Talking through my 10+ open model pieces from early 2026 and how they fit together. They're all trying to figure out where open models go next.
www.youtube.com/watch?v=hKIX...
Opus 4.7 has a new tokenizer.
This means it's also a new base model.
Glory days of pretraining still very much going.
The current pace of token-efficient reasoning improvements across minor Claude Opus/GPT model versions is pretty wild. All signs point to this continuing.
4.6 to 4.7 could've been presented as a fairly large model bump in the past with this plot.
i've considered it, but I already maintain a private discord for my blog and I don't know if I have bandwidth for more socials.
I spent some time trying to distill all the complex factors impacting open models -- economics, capabilities, distribution, policy, etc. -- into a clear list of beliefs. Here they are in full.
www.interconnects.ai/p/my-bets-on...
Do the work (and read my post contextualizing why all my recent efforts are important)!
It'll seem like a lot of your audience already knows everything if you've mentioned it once, but repetition is key. People are busy, algorithmic feeds are fickle, and people don't remember points/brands until they've seen it many times.
Many people, especially academics, too far too many projects and invest too little in each. The amount of work just on distributing each project is fairly high. If you don't work on distributing it, don't expect to get new readers.
It pays off so much on the downstream influence of your work to have direct audiences for it. These types of articles are a pure grind, but the directly convert hard earned reputation into success for projects.
One of my key strategies with Interconnects is to develop the practice of making my work obviously compelling to a wider audience, keeping them hooked over time and wondering what I'm up to, etc.
www.interconnects.ai/p/what-ive-b...
I'm going to add question & answer videos throughout the lecture to go deeper on topics that need it, and potentially cover some topics that are too recent and in flux to go in print. I expect 10-15 videos in total over the next few months.
Implementing RL Algorithms for LLMs | RLHF Course Lecture 4
youtu.be/i-AIMpZHgeg
Understanding Policy Gradient Algorithms for RL on LLMs | RLHF Course Lecture 3
youtu.be/K_Sj_-1BUMM
RLHF Foundations, IFT, Reward Modeling, Rejection Sampling | RLHF Course Lecture 2
youtu.be/4gIwiSPmQkU
RLHF and Post-training Overview | RLHF Book Course, Lecture 1
youtu.be/o6l6tJQgUg4
Playlist & welcome video: www.youtube.com/watch?v=jQPi...
Excited to launch the accompanying free RLHF Course for my book. To kick it off, I've released:
- Welcome video
- Lecture 1: Overview of RLHF & Post-training
- Lecture 2: IFT, Reward Models, Rejection Sampling
- Lecture 3: RL Math
- Lecture 4: RL Implementation
Landing page: rlhfbook.com/course
a bit over 7 days out from the Gemma 4 release and it's models are outpacing (slightly) the equivalent Qwen 3.5 models on downloads. Big numbers!
Great stuff happening as we start to build out the codebases for my RLHF book (sorry, I haven't had much time until now!). Very accessible to issues, emails, comments etc to make it better. I'm also going to need another dgx spark.
github.com/natolambert/...
In 5-10 years, as models get more expensive & capable, I see the funding structures and support for open models breaking down. We need to consider if we need other options of supporting the open ecosystem.
The inevitable need for an open model consortium
www.interconnects.ai/p/the-inevit...
Incredible video. Very pure experience in why so many of us love science. www.youtube.com/watch?v=oaXR...
If you’re not concerned about ai progress in some way you’re not paying attention imo
A lot of it is that I don't want to mislead people in what is going to be a lasting book or not. Is a tradeoff of hype for imo a more grounded book.
1. dont fall for anti open model fearmongering, but
2. acknowledge that AI capabilities are proceeding fast, and eventually there may be a reason to be more careful with open weight models
I don't think Mythos is that trigger, but I'm not 100% confident
www.interconnects.ai/p/claude-myt...
Full book online: rlhfbook.com
Buy on Amazon: amzn.to/4cwCDJQ
Buy on Manning (currently big pre order discount): www.manning.com/books/the-rl...
Thank you for all your support. More projects related to the book being announced soon 🎥 I'm excited to reconnect with the community through in-person book events this summer and fall.
Any other topic is too dynamic, where writing a book today would be immediately outdated. RLHF is largely being overshadowed by lots of other developments in AI, but will always be around and at the forefront of human-AI interactions. The topic deserves coverage in depth and this platform.