Introducing 🥚EGGROLL 🥚(Evolution Guided General Optimization via Low-rank Learning)! 🚀 Scaling backprop-free Evolution Strategies (ES) for billion-parameter models at large population sizes
⚡100x Training Throughput
🎯Fast Convergence
🔢Pure Int8 Pretraining of RNN LLMs
Late to the party (since I just took some time to spend with our two little ones) but luckily good science is timeless ;)
PQN puts Q-learning back on the map and now comes with a blog post + Colab demo! Also, congrats to the team for the spotlight at #ICLR2025
My group @FLAIR_Ox is recruiting a postdoc and looking for someone who can get started by the end of April. Deadline to apply is in one week (!), 19th of March at noon, so please help spread the word: my.corehr.com/pls/uoxrecru...
That's the first time that I see a video by chess.com cited in an accepted ICLR paper, in particular on handshakes vs. fist bumps during a chess competition ...
By Oxford, @jfoerst.bsky.social
Paper: openreview.net/forum?id=wFg...
Video: www.youtube.com/watch?v=6fS7...
@danielrensch.chess.com
@jfoerst.bsky.social take on how the community sees the ARC Challenge and how we evaluate models and use benchmarks nowadays is 👌.
#more_science_less_hype (please).
PS: Amazing discussion and good brain food, as usual with MLST.
Second #runconference @neuripsconf.bsky.social #NeurIPS2024 !
@jfoerst.bsky.social @ferranalet.bsky.social @adamjelley.bsky.social @enjeeneer.io
Same deal for tomorrow: 7am at
goo.gl/maps/8Z8eMrd...
Join us!
Apply here and list me as the _first_ supervisor: ox.ac.uk/admissions/g...
More information at foersterlab.com. Thanks a lot and happy applying!
🚨 PSA 🚨 Deadline to apply for your dream Phd in ML
@FLAIR_Ox
is coming up on the 2nd of December AOE. We work on compute-only scaling of LLMs, (meta/multi-agent) RL at the Hyperscale, Human-AI coordination, opponent-shaping for vaccine design, GenAI for finance & much more..
correct -- this runs on top of an open-source protocol and the UI is a Twitter clone. How hard can this be?
wth did we not go to an open-source and non-for profit alternative? en.wikipedia.org/wiki/Bluesky
sad times. Joking aside, have you tried pufferlib? I am really curious how it compares and contrasts to JAX RL line of work and haven't seen much direct comparison.
Let's try @josephsuarez.bsky.social
Candidates also need to apply for an Engineering DPhil by 2nd of Dec AOE (if they haven’t already) listing me as the supervisor, www.ox.ac.uk/admissions/g... The student should have an outstanding track record of academic excellence and relevant research experience.
Apply by emailing a CV, personal statement, and research proposal to “fair-flair-2024-applications@googlegroups.com” by 2nd of Dec AOE. Joint interviews will be held in January. Shortlisted candidates will also be invited to apply to FAIR.
The goal is to improve the generalisation abilities and data efficiency of GenAI, e.g. using RL and curriculum learning to train LLMs at the frontier of learnability.
For more details about our work, check out foersterlab.com and joao.science
Hello BlueSky! Joao Henriques (joao.science) and I are hiring a fully funded PhD student (UK/international) for the FAIR-Oxford program. The student will spend 50% of their time @UniofOxford and 50% @MetaAI (FAIR) in London, while completing a DPhil (Oxford PhD). Deadline: 2nd of Dec AOE!!
I got summoned 🫡
Hello world!