Introducing 🥚EGGROLL 🥚(Evolution Guided General Optimization via Low-rank Learning)! 🚀 Scaling backprop-free Evolution Strategies (ES) for billion-parameter models at large population sizes
⚡100x Training Throughput
🎯Fast Convergence
🔢Pure Int8 Pretraining of RNN LLMs
hello 🦋 ! I'll be at NeurIPS next week presenting our work on using learnability to select levels for RL autocurricula. If you're there, I would love to chat about curricula and RL generalisation more broadly. Please DM if you'd like to grab a coffee :)
The good reviewer party™, a party at conferences for reviewers whose ACs said they did a great job. ML operates on FOMO so I think it'd work
Hello BlueSky! Joao Henriques (joao.science) and I are hiring a fully funded PhD student (UK/international) for the FAIR-Oxford program. The student will spend 50% of their time @UniofOxford and 50% @MetaAI (FAIR) in London, while completing a DPhil (Oxford PhD). Deadline: 2nd of Dec AOE!!