Wake up Samurai, RLVG IS BACK! ๐
We are happy to announce the awesome speakers of the second edition of the RLVG workshop: @alexkearney.com, @marloscmachado.bsky.social, Martin Singh-Bloom, Sam Devlin, Pete Wurman and @neuroai.bsky.social!
Join us August 15th at @rl-conference.bsky.social!
This work was led by Diego Gomez (tarod13.github.io), in collaboration with
@michaelhbowling.bsky.social and @jquinnlee.bsky.social. What an environment we have here at @ualberta.bsky.social and @amiithinks.bsky.social!
Preprint: biorxiv.org/content/10.1...
Blog post: medium.com/@marlos.chol...
Are these results real? We then replicated experiments that measured cell activations in the hippocampus of mice during remapping experiments. We observed the NNs we trained performing quite similarly to the neural data from previous work! ๐คฏ๐คฏ๐คฏ
Ref: www.cell.com/neuron/fullt...
In other words, the successor representation shows up immediately before grid cells, similarly to how the hippocampus (where place cells are) comes after the entorhinal cortex (to understand the direction, think about what influences what when performing backprop in an ANN).
Moreover, we see activations resembling place cells in the 3rd layer and border and corner cells in the 1st and 2nd layers (and many other cell types). These results showed the formation of a representational loop consistent with anatomical and functional circuits in the brain.
First, we showed that when training a neural network to estimate, from high-dimensional observations, the eigenvectors of the successor representation (or graph Laplacian), we see activation patterns resembling those in the entorhinal cortex. In fact, we see many of those!
A couple of months ago, we released a preprint of one of my favourite papers Iโve ever written. It lies at the intersection of representation learning and neuroscience. I have now written a blog post about it.
Preprint: biorxiv.org/content/10.1...
Blog post: medium.com/@marlos.chol...
"Canada Impact+ Research Chairs programโa new $1 billion investment that will provide Canadian institutions the opportunity to recruit top-tier international researchers with expertise in key areas ..."
www.canada.ca/en/innovatio...
Thrilled to start 2026 as faculty in Psych & CS
@ualberta.bsky.social + Amii.ca Fellow! ๐ฅณ Recruiting students to develop theories of cognition in natural & artificial systems ๐ค๐ญ๐ง . Find me at #NeurIPS2025 workshops (speaking coginterp.github.io/neurips2025 & organising @dataonbrainmind.bsky.social)
The Computing Science Dept. at the University of Alberta has multiple faculty job openings. Please share this broadly. We have a great environment!
- CS Theory: tinyurl.com/zrh9mk69
- Network/Cyber Security: tinyurl.com/renxazzy
- Robotics/CV/Graphics: tinyurl.com/ypcsfbff
The Department of Computing Science at the University of Alberta at the University of Alberta has an opening for another tenure-track faculty in robotics. Please, spread the word.
I can attest to how awesome our department and @amiithinks.bsky.social are!
(Official job posting coming soon.)
Ratatouille (2007)
This paper has now been accepted @neuripsconf.bsky.social !
Huge congratulations, Hon Tik (Rick) Tse and Siddarth Chandrasekar.
2/2: โConquerors live in dread of the day when they are shown to be, not superior, but simply lucky.โ
โ N.K. Jemisin, The Stone Sky
1/2: But there are none so frightened, or so strange in their fear, as conquerors. They conjure phantoms endlessly, terrified that their victims will someday do back what was done to themโeven if, in truth, their victims couldnโt care less about such pettiness and have moved on.โ
Excited to announce the RLC best paper awards! Like last year, we wanted to highlight the many excellent ways you can do research.
rl-conference.cc/RLC2025Award...
* RLC Journal to Conference Track:*
(Originally published at TMLR)
- Deep RL track (Thu): AGaLiTe: Approximate Gated Linear Transformers for Online Reinforcement Learning by S. Pramanik
* RLC Full Papers:*
(These are great papers!)
- Deep RL track (Thu): Deep Reinforcement Learning with Gradient Eligibility Traces by E. Elelimy
- Foundations track (Fri): An Analysis of Action-Value Temporal-Difference Methods That Learn State Values by B. Daley and P. Nagarajan
* RLC Workshop Papers (2/2):*
Inductive Biases in RL
sites.google.com/view/ibrl-wo...
- A Study of Value-Aware Eigenoptions by H. Kotamreddy
* RLC Workshop Papers (1/2):*
RL Beyond Rewards
rlbrew2-workshop.github.io
- Tue 11:59 (spotlight talk): Towards An Option Basis To Optimize All Rewards by S. Chandrasekar
- The World Is Bigger: A Computationally-Embedded Perspective on the Big World Hypothesis by A. Lewandowsi
Here's what our group will be presenting at RLC'25.
* Invited Talks at Workshops:*
Tue 10:00: The Causal RL Workshop sites.google.com/uci.edu/crlw...
Tue 14:30: Inductive Biases in RL (IBRL) Workshop
sites.google.com/view/ibrl-wo...
Tue 15:00: Panel Discussion at IBRL Workshop
RLC starts tomorrow here in Edmonton. I couldn't be more excited! It has a fantastic roll of speakers, great papers, and workshops. And this time, it is in Edmonton ๐
@rl-conference.bsky.social is my favourite conference, and no, it is not because I am one of its organizers this year.
This was a great long-term effort from @martinklissarov.bsky.social, Akhil Bagaria, and @ray-luo.bsky.social, and it led to a great overview of the ideas behind leveraging temporal abstractions in AI.
If anything, I think this is very useful resource for anyone interested in this field!
To align better with workshop acceptance dates, ๐๐๐ ๐ข๐ฌ ๐๐ฑ๐ญ๐๐ง๐๐ข๐ง๐ ๐ข๐ญ๐ฌ ๐๐๐ซ๐ฅ๐ฒ ๐ซ๐๐ ๐ข๐ฌ๐ญ๐ซ๐๐ญ๐ข๐จ๐ง ๐๐๐๐๐ฅ๐ข๐ง๐ ๐ญ๐จ ๐๐ฎ๐ง๐ ๐๐๐ซ๐!
9/9: I genuinely think AgarCL might unlock new research avenues in CRL, including loss of plasticity, exploration, representation learning, and more. I do hope you consider using it.
Repo: github.com/machado-rese...
Website: agarcl.github.io
Preprint: arxiv.org/abs/2505.18347
8/9: Well, if you are still interested, you should probably consider reading the paper, but it is interesting to see that most of the agents we considered were able to reach human-level performance only in the most benign settings. And we did use a lot of computing here!
7/9: Through mini-games, we tried to quantify and isolate some of the challenges AgarCL poses, including partial observability, non-stationarity, exploration, hyperparameter tuning, and the non-episodic nature of the environment (so easy to forget!). Where do our agents "break"?
6/9: Importantly, this is a challenge problem that forces us to deal with many problems we often avoid, such as hyperparameter sweeps and exploration in CRL.
It is perhaps no surprise that the classic algorithms we considered couldn't really make much progress in the full game.
5/9: Over time, even the agent's observation will change, as the camera needs to zoom out to accommodate more agents; not to mention that there are other agents in the environment. I'm very excited about AgarCL because I think it allows us to ask questions we couldn't before.
4/9: AgarCL is an adaptation of agar.io, a game with simple mechanics that lead to complex interactions. It's non-episodic, and a key aspect is that the agent dynamics change as it accumulates mass: It becomes slower, gains new affordances, sheds more mass, etc.