A symbolic representation of tasks is key for compositionality. Our goal-conditioned hierarchical #ReinforcementLearning algo STAR learns online a discrete representation of continuous sensorimotor space #ReachabilityAnalysis #ContinualLearning #ICLR2024 @iclr-conf.bsky.social
📽️ youtu.be/H5-ufMj2AE0

Talk like a graph: Encoding graphs for large language models

Graphs are everywhere, but LLMs are trained on text. In „Talk like a Graph“ (ICLR 2024), Google introduces methods for encoding graphs for LLMs, launches the GraphQA benchmark, and demonstrates how task phrasing & graph structure impact reasoning.
#ICLR2024 #AI #Graphs

Last year, we funded 250 authors and other contributors to attend #ICLR2024 in Vienna as part of this program. If you or your organization want to directly support contributors this year, please get in touch! Hope to see you in Singapore at #ICLR2025!

During #ICLR2024 rebuttal, the reviewers figured out they were looking for the wrong thing, but instead of adjusting their review scores, they lowered their confidence! They refer us to our own cited/published previous work (note, review was double blind) to consult that paper.

ICLR Invited Talk Copyright Fundamentals for AI ResearchersICLR 2024

Lots of discussion on copyright and AI right now. Check out Kate Downing’s #ICLR2024 Keynote on “Copyright Fundamentals for AI Researchers”. Lots of great insight there.

iclr.cc/virtual/2024...

At UKP Lab we research how AI hallucinations can be mitigated by merging LLMs. A paper on this topic by us,
@cs-tudarmstadt.bsky.social, hessian.AI, RIKEN AIP and Edinburgh University was presented by Nico Daheim and Thomas Möllenhoff at #ICLR2024.

Learn more in this 🧵:
bsky.app/profile/ukpl...

👋🏻 bye #ICLR2024 hello #CHI2024 🏝️

come chat w me & teammates @josephc.bsky.social @paopow.bsky.social Tal August @jbragg.bsky.social @axz.bsky.social about our work on LMs for supporting scientific research 🧐 @ai2.bsky.social

done w 🥐🍺 all about 🐠🐟 this week!!

Biased Causal Strength Judgments in Humans and Large Language Models Causal reasoning is a critical aspect of both human cognition and artificial intelligence (AI), playing a prominent role in understanding the relationships between events. Causal Bayesian Networks...

Excited to be at #ICLR2024 in Vienna to present our latest work on causal reasoning in humans and LLMs! 🧠🤖 We examined biases in causal judgments using Causal Bayesian Networks. Join us at the Re-Align workshop!
openreview.net/forum?id=544...

Strategy Talk mit Yoshua Bengio am Podium im BMK

Strategy Talk mit Yoshua Bengio im BMK

Strategy Talk mit Yoshua Bengio und Sektionschefin Henriette Spyra im BMK

Hoher Besuch im #BMK: Der kanadische 🇨🇦 #TuringAward Träger Yoshua Bengio ist anlässlich der #ICLR2024 derzeit in Wien & hat im #BMK vor ausgewählten Expertinnen einen Strategy Talk zu Risiken von #AI Technologien gehalten. #ArtificialIntelligence

Dobré město je takové, kde se vás neustále někdo snaží přejet na 🚲. Díky, Velká Zlá Společnosti za barevné polštářky, po 3. 🍻 je to nej. Tolik 🦐 jsem ještě neměl. Nechcete mě tu někdo zaměstnat? Umím chovat 🦙! (U stánků na levnou AI paradoxně nabízí hlavně práci v NYC) #iclr2024 #drunktoot

#ICLR2024

🧐How can we design neural networks that process other neural networks?
The answer is simple: represent them as graphs, a.k.a. neural graphs!
Join our oral presentation and our poster at #ICLR2024 for more details.
Oral session 4B @ Halle A7 and poster #77 session 4 @ Halle B.

Re-Align Workshop @ ICLR 2024

Travelling to Vienna for #ICLR2024 ! If you want to chat about efficient AI, dynamic representations, NeuroAI, hardware accelerators & neuromorphic, please reach out!

Also join for our Re-Align workshop representational-alignment.github.io

#MLSky #NeuroAI #CompNeuro #Neuroscience

And consider following the authors Nico Daheim, Thomas Möllenhoff, Edoardo M. Ponti, Iryna Gurevych and Mohammad Emtiyaz Khan (UKP Lab, @cs-tudarmstadt.bsky.social, @tuda.bsky.social, Hessian.ai, RIKEN AIP, University of Edinburgh). (9/9)

See you in Vienna! 🇦🇹 #ICLR2024

Model Merging by Uncertainty-Based Gradient Matching Models trained on different datasets can be merged by a weighted-averaging of their parameters, but why does it work and when can it fail? Here, we connect the inaccuracy of weighted-averaging to...

We provide open access to our code and results:

📄 Paper: arxiv.org/abs/2310.12808
💻 Code: github.com/UKPLab/iclr2...
🤗: huggingface.co/papers/2310....
(8/🧵) #ICLR2024 #NLProc

This principle also holds for removing data from models, for example reducing toxicity in LLMs by removing toxic training data or removing private training data without retraining.

(7/🧵) #ICLR2024 #NLProc

It also outperforms many existing schemes, such as Fisher Averaging. 🥇

(6/🧵) #ICLR2024 #NLProc

This new merging scheme improves robustness to scaling and performance!

(5/🧵) #ICLR2024 #NLProc

If the mismatch is small, we expect model merging to work well. 🥳

If the mismatch is large, it might fail. 🫣
But we can also reduce it! Using simple approximations, we arrive at a new, improved merging scheme. 🚀

(4/🧵) #ICLR2024 #NLProc

The principle is generally applicable to other schemes - for example Task Arithmetic has the following mismatch.

(3/🧵) #ICLR2024 #NLProc

When merging multiple models, we want to approximate a target model that has minimal loss on all data. 

We can quantify how close we can get to this target model in terms of their gradient mismatch.

(2/🧵) #ICLR2024 #NLProc

Model Merging has shown great success but key questions remain unresolved

✅ Why does it work?
❌ When can it fail?

We shed light on those questions by connecting inaccuracies of weighted-averaging to mismatches in the gradients.

🧵(1/9) #ICLR2024 #NLProc

📰  arxiv.org/abs/2310.12808

flying to #ICLR2024 in Vienna 🇦🇹 excited to chat:

🥐 evaluating book-length generation (BooookScore & FABLES) w/ @yapei.bsky.social Mohit Iyyer & Tanya Goyal!

🥐 pretraining data curation (Dolma 🍇), training open LMs (OLMo 🐙) & more w/ @soldaini.net & my team @ai2.bsky.social!

🔍How can we design neural networks that take neural network parameters as input?
🧪Our #ICLR2024 oral on "Graph Neural Networks for Learning Equivariant Representations of Neural Networks" answers this question!

📜:💻:🧵 [1/9

github.com/mkofinas/neura… arxiv.org/abs/2403.12143

Feature-aligned N-BEATS Project page of Feature-aligned N-BEATS

Excited to share that my debut paper, "Feature-aligned N-BEATS with Sinkhorn divergence", has been accepted as a spotlight at #ICLR2024! 🎉 Catch me presenting at Messe Wien Exhibition and Congress Center. 🇦🇹

Dive into the project details here: paper.leejoonhun.xyz/fan-beats.

A paper by Nico Daheim, Thomas Möllenhoff, Edoardo Maria Ponti, Iryna Gurevych and Mohammad Emtiyaz Khan has been accepted to this year's ICLR conference, taking place May 7-11 in Vienna 🇦🇹! Congratulations to all authors!

#ICLR2024 #NLProc

www.informatik.tu-darmstadt.de/ukp/ukp_home...

🎉 Super stoked that our work on "Graph Neural Networks for Learning Equivariant Representations of Neural Networks" was accepted as an oral in #ICLR2024!
Paper details and source code will follow soon.

TRAM is accepted to
#ICLR2024 as a Spotlight! See you in Vienna 🇦🇹! Thanks to @nsaphra.bsky.social, Pradeep Dasigi, Hao Peng and @ai2.bsky.social

Vision experiments, more discussion and visuals coming soon to the camera ready!

Re-Align Workshop @ ICLR 2024

🚨Call for Papers 🚨

The Re-Align Workshop is coming to #ICLR2024

Our Call for Papers is finally up! Come share your representational alignment work at our interdisciplinary workshop at ICLR in beautiful Vienna!
representational-alignment.github.io

#compneuro #MLSky #neuroscience #NeuroAI

1/4