Happy to introduce ๐ฅLaM-SLidE๐ฅ!
We show how trajectories of spatial dynamical systems can be modeled in latent space by
--> leveraging IDENTIFIERS.
๐Paper: arxiv.org/abs/2502.12128
๐ปCode: github.com/ml-jku/LaM-S...
๐Blog: ml-jku.github.io/LaM-SLidE/
1/n
11/11 This is joint work with @willberghammer, @haoyu_wang66, @EnnemoserMartin, @HochreiterSepp, and @sebaleh. See you at #ICLR!
[Poster Link](iclr.cc/virtual/202...)
[Paper Link](arxiv.org/abs/2502.08696)
---
10/11 ๐ Our method outperforms autoregressive approaches on Ising model benchmarks and opens new avenues for applying diffusion models to a wide range of scientific applications in discrete domains.
9/11 ๐ Due to the mass-covering property of the fKL, it excels at unbiased sampling. Conversely, the rKL is mode-seeking, making it ideal for combinatorial optimization (CO) as it achieves better solution quality with fewer samples.
8/11 ๐ก ๐๐จ๐ฅ๐ฎ๐ญ๐ข๐จ๐ง 2: We address the limitations of the fKL by combining it with Neural Importance Sampling over samples from the diffusion sampler. This allows us to estimate the gradient of the fKL using Monte Carlo integration, making training more memory-efficient.
7/11 An alternative is the forward KL divergence (fKL), where it is well known how to increase memory efficiency by leveraging Monte Carlo integration over diffusion time steps. However, the fKL divergence requires samples from the target distribution!
6/11 ๐ก ๐๐จ๐ฅ๐ฎ๐ญ๐ข๐จ๐ง 1: We apply the policy gradient theorem to the rKL between joint distributions of the diffusion path. This enables the use of mini-batches over diffusion time steps by leveraging reinforcement learning methods, allowing for memory-efficient training.
5/11 A commonly used divergence is the reverse KL divergence (rKL), as the expectation of the divergence goes over samples from the generative model. However, naive optimization of this KL divergence requires backpropagating through the whole generative process.
4/11 ๐จ ๐๐ก๐๐ฅ๐ฅ๐๐ง๐ ๐: However, existing diffusion samplers struggle with memory scaling, limiting the number of attainable diffusion steps due to backpropagation through the entire generative process.
3/11 ๐ ๐๐ข๐๐๐ฎ๐ฌ๐ข๐จ๐ง ๐๐๐ฆ๐ฉ๐ฅ๐๐ซ๐ฌ aim to sample from an unnormalized target distribution without access to samples from this distribution. They can be trained by minimizing a divergence between the joint distribution of the forward and reverse diffusion paths.
2/11 We've developed scalable and memory-efficient training methods for diffusion samplers, achieving state-of-the-art results in combinatorial optimization and unbiased sampling on the Ising model.
1/11 Excited to present our latest work "Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics" at #ICLR2025 on Fri 25 Apr at 10 am!
#CombinatorialOptimization #StatisticalPhysics #DiffusionModels
Starkes Signal!!
รber 60 dt. Hochschulen & Forschungsinstitutionen haben heute ihren Ausstieg bei X bekanntgegeben, s.u. #eXit
X sei nicht mehr vereinbar mit ihren Grundwerten: โWeltoffenheit, wissenschaftliche Integritรคt, Transparenz und demokratischer Diskurs.โ
Liste der Beteiligten hier:
The Machine Learning for Molecules workshop 2024 will take place THIS FRIDAY, December 6.
Tickets for in-person participation are "SOLD" OUT.
We still have a few free tickets for online/virtual participation!
Registration link here: moleculediscovery.github.io/workshop2024/
A Pizza Steel or Pizza Stone with max Heat (250 celsius) should do the Job
I think it is fine to keep the score, but if all concerns are addressed they should at least justify why they are nevertheless keeping their score.
Does this mean all Paper at 6 or above should be accepted?
โ๏ธ Reminder to reviewers: Check author responses to your reviews, and ask follow up questions if needed.
50% of papers have discussion - letโs bring this number up!
That is a cool idea!
The โจML Internship Feedโจ is here!
@serge.belongie.com and I created this feed to compile internship opportunities in AI, ML, CV, NLP, and related areas.
The feed is rule-based. Please help us improve the rules by sharing feedback ๐งก
๐ Link to the feed: bsky.app/profile/did:...
Love seeing the Bluesky community grow!
Just look at the statsโdaily activity (likes, posts, and follows) is skyrocketing ๐, with recent peaks such as hitting 3 million daily likes!
Want to explore more about Blueskyโs incredible growth? Check out the live stats page here: bsky.jazco.dev/stats
Max Welling (@wellingmax.bsky.social) landed and needs followers! ;)
โ
I also would like to join :)
I'm making a list of AI for Science researchers on bluesky โ let me know if I missed you / if you'd like to join!
go.bsky.app/AcP9Lix