Awesome to see @nvidia @NVIDIAAI using our research for their open-source long-context models.

Talking about DeepSeek and their connection to Tsinghua. Tsinghua and CMU have an older (2017) but still great series on high-performance parallel computing. The playlist can be found here: https://buff.ly/4gktKSd

Bridging the Data Provenance Gap Across Text, Speech and Video Progress in AI is driven largely by the scale and quality of training data. Despite this, there is a deficit of empirical analysis examining the attributes of well-established datasets beyond text.…

Paper link: https://buff.ly/3WtuwWi

Happy to anounce that our paper, Bridging the Data Provenance Gap Across Text, Speech, and Video, was accepted to @iclr_conf. #ICLR2025

I have an extremely easy evaluation that currently all top models achieve a 0% on. This is the easiest set of evaluations in our entire suite. AGI would be able to solve the hardest problems effortlessly. Once o3 becomes available in the API, I will put out a public baseline.

Does anyone have any ideas why T5 or CLIP is being used for text encoding in diffusion training instead of a much stronger encoder or embedding model?

There is absolutely no shortage of pre-training data.

GitHub - aws/aws-parallelcluster: AWS ParallelCluster is an AWS supported Open Source cluster management tool to deploy and manage HPC clusters in the AWS cloud. AWS ParallelCluster is an AWS supported Open Source cluster management tool to deploy and manage HPC clusters in the AWS cloud. - aws/aws-parallelcluster

AWS ParallelCluster is honestly such an incredibly useful tool for large-scale distributed training:

CompVis/cleandift · Hugging Face We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Hugging Face repo: huggingface.co/CompVis/clea...

GitHub repo: github.com/CompVis/clea...

Paper: compvis.github.io/cleandift/st...

Be sure to check out this awesome work by @stefanabaumann.bsky.social, @rmsnorm.bsky.social, and @koljabauer.bsky.social.