Wrote up some notes in my most recent blog post on how images are passed into LLMs.
drchrislevy.github.io/posts/vllms/...
Inspired by recent work/posts from @merve.bsky.social and @sebastianraschka.com
Transformers: 4.36.2 --->4.46.3
Torch: torch-2.1.2 ---> torch-2.5.1
Look at that memory drop! This is a production workload running some fine-tuned transformer encoder models. I didn't look through all the release notes yet for the reason. Anyone from @hf.co know off the top of their head?
I still haven’t tried it. Wanting to though. First for personal use then maybe even for work with the main project I work on. Huge env managed by poetry at the moment.
👋