forgot the hashtag #neurips2024 😅

Hey! I will be at NeurIPS @neuripsconf.bsky.social this week (friday 13th) presenting our paper VeLoRA.

tldr; simple method for compressing gradients with a coarse reconstruction during the backwards pass. Significant memory reductions while being complimentary to LoRA!

github.com/roymiles/VeL...

Added!

Added!

Hey! I have added you

Added!

Hey! I have added you

Hey! I have added you

Cool! I have added you

Of course! I have added you now

Cool! I have added you.

Haha I wasn't really thinking about it like that. In either case I have added you to this one too.

Added you in 😁

Hey, I have added you!
oops yea I did see that one. Perhaps this one can be a bit more broader with other modalities.

Added both of you!

Added!

Cool, added!

Hey! Yea yours probably overlaps a lot 😅 I have added you and will add some others from there too.

Sweet, I have added you!

I made a starter pack for people working or interested in multi-modality learning.

It would be good to add lots more people so do comment and I'll add!

go.bsky.app/97fAH2N

Our BMVC poster is today!

Knowledge distillation can work in cross-task settings and even with a randomly initialised teacher! Our inverted projection decomposes into knowledge transfer and spectral regularisation, enabling teacher-free distillation with improvements on ImageNet-1K at no extra cost

Hey, this is great and relevant for me. Would love to join if there is still space!

This is really great thanks! Could I be added to the AI starter pack too 😅?

Would be great to be added too!

Some related work worth checking out if you find this cool.

mbaradad.github.io/shaders21k/ - learning good visual features from procedurally generated images.
arxiv.org/abs/2403.14494 - distillation from randomly weighted teachers.

Sounds great! Would love to join if there is space.

p.s. please add me to any starter packs! 😁

Hi Bluesky!

I'm currently working on multi-modality learning and efficient ML (mobile devices).

Hoping to regularly post about any interesting topics I find cool (ai, maths, physics).

Looking forward to seeing how this platforms grows and meeting you all!

Hi Kostas!
Could you add me as well? :D