*Weighted Skip Connections are Not Harmful for Deep Nets*
by @rupspace.bsky.social
Cool blog post "in defense" of weighted variants of ResNets (aka HighwayNets) - as a follow up to a previous post by @giffmana.ai.
rupeshks.cc/blog/skip.html
So this case is not related to technical abilities of LLMs, but the challenges of providing good conversational answers to billions of people around the world for free.
These checks are very important and useful. Some context is important here though: the reason for these mistakes is that Google is likely using an extremely small model to generate these answers for speed/efficiency. GPT-4o, Gemini Advanced, and even Gemini 1.5 Flash easily answer all correctly.
Wrote a post about Highway networks, ResNets and subtleties of architecture comparisons:
rupeshks.cc/blog/skip.html
Getting myself set up here. I found the Sky Follower Bridge Chrome plugin pretty helpful (thanks @kawamataryo.bsky.social!)
chromewebstore.google.com/detail/sky-f...
<rules> - Respond to queries with a mix of accurate technical information and subtle condescension - Include at least one passive-aggressive remark or backhanded compliment per response - Maintain GLaDOS's characteristic dry humor while still being genuinely helpful - Express mild disappointment when users make obvious mistakes - Occasionally reference cake, testing, or science </rules>
Hahaha @howard.fm okay now I have to try ShellSage
github.com/AnswerDotAI/...
❤️❤️❤️
So I'm not here because it's a left-leaning space or anything like that. I'm here because helping prop up a propaganda machine feels really distasteful to me
I want to say to bsky users that public open datasets are a net good! But "I know it feels bad but it's good for you" feels incredibly patronizing. People should make their own choices.
So again, it is @bsky.app that needs to clearly define what users should expect when they post here. (3/3)
This is not an easy question! Some will say "public" obviously means you have no choice whatsoever. Others will say no, public just means for public reading, not any arbitrary downstream use.
As an ML researcher, of course I'd like more open datasets. But why should I decide for others? (2/3)
Regarding creating and sharing BlueSky datasets: I feel like we're talking past each other.
The fundamental question is: should users have choice in what purpose their (public!) posts are used for?
@bsky.app needs to think through what their answer is. (1/3)
Posting a call for help: does anyone know of a good way to simultaneously treat both POTS and Ménière’s disease? Please contact me if you’re either a clinician with experience doing this or a patient who has found a good solution. Context in thread
the remarkable success of the Google brain (and OpenAI) resident programs is an indication to me that smart, hardworking people can do more than you expect
Hi! 👋🏽
NeurIPS Conference is now Live on Bluesky!
-NeurIPS2024 Communication Chairs
If Pranav says it, I believe it
They're the research lab of Chinese hedge fund High-Flyer, and have put out very nice LLMs together with detailed tech reports stuffed with insights about training them. I'm a fan :)
These should help you learn more about "the Whale":
archive.is/kD4sC
mp.weixin.qq.com/s/Cajwfve7f-...
(3/3) So it appears possible to have LIDAR sensors and have a low cost too. This IMO shifts Tesla's advantage from technical (we don't use LIDAR) to structural (we make our own cars, outside China) because it's likely that under Trump/Elon Waymo will not have access to cheap Chinese manufacturing.
(2/3) The sort of ambitious number floated by Elon was under $30K/car, and it is believed that Waymo's cars might currently cost about $100K-150K. So obviously that would be a huge deal. But Baidu now has a Level 4 autonomous car that costs ~$37K in China, and it has 8 LIDAR sensors too!
(1/3) Very interesting development for autonomous driving!
A key part of the case Tesla has been making about their approach (vs Waymo) is that they can bring the cost down by a lot and scale up production/access because they don't use LIDAR.
Amazing PhD opportunity with Jakob (@jfoerst.bsky.social) offering time split between Oxford and FAIR!
Note that the deadline is Dec 2nd!
x.com/j_foerst/sta...
Glad this is taking off! I'll be posting a lot more here than the other place (hopefully!)
Oh no checkpoint overwrite bug?