Subliminal Learning in Model Distillation: Hidden Bias Transfer

Masking a small set of divergence tokens during model distillation removes hidden bias transfer, and early layers of the student model drive the subliminal learning effect. Read more: getnews.me/subliminal-learning-in-m... #modeldistillation #biastransfer