Value-based Knowledge Distillation Boosts Preference Optimization

TVKD adds a soft reward from a teacher model’s value function to Direct Preference Optimization, boosting performance on benchmarks without extra rollouts. Read more: getnews.me/value-based-knowledge-di... #knowledgedistillation #preferencoptimization