Meta Pseudo Labels
Flexible Teachers, Smarter Students: Meta Pseudo Labels improves knowledge distillation.
Human teachers can teach more effectively by adjusting their methods in response to student feedback. It turns out that teacher networks can do the same.