DEIT
the teacher can be either transformer or convnet.
DINO:
Emerging Properties in Self-Supervised Vision Transformers
BEIT: BERT Pre-Training of Image Transformers
MLP-Mixer
