student distillation

[leilamalihi]

Hello,
Thanks for your nice repository. I want to distill knowledge from different teacher to student, how can i do that? also the output of that is in which format?that is a model or orderdict file?

Thanks

[yoshitomo-matsubara]

Hello @leilamalihi

I do not well understand your questions, but you can specify teacher and student models in a yaml config file.
For instance, this will use DenseNet BC K12 (depth=100) as a teacher and ResNet-20 as a student model.
https://github.com/yoshitomo-matsubara/torchdistill/blob/main/configs/legacy/sample/cifar10/kd/resnet20_from_densenet_bc_k12_depth100-hyperparameter_tuning.yaml

Close this discussion and "Mark as answer" if it answers your question.