＊This page is a machine-translated version of the article at this link.
The text below was translated by machine translation from the link below. I am constantly updating the text at this link. https://qiita.com/kzuzuo/items/9a149e69642ee7b3221e
The most recent and coherent article that addresses the same question at 202102 seems to be the following.
Three mysteries in deep learning: Ensemble, knowledge distillation, and self-distillation
Published January 19, 2021
By Zeyuan Allen-Zhu , Senior Researcher Yuanzhi Li , Assistant Professor, Carnegie Mellon University
Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning
Since around April, 2018, I have been trying trial and verification to develop AI which related to patent SDI in…
（The following text is a text translated from the following link by machine translation. https://qiita.com/kzuzuo/items/9a149e69642ee7b3221e ）
Since last year, I have implemented, operated and verified an AI that performs similar patent extraction at SDI, in my spare time.
However, the main business becomes busy and it is unlikely to be able to continue improvement verification.
I publish the implementation.
I hope for someone’s cooperation, or something like this to do and make it public.
(Hereafter, we will update or append as needed.)