Akira’s Machine Learning news — #21
Summary for Week 26, 2021 (June 27~)
Featured Paper/News in This Week.
- In pre-training, the Winning ticket of the lottery hypothesis seems to exist regardless of whether it is supervised or unsupervised. Since pre-training models are usually very huge, it may be possible to use a winning ticket (a small and fast network).
- A zero-shot approach to instance segmentation has been proposed. Since real-world tasks require the detection of new objects, it may be used more in the real world if more research is done.
Machine Learning in the Real World
- Amazon has opened a store where customers can pay without going through a cash register. Such unmanned stores are a hot topic somewhere every year, and there was an image of them going out of business soon, but what will happen this time?
- There seems to be a tool for monitoring model bias. Since model bias can cause big problems, I think such tools will become more and more necessary as the actual use of such ML use case increases.
Papers
- A method to quantize a model without data is proposed. I feel that it is very practical to be able to reduce the size of the model if you only have the model.
— — — — — — — — — — — — — — — — — — –
In the following sections, I will introduce various articles and papers not only on the above contents but also on the following five topics.
- Featured Paper/News in This Week
- Machine Learning Use Case
- Papers
- Articles related to machine learning technology
- Other Topics
— — — — — — — — — — — — — — — — — — –
1. Featured Paper/News in This Week
Winning tickets in pre-training are transferable — arxiv.org
[2012.06908] The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models
A study examining the effectiveness of Winning Ticket (a small useful subnetwork dominant in accuracy in large networks in the lottery ticket hypothesis) of pre-training on the transfer learning performance. The results show that Winning Ticket is present regardless of whether pre-training is supervised or unsupervised.
Zero-Shot Instance Segmentation — arxiv.org
[2104.06601] Zero-Shot Instance Segmentation
They propose a task of zero-shot instance segmentation, which performs segmentation of a new class with zero-shots (no parameter updates during inference). They believe that the background representation is the key, so they use the rich background representation vector to learn the mask and the zero-shot detection head representation.
— — — — — — — — — — — — — — — — — — –
2. Machine Learning Use case
Visualizing Model Biases — siliconangle.com
The biases of machine learning models have received a lot of attention lately, but they are very difficult to remove; Fiddler Labs provides a way to centralize this information and provides tools for continuous monitoring.
Amazon Opens Store That Lets You Pay Without Going Through a Cash Register — siliconangle.com
Amazon has opened a store that uses machine learning and a number of sensors to allow customers to buy fresh food without going through a cash register, allowing them to make a payment by simply holding up their hand for one second.
— — — — — — — — — — — — — — — — — — –
3. Papers
Knowledge distillation that allows knowledge to be transferred between different stages — arxiv.org
[2104.09044] Distilling Knowledge via Knowledge Review
In previous studies, knowledge distillation was based on the comparison of information from the same stage between student and teacher models. In this study, they propose a “knowledge review” approach to knowledge distillation that takes into account information from different stages. The distillation effect was greatly improved.
Use Transformer to classify fine grained actions — arxiv.org
[2104.09496] Temporal Query Networks for Fine-grained Video Understanding
They propose a transformer-based model TQN for fine-grained action detection that classifies fine-grained actions in videos, using a DETR-like query strategy to identify each action. They achieved SotA performance on three fine-grained action detection datasets.
A data-free quantization method using adversarial learning — arxiv.org
[2103.15263] Zero-shot Adversarial Quantization
When quantizing a model, it is usually necessary to use training data to adjust the quantized model, but they propose ZAQ, a quantization method that does not require data using an adversarial learning framework. It learns by bringing the output, including the intermediate layers, closer together in a high-precision model and a quantized model using the generated data.
Allowing individual losses to be optimized in style transformations — arxiv.org
[2104.10064] Style-Aware Normalized Loss for Improving Arbitrary Style Transfer
Identified that the loss of individual data was being averaged and updated in batches, which was having a negative effect on the style conversion. They avoided the problem by normalizing using an upper bound on the losses. They applied this to various methods to see the effect.
— — — — — — — — — — — — — — — — — — –
4. Articles related to machine learning technology
Explanatory article on score-based generative models — yang-song.github.io
While describing the differences between the likelihood-based generative model and the score-based model, this article explains how the score function can be inaccurate in areas where there is no data, and how to overcome this.
— — — — — — — — — — — — — — — — — — –
5. Other Topics
Repository of U²-Net — github.com
The U²-Net repository. Various pre-trained models are provided, and segmentation can be done immediately.
— — — — — — — — — — — — — — — — — — –
Past Newsletters
— — — — — — — — — — — — — — — — — — –
🌟I post weekly newsletters! Please subscribe!🌟
— — — — — — — — — — — — — — — — — — –
About Me
Manufacturing Engineer/Machine Learning Engineer/Data Scientist / Master of Science in Physics / http://github.com/AkiraTOSEI/
Twitter, I post one-sentence paper commentary.
