Akira’s Machine Learning news — #24
Week 31, 2021 (Aug 8~)
Featured Paper/News in This Week.
- There is an article that says that the AI tools created for COVID-19 was completely useless. The main cause seems to be leakage and data quality, but it reminds me that it is difficult to create something that works well in a real environment.
- In the case of crowd counting, there has been research using synthetic data. Since deep learning requires a large amount of data, there is also an example of using data from the game GTA5 for 3D object detection, and the use of synthetic data, where annotations are accurate and a large amount of data is available, may become more popular.
Machine Learning in the Real World
- There is an article out describing what is being done with AI-based attacks. The attackers are using Deep Fake, for example, but there is no investment in defense.
- It seems that they are applying machine learning to physics simulations. In large systems, the computational cost seems to be lower than direct computation. A study also shows that the computational cost advantage can be achieved in computing the Hamiltonian of large systems.
Papers
- In a setting where there are errors in the labels, a method is proposed that uses self-supervised learning to discriminate between in-distribution and out-of-distribution labels. In recent years, there has been an increase in research on augmenting supervised learning with self-supervised learning. However, the computational cost seems to be a problem.
— — — — — — — — — — — — — — — — — — –
In the following sections, I will introduce various articles and papers not only on the above contents but also on the following five topics.
- Featured Paper/News in This Week
- Machine Learning Use Case
- Papers
- Articles related to machine learning technology
- Other Topics
— — — — — — — — — — — — — — — — — — –
1. Featured Paper/News in This Week
Crowd Counting Dataset with Synthetic Data — openaccess.thecvf.com
[Cross-View Cross-Scene Multi-View Crowd Counting]
A crowd counting task with various multiple camera configurations (CVCS) is proposed. Since annotation is very costly in such a problem setting, they proposed a CVCS dataset with synthetic data. By Fine-Tuning with unsupervised domain adaptation, they proved that it can be applied to real-world data.
AI tools were Useless in the Fight Against Covid-19 — www.technologyreview.com
An article reporting that not all of the 415 AI tools developed for Covid-19 were at the level of clinical applicability. Many had poor data quality or problems with leakage (where training and evaluation data contain the same data, so performance appears better than it really is).
— — — — — — — — — — — — — — — — — — –
2. Machine Learning Use case
The Threat from Offensive AI — venturebeat.com
An article discussing what could be caused by offensive AI. It points out that attackers may use Deep Fake for phishing scams and reverse engineering to steal algorithms, but there has been no investment in defending against them.
Physics Simulation and Machine Learning — spectrum.ieee.org
Physics simulations are powerful, but they require a very large amount of computation when trying to calculate huge atomic models. If you use machine learning to approximate the simulation, you may be able to calculate very large systems because the computation time increases only linearly even if the number of atoms increases exponentially.
— — — — — — — — — — — — — — — — — — –
3. Papers
Explicitly incorporate the person-context-person triad into the model — arxiv.org
[2006.07976] Actor-Context-Actor Relation Network for Spatio-Temporal Action Localization
They proposed ACAR-Net, which explicitly incorporates the person-context-person triad into the model for action recognition, because it is often impossible to infer a person’s behavior from a person-context pair alone. In the AVA-Kinetics of the ActivityNet Challenge 2020, ACAR-Net won the first place, significantly outperforming other methods.
An ensemble technique that uses masks to manipulate joins. — arxiv.org
[2012.08334] Masksembles for Uncertainty Estimation
Ensemble is used to calculate uncertainty, but it is expensive. They proposed a method of ensembling by pseudo-manipulating the model coupling using masks, and confirmed its effectiveness on ImageNet and CIFAR-10.
Using contrastive learning to obtain category-level alignment — openaccess.thecvf.com
[Partially View-aligned Representation Learning with Noise-robust Contrastive Loss]
In MvRL (Multi-view representation learning), which is a task to obtain useful representations for clustering and classification from multi-view/multimodal, there is a problem that only a part of the data is aligned. Therefore, they proposed MvCLN, which uses contrast learning, and proposed a method to align at the category level.
Use contrastive learning to find mislabeling. — arxiv.org
[2103.13029] Jo-SRC: A Contrastive Approach for Combating Noisy Labels
Conventional methods for noisy-label problem ignores the proportion of noise that varies from one mini-batch to another. For this reason, they proposed Jo-SRC, which uses contrast learning to label each data as Clean, in-distribution, or out-of-distribution, and uses only Clean to train. We confirmed the effectiveness of the method in a variety of settings.
Explicitly incorporating resistance to adversarial attacks into the NAS
[2012.06122] DSRNA: Differentiable Search of Robust Neural Architectures
A study of explicitly incorporating resistance to adversarial attacks into the NAS. Exploring robust architectures by minimizing not only loss in validation data, but also a combination of differentiable metrics that measure robustness to adversarial attacks.
Letting the model take into account the ambiguity of labels in facial expression recognition — arxiv.org
[2104.00232] Dive into Ambiguity: Latent Distribution Mining and Pairwise Uncertainty Estimation for Facial Expression Recognition
Facial expression recognition has ambiguity because it is subjectively annotated. They propose a method learning ambiguity to avoid costly inference by constraining the network to classify C-1 classes in the same way as the combined network of C branches during training. Gaining SotA performance in AffectNet.
— — — — — — — — — — — — — — — — — — –
4. Articles related to machine learning technology
MIT Deep Learning Lectures — www.youtube.com
[MIT 6.S191: Introduction to Deep Learning]
A list of videos on deep learning at MIT, covering not only the basics of CNN, RNN, reinforcement learning, generative models, etc., but also fairness, current topics, and limitations of deep learning.
Explanatory article on MDETR
An article on MDETR, which combines language learning and DETR. By performing multimodal learning combining languages, it is possible to detect categories that do not exist in the training data, such as “pink elephants”.
— — — — — — — — — — — — — — — — — — –
5. Other Topics
Templates for projects that combine Hydra and Pytorch-lightning — github.com
A repository of project templates that combine Hydra that simplifies the control of experiments by parameters and Pytorch lightning, a wrapper for Pytorch . Unnecessary functions can be easily removed from the pipeline and reconfigured.
— — — — — — — — — — — — — — — — — — –
🌟I post weekly newsletters! Please subscribe!🌟
— — — — — — — — — — — — — — — — — — –
About Me
Manufacturing Engineer/Machine Learning Engineer/Data Scientist / Master of Science in Physics / http://github.com/AkiraTOSEI/
Twitter, I post one-sentence paper commentary.
