Akira’s ML news #September,2020
Here are some of the papers and articles that I found particularly interesting I read in September 2020.
- Machine Learning Papers
- Technical Articles
- Examples of Machine Learning use cases
- Other topics
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — —
1. Machine Learning Papers
— —
Removing a person from a video
Flow-edge Guided Video Completion
https://arxiv.org/abs/2009.01835
This is a study of video completion using flow. The key points of this method are: 1) Flow completion by detecting the missing edges and connecting them, 2) Obtaining the invisible parts of the video by using a slightly distant frame, and 3) Preventing the seams by using the gradient. Their method is quantitatively better than previous studies and can even eliminate people from the video.
Decoding Brain Activity for Language Generation
Brain2Word: Decoding Brain Activity for Language Generation
https://arxiv.org/abs/2009.04765
A study to understand the letters seen from an fMRI image. fMRI images were used as input to generate GloVE embedding vectors of the five leading candidates from 180 candidate words. Then they do the same thing in GPT-2 with 50257 words and compare the two to solve as a language model. The accuracy is greatly improved compared to the existing methods.
Theorem Proving with Language Modeling
Generative Language Modeling for Automated Theorem Proving
https://arxiv.org/abs/2009.03393
They proposed GPT-f, which automatically proves theorems using a transformer. The proofs of theorems are defined as a language model such as [GOAL,(theorem),PROOFSTEP,…]. They have succeeded in shortening the theorem proofs with 23 theorems compared to existing ones.
Game Balancing with AlphaZero
Assessing Game Balance with AlphaZero: Exploring Alternative Rule Sets in Chess
https://arxiv.org/abs/2009.04374
This is an attempt to assess game balance using AlphaZero. To train AlphaZero on the chess mimics with minor rule changes such as no castling, we can see how experienced players view the game. In the history of chess, the rules have been repeatedly modified and settled on the current rules. Other games can be made to simulate such a thing with AlphaZero, they said.
Retiming People in Video
Layered Neural Rendering for Retiming People in Video
https://arxiv.org/abs/2009.07833
This model allows you to change the timing (start timing and speed of movement) of the people in the video. First, the each person, including occlusion, is separated from the background. Then the background and each person’s information are combined into a feature, and combined together to generate. They are able to change the timing of the water splashes and other features that accompany the movements of the people.
Reduce the size of BigGAN using the Wavelet transform
not-so-BigGAN: Generating High-Fidelity Images on a Small Compute Budge
https://arxiv.org/abs/2009.04433
This is a research to generate a high-fidelity image using GANs with a small amount of computational resources by reproducing the image from the low-frequency information only using the Wavelet Transform (WT). Specifically, the lost high-frequency information is reconstructed by NN and the inverse Wavelet Transform (iWT) is applied. Although the quality of the generated images decreases a little, they have succeeded in reducing the computational resources from TPUx256 to GPUx4
Activation function to determine applicably whether it is active
Activate or Not: Learning Customized Activation
https://arxiv.org/abs/2009.04759
They treated ReLU and Swish in a unified manner and proposed the activation function ACtivationOrNot (ACON) as its general form. The activation function is composed of multiple learnable parameters, and they can freely change whether it is activated or not. They confirmed that this method improves the accuracy of object detection and image retrieval.
GANs that can generate rare weather data at will
ExGAN: Adversarial Generation of Extreme Samples
https://arxiv.org/abs/2009.08454
While GANs usually train and generate only typical data, ExGANs can generate low-probability data in a controlled manner; only the extremes of the samples generated by GANs are added to the dataset to bias the data, and then use them to train the GAN conditional on the extreme value. They confirmed that they can generate the data successfully in the weather data.
Achieve both accuracy and speed by dividing filters into blocks to prune
YOLObile: Real-Time Object Detection on Mobile Devices via Compression-Compilation Co-Design
https://arxiv.org/abs/2009.05697
In order to achieve both high accuracy and low latency for mobile ,they proposed a block-puhched pruning method that divides each layer into blocks and learns different pruning patterns in each block. They succeeded in increasing the speed while maintaining the accuracy. The GPU is used to compute the convolutional layer, while the CPU is used to compute the other layers, to further improve the performance.
Control the generated images by understanding the role of each unit
Understanding the Role of Individual Units in a Deep Neural Network
https://arxiv.org/abs/2009.05041
Research to interpret the role of each unit in DNNs. They found that there are units that learn concepts such as “tree” without explicitly giving them to the DNN. In GAN, They have succeeded in reducing trees from the image and attaching doors to buildings by manipulating the units that control each concept.
Transformers Survey
Efficient Transformers: A Survey
https://arxiv.org/abs/2009.06732
A survey paper on the improvement system of the Transformer model, which has been spreading rapidly in recent years, especially in natural language processing. It is summarized in terms of memory , usage pattern of attention, and so on. It gives a good overview of the flow of the paper from the figures and discussions.
Density functional theory and Machine Learning
Kohn-Sham equations as regularizer: building prior knowledge into machine-learned physics
https://arxiv.org/abs/2009.08551
In the approximating physical simulations using density functional theory with neural networks,physical constraints can be imposed on ML by treating the Kohn-Sham equation as a differentiable model.This greatly improves the accuracy of XC term calculations
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — —
2. Technical Articles
— — — —
An interview with Anthony Goldbloom, founder of Kaggle
An interview with Anthony Goldbloom, founder of Kaggle, about Kaggle’s competition proved the usefulness of Random Forest, how GBDT isn’t that popular in academia, but it works well with structured data, and the introduction of Kaggle Notebook and Kaggle Dataset creates an environment where many people are free to express their insights.
Times Series Prediction models with COVID-19
It was known that time series forecasting models such as demand forecasting models would fail due to COVID19, but there is a debate on Reddit about how to deal with it.There are some opinions such, COVID-19 was treated as a seasonal factor with a large impact because the trend was equivalent to a seasonal decline, and so on.
