Open in app
Sign inGet started
Go to the profile of Kevin Shen
Kevin Shen in Mini Distill

Effect of batch size on training dynamics

This is a longer blogpost where I discuss results of experiments I ran myself.

In this experiment, I investigate the effect of batch size on training dynamics. The metric we will focus on is the generalization gap which is defined…

Read more…
23 responses
Go to the profile of Kevin Shen
Kevin Shen in Mini Distill

(PPS) Deformable Convolutional Networks

The basic idea of this paper is to give the convolution and pooling layers the ability to model different orientations and scales of objects in images. They do this by making the shape of the convolution filter learnable. For intuition, consider the…

Read more…
Go to the profile of Kevin Shen
Kevin Shen in Mini Distill

(PPS) Cascade Residual Learning: A Two-stage Convolutional Neural Network for Stereo Matching

This paper tackles stereo matching which is the task of matching pixels from two images taken with two different cameras to deduce depth information. This is similar to…

Read more…
Go to the profile of Kevin Shen
Kevin Shen in Mini Distill

Discrete Optimization: beyond REINFORCE

Gumbel Softmax (a.k.a. Concrete distribution)

While it might still be too early to start talking about the next A.I. winter, researchers are starting to see the limits of machine learning’s current set of…

Read more…
3 responses
Go to the profile of Kevin Shen
Kevin Shen in Mini Distill

(PPS) Dynamic Routing Between Capsules

Capsules is one of the recent works coming from Geoff Hinton which is why it’s so hyped up…

Read more…
Go to the profile of Kevin Shen
Kevin Shen in Mini Distill

(PPS) DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

This paper deals with semantic segmentation: the task of labeling pixels in an image by their object classes. The goal of semantic…

Read more…
Go to the profile of Kevin Shen
Kevin Shen in Mini Distill

(PPS) Efficient Deep Learning for Stereo Matching

Read more…
1 response
About
Mini Distill
Mini Distill
This is a space where I write short summaries or extended tutorials of interesting papers I've read. The language is intended to be informal as to minimize the time between reading and understanding.
More information
Followers
235
Elsewhere