Become a member
Sign in
Tambet Matiisen
Tambet Matiisen

Tambet Matiisen

9 Following
6 Followers
  • Profile
  • Highlights

Highlighted by Tambet Matiisen

See more

From An Overview of ResNet and its Variants by Vincent Fung

As a matter of fact, ResNet was not the first to make use of shortcut connections, Highway Network [5] introduced gated shortcut connections. These parameterized gates control how much information is allowed to flow across the shortcut. Similar idea can be found in the Long Term Short Memory (LSTM) [6] cell, in which there is a parameterized forget gate that controls how much information will flow to the next time step. Therefore, ResNet can be thought of as a special case of Highway Network.