Review — Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction

Split-Brain Auto for Self-Supervised Learning, Outperforms Jigsaw Puzzles, Context Prediction, ALI/BiGAN, L³-Net, Context Encoders, etc.

Sik-Ho Tsang
Sep 5 · 4 min read
Proposed Split-Brain Auto (Bottom) vs Traditional Autoencoder, e.g. Stacked Denoising Autoencoder (Top)

In this paper, Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction, (Split-Brain Auto), by Berkeley AI Research (BAIR) Laboratory, University of California, is reviewed. In this paper:

This is a paper in 2017 CVPR with over 400 citations. (Sik-Ho Tsang @ Medium)


1. Split-Brain Autoencoders (Split-Brain Auto)

Split-Brain Autoencoders applied to various domains

1.1. Cross-Channel Encoders

By performing this pretext task of predicting X2 from X1, we hope to achieve a representation F(X1) which contains high-level abstractions or semantics.

1.2. Split-Brain Autoencoders as Aggregated Cross-Channel Encoders

By concatenating the representations layer-wise, Fl = {Fl1, Fl2}, a representation F is achieved which is pretrained on full input tensor X.

If F is a CNN of a desired fixed size, e.g., AlexNet, we can design the subnetworks F1, F2 by splitting each layer of the network F in half, along the channel dimension.

1.3. Alternative Aggregation Technique

However, it is found that the proposed Split-Brain Auto (Section 1.2) outperforms the above two alternatives (Section 1.3).

2. Experimental Results

2.1. ImageNet

Task Generalization on ImageNet Classification

Split-Brain Auto (cl, cl), cl means using classification loss, outperforms all variants and all self-supervised learning approaches such as Jigsaw Puzzles [30], Context Prediction [7], Ali [8]/BiGAN, Context Encoders [34] and Colorization [47].

2.2. Places

Dataset & Task Generalization on Places Classification

Similar results are obtained for Places Classification, it outperforms such as Jigsaw Puzzles [30], Context Prediction [7], L³-Net [45], Context Encoders [34] and Colorization [47].


Task and Dataset Generalization on PASCAL VOC

The proposed method, Split-Brain Auto (cl, cl), achieves state-of-the-art performance on almost all established self-supervision benchmarks.

There are still other results in the paper. If interested, please feel free to read the paper. Hope I can write a story about Jigsaw Puzzles in the coming future.

Geek Culture

Proud to geek out. Follow to join our +1.5M monthly readers.