Straight forward progression. We first didn't learn anything. Then just the classifier. Then just the ConvNet params. Now also architecture.

 — @karpathy