← Back to feed

Deep pretraining before AlexNet: the table is set

Before 2012, Hinton, Bengio, and LeCun pushed greedy pretraining, autoencoders, and evidence that deep nets could learn.

Deep pretraining before AlexNet: the table is set

Between 2006 and 2011, Geoffrey Hinton and collaborators popularized deep belief nets and layer-wise pretraining; Yoshua Bengio and Yann LeCun contributed theory, CNNs, and optimization. The “Deep Learning Conspiracy” still fought for acceptance.

Speech and digit results suggested depth mattered, but a large-scale vision media hit was missing.

That hit arrived with AlexNet at ILSVRC 2012. Without the prior decade of stubborn work, the boom would have been impossible.