My bet is on Chomsky

My bet is Chomsky is right in the Chomsky vs Norvig debate and Geoffrey Hinton just provided some amazing fuel on the fire.

If you don’t know what the Chomsky vs Norvig debate is, it started with Chomsky deriding AI researchers about heavily relying on opaque probabilistic models for speech and vision instead of doing “real” science to figure out how the brain actually works. Norvig provided a rebuttal where he claimed that because the probabilistic models are so successful, it “is evidence (but not proof) of a scientifically successful model.”

And if you have been witness the huge success that Deep Learning has had for processing text and images you might have agreed that Norvig is right. That there must be something deeply right about these probabilistic models. However, most researches in the know will tell you that Deep Learning is highly problematic because it requires a huge amount of data to train a good system. I have believed that because of how these systems are trained is so different from how the brain learns they simply cannot be evidence of a scientifically correct model. These Deep Learning systems are also easily fooled once trained.

Then comes Geoffrey Hinton, who is famous for helping ignite the Deep Learning revolution, with a new model he calls the Capsule model which is based on how he thinks the brain processes images. His model uses unsupervised learning to extract low level features into a linear manifold, and from there training labels requires very few examples. He has essentially come up with a system which he says “does inverse computer graphics”. Where pixels are turned into objects and their poses. The initial results of the new system are incredibly exciting and Wired did a nice write up about it.

What is exciting to me is that Geoffrey is doing real science. He has come up with a theory about how the brain must do image processing and created a computer model to validate the theory. He brings a lot of old ideas back into the spotlight. Not only did he come up with a computer model, but one that could possibly blow away any of the existing probabilistic models by requiring orders of magnitude fewer examples with the same or better performance. This is the kind of science Chomsky says needs to happen and I believe Mr Hinton has just shifted the debate.