This is my second post about the history of neural networks. You'll find the first one here.
1990 - A new winter
Already towards the end of 1980 it became clear that the backpropagation algorithm would prove to be a problem. Because this failed because of deep neural networks with many layers. Many in the research community lost their interest in neural networks again. This, since there was a new method on the rise: the "Support Vector Machines". Support vector machines are a supervised learning models which analyzes and classifys data. [1]
This image shows how a support vector machine classifies a data set in two different groups.
Source: Cyc
In the year 2002 a support vector machine achieved great results in the classification of handwritten digits.
The error rate was only 0.56%! [2]
2006 - The Era of Deep Learning
In 2006, a paper entitled "A fast learning algorithm for deep belief nets" was published. In it, a method was presented with which deep neural networks can be trained quickly. Nevertheless, the speed of the computers was still a limiting factor. Rajat Raina, Anand Madhavan and Andrew Y. Ng published 2009 their work "Large-scale Deep Unsupervised Learning using Graphics Processors". They reached a performance increase of up to 70% through the use of graphics cards[3]. Since then, the interest in Deep Learning has grown with numerous companies, including Google, IBM and Microsoft[1].
Today
Everyday new amazing papers come out and show us what amazing things computer are capable to do.
Sources
[1] Kurenkov, Andrey. A ’Brief’ History of Neural Nets and Deep Learning, Part 4. url: http://www.andreykurenkov.com/writing/a- briefhistory-
of-neural-nets-and-deep-learning-part-4/.
[2] http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.88.9924&rep=rep1&type=pdf
[3] Raina, Rajat und Madhavan, Anand und Ng, Andrew Y. “Large-scale deep unsupervised learning using graphics processors”. In: Proceedings of the 26th annual international conference on machine learning. ACM. 2009, S. 873–880.