Códigos usados em sala de aula
Implementação de redes neurais artificias usando o pyBrain
Soluções de Exercícios
Implementação de dropout (Exercício 3)
Autoencoder (Exercício 4)
Exemplo de redes neurais recorrentes (Exercício 5)
MNIST (Exercício 6)
Referências
Pattern Recognition and Machine Learning - Christopher Bishop [Seções 5.1 a 5.5]
Neural networks - Haykin [Capítulo 4]
Deep learning - Ian Goodfellow, Yoshua Bengio and Aaron Courville [Capítulo 6, 7, 8, 9, 10, 11, 12, 14]
LeCun, Y., Bengio, Y. and Hinton, G. E. (2015) Deep Learning Nature, Vol. 521, pp 436-444.
Hinton, G. E. (2007) Learning Multiple Layers of Representation. Trends in Cognitive Sciences, Vol. 11, pp 428-434.
Quoc V. Le A Tutorial on Deep Learning Part 1: Nonlinear Classifiers and The Backpropagation Algorithm
Quoc V. Le A Tutorial on Deep Learning Part 2: Autoencoders, Convolutional Neural Networks and Recurrent Neural Networks
Referências Complementares
Razvan Pascanu, Çağlar Gülçehre, Kyunghyun Cho and Yoshua Bengio, How to Construct Deep Recurrent Neural Networks, in: International Conference on Learning Representations 2014(Conference Track), 2014
Guillaume Alain and Yoshua Bengio, What Regularized Auto-Encoders Learn from the Data-Generating Distribution (2014), in: Journal of Machine Learning Research, 15(3563-3593)
Hinton, G. E. Where do features come from?. Cognitive Science, Vol. 38(6), pp 1078-1101.
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. and Salakhutdinov, R. Dropout: A simple way to prevent neural networks from overfitting The Journal of Machine Learning Research, 15(1), pp 1929-1958.
Sutskever, I., Martens, J., Dahl, G. and Hinton, G. E. On the importance of momentum and initialization in deep learning In 30th International Conference on Machine Learning, Atlanta, USA, 2013.
Yoshua Bengio and Aaron Courville, Deep Learning of Representations, in: Handbook on Neural Information Processing, Springer: Berlin Heidelberg, 2013
Çağlar Gülçehre and Yoshua Bengio, Knowledge Matters: Importance of Prior Information for Optimization, in: International Conference on Learning Representations (ICLR'2013), 2013
Yoshua Bengio, Aaron Courville and Pascal Vincent, Representation Learning: A Review and New Perspectives (2013), in: Pattern Analysis and Machine Intelligence, IEEE Transactions on, 35:8(1798-1828)
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I. and Salakhutdinov, R. R. Improving neural networks by preventing co-adaptation of feature detectors http://arxiv.org/abs/1207.0580, 2012
Suskever, I., Martens, J. and Hinton, G. E. Generating Text with Recurrent Neural Networks. Proc. 28th International Conference on Machine Learning, Seattle, 2011.
Nicolas Le Roux and Yoshua Bengio, Deep Belief Networks are Compact Universal Approximators (2010), in: Neural Computation, 22:8(2192-2207)
Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pierre-Antoine Manzagol, Pascal Vincent, Samy Bengio; Why Does Unsupervised Pre-training Help Deep Learning? Journal of Machine Learning Research, 11(Feb):625−660, 2010.
Hugo Larochelle, Yoshua Bengio, Jerome Louradour and Pascal Lamblin, Exploring Strategies for Training Deep Neural Networks (2009), in: Journal of Machine Learning Research, 10(1--40)
Yoshua Bengio, Learning deep architectures for AI (2009), in: Foundations and Trends in Machine Learning, 2:1(1--127)
van der Maaten, L. J. P. and Hinton, G. E. Visualizing Data using t-SNE. Journal of Machine Learning Research, Vol 9, (Nov) pp 2579-2605, 2008.
Hinton. G. E. What kind of a graphical model is the brain? International Joint Conference on Artificial Intelligence 2005, Edinburgh.
Yoshua Bengio, Gradient-Based Optimization of Hyperparameters (2000), in: Neural Computation, 12:8(1889--1900)
Hinton, G.E. Supervised learning in multilayer neural networks in The MIT Encyclopedia of the Cognitive Sciences Editors: Robert A. Wilson and Frank C. Keil The MIT Press, 1999.
Hinton, G. E., Plaut, D. C. and Shallice, T. Simulating brain damage Scientific American, 1993.
Nowlan. S. J. and Hinton, G. E. Simplifying neural networks by soft weight sharing.
Neural Computation, 4, 173-193.
Hinton, G.E. How neural networks learn from experience. Scientific American, September 1992.
Rumelhart, D. E., Hinton, G. E., and Williams, R. J. Learning representations by back-propagating errors.
Nature, 323, 533--536, 1986.
Hinton, G. E. Learning distributed representations of concepts. Proceedings of the Eighth Annual Conference of the Cognitive Science Society, Amherst, Mass, 1986.
Hinton, G. E., McClelland, J. L., and Rumelhart, D. E. Distributed representations. In Rumelhart, D. E. and McClelland, J. L., editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1: Foundations, MIT Press, Cambridge, MA. pp 77-109, 1986.
Rumelhart, D. E., Hinton, G. E., and Williams, R. J.
Learning internal representations by error propagation.
In Rumelhart, D. E. and McClelland, J. L., editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1: Foundations, MIT Press, Cambridge, MA. pp 318-362, 1986.
