深度学习-综述

1、定义和背景：

1.1 深度学习（DL）有各种相近的定义或者高层次描述

定义2：Deep Learning is a new area of Machine Learning research, which has been introduced with the objective of moving Machine Learning closer to one of its original goals: Artificial Intelligence. Deep Learning is about learning multiple levels of representation and abstraction that help to make sense of data such as images, sound, and text.（参见https:///lisa-lab/DeepLearningTutorials）自2006年以来，深度学习（deep learning）（也通常叫做深层结构学习或分层学习）已经成为机器学习领域的一个新兴领域（Hinton et al., 2006; Bengio, 2009 ）.在过去几年中，深度学习技术的发展已经对信号和信息过程领域产生广泛的影响，并将继续影响到机器学习和人工智能的其它关键领域；参见综述文章（Bengio et al., 2013; Hinton et al., 2012; Yu and Deng, 2011; Deng, 2011; Arel et al., 2010 ）.最近，已有一系列的致力于关于深度学习以及应用的研讨会和特别会议。包括:

the 2013 ICASSP’s special session on New Types of Deep Neural Network Learning for Speech Recognition and Related Applications,

the 2010, 2011, and 2012 NIPS Workshops on Deep Learning and Unsupervised Feature Learning,

the 2013 ICML Workshop on Deep Learning for Audio, Speech, and Language Processing;

the 2012 ICML Workshop on Representation Learning,

the 2011 ICML Workshop on Learning Architectures, Representations, and Optimization for Speech and Visual Information Processing,

the 2009 ICML Workshop on Learning Feature Hierarchies,

the 2009 NIPS Workshop on Deep Learning for Speech Recognition and Related Applications, the 2008 NIPS Deep Learning Workshop,

the 2012 ICASSP tutorial on Deep Learning for Signal and Information Processing, the special section on Deep Learning for Speech and Language Processing in IEEE Transactions on Audio, Speech, and Language Processing (January 2012), and the special issue on Learning Deep Architectures in IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI, 2013).

一些DL领域活跃的实验室和研究团队包括：

University of Toronto - Machine Learning Group (Geoff Hinton, Rich Zemel, Ruslan Salakhutdinov, Brendan Frey, Radford Neal)

Université de Montréal - Lisa Lab (Yoshua Bengio, Pascal Vincent, Aaron Courville, Roland Memisevic)

New York University –Yann Lecun‘s and Rob Fergus‘ group

Stanford University –Andrew Ng‘s group

UBC –Nando de Freitas‘s group

Google Research–Jeff Dean, Samy Bengio, Jason Weston, Marc’Aurelio Ranzato, Dumitru Erhan, Quoc Le et al

Microsoft Research –Li Deng et al

SUPSI –IDSIA(Schmidhuber’s group)

UC Berkeley –Bruno Olshausen‘s group

University of Washington –Pedro Domingos‘ group

IDIAP Research Institute - Ronan Collobert‘s group

University of California Merced –Miguel A. Carreira-Perpinan‘s group

University of Helsinki - Aapo Hyvärinen‘s Neuroinformatics group

Université de Sherbrooke –Hugo Larochelle‘s group

University of Guelph –Graham Taylor‘s group

University of Michigan –Honglak Lee‘s group

Technical University of Berlin –Klaus-Robert Muller‘s group

Baidu –Kai Yu‘s group

Aalto University –Juha Karhunen‘s group

U. Amsterdam –Max Welling‘s group

U. California Irvine –Pierre Baldi‘s group

Ghent University –Benjamin Shrauwen‘s group

University of Tennessee –Itamar Arel‘s group

IBM Research –Brian Kingsbury et al

University of Bonn –Sven Behnke’s group