Unsupervised feature learning for audio classification using convolutional deep belief networks

Unsupervised feature learning fo r audio classification using con volutional deep belief networks Honglak Lee, Yan Largman, Peter Pham and Andr ew Y. Ng Presented by Bo Chen, 5.7,201

Upload
jeroen
Category

Documents
view
96
download
0

Embed Size (px):

description

Unsupervised feature learning for audio classification using convolutional deep belief networks. Honglak Lee, Yan Largman, Peter Pham and Andrew Y. Ng. Presented by Bo Chen, 5.7,2010. Outline. 1. What’s Deep Learning? 2. Why use Deep Learning? 3. Foundations of Deep Learning - PowerPoint PPT Presentation

Transcript of Unsupervised feature learning for audio classification using convolutional deep belief networks

Unsupervised feature learning for audio classification using convolutional deep belief net

works

Honglak Lee, Yan Largman, Peter Pham and Andrew Y. Ng

Presented by Bo Chen, 5.7,2010

Page 2: Unsupervised feature learning for audio classification using convolutional deep belief networks

Outline

• 1. What’s Deep Learning?

• 2. Why use Deep Learning?

• 3. Foundations of Deep Learning

• 4. Convolutional Deep Belief Networks

• 5. Results

Page 3: Unsupervised feature learning for audio classification using convolutional deep belief networks

Deep Architecture

• Deep architectures: compositions of many layers of adaptive non-linear components.

Difficulty: parameter searching (local minima)

• Deep belief nets: probabilistic generative models that are composed of multiple layers of stochastic, latent variables. (Hinton et al., 2006)

Deep Learning Wiki

Page 4: Unsupervised feature learning for audio classification using convolutional deep belief networks

Why Use Deep Learning

• Insufficient depth can hurt Usually our experiences tell us that one-layer machine only gives us

a set of general dictionary elements, unless a huge number of dictionary elements.

• The brain has a deep architecture• Cognitive processes seem deep• Learn a feature hierarchies or the complicated fu

nctions that can represent high-level abstractions

For example, PixelsEdgletsMotifsPartsObjectsScenes

Some from Yoshua Bengio’s course notes and Yann Lecun, et.al.,2010

Page 5: Unsupervised feature learning for audio classification using convolutional deep belief networks

One-layer dictionary

30 16x16 dictionary elementsand reconstructed images

250 16x16 dictionary elementsand reconstructed images

Page 6: Unsupervised feature learning for audio classification using convolutional deep belief networks

Restricted Boltzmann Machine

Figure from R Salakhutdinov et. al.

Energy functionBinary-valued

Real-valued

Contrastive divergence is used to solve the problem. (Hinton et al., 2006)

Page 7: Unsupervised feature learning for audio classification using convolutional deep belief networks

Deep Architectures

RBM in the different layers can be independently trained.

Page 8: Unsupervised feature learning for audio classification using convolutional deep belief networks

Convolutional Network Architecture

Figure from Yann LeCun et. al, 1998

Intuitively, in each layer the weight matrix will catch the most consistent ‘structures’ through all of the images.

Page 9: Unsupervised feature learning for audio classification using convolutional deep belief networks

3-dimensional Dictionary elements in the second layer

The dictionary element in the second layeris a 3-dimensional matrix.

D: the first-layer dictionary element E: the second-layer dictionary elementS: the convolution of the image and the first-layer elements.

Page 10: Unsupervised feature learning for audio classification using convolutional deep belief networks

Convolutional RBM with Probabilistic Max-Pooling Layer

Max-pooling Layer

Convolutional Deep Belief Networks

: the weight matrixConnecting poolingunit Pk to detection unit H’l.

Results on Natural Images

Page 13: Unsupervised feature learning for audio classification using convolutional deep belief networks

Results Caltech101 Images

Unsupervised learning of hierarchical representations with …rgrosse/cacm2011-cdbn.pdf · 2017. 8. 6. · our model, the convolutional DBN (CDBN), where weights are shared among

Evolution of features in Computer Visionrm17770/mlreading/dcgan_pres.pdf · DCGAN: Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks ConGAN:

CONVOLUTIONAL DEEP BELIEF NETWORKSdasgupta/254-deep/emanuele.pdf · Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations probabilistic

5LSM0: Convolutional neural networks for computer visionvca.ele.tue.nl/sites/default/files/5LSM0_Module_09 Unsupervised learning.pdfZhu et al. "Unpaired image‐to‐image translation

Evolving Deep Unsupervised Convolutional Networks for ...people.idsia.ch/~koutnik/papers/koutnik2014gecco.pdf · Figure 1: The Unsupervised Learning { Evolutionary Reinforcement Learning

RESTRICTED BOLTZMANN MACHINESafb/classes/CS7616-Spring... · Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. Honglak Lee, Roger

Convolutional Deep Belief Networks for Scalable ...cv.znu.ac.ir/afsharchim/lectures/ML/CNNExample.pdf · Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical

Perception-link Behavior Model€¦ · [1] H. Lee et al, “Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations,” in Proceedings

Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations Honglak Lee, Roger Grosse, Rajesh Ranganath, and.

Unsupervised Prostate Cancer Detection on H&E using ... · We propose an unsupervised method using self-clustering convolutional adversarial autoencoders to classify prostate tissue

Preserving differential privacy in convolutional deep ...ix.cs.uoregon.edu/~dou/research/papers/MLJ2017.pdfPreserving differential privacy in convolutional deep belief networks ...

Convolutional*Deep*Belief*Networks*for*Scalable ...swadhin/reading_group/... · Convolutional*Deep*Belief*Networks*for*Scalable*Unsupervised*Learning*of*Hierarchical*Representations

Image: used under license from shutterstock · 2019. 3. 27. · Unsupervised Learning of Hierarchical Representations with Convolutional Deep Belief Networks, Honglak et al., ACM

Bildverarbeitung beim Laserstrahlschweißen · [2] Lee et al. - Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations [3] Springenberg

Local Convolutional Features With Unsupervised …openaccess.thecvf.com/content_iccv_2015/papers/Paulin...Local Convolutional Features with Unsupervised Training for Image Retrieval

End-to-End Unsupervised Deformable Image Registration with ...End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network Bob D. de Vos1, Floris F. Berendsen

A probabilistic model for recursive factorized image features.sergeykarayev.com/files/cvpr2011_slides.pdf · Convolutional Deep Belief Nets Stacked layers, each consisting of feature

Convolutional Neural Network Application in Biomedical Signalsjcsitnet.com/journals/jcsit/Vol_6_No_2_December_2018/5.pdf · 2019. 1. 15. · learning such as Auto-encoders, Deep Belief

Convolutional Deep Belief Networks for Scalable ... · •Convolutional Restricted Boltzmann Machine –Probabilistic max-pooling •Convolutional Deep Belief Networks –Scalable

Local Convolutional Features with Unsupervised Training ... · Local Convolutional Features with Unsupervised Training for Image Retrieval Mattis Paulin 1Matthijs Douze Zaid Harchaoui;2

Unsupervised feature learning for audio classification using convolutional deep belief networks

Documents

Transcript of Unsupervised feature learning for audio classification using convolutional deep belief networks