Machine LearningThe Cutting Edge

Sergii ShelpukDirector, Data Science SoftServe,

Inc.sshel@softserveinc.com

Classification Problem

Recognize what is a bike and what is a moon

Classification Problem

Classifier

©A. Ng

Classification Problem

pixelintensity

Classification Problem

Raw data does not represent the picture well. You need some smart features

contains wheels

contains seas

Feature Extraction

Classifier

Feat

ure

extr

acto

r

©A. Ng

Feature Extraction

Can we do better?

Neural Networks

a

a

a

a

a

a

a

a

a

a

a

a

a

afeat

ures bike

moon

Neural Networks

Neural Networks

aX

a0

a1

a2

w0

w1

w2

Activation function:aX = f(a0, a1, a2, w0, w1, w2)

Example (logistic):aX = 1 / (1 + e-(a0*w0+a1*w1+a2*w2))

Autoencoder

Autoencoder

© H. Lee et al.

Autoencoder

© Q Le et al.

Deep Learning Neural Network

Pre-trained as AutoencoderTypical classification

neural network

Moon

Deep Learning Neural Network

Vide

oText/N

LPIm

ages

©A. Ng

Deep Learning Neural Network

Hints and Tips Using unlabeled data Avoiding overfitting Computational efficiency

Using Unlabeled Data

wheels

handlebar

Avoiding Overfitting

Sparsity constraint limits variance of autoencoder

Avoiding Overfitting

Dropout ensures generalization of the neural network

Computational Efficiency

Thousands of cores Base Clock: 300-900 MHz Memory: 2-6 Gb Performance: up to 3.5 Tflops Instruction-level parallelism Shared memory Up to 4 devices in cluster

GPU computing provides cheapest computational power

Feature Learning: MNIST

Data:

Features:

Feature Learning: Galaxy Zoo

Data: Features:

Thank you!

?