Image Classification with Neural Networks

Advisor

Prof. Xifeng Yan, Ph.D

Mentor

Fangqiu Han

Bruce LiuComputer Engineering

NSF Award 0954125

Neural Networks What is a neural network?

Image Classification

Applications

• Search

• Autonomous Driving

• Speech Recognition

• Artificial Intelligence

Building a Neural Network

Using MATLAB

• Use gradient descent

• Learn backpropagation

algorithm

Build a Sparse Autoencoder

*Source: Machine Learning, Coursera.org, Andrew Ng

*

*

Image Classification

𝒙𝟏

𝑎𝟏1

𝑎21

⋮𝑎𝑚1

𝒙𝟏

𝑏11

𝑏21

⋮𝑏𝑚1

y

0

0

⋮0

y

1

1

⋮1

DataLabel

𝒙𝟐 … 𝒙𝒏

𝑎𝟏2 … 𝑎𝟏𝑛

𝑎22 ⋮ ⋮

⋮ ⋮ ⋮

𝑎𝑚2 … 𝑎𝑚𝑛

𝒙𝟐 … 𝒙𝒏

𝑏12 … 𝑏1𝑛

𝑏22 ⋮ ⋮

⋮ ⋮ ⋮

𝑏𝑚2 … 𝑏𝑚𝑛

Image Data

𝑥1

𝑥2

Repeat{ } until minimized 𝐽(𝜃)

Calculations

Cost Function:

Gradient Descent:

𝒙𝟏

𝒙𝟐

𝜃0𝜃1

𝐽(𝜃)

*Source: Machine Learning, Coursera.org, Andrew Ng

*

Magnitude of any given boundary parameter to actual data points

Overfitting and Regularization

Gradient Descent 𝜃0 + 𝜃1𝑥1 + 𝜃2𝑥2

2 + 𝜃3𝑥33+ 𝜃4𝑥4

4 +… + 𝜃𝑛𝑥𝑛𝑛

𝐽(𝜃)

Modify Cost Function

+𝜆

2𝑚

𝑖

𝑚

𝜃𝑗2

Gradient Descent

0 0 0

Multi-layer Network

Layer 1 Layer 2 Layer 3 Layer 4

Forward Propagate

Back Propagate

𝑥 Θ(1)Θ(2) Θ(3)

𝜹(𝟒)𝜹(𝟑)𝜹(𝟐)

Want:

𝜹(𝑳) =𝝏

𝝏𝜽𝑱(𝜽)

𝑎 1 = 𝑥𝑧 2 = 𝛩 1 𝑎 1

𝑎 2 = 𝑔 𝑧 2

𝑧 3 = 𝛩 2 𝑎 2

𝑎 3 = 𝑔 𝑧 3

𝑧 4 = 𝛩 3 𝑎 3

𝑎 4 = 𝑔 𝑧 4

𝜹 𝟒 = 𝑎 4 − 𝑦

𝜹 𝟑 = 𝛩 3𝑇𝜹 𝟒 .∗ 𝑔′ 𝑧(3)

𝜹 𝟐 = 𝛩 2𝑇𝜹 𝟑 .∗ 𝑔′ 𝑧(2)

Progress

Cost Function Logistic Regression

Some Conclusions..

Management of data

Choosing good parameters

A long way to go…

Plans for the future

Building and training different types

of neural networks

Enhance the accuracy

Acknowledgements

Advisor

Prof. Xifeng Yan, Ph.DMentor

Fangqiu Han

Thank you wonderful people at INSET