Caffe:Deep learningFramework

Ramin FahimiPyCon 2016 , IUST

Many contents has been taken from Caffe CVPR’15 tutorial and CS231n lectures, Stanford.

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Took million years from nature to form effective visual recognition system.

It didn’t happened in one night!

Evolution.

Computer Vision1. Controlling processes: an industrial robot2. Navigation: an autonomous vehicle3. Detecting events: visual surveillance4. Organizing information: indexing databases of

images and image sequences5. Modeling objects or environments: medical

image analysis or topographical modeling6. Interaction: input to a device for computer-

human interaction7. Automatic inspection in manufacturing

applications.

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

ILSVRC – Visual recognition challenges

What is neural network?

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

What is deep learning? (DL)

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Input image Weights Loss

The number of neurons in each layer is given by 253440, 186624, 64896, 64896, 43264, 4096, 4096, 1000

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Why?What changed?

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Why?What changed?

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

1. Improvements in hardware 2. Data Size3. Initialization4. Successfully applying back propagation5. Many other things

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Training Deep networks

Back propagation

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Back propagation – cont.

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Frameworks

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Use Cases

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

• Extract AlexNet or VGG features? Use Caffe• Fine-tune AlexNet for new classes? Use Caffe• Image Captioning with fine-tuning?

o Need pre-trained models (Caffe, Torch, Lasagne) o Need RNNs (Torch or Lasagne) o Use Torch or Lasagna

• Segmentation? (Classify every pixel) o Use Caffe If loss function exists in Caffe else Use Torch

• Object Detection? o Need pre-trained model (Torch, Caffe, Lasagne) o Need lots of custom imperative code (NOT Lasagne), Use Caffe or Torch

Use Cases – Cont.

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

• Feature extraction / fine-tuning existing models: Use Caffe • Complex uses of pre-trained models: Use Lasagne or Torch • Write your own layers: Use Torch • Crazy RNNs: Use Theano or Tensorflow • Huge model, need model parallelism: Use TensorFlow

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Why Caffe?

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

§ Expression: models + optimizations are plaintext schemas, not code. § Speed: for state-of-the-art models and massive data. § Modularity: to extend to new tasks and architectures. § Openness: common code and reference models for reproducibility. § Community: joint discussion, development, and modeling

● Frameworks are more alike than different o All express deep models o All are nicely open-source o All include scripting for hacking and prototyping

● No strict winners – experiment and choose the framework that best fits your work

Open model collections

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

• The Caffe model zoo contains open collection of deep models

o VGG ILSVRC14 + Devil models in the zooo Network-in-Network / CCCP model in the zooo MIT Places scene recognition model in the zoo

• help disseminate and reproduce research• bundled tools for loading and publishing models • Share Your Models! with your citation + license of course

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Blobs

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Models and solvers are schema, not code

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Prototxt : Define Layer

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

References.

Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

[1]. http://caffe.berkeleyvision.org/[1]. http://cs231n.stanford.edu/[2]. http://www.panderson.me/images/Caffe.pdf

Thanks for your attention.Questions?