Deep Learning on iOS #360iDev

360|iDev 2017

Shuichi Tsutsumi @shu223

Deep Learning on iOS

Overview• How to implement “Deep Learning” on iOS

Metal Performance Shaders (MPSCNN) Accelerate (BNNS)

Core ML

Vision

Your App

Why so exciting? !

AIphaGo

Cancer Detection

Self-driving Car

AutoDraw

Pose Estimation

http://qiita.com/nnn112358/items/a4490d85dac5827db53b

Frontal View Synthesis

AutoHair

AutoDraw Pose Estimation

Frontal View AutoHair

AutoDraw Pose Estimation

Frontal View AutoHair

This evolutional tech works on iOS"

Deep Learning ON iOS

= works on iOS devices

(Demo)

Image

Result60 times / sec

xUsers

#

#

TOOTHPASTE

Train InferenceTrained Params

iOS ML frameworks

Implementation

"""Core ML"""

The largest hall was 100% full for the Core ML session.


Core ML

Vision

Your App

iOS 11

iOS 10

Your App

Metal Performance Shaders (MPSCNN) Accelerate (BNNS) iOS 10

• Optimized for GPU (by ) and CPU (by Accelerate) • Available for iOS 10, too • Basically any ML tools can be used to train the models

Still works!


Your App

GPU CPU

MPSCNN 900 results

Core ML 160,000 results

3 steps to implement w/ MPSCNN

How to implement w/ MPSCNN

Step 1: Create the model

Train InferenceTrained Params

-Which tools can be used for the training? -What kind of formats can be used to pass the pre-trained params?

Which ML tools can be used? : ANY What model format can be used? : ANY

Model (Trained Params)

dat

Train

ML Tools



dat

Train

ML Tools

hdf5

•The “.dat” files are common binary files. - Not specific for iOS or MPS.

•Contains the trained params - Weights / Biases

•Any other format can be used as long as it can be read by the iOS app. - c.f. hdf5



dat

Train

ML Tools

hdf5

•Any tools which can train CNN, and export the params


Step 2: Implement the network

MPSCNNConvolution

MPSCNNFullyConnected

MPSCNNPooling

MPSCNNConvolution

MPSCNNConvolution

MPSCNNPooling

MPSCNNConvolution


MPSCNNPooling

MPSCNNConvolution

MPSCNNConvolution

MPSCNNPooling

• Almost same name -> Easy to find• Complicated Maths or GPU optimization are encapsulated

Classes corresponding to each CNN layers are provided


Step 3: Implement the inference

Input image

MPSImage

Result (UInt, etc.)

CNN • Implemented in Step 2 • Trained params (created in Step 1)

are loaded.

Do something

• Step 1: Create the model - Any ML tools can be used

- Any file format can be used for the trained params

• Step 2: Implement the network - Classes corresponding to each CNN layer are provided

• Step 3: Implement the inference - Input to the CNN, and output from the CNN

Demo : Logo detection

Increased by 70 times

Trained Params

dat

f.write(session.run(w_conv1_p).tobytes())

(Demo)

CNN for the logo detection

GoogLeNet (Inception v3)

GoogLeNet (Inception v3) Apple’s implementation w/ MPSCNN

Inception3Net.swift

2,000 lines&Only for the CNN

Core ML

Development Flow w/ MPSCNNML Tools

Train

some format

Trained Params

dat

Extract

Parse

dat

MPSCNNConvolution


Implement Network 2,000

lines&App

Development Flow w/ Core MLML Tools

Train

some format

Trained Params

dat

Extract

Parse

dat

MPSCNNConvolution


Implement NetworkApp 2,000

lines&

1) Convert w/ coremltools

2) Drag & Drop

some format

Generate

Input image

MPSImage

Result (UInt, etc.)

CNN

Do something

let size = MTLSize(width: inputWidth, height: inputHeight, depth: 1) let region = MTLRegion(origin: MTLOrigin(x: 0, y: 0, z: 0), size: size) network.srcImage.texture.replace( region: region, mipmapLevel: 0, slice: 0, withBytes: context.data!, bytesPerRow: inputWidth, bytesPerImage: 0)

Need to know Metal to use MPSCNN

let origin = MTLOrigin(x: 0, y: 0, z: 0) let size = MTLSize(width: 1, height: 1, depth: 1) finalLayer.texture.getBytes(&(result_half_array[4*i]), bytesPerRow: MemoryLayout<UInt16>.size*1*4, bytesPerImage: MemoryLayout<UInt16>.size*1*1*4, from: MTLRegion(origin: origin, size: size), mipmapLevel: 0, slice: i)

MPSCNN Accelerate (BNNS)

Core ML

Vision

Your App

Input image

MPSImage

Results

CNN

Do something

let ciImage = CIImage(cvPixelBuffer: imageBuffer)let handler = VNImageRequestHandler(ciImage: ciImage)try! handler.perform([self.coremlRequest])

Don’t need to touch Metal to use Vision

guard let results = request.results as? [VNClassificationObservation] else { return }guard let best = results.first?.identifier else { return }


Your App


Core ML

Vision

Your App


Core ML

Vision

Your App

How about BNNS?

Deep Learning on CPU?!

CPU: 40 days GPU: 6 days

Deep Learning on CPU?!

Asked @ Lab in WWDC17

“How can we utilize both MPSCNN and BNNS?”

' “Basically use MPSCNN.”

“OK, but, when should I choose BNNS?”

' “I don’t know.”

• He added “watchOS might be a case on that you should use BNNS.”

• Because watchOS doesn’t support MPSCNN, but supports BNNS. (I haven’t tried yet.)

My current understanding:• The cost for passing data between CPU and GPU is

not small • When the network is small, the CPU <-> GPU cost

might be bigger than the benefit of parallel processing.

BNNS might be better when the network is small.

Recap• Why is “Deep Learning on iOS” exciting? • How to implement “Deep Learning” on iOS

- w/ MPSCNN (iOS 10)

- w/ Core ML & Vision (iOS 11)

- When to choose BNNS

MPSCNN BNNS

Core ML

Vision

Thank you!

https://github.com/shu223

Deep Learning on iOS #360iDev

Technology

Transcript of Deep Learning on iOS #360iDev