JavaOne 2016 - Faces Counter

All Your Faces Belong To UsBuilding an Open Face Recognition Platform

Copyright © 2016 Accenture All rights reserved. 2

About Us

Senior Technology ArchitectAccenture SpainGlobal Java Community LeadTwitter: @_deorsGitHub: deors

Senior Technology AnalystAccenture SpainGlobal Java SMETwitter: @locoporf1GitHub: locoporf1


Related Sessions in Java One 2016Monday, Sep 19 Tuesday, Sep 20 Wednesday, Sep 21Introduction to Java ME 8 [CON3189]

Java ME and Single-Board Computers for Creating Industrial Middleware [CON3187]

All Your Faces Belong to Us: Building an Open Face Recognition Platform [CON6217]

12:30 p.m. - 1:30 p.m 2:30 p.m. - 3:30 p.m 3:00 p.m. - 4:00 p.m

Hilton - Golden Gate 6/7/8 Hilton - Golden Gate 6/7/8 Hilton - Golden Gate 6/7/8

Kevin HookeJulio Palma

Jorge HidalgoJulio Palma

Jorge HidalgoMariano Rodriguez


• Internet of Things• Applications

• OpenCV library• Cascade Classifier for Object Detection

• Lbp vs Haar• Java API• Object Recognition Sequence Diagram

• MQTT protocol• Quality of Service

• Faces counter• Solution schema• Development environment

• Demo live!

Building an Open Face Recognition Platform


Internet Of Things

• Objects: Pressure sensor in tires Monitor implant in hearts GPS in cabs Thermometers in offices …

• With unique identifier• Connected to internet• Transfer data without requiring

human interaction

Author


IoT Applications

• Smart Cities: smart street lights, traffic control, surveilance cameras, …

• Healthcare: remote monitoring, ambulance telemetry, drug tracking, …

• Smart Manufacturing: flow optimization, real time inventory, employee safety, …

• Wearable: entertaiment, fitness, smart watch, location & tracking, …

• Automotive: wire replacement, telemetry, predictive maintenance, …

• Building & Home automation: light & temperature control, access control, energy optimization, …

Smart homesAppliances

Wearables

Industrial internet

Healthcare Smart places

Service Industries

Smart citiesP


OpenCV Library

• Coded in C/C++• Free for academic or commercial use• Supports Windows, Linux, Mac OS, iOS and Android• Focused on real-time applications: optimized using hardware

accelaration and multi-core processing• C++, Phyton and Java interfaces• Current version: 2.4.13 (May, 2016)• Repository: https://github.com/opencv/opencv

https://github.com/opencv/opencv


Cascade Classifier

1. A XML file applied to every frame of the video stream to detect a particular object

2. OpenCV distribution contains some examples to detect eyes, cats, human faces, …

3. Can be generated to detect any object by training


Cascade Classifier Types

LBP Faster Calculations in integers Best option for

embedded/mobile

HAAR More accurate Calculations in floats

Falses Positives

Lost Frames / sec

LBP 0 2 5.72

HAAR 2 0 4.24


• org.opencv.core• Mat

• org.opencv.videoio• VideoCapture

• org.opencv.objdetect• CascadeClassifier

• org.opencv.imgproc• Imgproc

• org.opencv.imgcodecs• Imgcodecs

OpenCV Java API


Object Detection Sequence Diagram

read

VideoCapture

Mat

Imgproc

cvtColor

equalizeHist

CascadeClassfier

detectMultiScale

Imgcodecs

imwrite

new

new

rectangle


Low network bandwidth Low power usage Perfect for IoT Publish / Subscribe mechanism Can use WebSockets Downloads:

Client: http://www.eclipse.org/paho/ Broker:

http://projects.eclipse.org/projects/technology.mosquitto

MQTT Protocol

MQTT Broker

Topic

Device 1

publish

Device 2

Device N

subscribe

subscribe…

http://www.eclipse.org/paho/




MQTT: Quality of Service

Publisher Broker Subscriber

publish

0: At most once Guarantees a best effort delivery It won’t be acknowledged by the

receiver or stored and redelivered by the sender

Fire and forgetpublish




publish

1: At least once Guarantees that a message will be

delivered at least once to the receiver

But the message can also be delivered more than once

Application must be tolerating duplicates and process them accordingly

publish

puback

store message

delete message




publish

1: Exactly once Guarantees that each message is

received only once by the counterpart

Safest and also the slowest quality of service level

publishpubrec

pubrel

pubcomp

store message

store message


Face Detector Device

• Raspberry PI (model 3)• Raspicam (V2) ~ 60 $}


Faces Dectector Solution Schema (I)

…

Face detector devices

publish

MQTT Broker Web Server

subscribe

MQTT MQTT

WS


Face Counter Solution Schema (II)

Java RuntimeFacesDetector App

OpenCV MQTT Paho

JNI

OpenCV

Raspicam drivers

publish subscribe

MQTT broker Java Runtime

Spring Boot

Monitor WebApp

Vaadin


Setting up a Raspberry PI: Install dependecies

sudo apt-get update

sudo apt-get install cmake ant build-essential pkg-config libpng12-0 libpng12-dev libpng++-dev libpng3 libpnglite-dev zlib1g-dbg zlib1g zlib1g-dev pngtools libtiff4 libtiffxx0c2 libtiff-tools libjpeg8 libjpeg8-dev libjpeg8-dbg libjpeg-progs libavcodec-dev libavformat-dev libgstreamer0.10-0-dbg libgstreamer0.10-0libgstreamer0.10-dev libunicap2 libunicap2-dev libdc1394-22-dev libdc1394-22libdc1394-utils swig libv4l-0 libv4l-dev


Setting up a Raspberry PI: Enviroment variables

Edit ~/.bashrc file to add the following lines:export JAVA_HOME=/usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/export ANT_HOME=/usr/share/ant/


Setting up a Raspberry PI: Download OpenCV source code

wget https://codeload.github.com/Itseez/opencv/zip/3.1.0mv 3.1.0 opencv-3.1.0.zipunzip opencv-3.1.0.zip

https://codeload.github.com/Itseez/opencv/zip/3.1.0


Setting up a Raspberry PI: Create Makefile

cd opencv-3.1.0mkdir buildcd buildcmake -DCMAKE_BUILD_TYPE=RELEASE -DWITH_OPENCL=OFF -D BUILD_PERF_TESTS=OFF -DBUILD_SHARED_LIBS=OFF -DJAVA_INCLUDE_PATH=$JAVA_HOME/include -DJAVA_AWT_LIBRARY=$JAVA_HOME/jre/lib/amd64/libawt.so -DJAVA_JVM_LIBRARY=$JAVA_HOME/jre/lib/arm/server/libjvm.so -DCMAKE_INSTALL_PREFIX=/usr/local ..


Setting up a Raspberry PI: Compile, install and check• make install• ATTENTION!!! This process can take several hours!• Check if exist: build/libs/libopencv_java310.so and build/bin/opencv-

310.jar have been created!!!


Setting up a Raspberry PI: Install raspicam drivers• curl http://www.linux-projects.org/listing/uv4l_repo/lrkey.asc |

sudo agt-key add –• Edit file /etc/apt/source.list to add the following line: deb http://www.linux-projects.org/listing/uv4l_repo/raspbian/ jessie main• sudo apt-get update• sudo apt-get install uv4l uv4l-raspicam• sudo apt-get install uv4l-raspicam-extras• sudo raspi-config (Enable raspicam interface)• sudo rpi-update (Update firmware)

http://www.linux-projects.org/listing/uv4l_repo/lrkey.asc

http://www.linux-projects.org/listing/uv4l_repo/raspbian/


Setting up a Raspberry PI: Transfer files & run

• Using ftp client tool copy:• Faces detector properties and JAR files• MQTT Paho JAR file

• Edit properties file to customize• Run java -Djava.library.path=build/libs/libopencv_java310.so -jar

faces-counter.jar faces-counter.properties


Development Enviroment

Remote Mode (option I):• Oracle JDK • NetBeans• Putty• FileZilla (client)

Remote Mode (option II):• Oracle Java ME SDK• NetBeans

“Embedded” Mode• BlueJ or Geany• Oracle JDK


• Faces Detector: https://github.com/locoporf1/faces-detector

• WebApp Monitor: https://github.com/locoporf1/faces-detector-server

Face Detector App Source Code

https://github.com/locoporf1/faces-detector

https://github.com/locoporf1/faces-detector-server

28

Question & Answers

29

Live DEMO !!!

JavaOne 2016 - Faces Counter

Technology

Transcript of JavaOne 2016 - Faces Counter