VIDEO AND IMAGE ANALYTICS:

A KEY PART OF AN ANALYTICS STRATEGY

Mike Furst, Director, Video and Image Analytics Lab, PARC

PARC IS AN INNOVATION PARTNER

2

Breakthrough innovation: from problem to prototype

Ideation — novel and customized technological solutions

Visioning and roadmapping — how to get there from here?

Tech creation and development — deep expertise & ongoing research

45 Years of The Business of Breakthroughs®

PARC TODAY

Six INNOVATION FOCUS AREAS

123 DIGITAL

MANUFACTURING

4

PRINTED ELECTRONICS

5

META MATERIALS

VIDEO ANALYTICS6

DATA ANALYTICS

CONTENT CENTRIC

NETWORKING

1

3

2

TODAY’S GOALS

VIDEO & IMAGE

ANALYTICS

5

Video and Images: The Other Big Data

Getting Started

Innovation Examples: The Right tools for the job

6

BIG DATA AND ANALYTICS ARE TRANSFORMING THE WAY

THE WORLD WORKS

TransportationRetailHealthcare

SecurityEducationManufacturing

PARC | 7

VIDEO AND IMAGES – “THE OTHER BIG DATA”

Your activity

What you

buy

Who you like

How you

perform

How you get

there

What motivates

you?

What you

consume

Structured

Specific

Transaction

Computational

What you

considered

What you (and they) saw

Your surroundingsWho was nearby

Rich in content

Flexible

Existing?

…and growing @ incredible rates

Video

TAXONOMY OF CAPABILITIES IN VIDEO ANALYTICS

PARC | 8

Rare Event Detection/ Identification:Tracking / Timing / Scene Analysis:

Video parking /

Que timing

In-store item and

people tracking

Vandalism /

People + Actions

Unexpected driving Event-driven Video

CompressionMonocular speed

estimation

Anomaly detection

with single and

multiple objects and

trajectories

Object Detection/ Identification: Event Detection/ Identification:

Industry Leading

License Plate

Recognition

Red Light ViolationParking Occupancy

Biometrics

Vehicle and

OccupancyPeople in Scenes

Signal Extraction:

Visually Assessed

Disease Screening

& fine motion extraction

Heart rate &

HR Variability

Fundamentals

Scene Analysis

People Monitoring and First-person Vision Surveillance → Interpretation

Operational

analytics

Process AnalyticsDriver Gaze

Monitoring

Expression and

EmotionClassifying Group

BehaviorStress Monitoring Wearable-assisted

Process Automation

People &

Behavior

Analysis

Semantic Understanding of People and Scenes

VIDEO AND IMAGES FROM “3+1 PERSPECTIVES” ARE

OPTIONS TO BE CONSIDERED

Xerox Confidential9

Fixed

Mobile 1st

Person

Scale

(1-N)

Mobile

3rd

Person

Surveillance

Fixed 3rd Person

Wearable and

Passive 1st person

Handheld or Mounted

Active 3rd person

TECHNICAL LANDSCAPE TO BE NAVIGATED

Technology and Architectural Options

• Suitability and customization of computer vision and analytics algorithms

• Capture, storage, processing (distributed <–> centralized)

Working “in the wild”

• Performance in non-laboratory conditions

Accuracy Required

• 6σ Accuracy?

• Fully-automated or collaborative with “human-in-the-loop”

Real-time?

• Action to be taken: in the moment, in the minute, later today, this month

10

ENGAGE WITH PARC FOR VIDEO ANALYTICS

• Identify the Opportunity and Potential

• Assess Options Available

• Learn from other Examples

• Scope Proof-of-Concept (PoC) Project

• Execute PoC and Validate Business Impact

• Demonstrate, Deliver, & Scale

11

Wo

rksh

op

1

3

2

TODAY’S GOALS

VIDEO & IMAGE

ANALYTICS

12

Video and Images: The Other Big Data

Getting Started

Innovation Examples: The Right tools for the job

EXAMPLES WILL SPAN FROM CONCEPT TO TRANSFER

• Identify the Opportunity and Potential

• Assess Options Available

• Learn from other Examples

• Scope Proof-of-Concept (PoC) Project

• Execute PoC and Validate Business Impact

• Demonstrate, Deliver, & Scale

13

Wo

rksh

op

TRANSPORTATION : EXAMPLE CLIENT ENGAGEMENTS

• Identify the Opportunity and Potential

• Assess Options Available

• Learn from other Examples

• Scope Proof-of-Concept (PoC) Project

• Execute PoC and Validate Business Impact

• Demonstrate, Deliver, & Scale

14

Wo

rksh

op

TRANSPORTATION:

VIDEO ANALYTICS ADDRESS MARKET NEEDS

Efficiency Safety & Security Predictability

Operations

Inspection

Usage Tracking

Situational Awareness

Accident Prevention

Asset Protection

Flow / Mobility

Parking

Public Transport

CONGESTION: IMPROVE TRAFFIC FLOW

16

Current State:

• Tolling increasingly relied on to support highway systems

• Booths impede traffic flow / Transponders require distributed infrastructure

Desired State:

• All video tolling with no lost revenue AND no false violations

PARC | 16

PARC | 17

Approach:

• Incorporating plate rules enhances performance

• Synthetic training data speeds scaling and adaptation

• Leverage “human-in-the-loop” to reach performance

• Design in feedback for ongoing improvement

Bala et al., SPIE-EI 2012

Challenges:

• Hundreds of plate designs with yearly expansion

• Environmental conditions

• Accuracy required

EXAMPLE: IMPROVE TRAFFIC FLOW WITH IMAGE-BASED LICENSE PLATE RECOGNITION

Result:

• State-of-the-art performance

• Millions of plate / month

CONGESTION: EFFECTIVE MANAGED LANES CAN IMPROVE

OVERALL TRAFFIC FLOW

Traffic Congestion:• $87B/yr Wasted Time & Fuel•Green House Gas Emissions• $$ to build out new Roadway

HOV: “High Occupancy Vehicle” LanesHOT: “High Occupancy Tolling”• Vehicle Occupancy Detection• License Plate Recognition• Dynamic Incentive Pricing

Agency Response

PARC | 18

HOV lane in Toronto,

Canada (Hwy 404)

Road congestion near

Ontario, Canada (Hwy 401)

Current State

PARC | 19

EXAMPLE: EFFECTIVE HIGH-OCCUPANCY VEHICLE SYSTEMS

Approach:

• Customized combination of technologies

• Simplify design with a priori knowledge

• Classification vs. face detection

Challenges:

• Lighting and environmental conditions

• Passenger compartment illumination

• Variation in passenger pose and obstructions

• Variation in vehicle make / model

• Accuracy required in near real-time

Result:

• First operationally effective system at required

accuracy and yield

• New market for client and industry

RETAIL: EXAMPLE CLIENT ENGAGEMENTS

• Identify the Opportunity and Potential

• Assess Options Available

• Learn from other Examples

• Scope Proof-of-Concept (PoC) Project

• Execute PoC and Validate Business Impact

• Demonstrate, Deliver, & Scale

20

Wo

rksh

op

RETAIL:

VIDEO ANALYTICS ADDRESSES MARKET NEEDS

Operational EffectivenessRight Info at Right Time at

Right PlaceConnecting “Clicks” and

“Bricks”

21

Complete Guest Picture

Guest Experience Management

Guest Interactions

Situational Awareness

Capture and Signage

Associate Effectiveness

Guest Detection and Tracking

Layout and Product Optimization

Inventory Management

VIDEO ANALYTICS PROVIDES RICH DATA ENABLING

DATA -> INSIGHTS -> ACTIONS

Video Analytics Generated Data

Data

Fusion

• Operations

• Business Intelligence

Customer Loyalty Data

Sales, Inventory, Supply Chain Data

Other Real-Time Tracking Technology Data

PARC | 22

GUEST TIMING SOLUTION: IMPROVED SITUATIONAL

AWARENESS USING VIDEO ANALYTICS

Real-Time

Dashboard

Drives Productivity:

• Shifts Compete

• Managers Compete

• Stores Compete

PARC | 23

Order-to-Fulfillment KPIs:

1. Pre-Order queue time

2. Order time

3. Pay time

4. Present time

5. Total Experience Time

Order-to-Fulfillment Time Measurement Proof-of-Concept

STEP 1

STEP 3STEP 2

PARC | 24

GUEST & CREW BEHAVIOR DETECTION:

CLICKS-TO-BRICKS USING VIDEO ANALYTICS

Enables Next

Generation

Retail Analytics

Foot Traffic Dwell Guest Touch

Video Analytics work in

progress: Guest

Tracking, Timing, &

Product Interaction

BUSINESS PROCESS OUTSOURCING:

EXAMPLE FROM XEROX

• Identify the Opportunity and Potential

• Assess Options Available

• Learn from other Examples

• Scope Proof-of-Concept (PoC) Project

• Execute PoC and Validate Business Impact

• Demonstrate, Deliver, & Scale

25

Wo

rksh

op

ACCELERATING DOCUMENT WORKFLOWS:

XEROX SAFE COURIER

• Mobile app bringing video analytics to secure document

workflows on mobile, smart phone platforms

• Intelligent photo capture

• Automatic document

detection, recognition

• Optimized user experience

• Secure data transmission

26

SOME EXAMPLES FROM RECENT EXPLORATORY PROJECTS IN

HEALTHCARE AND INDUSTRIAL MARKET SEGMENTS

• Identify the Opportunity and Potential

• Assess Options Available

• Learn from other Examples

• Scope Proof-of-Concept (PoC) Project

• Execute PoC and Validate Business Impact

• Demonstrate, Deliver, & Scale

27

Wo

rksh

op

HEALTHCARE:

VIDEO ANALYTICS ADDRESSES MARKET NEEDS

Excellent Health Outcomes Control Costs

28

Remote and Mobile Patient Monitoring

Patient Tracking

Novel Sensing and Diagnostics

Safety and Security

Procedural Compliance

Process Optimization

INDUSTRIAL:

VIDEO ANALYTICS ADDRESSES MARKET NEEDS

Measure & improve operational efficiency

Ensure Safety and process compliance

29

Process Optimization

Automation Enhancements

Operations Business Intelligence

Safety and Security

Procedural Compliance

Maintenance Early-warning

1

3

2

TODAY’S GOALS

VIDEO & IMAGE

ANALYTICS

30

Video and Images: The Other Big Data

Getting Started

Innovation Examples: The Right tools for the job

START WITH THE QUESTION IN THE EXPLORE WOKSHOP

• Identify the Opportunity and Potential

• Assess Options Available

• Learn from other Examples

• Scope Proof-of-Concept (PoC) Project

• Execute PoC and Validate Business Impact

• Demonstrate, Deliver, & Scale

31

Wo

rksh

op

A GOOD TOOLBOX TO NAVIGATE THE TECHNICAL AND

OPPORTUNITY LANDSCAPE

PARC | 32

• Appropriate State-of-the-Art Algorithms and Methods

• Rapid Prototyping Capability

• System Optimization:

• Camera Networks

• Processing Architecture

• Computer Vision Algorithms

• Design for Six Sigma Development Process

• Expertise in Robust Large Scale Deployment

• Breakthroughs

• Proof-of-Concept

• Path to Delivery

Corporate & Divisional Innovation

Transportation, Logistics, and Manufacturing

Automotive & Aerospace

High Tech & Telecommunications

Financial Services & Marketing

Construction & Infrastructure

Consumer Products & Retail

Energy & Natural Resources

HealthcareMedia &

Entertainment

PARC ENGAGES MANY INDUSTRIES

CONTACT US TO EXPLORE HOW WE CAN HELP

34

Breakthrough innovation: from problem to prototype

Ideation — novel and customized technological solutions

Visioning and roadmapping — how to get there from here?

Tech creation and development — deep expertise & ongoing research

www.parc.com

https://ninesights.ninesigma.com/web/parc-gallery

For More Information:

THANK YOU.