Intelligent Video Analysis System

Based on

GPU and Distributed Architecture

Dr. Shiliang PU Hikivision Research Institute

High Resolution VS storage

Complexity VS Accuracy

Mass data VS efficiency

Mid-size City, about 22,000 cameras

316PB/year Precious video content service under complex situation

Challenge in Video Surveillance

Surveillance video content analysis

Object detection

• Human

• Vehicle

• others

Feature

• Human feature

• Vehicle feature

Identification

• Human body

• Face

• Vehicle

Surveillance video content understanding framework

Challenge in Video Surveillance

Traditional algorithm can understand simple or standard scene content

车型

Sun blade closed

Phone calling

White

Safe Belt

Car

Ford Fiesta

皖A？？66R

Glass worn

Male

Clothes color

Teenage

Height

……

Traditional algorithm fails in such complex scene content, which is very common in public surveillance.

Revolution By Deep Learning in

Surveillance

Deep Learning in Surveillance

Traditional algorithm Deep learning

Deep Learning in Surveillance

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

overall passenger

channel

indoor public

area

sunny

day

rainny

day

winter summer

Pedestrian detection Recall rate, fppi = 0.1

Traditional Deep learning

Deep Learning in Surveillance

Clothes type Riding Safe belt

not fastened Phone calling

backpack Hat Hanging bag Mask

Deep Learning in Surveillance

70

75

80

85

90

95

100

vehicle color brand model sun blade safe belt phone calling

Vehicle feature accuracy increased by Deep Learning

traditional algorithm deep learning

Deep Learning in Surveillance

Identity?

Deep Learning in Surveillance

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Rank1 Rank10 Rank20 Rank30 Rank40 Rank50

Face Recognition Rank in 1 million enroll dataset

Traditional Deep Learning

Deep Learning in Surveillance

Vehicle retrieval based on image comparison

Deep Learning in Surveillance

0.0000

0.1000

0.2000

0.3000

0.4000

0.5000

0.6000

0.7000

0.8000

0.9000

1.0000

Rank10 Rank25 Rank50 Rank100

Vehicle Image retrieval

Traditional Deep Learning

These are what we need!

BUT……

Limitation on Deep Learning

High computing performance Objects detection in surveillance video require 2T Flops/sec, which needs support from high-performance computing hardware.

High cost Price of GPU-based server is significant higher than general server.

High energy consumption General server costs around 9000KWh per channel every year.

GPU solution based on distributed

architecture

Tegra

Hikvision-Blade Server Base on GPUs

Advantage of Hikvision Blade Server

System stability meets industry requirement based on low-cost chip, based on distributed-computing architecture.

1

1-1 1-2 1-3

1-1-1 1-1-2 1-1-3

Advantage of Hikvision Blade Server

16,000

14,000 14,000

300 550

8050

0

2,000

4,000

6,000

8,000

10,000

12,000

14,000

16,000

18,000

Blade Tesla M40*2 General Server

Performance/power ratio

Performance（Gflops） Power（W）

High performance Low cost Low power consumption

Advantage of Hikvision Blade Server

Flexibility-for different product forms

Smart Server

Smart IPC

Smart NVR

Sensing

Storage

Application

Intelligent Video compressing standard

Surveillance Video Compressing Standard

General Video Compressing Standard

Background frame

IVA

Bit rate equalization

Smart 264

Intelligent Video compressing standard

24 hours typical surveillance scene contrast rate at a consistent subjective

quality case

outdoor busy free

H.264 1855Kbps 1245Kbps

Smart264 419Kbps 164Kbps

Promotion 4.43 7.57

indoor busy free

H.264 3448Kbps 1715Kbps

Smart264 945Kbps 315Kbps

Promotion 3.65 5.45

Intelligent Video compressing standard H

.264/H

.265

Sm

art

264

100%

H.264-3830kbps

Smart264-683kbps

17.8%

H.2651920kbps

50%

Video structured description

Human:

male female wear glasses

riding backpack handbag

Vehicle:

driver

driver’s sun visor

copilot

copilot’s sun visor

safe belt fastened/not

phone calling

Security Big Data framework

Non-structured data Structured data

Cloud storage

01

High speed data bus

High speed data bus

Distributed file database

Memory computing

Data mining

Fulltext database

Police Traffic

Other market

Big data manager platform

Big data service

Collecting mass data(video, image, alarm, GPS). Extracting structured data from video and images.

Offering high speed service, like data searching, analyzing and statistics.

Cloud analysis

Advantages from Security Big Data

Small size

Million data level

Low speed

Slow feature

extraction Low accuracy

Long time cost.

>10 nins

Issues on traditional system

01 Cloud analysis handles mass-data computing problem

02 Big data architecture handles above billions level data

03 Spark memory computing offers second degree service

04 Deep learning increases computing accuracy

Smart traffic

Police City

management

…… Smart city

Statistic Alarm Analysis

Q O S D ! f

F 8 D 6 A 1 0

F 5 4 F j u

* K 1 Y ^ g

Data inquiry

Security Big Data application

Security Big Data depth application

Case study – Billion-level image search engine

Image search based on image feature extraction and comparison,

based on billion-level vehicle images.

Case study- Face recognition system

Lost elder found in 5 seconds.

Future

Multi-sensor increases data dimensions.

Unsupervised learning in video surveillance

Optimized neural network framework

THE END

HIKVISION Internal