Exploiting Semantics for Sensor Re-Calibration in Event Detection Systems

Ronen Vaisenberg, Shengyue Ji, Bijit Hore, Sharad Mehrotra and Nalini Venkatasubramanian

School of Information and Computer SciencesUniversity of California - Irvine

Outline

Motivation – Self Managed Sensors

Event detection systems Task Challenge Our approach

Exploiting semantics in Event Detection Systems Algorithms Analysis Experiments and results

Motivation – Self Managed Sensors

Managing Networks of Sensors can be very Challenging when sensors are: Distant and isolated Deployed in Large numbers

Human intervention is not desirable.

Example 1 – Isolated/Distant Sensors

Microwave radio-meters are used to provide continuous measurements of integrated water vapor (IWV) and integrated liquid water (ILW).

Deployed in isolation. (Rural Oklahoma and Kansas, the north slope of Alaska).

Physical changes in the sensor: Operates Based on mirrors that tend to slip as much as 1 degree on their stepper motor shafts due to continuous use.

Human intervention is not desirable,

Too far for frequent visits

Example 3 – Network with Large Number of Video Sensors

Video surveillance systems are being used in a variety of public spaces such as metro stations, airports, shipping docks, etc.

Example 3 – Network with Large Number of Video Sensors (Cont.) Changes in the field of view

Human intervention is not desirable,

Too many cameras

Outline

Motivation – Self Managed Sensors

Event detection systems Task Challenge Our approach

Exploiting semantics in Event Detection Systems Algorithms Analysis Experiments and results

The System Monitored – The context: Event Detection Systems (EDS)

The fundamental concept underlying EDS’s is the gathering, managing, analyzing, and interpreting of different information streams in a timely manner to recognize potential incidents early enough to respond effectively.

In most cases, event detection systems gather streams from sensors. Each sensor monitors a system of interest.

This system can be modeled.

The System Monitored – The Modeling: Finite State Machine Many systems of interest and observed environments can be

modeled as Finite State Machines. For example: Level of pressure (“Critical", ”High", “Medium", “Low") on the

foundations of a bridge are of crucial importance to maintenance authorities.

Traffic light states (“Red”, “Orange”, ”Green”)

The Sensor’s Task Event Detection Task Given an FSM representation of a system,

containing states S={s1..sn} generate an accurate time-series:

Coffee-Level Example – A More Complicated System

Application: detect when there is fresh coffee. Can be modeled as a finite state machine

Challenge: changes in view.

System Evolution – Change of CameraExample of Small Change of View

State A

State B

State C

State D

Timeline

System EvolutionTime

State A

State D

State D

State D

The Challenge: Recover from “System Evolution”

State A

State B

State C

State D

Timeline

System EvolutionTime

State A

State B

State D

State C

Semantics to the rescue

Semantics (one of the definitions) The meanings assigned to symbols and sets of symbols in a language.

If a computer understands the semantics of a system, it understands the meaning, rather than just interpreting a set of features.

The Proposed Solution

State A

State B

State C

State D

Timeline

System EvolutionTime

State A

State B

State D

State C

Semantic Model States of the FSM: Red, Orange, Green) State transition: {G->O,O->R,R->G} Semantics used for re-calibration: transition times between states should be in [avg-2std, avg+2std].

  Avg Std avg-2std avg+2std

G->O 14.75 6.185 2.380683123 27.11931688

O->R 3.25 0.957 1.335145784 5.164854216

R->G 15 4.761 5.478095429 24.52190457

Directory

Motivation – Self Managed Sensors

Event detection systems Task Challenge Our approach

Exploiting semantics in Event Detection Systems Algorithms Analysis Experiments and results

Information Flow – Regular Detection Process

Perform State Detection

User Specified Prediction

Parameters

Time Series State Data

Real TimeImage Input

Output SensingAuxiliary Services

Extract Features

Parameterized Prediction

State A

State B

State C

State D

…..…..…..…..ti=“Red”

Information Flow – Semantics Based Detection Process

Perform State Detection

User Specified Prediction

Parameters

Time Series State Data

Real TimeImage Input

Output SensingAuxiliary Services

Extract Features

Parameterized Prediction

State A

State B

State C

State D

Sliding Window of Images

…..…..ti=“Red”

Manage Semantic Module

System Evolution Occurred

Detect System

Evolution

No

Perform Recalibration

Yes

Auto GeneratedParameters

Semantic

State AState BState D State C

Parameter Recalibration

Search in the space of parameters for new parameters that optimize consistency of the stream generated with the model.

Forall p1,p2,..,pn do Use p1,p2,..,pn to generate a new stream based on features in the buffer Evaluate the previous stream against the semantic model return p1,p2,…,pn if consistent

DEMO: Searching for the correct field of viewUsing Semantics Only Search in the space of parameters for new parameters that

optimize consistency of the stream generated with the model.Forall p1,p2,..,pn do Use p1,p2,..,pn to generate a new stream based on features in the buffer Evaluate the previous stream against the semantic model return p1,p2,…,pn if consistent

Forall different fields of view do Extract features from the field of view from all images in buffer. Cluster features extracted to m clusters (m is the number of states). Forall possible m! state assignments to the different states do Use labeled centroids of clusters to generate a new state stream Evaluate the previous stream against the semantic model return p1,p2,…,pn if consistent

Detect System Evolution

Evaluate the consistency of the last M time series data elements generated using current prediction model.

The consistency value found in the previous step is compared to a predefined threshold.

Forall (M) State Transitions (si→sj) determined based on features in buffer do

avg ← get average transition time from model for (si→sj) stdv ← get standard deviation time from model for (si→sj)

If transition time for (si→sj) is not in [avg-2*stdv , avg+2*stdv] faults++

if faults/(number of State Transitions)>threshold declare “system evolution”

Probabilistic Analysis – Probability to avoid re-calibration under normal operation

(No System Evolution)

Probability of the observations collected in the WindowBuffer to have less than C=ThresholdConsistent percent of deviations observing a system with p = 4.4% probability to generate a transition outside the 2 stdv range.

Experimental Results - Coffee-Level Detection

Data Collection Two cameras: From June 11th to June 13th

Record an image when changes are detected anywhere in view 1174 images (60MB) were collected Number of actual transition transitions: 203

Bounding box adjusted correctly in both cameras when changed to make the prediction algorithm confuse two states

Empty↔Off

Empty↔Half-Full

Experimental Results - Coffee-Level Detection

After semantic based calibration, detection is much more accurate

Future Work

Multi sensor systems Incorporating better models for anomaly

detection Incorporating features in the process of re-

calibration

Questions/Comments?