Soft Sensor for Faulty Measurements Detection and

Reconstruction in Urban Traffic

Department of Adaptive systems, Institute of Information Theory and Automation, June 2010, Prague

Outline

Problem description Soft sensors Gaussian Process models Soft sensor for faulty measurement

detection and reconstruction Conclusions

Outline

Problem description Soft sensors Gaussian Process models Soft sensor for faulty measurement

detection and reconstruction Conclusions

Problem description

Traffic crossroad - count of vehicles Inductive loop is a popular choice Devastating for traffic control

system Failure detection and recovery of

sensor signal

Example of controlled network (Zličin shopping centre, Prague)

Sensors on crossroadsFailure: control system has no means to reactPossible solution: soft sensor for failure detection and signal reconstruction

Soft sensors

Models that provide estimation of another variable

`Soft sensor’: process engineering mainly

Applications in various engineering fields Model-driven, data-driven soft sensors Issues:

missing data, data outliers, drifting data, data co-linearity, different sampling rates, measurement delays.

Outline

Problem description Soft sensors Gaussian Process models Soft sensor for faulty measurement

detection and reconstruction Conclusions

Probabilistic (Bayes) nonparametric model.

GP model determined by: • Input/output data (data points, not

signals)

(learning data – identification data):• Covariance matrix:

GP model

Covariance function:• functional part and noise part• stationary/unstationary, periodic/nonperiodic,

etc.• Expreses prior knowledge about system

properties, • frequently: Gaussian covariance function

» smooth function» stationary function

Covariance function

Identification of GP model = optimisation of covariance function parameters • Cost function: maximum likelihood of

data for learning

Hyperparameters

GP model prediction

Prediction of the output based on similarity test input – training inputsOutput: normal distribution

•Predicted mean •Prediction variance

-2 +2

Static illustrative example

Static example: 9 learning points: Prediction

Rare data density increased variance (higher uncertainty).

-1.5 -1 -0.5 0 0.5 1 1.5 2-4

-2

0

2

4

6

8

xy

Nonlinear function to be modelled from learning points

y=f(x)

Learning points

-1.5 -1 -0.5 0 0.5 1 1.5 2-6

-4

-2

0

2

4

6

8

10

x

y

Nonlinear fuction and GP model

-1.5 -1 -0.5 0 0.5 1 1.5 20

2

4

6

x

e

Prediction error and double standard deviation of prediction

2|e|

Learning points

2f(x)

GP model attributes (vs. e.g. ANN) Smaller number of parameters Measure of confidence in prediction, depending on data Data smoothing Incorporation of prior knowledge * Easy to use (engineering practice)

Computational cost increases with amount of data Recent method, still in development Nonparametrical model

* (also possible in some other models)

Outline

Problem description Soft sensors Gaussian Process models Soft sensor for faulty measurement

detection and reconstruction Conclusions

The profile of vehicle arrival data

Modelling

One working day for estimation data Different working day for validation

data Validation based regressor selection the fourth order AR model (four delayed output values as regressors) Gaussian+constant covariance function Residuals of predictions with 3 band

50 100 150 200 250 300 350 400 450-10

-5

0

5

10

15

20

Samples

Que

ue le

ngth

3measurements

50 100 150 200 250 300 350 400 4500

5

10

Samples

e

3

|e|

-8 -6 -4 -2 0 2 4 6 80

20

40

60

80

100

120

Value of residual

Residuals distribution

Estimation

50 100 150 200 250 300 350 400 450-10

-5

0

5

10

15

20

Samples

Que

ue le

ngth

3measurements

50 100 150 200 250 300 350 400 4500

5

10

Samples

e

3

|e|

-8 -6 -4 -2 0 2 4 6 80

10

20

30

40

50

60

70

80

90

100

Value of residual

Residuals' distribution

Validation

Proposed algorithm for detecting irregularities and for reconstruction the data with prediction

Sensor fault: longer lasting outliers.

The comparison of MRSE for k-step-ahead predictions

Purposiveness of the obtained model(the measure of measurement validity, close-enough

prediction, fast calculation, model robustness)

0 5 10 15 20 250.46

0.48

0.5

0.52

0.54

0.56

0.58

0.6

0.62

Prediction steps

MR

SE

Soft sensor applied on faulty data

50 100 150 200 250 300 350 400 450-20

0

20

40

60

80

Samples

Que

ue le

ngth

3measurements

50 100 150 200 250 300 350 400 4500

5

10

15

Samples

Que

ue le

ngth

corr. measurements

Conclusions

Soft sensors: promising for FD and signal reconstruction.

GP models: excessive noise, outliers, no delay in prediction, measure of prediction confidence.

The excessive noise limits the possibility to develop better predictor.

Traffic sensor problem successfully solved for working days.