MES

Data Acquisition, Analysis

Why

Tracking & Tracing

When Execution

What

Resource

Who

Specification

How

Unrestricted © Siemens AG 2015. All rights reserved

Semantic-guided Feature Selection

for Industrial Automation Systems

M. Ringsquandl, S. Lamparter, S. Brandt, T. Hubauer, R. Lepratti

Page 2 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Summary

• Evaluation Results

• Production Cycle Time Forecasting

Use Case

• Linear Model-Embedded Feature Selection

• Semantic-guided Feature Selection

• Industrial Feature Ontology

Our Approach

• Data Access and Analytics

• Industrial Automation Systems

Introduction

Outline

Page 3 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Summary

• Evaluation Results

• Production Cycle Time Forecasting

Use Case

• Linear Model-Embedded Feature Selection

• Semantic-guided Feature Selection

• Industrial Feature Ontology

Our Approach

• Data Access and Analytics

• Industrial Automation Systems

Introduction

Outline

Page 4 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Industrial Automation Systems

Layered Architecture

Layer 1 – Field Device Layer

Production

Instruments

Identification

Systems

Drive

Systems

Power

Supplies

Field Devices

Electr. & Mech. Engineering

Knowledge

Page 5 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Industrial Automation Systems

Layered Architecture

Layer 2 – Control Layer

Real-time Control Industrial

Communication

Human-Machine

Interfaces

Switching

Technology

Control Layer

Field Devices

Electr. & Mech. Engineering

Knowledge

Control & Automation

Engineering Knowledge

Page 6 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Industrial Automation Systems

Layered Architecture

Layer 3 – Supervisory Layer

Control Layer

Field Devices

Engineering

Stations

Energy

Management

Asset

Management

Data Acquisition

Systems

Supervisory Layer

Electr. & Mech. Engineering

Knowledge

Control & Automation

Engineering Knowledge

IT-System Knowledge

Page 7 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Industrial Automation Systems

Layered Architecture

Layer 4 – Management Layer

Control Layer

Field Devices

Supervisory Layer

Management Layer

Operations

Management Plant Engineering

Production

Execution

Manufacturing

Intelligence

Electr. & Mech. Engineering

Knowledge

Control & Automation

Engineering Knowledge

IT-System Knowledge

Manufacturing Operations

Knowledge

Page 8 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

MES

Introduction

Industrial Automation Systems

Data Collection on Manufacturing Operations Layer

Manufacturing Operations Management

Quality Inventory

Maintenance Production

ERP

Observation

Motor Torque

Conveyor Motor

2015-03-01T12:31:00

Door Assembly

Torquemeter

1200

featu

reO

fIn

tere

st

observ

edP

rop

ert

y

Contextualize as

Unified Semantic Data Model

Thousands of Tags

and Events

Control Layer

Field Devices

Supervisory Layer

Page 9 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Summary

• Evaluation Results

• Production Cycle Time Forecasting

Use Case

• Linear Model-Embedded Feature Selection

• Semantic-guided Feature Selection

• Industrial Feature Ontology

Our Approach

• Data Access and Analytics

• Industrial Automation Systems

Introduction

Outline

Page 10 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Data Access and Analytics

Using Domain Knowledge

Going beyond Ontology-based Data Access (see [1])

Historic and

Real-time data

Data Access

Control Layer

Field Devices

Supervisory Layer

Management Layer

ETL

Analytics

OBDA

Do

ma

in K

no

wle

dg

e

Domain Knowledge

?

Page 11 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Data Access and Analytics

Application of Machine Learning Models

High-dimensional and Linked Data – Select optimal subset of features, cf. [2]

Manufacturing Operations Management

Quality Inventory

Maintenance Production

F

S

Fi

Model Feature

Selection

Model Fitting

Do we need

to check all

of them?

Page 12 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Summary

• Evaluation Results

• Production Cycle Time Forecasting

Use Case

• Linear Model-Embedded Feature Selection

• Semantic-guided Feature Selection

• Industrial Feature Ontology

Our Approach

• Data Access and Analytics

• Industrial Automation Systems

Introduction

Outline

Page 13 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Summary

• Evaluation Results

• Production Cycle Time Forecasting

Use Case

• Linear Model-Embedded Feature Selection

• Semantic-guided Feature Selection

• Industrial Feature Ontology

Our Approach

• Data Access and Analytics

• Industrial Automation Systems

Introduction

Outline

Page 14 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Approach

Industrial Feature Ontology

Extension of Semantic Sensor Network Ontology (see [3])

Do

ma

in K

no

wle

dg

e

Legacy

Model

Legacy

Model

Legacy

Model

Motor Temperature dependsOn

Motor Speed

Model dependencies

between data

Page 15 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Summary

• Evaluation Results

• Production Cycle Time Forecasting

Use Case

• Linear Model-Embedded Feature Selection

• Semantic-guided Feature Selection

• Industrial Feature Ontology

Our Approach

• Data Access and Analytics

• Industrial Automation Systems

Introduction

Outline

Page 16 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Approach

Semantic-guided Feature Selection

Feature Ontology reduces Feature Space without looking at actual data

Fe

atu

re O

nto

log

y

Legacy

Model

Legacy

Model

Legacy

Model

Response – Define

dependencies on the

variable we want to

predict

Page 17 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Approach

Semantic-guided Feature Selection

Role chain axioms propagating feature dependencies

Fe

atu

re O

nto

log

y

Legacy

Model

Legacy

Model

Legacy

Model

Page 18 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Approach

Semantic-guided Feature Selection

Conceptual definition of relevant features

Invoke reasoner before Data Access

Page 19 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Summary

• Evaluation Results

• Production Cycle Time Forecasting

Use Case

• Linear Model-Embedded Feature Selection

• Semantic-guided Feature Selection

• Industrial Feature Ontology

Our Approach

• Data Access and Analytics

• Industrial Automation Systems

Introduction

Outline

Page 20 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Approach

Embedded Model Feature Selection

Ontology as

RDF-Graph

• Lasso Regularization:

• Graph Kernel Lasso:

• Linear Model:

Calculate Graph Kernel Matrix

Based on sub-graphs

Augment Linear Model with a semantic regularization term (see [4])

„Semantic“

Bias

What if we still want a

sparse solution?

• Use Laplacian of Graph Kernel Matrix:

“Degree – Adjacency Matrix”

Page 21 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Summary

• Evaluation Results

• Production Cycle Time Forecasting

Use Case

• Linear Model-Embedded Feature Selection

• Semantic-guided Feature Selection

• Industrial Feature Ontology

Our Approach

• Data Access and Analytics

• Industrial Automation Systems

Introduction

Outline

Page 22 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Summary

• Evaluation Results

• Production Cycle Time Forecasting

Use Case

• Linear Model-Embedded Feature Selection

• Semantic-guided Feature Selection

• Industrial Feature Ontology

Our Approach

• Data Access and Analytics

• Industrial Automation Systems

Introduction

Outline

Page 23 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Cycle Time Forecasting

Outgoing

Goods Packaging Assembly Conveying Loading Quality Test

predict

Cycle Time

Use Linear Model to estimate time until product is finished

Feature

Selection

Model

Fitting

• Collect data from different

layers and processes

• Contextualize w.r.t.

product (cycle time)

Fe

atu

re O

nto

log

y

Page 24 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Summary

• Evaluation Results

• Production Cycle Time Forecasting

Use Case

• Linear Model-Embedded Feature Selection

• Semantic-guided Feature Selection

• Industrial Feature Ontology

Our Approach

• Data Access and Analytics

• Industrial Automation Systems

Introduction

Outline

Page 25 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Evaluation Results

Semantic Feature Selection

Feature selection performance

• Technomatix Plant Simulation Data Set:

47

29

18

0

10

20

30

40

50

No Feature Selection Semantic Feature Selection

P-value based Selection

Features

0.08 0.06

0.00

0.10

0.20

No Feature Selection Semantic Feature Selection

P-value based Selection

1.36E+11

1000

1E+11

2E+11

Normalized Model Error

Product Type Conveyor Speed Control Alarms … Cycle Time

47 Features

20

00

In

sta

nce

s

Page 26 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Evaluation Results

Embedded Feature Selection

22.9

42.9 42.8 43.8

0

10

20

30

40

50

Lasso ElsaticNet Graph Lasso Graph Kernel Lasso

Features

0.48 0.46

0.54

0.43

0.00

0.10

0.20

0.30

0.40

0.50

0.60

Lasso ElasticNet Graph Lasso Graph Kernel Lasso

Normalized Model Error

Embedded Feature Selection and Model Performance

• Small sample size n=40

• Results based on 10-Fold Cross-Validation

Product Type Conveyor Speed Control Alarms … Cycle Time

47 Features

40

In

sta

nce

s

Page 27 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Summary

• Evaluation Results

• Production Cycle Time Forecasting

Use Case

• Linear Model-Embedded Feature Selection

• Semantic-guided Feature Selection

• Industrial Feature Ontology

Our Approach

• Data Access and Analytics

• Industrial Automation Systems

Introduction

Outline

Page 28 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Summary

Major Takeaways

Semantic Feature Selection

• Domain knowledge from legacy models allows us to capture known dependencies between variables

• We can perform feature selection via semantic reasoning – without looking at the data

It gives competitive results

It reduces number crunching efforts

Embedded Model Feature Selection

• Extended graph regularization leverages from known dependencies

They introduce a “semantic bias” to learning of hypothesis

Can help to boost performance for small data sets

Page 29 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Outlook

Possibilities for Future Work

Ontology-Based Data Access

• Integrate Feature Selection directly into Query Answering

Additional Sources of Domain Knowledge

• Extract further dependencies from Product-Lifecycle and Engineering Systems

Evaluate on real-life plant data

• Apply techniques on real-life large-scale automation systems

Page 30 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved

Literature

[1] M. Rodríguez-Muro, R. Kontchakov, and M. Zakharyaschev, “Ontology-based data access: Ontop of

databases,” in Proc. of the 12th Int. Sem. Web Conf., 2013.

[2] J. Tang, S. Alelyani, and H. Liu, “Feature Selection for Classification: A Review. 2013”

[3] M. Compton, P. Barnaghi, L. Bermudez, R. García-Castro, O. Corcho, S. Cox, J. Graybeal, M. Hauswirth, C.

Henson, A. Herzog, V. Huang, K. Janowicz, W. D. Kelsey, D. Le Phuoc, L. Lefort, M. Leggieri, H. Neuhaus, A.

Nikolov, K. Page, A. Passant, A. Sheth, and K. Taylor, “The SSN ontology of the W3C semantic sensor network

incubator group,” Web Semant. Sci. Serv. Agents World Wide Web, vol. 17, pp. 25–32, Dec. 2012.

[4] C. Li and H. Li, “Network-constrained regularization and variable selection for analysis of genomic data,”

Bioinformatics, vol. 24, no. 9, pp. 1175–1182, 2008.