1

Stepping Towards the

Industrial 6th Sense Department of Chemical & Process Engineering

Department of Electrical and Electronics Engineering

Frontline Researchers (alpha.):

Dr M. Abdelwahab, Dr B. Dorneanu, Dr R. Hang, Mr W. Lee, Dr A.

Mohamed and Dr F. Yang

2018 Report

2

Table of Contents

Work package 1: Towards holistic system models .................................. 4

The industrial 6th sense concept ............................................................................................... 5

Towards holistic system model ................................................................................................. 6

Task 1.1 Plant-wide model development .............................................................................. 6

Case study 1: CPE mini plant ................................................................................................... 6

Dynamic model ..................................................................................................................... 7

Reduced model..................................................................................................................... 7

Task 3.2 Autonomous operations & decision making .......................................................... 7

Case study 2: Towards cooperative control ......................................................................... 8

Future work .................................................................................................................................. 9

Deliverables ................................................................................................................................ 9

References ................................................................................................................................. 10

Work package 2: Resilient network communication and data transfer

towards massive connectivity ...............................................................12

Project Objectives for WP2 ................................................................................................... 13

Research ........................................................................................................................... 13

Current Progress ............................................................................................................. 13

Future Deliverables and Timeline ............................................................................... 15

Resilient Communication Network and Data Transfer ...................................................... 16

Research ........................................................................................................................... 17

Current Progress ............................................................................................................. 17

Future Deliverables and Timeline ............................................................................... 20

Work package 3: Towards autonomous operation and data management

of heterogeneous, concurrent 6S networks ............................................22

Introduction ................................................................................................................................ 23

Targets ....................................................................................................................................... 23

3

Progress ...................................................................................................................................... 23

Deliverables .............................................................................................................................. 24

Work package 4: Towards the industrial virtual plants .........................26

Overview of Virtual Reality ................................................................................................... 27

Virtual Reality for Chemical Engineering ............................................................................ 27

Project Aims ............................................................................................................................... 28

Current Progress ....................................................................................................................... 28

Project Deliverables and Estimated Time ............................................................................ 31

Research Novelties and Publication Estimation .................................................................. 31

Digital-twin project objectives ............................................................................................... 33

Digital-twin Research Plan...................................................................................................... 33

4

Tab

1

Tab

1

The industrial 6th sense concept

Towards holistic system model

Task 1.1 Plant-wide model development

Case study 1: CPE mini plant

Dynamic model

Reduced model

Task 3.2 Autonomous operations & decision making

Case study 2: Towards cooperative control

Future work

References

Work package 1: Towards holistic system models

5

• The Industrial 6th Sense Concept

We, human beings acquire information from our surroundings through our sensory

receptors of vision, sound, smell, touch and taste – the five senses. The sensory stimulus

is converted to electrical signals as nerve impulse data communicated with our brain.

What is intriguing is the communication network. When one or more senses fail

(impairment), we can re-establish communication and improve our other senses to

protect us from incoming dangers. Furthermore, we have developed the mechanism of

“reasoning”, effectively analysing the present data and generating a vision of the

future, which we might call our 6th Sense (6S).

Is it possible to develop a 6S technology to predict a catastrophic disaster? Industrial

processes are already equipped with five senses:

➢ “Hearing” from acoustic sensors

➢ “Smelling” from gas and liquid sensors

➢ “Seeing” from cameras

➢ “Touching” from vibration sensors

➢ “Tasting” from composition monitors

The 6S can be achieved by forming a network which is self-adaptive and self-

repairing, carrying out deep-thinking analysis with even limited data, and predicting

the sequence of events via integrated system modelling.

Figure 1: Work packages of the 6th Sense project

Application:Planning, Scheduling, (Online ) Control,

Parameter estimation

De

cisi

on

mak

ing

syst

em

Knowledge

Co

mm

un

icat

ion

net

wo

rk

Process

Sensor network

WP2 WP3

Expertknowledge

Libraryof models

Behaviourdata

WP4

Virtual plant

Application:Control, Maintenance, Planning & Scheduling

6

This project (Figure 1) is the first step towards developing a 6S technology for

industrial processes by bringing together research expertise in process systems

engineering, wireless communication network, robotic and autonomous systems. The 6S

technology developed in this project could be further explored to a wide range of

industrial and manufacturing processes.

• Towards holistic system models – WP1

Development of a systematic model- and simulation based decision-making

framework enabling connectivity of systems based on multi-criteria analysis and

selection of operational, design and safety options while ensuring smooth running of

large and complex infrastructures.

• Task 1.1 Plant-wide model development

To achieve the goal of a plant-wide simulator that will be implemented in

collaboration with our industrial partners, a multilevel and multiscale modelling

approach using first-principles models in interaction with phenomenological and

empirical models at different length and time scales.

➢ Case study 1: CPE Mini plant

The mini plant (Figure 2) produces sodium ion solution as sodium chloride (saline)

solution for sale to fine chemical, pharmaceutical, and food industry. The raw material

is sodium chloride (NaCl) contaminated with calcium chloride (CaCl2), which is pre-

mixed with a stoichiometrically calculated amount of sodium bicarbonate (NaHCO3).

This feedstock is fed into a reactor vessel charged with pre-heated water and the

reaction temperature is maintained at 65 0C.

7

Figure 2: Simplified structure of the CPE mini plant

A dynamic model is developed for the mini plant using simulation software. To reduce

the computational effort required for computationally intensive topics, a structure-

retaining model reduction (Dorneanu, 2009) is applied to the full dynamic model.

.1. Dynamic model

A dynamic model for the simplified process presented in Figure 2 is developed using

Aspen Plus® Dynamics (Dorneanu et al, 2018). The reactor and the CO2 adsorption

column are modelled as dynamic, while all the other units in the process are assumed

to be instantaneous.

.2. Reduced model

The dynamic model is decomposed into models of individual units by cutting streams

of material. These models are then linearized, and a balanced model-order reduction

is applied to obtain their reduced models.

The linearization leads to a model with 9 states, which is reduced using Matlab® to a

model with 6 states using model-order reduction via balanced realization (Dorneanu

et al, 2018).

Filter

Filter

Reactor

Absorption column

NaCl (aq)

NaCl (aq)

NaCl (aq)

CaCO3 (s)

CaCO3 (s)

CaCO3 (s)

CaCO3 (s)

CO2 (g) CO2 (aq)NaHCO3 (s)

dosing agent

H2O (l)

NaCl (s) & CaCl2 (s)

8

Figure 3: Response of the reactor level to disturbance on the flowrate of NaHCO3

(Dorneanu et al, 2018)

The agreement between the full dynamic model and the linear model, and between

the linear model and the reduced model are presented in Figure 3. The graphs show

the changes in the reactor level when a change is applied on the NaHCO3 feed

flowrate. The results are presented as variation from the steady state values.

• Task 3.2. Autonomous operations & decision making

Robust control mechanisms are imperative for the envisioned system envisioned in this

project. Given its complexity, hierarchical subsumption-based control characteristics

are investigated. A major novelty is that the overall control architecture will remain

distributed, rather than centralized.

➢ Case study 2: Towards cooperative control

Modern large-scale industrial plants are complex dynamical systems which consist of

highly interconnected and interdependent elements. The control of such systems is

usually achieved via hierarchical multi-layered structures benefitting from their

inherent properties such as modularity, scalability, adaptability, flexibility, and

robustness (Stanković et al, 2008). However, with the rapid development of Industry

4.0, there are continuously new requirements for controllers to meet. In the control

layer, the controllers should automatically adapt to the changed systems structure,

process parameters, and production planning with high efficiency and good quality

(Dorneanu et al, 2018). Thus, highly flexible dynamic optimal control with high

performance is required.

The key idea of cooperative control (Figure 4) is that a team of agents (or dynamic

systems) must be able to respond to unanticipated changes in the environment by

collaborating towards a common objective without the use of a centralized coordinator

or optimizer (Thorpe, 2018).

-6

-4

-2

0

2

4

6

0 10 20 30 40 50

Mas

s fl

ow

/ [%

]

Time / [h]

Stream NaHCO3

Nonlinear Model Aspen Linear Model Aspen

-10

-5

0

5

0 10 20 30 40 50

Leve

l / [

%]

Time / [h]

Level

Nonlinear Model Aspen Linear Model Aspen

-10

-8

-6

-4

-2

0

2

0 10 20 30 40 50

Leve

l / [

%]

Time / [h]

Level

Linear Model

Reduced Model Truncation 6 states

9

Figure 4: Cooperative control strategy

Using the reduced models developed in the previous section, a cooperative MPC

strategy is applied to control the reactor level (Figure 5).

Figure 5: Cooperative MPC results (Dorneanu et al, 2018)

Process

Sensor network

Control

U1 U3

U2

Un…

S1m

…

S1m

S12

S11

S1S1m

…

S2m

S22

S21

S2S1m

…

S3m

S32

S31

S3S1m

…

Snm

Sn2

Sn1

Sn

C1p…C11 C12

C1

C2p…C21 C22

C2

C3p…C31 C32

C3

Cnp…Cn1 Cn2

Cn

…

…

MPC1 MPC2 MPC3 MPCn

Message broker

…

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

0 5 10 15 20 25

Mas

s fl

ow

/ [

%]

Time / [h]

Adsorption column outlet stream

-0.5

0

0.5

1

1.5

2

2.5

3

3.5

0 5 10 15 20 25

Leve

l / [

%]

Time / [h]

Reactor

-2

0

2

4

6

8

10

0 5 10 15 20 25

Leve

l / [

%]

Time / [h]

Adsorption column

-6

-4

-2

0

2

4

6

0 5 10 15 20 25

Mas

s fl

ow

/ [%

]

Time / [h]

Stream NaHCO3

Sampling instant Sampling instant

Sampling instant Sampling instant

Reactor outlet Reactor

Column outlet Column

Mas

s fl

ow

[%

]M

ass

flo

w [

%]

Leve

l [%

]Le

vel [

%]

10

• Future work

➢ Task 1 Plant-wide model development and optimisation (Q2 2019)

While digital technologies enable massive productivity jumps and driving down costs,

there are still no approaches to model the way the communication and exchange of

information is integrated. Thus, a framework to model the resulting cyber-physical

system will be developed.

➢ Task 2 Fault propagation, detection and analysis (Q3 2019)

Improved quality and reliability of data obtained from sensors in WP2 and WP3 is

used here to detect and analyse fault propagation. Predictive analytics and machine

learning will be used to exploit real-time data to distinguish normal and abnormal

behaviour (Gupta, 2018). Mismatches with the model predictions will be used to

anticipate failures.

➢ Task 3 Autonomous operations & decision making – Experimental validation of the control strategy using the four-tank process (Q4 2019)

The cooperative control framework developed for the mini plant in the previous section

of this report will be validated experimentally using the four-tank process, a well-

known benchmark for assessment of control strategies. This will be done with

collaboration partners.

• Deliverables

.1. Abdelwahab, M., Dorneanu, B., Arellano-Garcia, H., Gao, Y., AIM 2019 Conference, Hong Kong, 8-12 July 2019, in preparation

.2. Dorneanu, B., Abdelwahab, M., Gao, Y., Arellano-Garcia, H., Smart monitoring systems for chemical engineering applications, Connected Everything 2019, Nottingham, 25-26 June 2019, in preparation

.3. Dorneanu, B., Abdelwahab, M., Gao, Y., Arellano-Garcia, H., Intelligent monitoring and control for chemical plants, Industry 4.0 Conference, Manchester, United Kingdom, 10-11 April 2019, in preparation

.4. Dorneanu, B., Abdelwahab, M., Ruan, H., Xiao, P., Gao, Y., Arellano-Garcia, H., Fault detection for chemical engineering, Industry 4.0 Conference, Manchester, 10-11 April 2019, in preparation

.5. Yentumi, R., Dorneanu, B., Arellano-Garcia, H., Modelling of a natural gas fired natural heater, ESCAPE 29, Eindhoven, The Netherlands, 16-19 June 2019

11

.6. Dorneanu, B., Arellano-Garcia, H., 2018, Towards cooperative-based control of chemical plants, Computer-Aided Chemical Engineering 43, 1087-1092

.7. Dorneanu, B., Gu, S., Arellano-Garcia, H., 2018, Digitalization of manufacturing, Connected Everything Conference 2018

.8. Dorneanu, B., Yentumi, R., Arellano-Garcia, 2019, Modelling and optimal operation of a natural gas fired natural draft heater, Industrial and Engineering Chemistry Research, submitted

.9. Dorneanu, B., Abdelwahab, M., Mohamed, A., Ruan, H., Yang, F., Gu, S., Gao, Y., Xiao, P., Arellano-Garcia, H., 2019, Review paper on the Industrial sixth sense, in preparation

.10. Arellano-Garcia, H., Barz, T., Dorneanu, B., Vassiliadis, V.S., 2019, Real-time feasibility of nonlinear model predictive control for semi-batch reactors subject to uncertainty and disturbances, in preparation

• References

Dorneanu, B., et al., 2009, Comp & Chem Eng 33, 699-711

Dorneanu, B., et al., 2018, CACE 43, 1087-1092

Gupta, A., 2018, CEP 114, 22-29

Stanković et al, 2008, Proc 47 IEEE CDC, 4364-4369

Thorpe, 2018, RDP2018-41994085.1, 1-4

12

Tab

2

Tab

2

Work package 2: Resilient network communication and data transfer

towards massive connectivity

13

• Project Objectives for WP2

The aim of this package is to develop a new type of intelligent data communication

system by integrating a wireless sensor network (WSN) model with a machine learning

(ML) model(s). The physical layout and topology of the WSNs model will be

depending on the physical distributions of the control system model, to be specific,

wireless sensors can be equipped to each type of chemical process elements or units

whose locations are strictly fixed inside of the pilot plant. And the machine learning

model is mainly depending on the tasks of the control system as well as the deployment

scale of the WSNs. This type of mixed model configuration will enable its capacities

of being self-adaptive to dynamic and extreme environment; self-protection when the

network security is violated; self-repairing when one or more of the sensors are even

damaged; able to carry out real-time analysis from measured and received data and

predict anomalies and future events.

• Research

The WSN model can transit between two sub-models, i.e. a static WSN model (without

a robot) and a mobile WSN model (with a robot).

➢ The static WSN model will use the star topology to enable communications between the central controller and the wireless sensors rather instead of making the sensors communicate with each other directly. This is also beneficial for implementing the hash-based NOMA 5G technique for the WSNs.

➢ The mobile WSN model is used whenever the robot is interacting with the WSN, which is also beneficial for monitoring and surveillance applications. In this case the overall WSN topology is varying constantly.

➢ The ML model(s) – deep neural networks (DNNs), neutral networks that can deal with a large range of features without much data preprocessing, perform high level end-to-end solution in large-scale WSNs scenarios and big data challenge. Capable of performing fault detections and predictions as well as sensor self-healings. Comparably, very little research work has been done on using the state-of-art deep learning approach for those applications or tackling with fault predictions more than just fault detections. Limitations: data consuming, training time consuming, hardware requirements etc.

14

• Current Progress

➢ An attempt to apply for equipment purchase funding from Capital Award Competition in August 2018.

➢ A purchase request for a new computer has been submitted to the ICS IT department by the end of Sep 2018. Due to various reasons the old quote was cancelled by Dell and we just submitted a new quote by October 2018.

➢ The data we already have: collected from 4 days (21st – 24th March) in 2017 and 13 days (19th – 23rd Feb 5th – 8th and 19th – 22nd March) in 2018. The data of day 1 (21st March 2017) as an example, has been studied with regards to both fault detections and predictions.

➢ Fault detections: two-stage detection method

Stage 1: Unsupervised learning to separate the data which are likely to be faulty, so the system knows what faulty data are like => K-mean clustering. (Fig. 2-1)

Stage 2: Conventional supervised learning methods to detect faults based on training on the clustered data and testing on the new data. Seven different conventional supervised learning methods are tested for this task, based on the data collected from 22nd to 24th March of 2017. (Fig. 2-2)

Fig. 2-1 Fig. 2-2

➢ Fault predictions => time-series analysis combined with recurrent neural networks (RNNs) - Long-short term memory (LSTM) deep neural network. To predict faults before they happened to avoid damages and losses.

15

classifier detection accuracy

logistic regression (Matlab) 97.5%

KNN (Matlab) 100.0%

DT (Matlab) 99.4%

linear discriminant analysis (Matlab) 95.4%

MLP with one hidden layer (-10-) (Matlab) 99.0%

linear support vector machines (SVM) (Matlab) 99.5%

DNN classifier with 3 hidden layers (-10-20-10-)

(Python)

99.9971%

Table. 2-1

• Future Deliverables and Timeline

➢ Get the new computer and set up the necessary software to perform ML for larger amount of data (most likely will be artificial data depending on the availability of the chemical plant). ---- estimated by Feb 2019

➢ When applying classification to fault detections, if we want to know where exactly the fault happened: more variables and higher complexity; multiclass multi-label classification. ---- estimated by March 2019

➢ More model/hyperparameters tuning for the proposed LSTM DNN and results on different performance metrics. More than one-time step prediction and performance evaluations. ---- estimated by May 2019

➢ Other functions if possible (e.g. self-healing after actual damage) can be investigated, this can be casted as a multivariate regression problem. ----estimated by July 2019

➢ ML integration with the WSN network. ----estimated by August 2019

➢ Final integration of both the ML and WSN model with the control system model. ----estimated by September 2019

16

Resilient Communication Network and Data Transfer

• Research

Wireless sensors are considered as one of the key enablers in industry and process

automation, more specifically in intelligent chemical/process plants. Unlike

conventional wired sensors, wireless sensors can be deployed in hazardous areas of

the plant in a plug-and-play mode and they reduce deployment and maintenance

overhead. It is envisioned that many sensors will be deployed in large-scale plants to

provide reliable, timely and up-to-date measurements, thus enabling fast and

accurate decisions by the control system. Some of these sensors may provide periodic

measurements with different inter-measurement times, i.e., traffic inter-arrival times,

while others will be configured to operate on an event-basis, for instance a

temperature sensor can be configured to transmit measurements when a certain

condition is met and suspends transmission otherwise. In such scenarios, reliable network

access, massive connectivity support and timely scheduling will become critical

considerations.

The legacy wireless systems have been designed primarily for human initiated mobile

broadband communications. They are highly suboptimal for latency-critical, narrow

band, short-bust, sporadic traffic (e.g., sensor measurement data, such as temperature,

pressure, humidity, etc.) generated by sensors. Consequently, a new design paradigm

is needed to support large numbers of heterogeneous sensing devices with diverse

requirements and unique traffic characteristics. Compared to the sensors in traditional

internet-of-things (IoT) networks, those deployed in the extreme environments need to

operate in harsh (sometimes hazardous) conditions, thus are prone to wear and tear,

and cannot be easily replaced, posing major challenges in designing resilient wireless

networks for reliable communications.

Recently, the first release of the fifth-generation (5G) cellular system has been

standardised, and key use cases for narrow-band communications is identified for

17

ultra-reliable and low latency communications (URLLC) and massive machine-type

communications (mMTC). The former is characterised by extremely low end-to-end

transmission latencies (1 ms user plane latency1 and 20 ms control plane latency2 [1],

[2]) and high reliability figures e.g., 99.999% success rate within 1 ms and 0 ms

mobility interruption time, while the latter, i.e., mMTC, supports extremely high

connection densities up to 1,000,000 devices per km2 [1], [2] with typically low data

rates. Fig. 2-3 maps the main parameters of each 5G use case.

Fig. 2-3: 5G use cases [3]

It can be noticed that low latency and high connection density are the main

requirements for the scenarios under consideration in the Stepping Towards the

Industrial 6th Sense project. For instance, the fault detection and prediction algorithm

require fast transmission of measurement data to provide timely decisions. In addition,

it is envisioned that large number of sensors will be deployed within a plant, and the

same network can support multiple plants/factories. In other words, both the mMTC

and the URLLC system characteristics should be taken into account for the scenarios

under investigation.

1 User plane latency is the contribution of the radio network to the time from when the source sends a packet to when the destination

receives it. 2 Control plane latency refers to the transition time from a most “battery efficient” state (e.g. Idle state) to the start of continuous data

transfer (e.g. Active state).

18

• Current Progress

We consider a centralised control mechanism where the sensors are connected to a

fusion node via wireless links. The latter can also be used to send commands to

actuators within the plant as can be seen in Fig. 2-4. Consequently, the measurements

are transmitted in the uplink to the control unit with the commands being transmitted in

the downlink. The network consists of a heterogenous set of periodic and event-

triggered sensors with mixed requirements, characteristics and traffic models. A low

data rate is considered for all sensors. In addition, most sensors are assumed to be

static, i.e., deployed in fixed locations. Since humanoid robots and autonomous robots

are considered as an element of the control and measurement system, we assume that

a small number of sensors are moving with a low speed, e.g., ≤ 3 km/hr.

Fig. 2-4: Communication network and transmission directions

Considering heterogeneity of the plant and the associated sensors, a statistical model

rather than a deterministic model is chosen for the sensor transmission events. The

number of incoming packets (or events when each event generates a single packet)

per unit of time follows the Poisson distribution while the packet interval is modelled

19

as an exponential distribution. This results in probability-based transmissions that can

be controlled by the arrival rate and the inter-arrival time. Fig. 2-5 shows an example

of the probability-based transmission.

(a) (b) (c)

Fig. 2-5: Example of sensor transmission probability for (a) high periodicity sensor or

high probability events. (b) medium periodicity sensor or medium probability event.

(c) low periodicity sensor or low probability events.

Given this traffic model coupled with the network model in Fig. 2-4, work package

project will propose a low latency network access scheme for wireless sensors networks

in intelligent chemical/process plants. The proposed scheme considers the 5G frame

structure and allows massive number of sensors to access the network simultaneously

without collisions to request resources for measurement transmission. A low overhead

and low latency approach will be developed, and simulation along with theoretical

evaluations will be used to show gains of the proposed over conventional access

schemes.

Once the measurement data is received at the network side, it will be routed to the

control unit for further processing. This data will be used for fault detection and

prediction, as well as to optimise the plant operating parameters as can be seen in

Fig. 2-6. Decisions of the control unit will be routed to the 5G base station (known as

g-NodeB) and then sent to the wireless actuators.

20

Fig. 2-6: High level overview of the overall system model

• Future Deliverables and Timeline

• Research Plan

➢ Validating the traffic/event model with measurements from the pilot plant.

➢ Incorporating the 5G Industrial IoT channel model.

➢ Developing a low access delay scheme with redundancies for reliability.

➢ Investigating both orthogonal multiple access (OMA) for stand-alone mini-plants, and non-orthogonal multiple access (NOMA) for real-world deployments with a network covering multiple plant.

➢ Aligning the requirements and the models with the 5G URLLC and mMTC use cases with slices for stand-alone and ad-hoc deployments.

Deliverables:

➢ Abdelrahim Mohamed, Hang Ruan, Pei Xiao, Rahim Tafazolli, “The Role of 5G in Stepping Towards the Industrial Sixth Sense,” in IEEE Communications Magazine, to be submitted by June 2019.

21

➢ Low access latency support for massive sensors and actuators in intelligent chemical plants, in IEEE International Conference on Communications, to be submitted by October 2019.

➢ Bogdan Dorneanu, Hang Ruan, Abdelrahim Mohamed, Mohamed Heshmat, Yang Gao, Pei Xiao, Harvey Arellano-Garcia, “Fault Detection, Prediction, and Self-Healing in Chemical Processes Over Wireless Networks: A Step Towards the Industrial Sixth Sense,” under review in Industry 4.0 Conference, January 2019.

➢ Bogdan Dorneanu, Mohamed Heshmat, Abdelrahim Mohamed, Hang Ruan, Fengwei Yang, Sai Gu, Yang Gao, Pei Xiao, Harvey Arellano-Garcia, 2019,

Review paper on the Industrial sixth sense, in preparation.

➢ Abdelrahim Mohamed, Muhammad Imran, Pei Xiao, and Rahim Tafazolli, "Memory-full context-aware predictive mobility management in dual connectivity 5G networks," in IEEE Access Journal, vol. 6, 2018, pp. 9655-9666.

➢ Hybrid mMTC and URLLC Slice for Intelligent Industrial Plants in the 5G Era, in IEEE Transactions on Wireless Communications, to be submitted by December 2019.

➢ Industrial Sixth Sense: Enablers and Schemes, in IEEE Transactions on Industrial Informatics, to be submitted by June 2020

➢ 3GPP 5G standard contribution, by 2020.

➢ In parallel, ad-hoc papers with interim results will be prepared throughout the project period.

References

[1] International Telecommunication Union, “Report ITU-R M.2410-0: Minimum

requirements related to technical performance for IMT-2020 radio interface(s),” Tech.

Rep. November 2017.

[2] Nokia, “Self-Evaluation: URLLC and mMTC evaluation results,” in 3GPP RAN

Workshop on 3GPP submission towards IMT-2020, Brussels, Belgium, October 2018.

[3] 3GPP, “Industry Vision and Schedule for the New Radio Part of the Next

Generation Radio Technology,” in 3GPP RAN workshop on 5G, Phoenix, USA,

September 2015.

22

Tab

3

Tab

3

Introduction

Targets

Progress

Deliverables

Work package 3: Towards autonomous operation and data management

of heterogeneous, concurrent 6S networks

23

• Introduction:

As modern chemical processes evolve into extremely large and complex systems. The

structural scale and the dynamic complexity make it challenging for operators to infer

the conditions in the plant quickly and make timely decisions, especially during

abnormal situations. We need that technology that could help in preventing human

error and stop chain reactions that can transform small incidents into catastrophic

failure. The sixth sense technology aims to analysis the present data and generating

a vision of the future. This could be achieved by forming an integrated system

modelling, self-adaptive and self-repairing sensing network, and autonomous

software architecture that exhibit rapid information selection, scene understanding

and decision-making capabilities.

• Targets:

➢ Design of knowledge representation software that integrate the normal chemical process engineering knowledge and our chemical plant design and operational models.

➢ Design of autonomous software architectures that exhibit rapid information selection, scene understanding and decision-making capabilities.

➢ Development of the inspector robot that could build spatial-temporal model of the working environment (anomaly detection) or execute some inspection tasks defined by the main system.

➢ Development of IoT wireless sensors board for the environment and gas measurements which is compatible with the ROS framework (the robot software).

➢ Exploring other sensors like thermal or gas leakage detector camera for better inspection tasks (within ROS framework).

➢ Setting up a communication channel between the inspector robot and the VR model, where the robot provides camera feed, sensors reading and robot location information.

• Progress:

➢ Defining the structure of the knowledge representation and the autonomous decision-making software.

24

➢ Equipment of the inspector robot with the required sensors for the navigation and the visual inspection tasks (the Pioneer-3at robot equipped with a 2d SICK laser range finder and RealSense RGBD camera).

➢ Development of the inspector robot ROS packages: robot description, sensors description, sensors node and customized robust navigation software.

➢ Development of IoT wireless sensors board for the environment and gas measurements that is compatible with the ROS framework.

➢ Progress has been made for setting up the communication channel between the inspector robot and the VR model.

The inspector mobile robot The wireless gas sensors board

• Deliverables:

➢ Testing the software structure and the communication with the other work packages and so we could develop a working demo. (April 2019, publication, demonstration)

➢ Finalizing the communication channel between the inspector robot and the VR model. (May 2019)

➢ Making a controlled experiment for gas leakage detection and environment measurement record using IoT wireless sensors board. (June 2019, publication, demonstration)

➢ Exploring other sensors for inspection and preparing the required software to be compatible with the robot software. (July 2019)

➢ Scheduling a frequent inspector robot patrolling around the chemical plant and building a model for the working environment using the available sensors. (August 2019, publication, demonstration)

25

➢ Working with the knowledge representation software, customizing it for our objectives and integrating it with the chemical plant design and operation models. (January 2020, publication)

➢ Development of the decision-making system and the connection with knowledge representation software. (May 2020, publication)

➢ Integrating the full Sixth Sense system and the communication between the work packages and making practical testing. (October 2020, demonstration)

26

Tab

4

Tab

4

Overview of Virtual Reality

Virtual Reality for Chemical Engineering

Project Aims

Current Progress

Project Deliverables and Estimated Time

Research Novelties and Publication Estimation

Digital-twin project objectives

Digital-twin Research Plan

Work package 4: Towards the industrial virtual plants

27

• Overview of Virtual Reality

Virtual Reality (VR) is a technology for immersive simulation of a digital world in which

the user may immerse, visualise and potentially interact with the objects of their virtual

presence in that 3D world. Amongst all our senses, vision is arguably the dominant one,

and visual access (e.g. via stereoscope) to a digital world is the fundamental building

block of any successful modern VR design [1]. The concept of stereoscopy can be

traced back to C. Wheatstone and D. Brewester around the 1840s. However, it is only

around the end of last century, our technology becomes sufficiently advanced to

display a VR using computer rendering [2] [3].

VR in engineering has a long trail. Flight simulators have been used for decades to

train pilots for both commercial and military aviation. Arguably its digital world re-

creation is via computer screens in a closed booth which resembles cockpits rather than

stereoscopes; the concept is the same.

VR recently made an entrance to the architecture and construction industry [4]. One of

the big VR-stereoscope manufacturers, namely HTC, has announced its plan for a

hardware design which aligns to industry standards (e.g. with AECOM) and produces

specifically to the construction industry. A major part of the application will be training

and teaching [5].

• Virtual Reality for Chemical Engineering

The similarities between flight simulators and construction training are: the work

involved is complex; traditional teaching methods (e.g. reading materials and

lecturing) are not sufficient; a large degree of skills and knowledge are associated

with engineering hardware; the real application bears potential fatal danger, and

actual facilities are not easily accessible for training purposes due to various financial,

health and safety reasons.

When one considers these similarities, one will conclude that chemical engineering will

hugely benefit from the use of VR. Considering the current literature such as [1] which

associates virtual learning with processing engineering, we are still in the infancy of

designing and developing VR techniques for chemical engineering characteristics.

28

• Project Aims

This part of the project is to design and develop

➢ A complete 3D digital model resembles the Surrey Pilot Plant, including a complete list of instruments and a piping network.

➢ The 3D model (from the previous step) into VR via HTC Vive and to be available to other platforms such as smartphones, web browsers and portable devices (e.g. iPad).

➢ Full animation of flow dynamics within the piping network as well as a visualisation in main instruments such as the reactor.

➢ A full chemical model using commercial software (e.g. Aspen) that can calculate the chemical engineering dynamics from the Surrey Pilot Plant. This stage is related to Work package 1.

➢ A two-way connection between modelling software (e.g. Aspen) and the 3D model and a dynamic way to display the modelling results in VR graphically (so-called digital twin).

➢ A two-way connection between a digital control system and the 3D model, so interactions via VR or traditional mouse-keyboard from other platforms to potentially enable remote control of the actual Surrey Pilot Plant. This is

related to Work package 2, where an efficient network (i.e. 5G) is needed for massive sensor communication.

➢ Two-Way communication between a robot and the 3D model, where the robot provides camera feed, sensor reading and interaction from the 3D model (via VR or otherwise) can command the manoeuvre of the robot. This is related to Work package 3.

• Current Progress

➢ A 3D model has been developed from scratch using an open source software Blender. See Figure 1. Although there is an existing model made by one of the PhD students in the department, his polygon count exceeds the tolerant level even for a top-spec PC from the current market. Hence the rework and this also enables me to investigate and improve the graphics effects of 3D models.

29

➢ This model has been successfully migrated to web browsers via a technique called WebGL. This enables the user to remotely access the model as a

Figure 1: Left is a photo of the Surrey Pilot Plant and right is my 3D model.

Figure 2: The view of my 3D model within Firefox web browser.

30

demonstration and introduction to our Pilot Plant. See Figure 2. The URL of this website is http://industry-sixth-sense.eps.surrey.ac.uk/

➢ This model has also been migrated to portable devices such as iPads. See Figure 3.

➢ An animation technique, namely Shuriken Particle System, offered by Unity has been applied to demonstrate the water/solids/air flows inside the piping network, that is controlled by various valves and instruments.

➢ CCTV footage (currently from a webcam) has been included in the 3D model.

➢ Online videos (e.g. CPE welcoming video) can be included and displayed in the 3D model.

➢ Progress has been made to communicate with robots that are linked to Work package 3. This work is ongoing.

➢ Progress has been made to link with a digital chemical modelling software (i.e. Aspen Plus). This is related to Work package 1.

Figure 3: A photo showing the view of my 3D model displayed on a Mini iPad.

31

• Project Deliverables and Estimated Time

➢ To complete 3D modelling. There are still a few instruments missing, and various parts of the building need to be modelled to better resemble our Pilot Plant. (Estimated Time of Completion: 1 month, from the middle of Nov 2018 to Dec 2018)

➢ To complete flow animations in the piping network. I have developed some essential code to control and display the flow animation and have a half-dozen displaying as a demonstration. However, more work will be needed to complete the whole piping network. (Estimated Time of Completion: 2 months,

from early Jan 2019 to early Mar 2019)

➢ To complete communication with the robots. I have proposed to use a web server for the communication with the robots. The network traffic from the model (e.g. via VR) to the robots will be coordinates that indicates where the user would like the robots to move. From the reversed traffic, the model (e.g. via VR) will receive camera footage, robots’ latest position coordinates and sensor reading. (Estimated Time of Completion: 2 months, from Mar 2019 to May 2019)

➢ To complete the Aspen model of the current Pilot Plant. The current result from Work package 1 only contains three instruments, so quite a sizeable work is needed to complete such model. (Estimated Time of Completion: 5 months, from May 2019 to Oct 2019)

➢ To complete the communication to the Aspen model. (Estimated Time of Competition: 1 month, from Oct 2019 to Nov 2019).

➢ To complete the communication to the digital control system. Since the digital control system has not been installed to the chemical plant, the time of completion will be assessed and estimated once the system is online.

➢ To establish network communication via a 5G network. Since the 5G network is yet to be installed, the time of completion will be assessed later.

• Research Novelties and Publication Estimation

VR is a research field, not just an emerging information and communication technology.

The challenges based on VR is conceiving new ways to design information, new

narratives and storytelling for a medium in which possibilities have not been fully

explored.

32

From this project, Fengwei will, for the first time (to the author’s best knowledge), to

investigate

➢ The communication between the plant model (within VR, web browsers, smartphones and portable devices) and the Aspen model, as well as the interaction, data exchange and visualisations of the results.

➢ The communication between the plant model (within VR, web browsers, smartphones and portable devices) and a digital control system onsite. Particularly considering the network latency issue and how to derive a communication strategy to identify and resolve network packages loss.

➢ The integration of all three systems, namely the plant model, Aspen model and digital control system, as a brand-new term: digital triplets. The key issue here is what is the best illustration of the information to the user, particularly when the user is in an immersive environment.

I will aim to submit a conference paper to the Industry 4.0 Academia Summit 2019 by

the end of January 2019. I will collaborate further with Dr Abdelwahab and aim to

publish a journal paper in the mid of 2019 on process automation and remote-

inspection technology in the theme of Industry 4.0.

Bibliography

[1] D. Schofield, "Experiences with Virtual Learning," SBC Journal on 3D Interactive

Systems, pp. 18-26, 2012.

[2] Namrata Singh and Sarvpal Singh, "Virtual Reality: a brief survey," International

conference on information, communication & embedded systems, 2017.

[3] Jose Luis Rubio-Tamayo and Manuel Gertrudix Barrio and Francisco Garcia

Garcia, "Immersive environments and virtual reality: systematic review and

advances in communication, interaction and simulation," Multimodal Technologies

and Interaction, 2017.

[4] A. Comunale, "VR focus," 10 10 2017. [Online]. Available:

https://www.vrfocus.com/2017/10/virtual-reality-in-the-construction-industry/.

[Accessed 19 11 2018].

33

[5] D. Nugent, "AECOM," 15 11 2017. [Online]. Available:

https://www.aecom.com/press-releases/aecom-htc-announce-agreement-

develop-virtual-reality-technologies-architecture-engineering-construction-

industry/. [Accessed 19 11 2018].

• Digital-twin project objectives

The aim of this project is to develop an intelligent framework towards abnormal

situation management by using advanced simulator and data analytic technologies.

This smart framework enables the simulator the capabilities of migrating from any

operation conditions to actual plant status based real time measurements. Following

this, effective methodologies for abnormality prediction and diagnosis, alarm

management and process recovery can be further developed. Together they can

contribute a step change towards smart manufacturing and recently proposed digital

twin concept.

• Research Plan

➢ A data driven fault reconstruction methodology will be developed to automatically reconstruct abnormalities in simulator using isolated variables. Also, the performance of an offline simulated fault database in speeding up fault reconstruction process will be evaluated. Effectiveness and efficiency of this methodology will be evaluated on a benchmark problem i.e. the Tennessee Eastman process. ------------------------Estimated by end of February 2019

➢ Similarity metrics that is able to effectively recognize patterns between real measurements and simulated data will be developed. Its performance will be compared with existing metrics on both pilot scale and full-scale processes. Many research fields such as process control, optimization and fault diagnosis could benefit from this result. -----------------------Estimated by end of July 2019

➢ Incipient fault diagnosis method will be developed under the framework through real-time match of the plant measurements and offline simulated datasets. Its performance will be evaluated based on both small scale and large-scale processes. ---------------------------------Estimated by end of September 2019

➢ Under the framework, methods for alarm prediction will be developed, together with root cause of abnormality they can achieve alarm suppression. The TE process will be firstly tested. --------------Estimated by end of January 2020

34

➢ A methodology for process recovery will be developed by simulating different control actions as well as their combinations. This innovative model-based control methodology can have advantages over traditional model-based control methods under abnormal situations. -----------------------Estimated by end of July 2020