Post on 02-Feb-2016
description
September 2.001
ENGINEERING ETHERNET BASED SOLUTIONS
This presentation is based in GTD experience integrating control and supervision architectures for Science, Space,
Defense and Industrial applications.
By the Software Support Team (GTD St.Genis) and the Cryogenics Team (GTD BCN)
Projects with relevant GTD involvement: XMM-Newton Space Telescope, Eurofighter, Arianne V launcher, Spaceport Control-room and Accelerators (Grenoble and CERN)
ENGINEERING ETHERNET BASED SOLUTIONSBy the Software Support Team (GTD St.Genis) and
the Cryogenics Team (GTD BCN)
1) SOLUTIONS: … to a problem. This presentation discuss suitable problems for ETHERNET application.
2) ENGINEERING: No general solution (at least for the most relevant problems). Criteria to create suitable solutions, identify major constraints and defined the proper integration strategy are also introduced.
PROBLEM OVERVIEW1.1DisplayingLarge amount of data in human real-time scale (i.e.: seconds)RegisteringNon real-time large amount of data with high time-stamp accuracy… … …Processing/ControlVariable amount of data in process real-time** scale (1ms – 500ms)*Data Supply for:
- Displaying- Registering
… … …
*) Ranges for industrial (PLC based) control systems in GTD**)REAL-TIME: known, accurate - i.e. 1ms - & controllable
AcquisitionVariable amount of data in process real-time scale** (1ms – 500ms)*Data Supply for:
- Processing/Control- Displaying- Registering
… … …
RequestingReduced amount of data in human
time scale (seconds)Configuring/Backing-up
Large amount of data non real-time event-driven (i.e. start-up)
… … …Commanding
Variable amount of data in process real-time** scale (1ms – 500ms)*
… … …
ActuationVariable amount of data in process
real-time** scale (1ms – 500ms)*… … …
the Processthe Process
the Controlthe Control
the Supervisionthe Supervision
Synchronous / Time[ms]
Synchronous / Time[s]
Asynchronous
Asynchronous
Asynchronous
requestsrequests
commandscommands
records: records: events, trends, …events, trends, …
sub-samplingsub-sampling
statusstatus
data supplydata supply
data supplydata supply
the Processthe Process
the Controlthe Control
the Supervisionthe Supervision
PROBLEM OVERVIEW1.2
synchronous
asynchronous
Max Data Quantity
Time Schedule
Max Data Quantity
Time Limits data
Real-TimeNon
Real-Time
Bw1 Bw i Bw j Bw n
Application Domain ofEthernet as Fieldbus
PROBLEM OVERVIEW1.3
data
TIMETIMEacquisition processing
known& fixedT ± T
THE ENGINEERING APPROACHTHE CONCEPT
2.1
PROBLEM
URD
… …
TOPOLOGY
ARCHITECTUREADD
For Processes affecting multiple devices:• Nature (synchronous/asynchronous)• Real Time Constraints• Bandwidth (Bw)
FUNCTIONALITIES
validated 1 on 1
ETHERNETapproach
+ complexity+ performance+ new functionalities+ data availability*/richness+ flexibility
may block (partially or totally) Ethernet applicability
Synchronous(1 (typically Closed Control Loops signals – “feedback” and “output”) Real Time Requirements
0< Tx <10ms Tiny Bandwidth Limited (Field)bus participants Local Processing (eventually by distributing processing effort)
10ms< Tx <50ms Limited Bandwidth Limited (Filed)bus participants Deterministic FieldbusDevice’s efficiency for TCP/IP packaging is limited under 50ms
50ms < Tx Ethernet Applicable (Bw occupation 1:10) for “determinisms” New topologies (system redesign!) Greater Bw Packets oriented transmission for Bw optimization (Protocol “Overhead”)
Fieldbus still ApplicableLess Bw availableLess flexible topologies (several distance constraints)
1) Synchronous: Deterministic time control on processes concurrently running at different devices.
THE ENGINEERING APPROACHTHE CONCEPT
2.2
AsynchronousReal Time Requirements
0< Tx <10ms [i.e. Open Loop Control – Alarm Handling, Time-stamping, …] Tiny Bandwidth
Limited (Field)bus participants * Deterministic Fieldbus > Polling / Packages > Slow! Local Processing (eventually by distributing processing effort)
10ms< Tx <50ms [i.e. Open Loop Information refresh]* “Messages” better than “token”* Token > Polling / Packages limited efficiency (overhead) TCP/IP with PROTOCOL + Tx Policy
50ms < Tx [i.e. (very) large systems for info refresh, buffered data, configurations’ up/download, …]* fully package oriented Ethernet Best Choice (Bw occupation 1:10) for “confidence” New topologies (system redesign!) Greater Bw TCP/IP with PROTOCOL + Tx Policy
THE ENGINEERING APPROACHTHE CONCEPT
2.3
Real TimeRequirements
10ms 100ms50ms0
SYNCHRONOUS
ASYNCHRONOUS
for Increasing Bandwidth
ETHERNETTCP/IP
DeterministicFieldBus
suit
abili
ty
Reco
mm
en
dable
… a
t th
e lim
it
Not
reco
mm
en
dable
THE ENGINEERING APPROACHTHE CONCEPT
2.4
THE ENGINEERING APPROACHENGINEERING A SOLUTION
3.1
ETHERNETTCP/IP
10MBits100MBits
IndustrialControl& I/O
Devices
But … … …• What is the effective Bandwidth occupation?• What are the constraints?• How shall the information exchange be organized?
Ethernet(IEEE 802.3)
TCP/IP
MiddlewareServices
1
2
3
4
5
6
7
THE ENGINEERING APPROACHENGINEERING A SOLUTION
3.2
Physical
Data Link
Network
Transport
Session
Presentation
Application Layer
High Level Protocols
Information Organization
APPLICATION
throughput
More data managed (maybe not more knowledge!)
ENGINEERING: Superprotocol + BW management
Some constraints: somehow inherited structures
Some immaturity adding TCP/IP services to certain PLC’s Operating Systems.
Some immaturity degrees in the firmware connected to proprietary architectures.(is it becoming a PLC weakness?!?) h
ard
ware
THE ENGINEERING APPROACHENGINEERING A SOLUTION
A case study ... the Control System for the LHC Cryogenics
3.3
supervision layer
control layer
field layer
FIRST IDEA ... Derived literally from Specification
SynchronousTime Stamping10ms<9.000 Binaries( 275.000 Binar.)
VisualizationSynchronous~2s<50.000 Binaries<22.000 Floats( 700.000 Binar.)( 300.000 Floats)
SynchronousProcessTs=500ms(150ms)< 9.000 Binaries<15.000 Floats( 275.000 Binar.)( 105.000 Floats.)
EventsAsynchronous<acceptable delayfrom 700.000 BinariesMax 500 events/s( 4.000 events/s)
SynchronousProcessTs=500ms(100ms)< 3.500 Binaries< 3.500 Floats( 155.000 Binar.)( 160.000 Floats.)
AsynchronousManual Requests~1s Human Feeling< 3.500 Binaries( 155.000 Binar.)
13
Bw~8MBitsBw~10,5MBits
2
THE ENGINEERING APPROACHENGINEERING A SOLUTION
A case study ... the Control System for the LHC Cryogenics
3.4
IMPLEMENTATION (as it is today!)
VisualizationAsynchronous(worst case 2s)<50.000 Binaries<22.000 Floats( 700.000 Binar.)( 300.000 Floats)
ProcessAsynchronous(worst case 150ms)< 9.000 Binaries< 5.000 Floats( 275.000 Binar.)( 105.000 Floats.)
EventsAsynchronous<acceptable delayfrom 700.000 BinariesMax 500 events/s( 4.000 events/s)
ProcessAsynchronous(worst case 100ms)< 3.000 Binaries< 2.000 Floats( 155.000 Binar.)( 160.000 Floats.)
AsynchronousManual Requests~1s Human Feeling< 3.500 Binaries( 155.000 Binar.)
TIME STAMPINGTIME STAMPING
TIME STAMPINGTIME STAMPING
X 165
X 64
X 8(redundant)
2 x TCP/IP cards
**
**
1
2
3
4
SAMPLINGSAMPLING
3.5THE ENGINEERING APPROACHENGINEERING A SOLUTION
A case study ... the Control System for the LHC Cryogenics
summarizing the chosen approach …
1) Distribution of the Time-stamping process. Time-stamping at the source eliminates synchronous communication.>>> It is feasible to move to a full ETHERNET TCP/IP solution>>> Requires specific Supervision system
2) Duplication of networks at the Control Layer to make synchronous exchanges “deterministic” and/or reduced collisions (moreover, dimensioning of the number of devices)>>> Measure performances of the Hardware
3) Eliminate unnecessary throughputs by directly addressing the information item to the target (favored by the flexible Ethernet architecture)
4) Bandwidth Management: Synchronous >>> Asynchronous!(further explained)
3.6
TCP/IP 100MBitsOpen Modbus CPS211 00
CPU534 14NOE771 00NOE771 00
0
100
200
300
400
500
600
1 Port 2 Ports 4 Ports 8 Ports 16 Ports 24 Ports 30 Ports
Kbits/s
100 ms70 ms50 ms40 ms
0
5
10
15
20
25
30
35
40
45
50
1 Port 2 Ports 4 Ports 8 Ports 16 Ports 24 Ports 30 Ports
Percentage
100 ms70 ms50 ms40 ms
Percentage of the PLC cycle time occupied by communications related tasks [%] for n parallel ports/channelsand different PLC cycles.
Performance (Bw)[Kbit/s] for n parallel managed communication’s ports/channels.Curves represent different PLC cycles (40ms .. 100ms)
THE ENGINEERING APPROACHENGINEERING A SOLUTION
A case study ... the Control System for the LHC Cryogenics
HARDWARE evaluation (example … at the Control Layer)
3.7
0
5
10
15
20
25
30
35
40
45
50
0 100 200 300 400 500 600
Band width
Du
ty c
ycle
100 ms70 ms
50 ms40 ms
0
5
10
15
20
25
30
35
40
45
50
0 100 200 300 400 500 600
Band width
Du
ty c
ycle
1 MASTER2 MASTERS4 MASTERS8 MASTERS16 MASTERS24 MASTERS30 MASTERS
Duty Cycle / Band Width ratio for different PLC Cycles.
Duty Cycle / Band Width ratio for different number of parallel opened
Ports/channels (MASTERS)
Good conditions found for 16 parallel managed ports/channels in a 50ms PLC cycle, thus resulting in approx. 256 Kbits/s of bandwidth and less than 20% PLC cycle occupation due to communication’s management (approx. 10ms).
THE ENGINEERING APPROACHENGINEERING A SOLUTION
A case study ... the Control System for the LHC Cryogenics
HARDWARE evaluation (example … at the Control Layer)
Event Driven Protocols / Tx Policies:>>> Oriented to Asynchronous strategies
a) People use to neglect “worst case” conditionsb) Steady conditions shall be carefully understood
>>> Event Driven use to be optimized for reduced pieces of INFOWorst conditions use to mean: large quantities!
>>> Optimization is achieved by the probabilistic fact of “little changes”“worst case” and “close to worst case” are not negligible
>>> Oriented to Synchronous strategiesIt is feasible by:a) Dimensioning for the worst caseb) Sampling to regularize data flow
>>> What is the advantage?Bandwidth Management
a) To support concurrent, less demanding, Asynchronous communication processes. b) To reduce collision probability in steady conditions
3.8THE ENGINEERING APPROACHENGINEERING A SOLUTION
A case study ... the Control System for the LHC Cryogenics
PROTOCOL & BANDWIDTH MANAGEMENT
Ti
Tk
Ts
process
Ts
THE ENGINEERING APPROACHENGINEERING A SOLUTION
A case study ... the Control System for the LHC Cryogenics
3.9
IMPLEMENTATION (as it is today!)
Maximum 8 Control Units
(PCUs) for each DS
Maximum 4 Field
Interfaces (FIs) for
each PCU
8 Redundant Data Servers(Bi-Pentium)
EventsVisual
Trends
Requests
EventsVisual
Requests
Process
ProcessEventsVisualTrends
Process
ProcessRequests
SAMPLINGSAMPLING
TIME STAMPINGTIME STAMPING
TIME STAMPINGTIME STAMPING
Example PCU:Quantum 534 14 + 2x NOE 771 00Event Driven (by GTD) on Open Modbus
Steady “worst conditions” (= Polling)Bw = 256 Kbit/s available- needed for Superv. 200KBit/s- needed for Process.160KBit/s
Better than “worst conditions”- Bandwidth availability for true Asynchronous processes.
ENGINEERING ETHERNET BASED SOLUTIONSconclusions
synchronous
asynchronous
Max Data Quantity
Time Schedule
Max Data Quantity
Time Limits data
Real-TimeNon
Real-Time
Bw1 Bw i Bw j Bw n
Application Domain ofEthernet as Fieldbus
PROBLEM OVERVIEW1.3
data
TIMETIMEacquisition processing
known& fixedT ± T
Ethernet(IEEE 802.3)
TCP/ IP
MiddlewareServices
1
2
3
4
5
6
7
THE ENGINEERING APPROACHENGINEERING A SOLUTION
3.2
Physical
Data Link
Network
Transport
Session
Presentation
Application Layer
High Level Protocols
Information Organization
APPLICATION
throughput
More data managed (maybe not more knowledge!)
ENGINEERING: Superprotocol + BW management
Some constraints: somehow inherited structures
Some immaturity adding TCP/IP services to certain PLC’s Operating Systems.
Some immaturity degrees in the firmware connected to proprietary architectures.(is it becoming a PLC weakness?!?)h
ard
ware
1. Synchronous-Asynchronous2. Data Quantity3. Time Frame
Our recommendation:
Try to use Ethernet for1. Synchronous, over 75ms2. Asynchronous, over 25ms
Real TimeRequirements
10ms 100ms50ms0
SYNCHRONOUS
ASYNCHRONOUS
for Increasing Bandwidth
ETHERNETTCP/ I P
DeterministicFieldBus
suit
abili
ty
Reco
mm
endable
… a
t th
e lim
it
Not
reco
mm
endable
THE ENGINEERING APPROACHTHE CONCEPT
2.4
Still difficulties:
1. Certain immaturities2. Inherited standards3. Large/Complex applications (more than necessary !?!)
> Engineering Effort a) To adapt resources b) To create Tx policies c) To benefit from Ethernet added values.
Case Study:
Characterized by … 1. Application “size”2. Information flow3. Synchronous/Asynchronous
Ethernet TCP/IP not well suited for synchronous, fast processes
THE ENGINEERING APPROACHENGINEERING A SOLUTION
A case study ... the Control System for the LHC Cryogenics
3.9
IMPLEMENTATION (as it is today!)
Maximum 8 Control Units
(PCUs) for each DS
Maximum 4 Field
Interfaces (FIs) for
each PCU
8 Redundant Data Servers(Bi-Pentium)
EventsVisual
Trends
Requests
EventsVisual
Requests
Process
ProcessEventsVisualTrends
Process
ProcessRequests
SAMPLI NGSAMPLI NG
TIME STAMPINGTIME STAMPING
TIME STAMPINGTIME STAMPING
Example PCU:Quantum 534 14 + 2x NOE 771 00Event Driven (by GTD) on Open Modbus
Steady “worst conditions” (= Polling)Bw = 256 Kbit/s available- needed for Superv. 200KBit/s- needed for Process.160KBit/s
Better than “worst conditions”- Bandwidth availability for
true Asynchronous processes.
THE ENGINEERING APPROACHTHE CONCEPT
2.1
PROBLEM
URD
… …
TOPOLOGY
ARCHITECTUREADD
For Processes affecting multiple devices:• Nature (synchronous/ asynchronous)• Real Time Constraints• Bandwidth (Bw)
FUNCTIONALITIES
validated 1 on 1
ETHERNETapproach
+ complexity+ performance+ new functionalities+ data availability* /richness+ flexibility
may block (partially or totally) Ethernet applicability
September 2.001