Post on 23-Mar-2022
CONTROL SYSTEM CONSIDERATIONS CHP PLANTS
John Beaudry, PE
6/9/2014
Overview
• Types of Control Systems
• Reliability and Availability
• Strategy
• Network Protocols
• Cybersecurity issues
• Alarm Management
• Examples
Types of Controllers
• Direct Digital Control Systems
• Distributed Control Systems
• Programmable Logic Controllers
• SCADA
Direct Digital Systems (Building Automation Systems)
• Designed for HVAC Controls
– Built in Routines for Air Handlers, VAV Boxes, Energy Saving
– Network to Chillers, CRAC Units, Roof Top Air Handlers, etc.
– Scheduling for Occupancy
– ASHRAE 90.1, Energy Code Required Routines Built In.
• Tightly Integrated Graphics and Controls
• Reliability Is Not Designed Into Base Product
• Proprietary Marketing Limits Support
DDC Systems are Not the Normal System of Choice for CHP
Distributed Control Systems
• High Reliability
– Designed for Redundancy (Controllers and HMI) - $$$
• Tightly Integrated Graphics and Field Controllers
• Originally Replacement of Single Loop Control (Analog Control)
• Support network limited for some DCS vendors
Programmable Logic Controllers • Scalable from High Reliability to Lower Cost
– Relay Replacement (Low Cost) - $
– Redundant PLC’s (High Relability) - $$
• Originally Replacement of Relays
• Flexible Implementation
• Large Support Networks
Virtually No Difference Today Between DCS and PLC or PAC Hardware
• Graphics Representation of the Process
• Data Collection for Human-Machine Interface or HMI
• Historical Data
• Scalable from High Reliability to Lower Cost – Server Grade Redundant Equipment - $$
– Capable of Virtual Server Redundant Applications -$$
– Field Mounted Standalone Panels – Very Low Cost- $
• Flexible implementation – Custom Graphics
– Manufacturer Standard Graphics
• Third Party or PLC Vendor
SCADA
Reliability vs. Availability
• Reliability –the control system doesn’t shut down the
process upon a failure of the control system. Usually
economic impact.
• Availability - probability that the control system shuts
down the process when needed. Usually safety
related.
Reliability vs. Availability
• A highly reliable system may not be a safe system.
Both are desired, but there is a tradeoff between
reliability, availability, and cost.
• Balance of Plant high reliability (economic).
• Boiler Safeties high availability (safety)
• Processor and Network redundancy eliminates some
single points of failure and may increase the
availability and reliability.
• Triple Modular Redundancy is an approach to achieve
high reliability with high availability. TMR is very
expensive.
Reliability – Availability – Cost
Strategy of Controls
• Stick Built Controls Strategy – All Stick-Built Controls where single Plant Control System
– Vendors provide detailed Sequence of Operation description
– Advantages: • Common hardware software for all systems
– Risks: • No Single point of responsibility – potential finger pointing
• Higher cost of hardware, integration, engineering
• Vendor Skid Controls Strategy – Skid stand alone vendor skid controls
– May be integrated into larger Control System
– Advantages: • Vendors single point of responsibility
• Cost effective
• Stand alone operation
– Risks: • Hardware from multiple vendors
Vendor Skid Controls Specification
• PLC, or DCS – Common Platform
• Local HMI (Human Machine Interface) for Local Control
• Communication Media
– Ethernet
– Serial
• Communication Protocols
– Native to PLC, or DCS (Ethernet IP, Profibus)
– Common Protocols (i.e. Modbus RS-485, Modbus TCP/IP, BacNet)
• Redundacy
– PAC, PLC, or DCS
– Network communications to Plant Control System
• Interlocks to Plant Control System
– Hardwire vs. Networked
Networks
Ethernet
• Modbus TCP/IP
• Ethernet IP
• DNP3 LAN/WAN protocol
• IEC 61850 GOOSE
• ProfiNet
• BACnet IP
Serial
• Modbus RS-485
• Proprietary RS-485 Networks (AB DH+, Modbus+, Genius I/O)
• BACnet MSTP
• LonTalk
• ArcNet
Although all use Ethernet Media they do not talk nor coexist on the same network
Network Reliability
• Field I/O – Low bandwidth requirements but fast failover
– Device Level Redundancy (ring without a switch)
– Proprietary Rings (N-Ring, HIPER Ring, Turbo Ring)
– Proprietary Communications (i.e. Controlnet, Profibus DP)
• PLC to PLC Communication – Device Level Redundancy
– Proprietary Rings (N-Ring, HIPER Ring, Turbo Ring)
– Managed Switches
• PLC to SCADA Communication – High Bandwidth
– Managed Switches
Field Communication Issues
• Network Diagnostics – Software for failure conditions
– Alarming
• Distributed processing – Equipment such as VFD’s, MCC’s may have logic at the device.
– Loss of communication may result in motors not capable of being stopped
– Software may behave differently than hardwired devices (i.e. Auto over-
riding Hand control)
– Motor Fail Logic will not alarm if communications are lost.
Cyber Security
• Standard IT Security
– Require each individual to Login
– Change Passwords
– Limit access rights
• SCADA and PLC development applications should be limited to
qualified individuals
• Standard Login won’t allow devices to be stopped and started or
Setpoints changed
• Operator has rights to change setpoints, start and stop devices, but not
tune loops or change software
• Burner Management , HRSG Combustion Controls, Combustion
Turbine , Gas Compressors should require special access. May want
to limit changes to vendor.
• Lock out flash drives, CD’s, Email, Internet inside the Plant Control
System?
Cyber Security
• Network Security
– DMZ Level between Business Network and the Plant Control
– Firewalls capable of filtering on content and source.
– VLAN’s with MAC Address Limits on Control Network
• Patching SCADA/PLC Software and Firmware
– US Dept Homeland Security ICS-CERT identifies Hardware Vulnerabilities
– Firmware Updates may need to be scheduled around downtime.
• Limit Physical Access
Don’t Forget Alarm Management
• Refinery hardwire alarms ignored (horn wiring cut) when too
many/ too frequent alarms.
• Steam Turbine was loosing lube oil, but the operator didn’t see
the alarm because the large number of alarms. Close call to
loosing all lube oil.
• Large Semiconductor Plant Central Utility Plant couldn’t see
motor alarms, nor start/stop motors when DeviceNet network
failed. DeviceNet communication alarms were buried in the large
number of alarms.
• Cogen Plant lost SCADA communication because Ethernet failure
alarms weren’t noticed. Although redundant processors,
switches, SCADA servers, system lost complete visibility. Lost
ability to shut down HRSG from control room.
Alarm Management Strategies
• Only Alarm Important Values – Use Historical Trending
for Information
• Assign Alarm Priorities
• Categorize Alarms by Process Areas
• All Network Communications should be highest
priority alarms
• Alarm Inhibiting
• Conditional Alarming – Don’t alarm equipment if the equipment is not running
• i.e. Inhibit HRSG drum level if the HRSG is down
– RODI conductivity shouldn’t alarm when the product is being dumped
LA County Sanitation District Carson Plant
REDUNDANT PLC
CEMS DAHS
pentium III
SCADA VIEWNODE
pentium III
SCADA VIEWNODE
pentium III
ENGINEERING
WORKSTATION
pentium IIIpentium III
REDUNDANT
SCADA SERVERS
SCADA
NETWORK
A
SCADA
NETWORK
B
1 G FIBER
NETWORK
A NETWORK
B
POWER
Allen-BradleyQUALITY
RUN
BAT
I/O
Rs232
OK
RUN PROGREM
Logix5550
RXD
ETHERNET< >
TXD OK RXD
ETHERNET< >
TXD OKPR
I
REDUNDAN
CY MODULE
CO
M
O
K
O
K
O
K
O
K
O
K
O
K
O
K
POWER
Allen-BradleyQUALITY
RUN
BAT
I/O
Rs232
OK
RUN PROGREM
Logix5550
RXD
ETHERNET< >
TXD OK RXD
ETHERNET< >
TXD OKPR
I
REDUNDAN
CY MODULE
CO
M
O
K
O
K
O
K
O
K
O
K
O
K
O
K
REMOTE I/O PANEL
HRSG 1
REMOTE I/O PANEL
HRSG 2
REMOTE I/O PANEL
HRSG 3STEAM TURBINE AREA
WATER TREATMENT
AREA
GAS COMPRESSOR
AREA
REDUNDANT
CONTROLNET
GAS CONDITIONING
AREA
STEAM TURBINE PANEL
GAS TURBINE
CEMS
pentium III
Tank02
F1
F7
F2
F8
F3
F9
F4
F10
F5
F11
F6
F1212 102 31
High level Tank 02
M
P
P
P
P
P
P
Na
H2OCa
Li
PbCu
Waste
ON
Temp.:Speed:
123 gr56 m/s
ON
ON
ON
ON
ON
ON
OFF
ON
Temp.:
Press.:
Level:
123deg
12Pa
4,5m
PLANT J45
Output:
Tank 01
Tank 02
Tank 03
Tank 04
Converter
React. A
React. B
React. C
React. D
React. E
Level H234
Start/Stop
Auto
Manual
Force
Setpoint
Output
345
34
LL L H HH
GAS CONDITIONING
SKID
Redundant PLC’s
Redundant PLC’s
Redundant
Ethernet Network
To Vendor Skids
Redundant
Controlnet
To Remote I/O
Redundant
Ethernet Star
or Mesh Network Redundant
SCADA Server
HRSG Controls
In Balance of Plant
PLC
Motors Controls
Hardwired
Ethernet Network
To Steam Turbine
Ethernet IP
(native to PLC)
Hardwired
Safeties between
CTG and PLC
Hardwired I/O to CEM
Oklahoma University
CEMS DAHS
pentium III
SCADA COMPUTERS
pentium III
SCADA COMPUTERS
REDUNDANT PLC
POWER
Allen-BradleyQUALITY
RUN
BAT
I/O
Rs232
OK
RUN PROGREM
Logix5550
RXD
ETHERNET< >
TXD OK RXD
ETHERNET< >
TXD OK
P
R
I
REDU
NDAN
CY
MODU
LE
C
O
M
O
K
O
K
O
K
O
K
O
K
O
K
O
K
POWER
Allen-BradleyQUALITY
RUN
BAT
I/O
Rs232
OK
RUN PROGREM
Logix5550
RXD
ETHERNET< >
TXD OK RXD
ETHERNET< >
TXD OK
P
R
I
REDU
NDAN
CY
MODU
LE
C
O
M
O
K
O
K
O
K
O
K
O
K
O
K
O
K
SWITCH
SWITCH
SWITCH
VENDOR SKID
POWER
Allen-BradleyQUALITY
RUN
BAT
I/O
Rs232
OK
RUN PROGREM
Logix5550
RXD
ETHERNET< >
TXD OK RXD
ETHERNET< >
TXD OK
P
R
I
RED
UND
ANC
Y
MOD
ULE
C
O
M
O
K
O
K
O
K
O
K
O
K
O
K
O
K
QUALITY Allen-Bradley PanelView 600
VENDOR SKID
POWER
Allen-BradleyQUALITY
RUN
BAT
I/O
Rs232
OK
RUN PROGREM
Logix5550
RXD
ETHERNET< >
TXD OK RXD
ETHERNET< >
TXD OK
P
R
I
RED
UND
ANC
Y
MOD
ULE
C
O
M
O
K
O
K
O
K
O
K
O
K
O
K
O
K
QUALITY Allen-Bradley PanelView 600
HRSG BMS
POWER
Allen-BradleyQUALITY
RUN
BAT
I/O
Rs232
OK
RUN PROGREM
Logix5550
RXD
ETHERNET< >
TXD OK RXD
ETHERNET< >
TXD OK
P
R
I
RED
UND
ANC
Y
MOD
ULE
C
O
M
O
K
O
K
O
K
O
K
O
K
O
K
O
K
QUALITY Allen-Bradley PanelView 600
VENDOR SKID
POWER
Allen-BradleyQUALITY
RUN
BAT
I/O
Rs232
OK
RUN PROGREM
Logix5550
RXD
ETHERNET< >
TXD OK RXD
ETHERNET< >
TXD OK
P
R
I
RED
UND
ANC
Y
MOD
ULE
C
O
M
O
K
O
K
O
K
O
K
O
K
O
K
O
K
QUALITY Allen-Bradley PanelView 600
VENDOR SKID
POWER
Allen-BradleyQUALITY
RUN
BAT
I/O
Rs232
OK
RUN PROGREM
Logix5550
RXD
ETHERNET< >
TXD OK RXD
ETHERNET< >
TXD OK
P
R
I
RED
UND
ANC
Y
MOD
ULE
C
O
M
O
K
O
K
O
K
O
K
O
K
O
K
O
K
QUALITY Allen-Bradley PanelView 600
MCC
Profibus
Gateway
INPUTS
OUTPUTS
INPUTS
OUTPUTS
INPUTS
OUTPUTS
INPUTS
OUTPUTS
INPUTS
OUTPUTS
INPUTS
OUTPUTS
CEMS
POWER
Allen-BradleyQUALITY
RUN
BAT
I/O
Rs232
OK
RUN PROGREM
Logix5550
RXD
ETHERNET< >
TXD OK RXD
ETHERNET< >
TXD OK
P
R
I
RED
UND
ANC
Y
MOD
ULE
C
O
M
O
K
O
K
O
K
O
K
O
K
O
K
O
K
QUALITY Allen-Bradley PanelView 600
pentium III
Tank02
F1
F7
F2
F8
F3
F9
F4
F10
F5
F11
F6
F1212 102 31
High level Tank 02
M
P
P
P
P
P
P
Na
H2OCa
Li
PbCu
Waste
ON
Temp.:Speed:
123 gr56 m/s
ON
ON
ON
ON
ON
ON
OFF
ON
Temp.:
Press.:
Level:
123deg
12Pa
4,5m
PLANT J45
Output:
Tank 01
Tank 02
Tank 03
Tank 04
Converter
React. A
React. B
React. C
React. D
React. E
Level H234
Start/Stop
Auto
Manual
Force
Setpoint
Output
345
34
LL L H HH
15 KV/5KV
HRSG CCS
POWER
Allen-BradleyQUALITY
RUN
BAT
I/O
Rs232
OK
RUN PROGREM
Logix5550
RXD
ETHERNET< >
TXD OK RXD
ETHERNET< >
TXD OK
P
R
I
RED
UND
ANC
Y
MOD
ULE
C
O
M
O
K
O
K
O
K
O
K
O
K
O
K
O
K
QUALITY Allen-Bradley PanelView 600
CTG
POWER
Allen-BradleyQUALITY
RUN
BAT
I/O
Rs232
OK
RUN PROGREM
Logix5550
RXD
ETHERNET< >
TXD OK RXD
ETHERNET< >
TXD OK
P
R
I
RED
UND
ANC
Y
MOD
ULE
C
O
M
O
K
O
K
O
K
O
K
O
K
O
K
O
K
QUALITY Allen-Bradley PanelView 600
REDUNDANT PLC
POWER
Allen-BradleyQUALITY
RUN
BAT
I/O
Rs232
OK
RUN PROGREM
Logix5550
RXD
ETHERNET< >
TXD OK RXD
ETHERNET< >
TXD OK
P
R
I
REDU
NDAN
CY
MODU
LE
C
O
M
O
K
O
K
O
K
O
K
O
K
O
K
O
K
POWER
Allen-BradleyQUALITY
RUN
BAT
I/O
Rs232
OK
RUN PROGREM
Logix5550
RXD
ETHERNET< >
TXD OK RXD
ETHERNET< >
TXD OK
P
R
I
REDU
NDAN
CY
MODU
LE
C
O
M
O
K
O
K
O
K
O
K
O
K
O
K
O
K
FIREWALL
Redundant PLC
Chiller Plant Redundant PLC
Power Plant Redundant
Controlnet
To Remote I/O
High Speed Ethernet
Ring Between PLC’s
Vendor Skids, CEMS,
MCC’s, And Switchgear
Hard Wired I/O between
Duct Burner BMS and CTG
CEMS Standalone DAHS
Logic in MCC’s and
Switchgear
Overview
Further Reading
Security
• NIST Special Publication 800-82 - Guide to Industrial Control Systems
(ICS) Security
• Homeland Security - Recommended Practice: Improving Industrial
Control Systems Cybersecurity with Defense-In-Depth Strategies
• Homeland Security - Common Cybersecurity Vulnerabilities in Industrial
Control Systems
• ANSI/ISA-TR99.00.01-2007 - Security Technologies for Industrial
Automation and Control Systems
Reliability
• ANSI/ISA 84 Functional Safety: Safety Instrumented Systems for the
Process Industry Sector