ACME Inc (Inspired by Qualtech) · 2019-10-30 · Prime Infrastructure / DNA-C Prime...

ACME Inc (Inspired by Qualtech)

Connected Machines

© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential

Table of Content

Background and current State of Company and Architecture

Challenges for IT, OT, and Business and derived Targets

Project details and Technical Approach

Components for Hardware and Management

IIoT and Security, Guidelines

Best Practices

01.

02.

03.

04.

05.

06.


Current State of the Company

ACME Inc. offers a wide range of specialized manufacturing equipment, process capabilities and stainless steel components, which are manufactured and tailored to meet the specific needs of for instance agrofood, industrial, biopharmaceutical and brewing sectors.

Their products are core components for Clean-in-Place (CIP) systems for edible good production and range from tanks, high-temperature/short-time pasteurization (HTST), washing equipment and filtration solutions.

Industry: Manufacturing

Focus: Industrial Machines

Main Business: Multidivisional / Multibranch

Employees: ~ 300

Revenue:


Current Architecture of company

ACME Inc. produces machines which are core components of production chains – often with special requirements to hygiene. The production process requires operation, monitoring and management of dependent, multi-vendor installations.

ACME Inc. machines are sold and operated worldwide on customers premise.

The machines remained unconnected and there was no common segmentation scheme.

Machine telemetry was kept isolated and accessible only by local operators at the machine.

Machine Machine Machine Machine Machine Machine

Floor Floor Floor

Customer 1 Customer 2 Customer 3


Desired Business GoalsThe North Star

By integrating networking capabilities with their machines, OEMs can offer their customers immediate and long-term benefits. Machines equipped with managed switches can be integrated directly into existing network structures, providing:

• Machine Connectivity as the basis for data provisioning and control• Standard Telemetry acquisition from the machines and data provisioning as the basis for

monitoring, alarming and analytics• Production Process Monitoring to ensure that machines in the multi-vendor, dependent

production chain are operating normally• Quality Monitoring by acquiring inspection data across the multi-vendor production chain• Remote Support and Control to address manual effort and time-to-reaction

Be able to offer remote monitoring, predictive maintenance and other recurring revenue generating services that is only enabled by network enabled machines.

Machines with managed switches can be installed as easily as machines with unmanaged switches or no networking capabilities as it requires no initial configuration. Cisco Design-in program also allows the SIs to train on the latest networking technologies so that they can apply the best practices as they integrate the machines into the customers manufacturing network.

Machine Builder

SI


Challenges Identified with the current customer architecture / technologies used

OT BusinessIT

No form of remote support, remote operation, or monitoring possible.

Connectivity

Locked and inaccessible device telemetry prevents optimization of invested resources into production process

Data Availability

Incomplete visibility of multi-vendor production process telemetry

Data Availability

Machines operated isolated and unsupervised require manual inspection and prevent any form of preventive or prescriptive maintenance

Connectivity

Unconnected machines are unable to provision telemetry data securely, scalable and with an acceptable latency

Connectivity


Solution Details and DeliveryThe technology portfolio and its possible instantiations for connecting machines complies to the wide range of technical and business requirements per customer.

Rockwell Automation Control, Allen-Bradley® CompactLogix™ controllers, switches from Cisco IE or Rockwell Stratix series are core components of target architectures for a cell/area zone network solution to provide machine, machine ensemble or machine skid connectivity.

The most adequate software components and network design depends on the requirements on resiliency, complexity, initial size, scalability as well as to the functionality of the management software.

The solution integrates on an EtherNet/IP™ network and includes either Cisco IE 2000/3000/4000, or the corresponding Rockwell Stratix/Allen Bradley switches, organized in the most appropriate network topology.


New Architecture


Industrial Ethernet Switch CharacteristicsFeatures Cisco Industrial Ethernet (IE) Typical Non-Industrial Ethernet Switch

Form Factor / Mounting Options Din Rail, Panel and Rack Mount Rack Mount

Interface Options Port density 6-28 High port density

PoE Density / Max Power Port density 6-28 High port density

Power Supply OptionsAC and DC

DC input voltage range = 10 to 300AC and DC

DC input voltage range = 36 to 72

Converged Access

(Wired plus Wireless)No

Yes,

Mobility agent and Mobility controller

Environmental Design

• Fanless (no moving parts) vs Fans

• Operating Temperature Range

• Ingress Protection (IP) Rating

• Industry Certifications

Fanless

-40c to +60c

IP30 (models up to IP67)

Hardened for vibration, shock, surge, and noise immunity

Fans

-5c to +45c

IP XX (Not Specified, IP20 or less)

Enterprise class certifications

“Swap Drive” – Removable Flash Yes No

Dying Gasp - Upon loss of input power Yes No

Alarm Ports Yes No

Deterministic Ethernet

IEEE 802.1 TSNYes – Supported by IE 4000 and 5000 No


Features Cisco Industrial Ethernet (IE) Typical Non-Industrial Ethernet Switch

Industrial Protocols - Management EtherNet/IP CIP, Profinet, Modbus TCP Not available

Industrial Protocols – High Availability REP, MRP, Flexlink, PRP, HSR REP (slower convergence time), Flexlink

Smart-port Macros

IE Smart-port macros (Qty 32):QoS policies, IED, PTP, CIP, HMI etc…

Enterprise (qty 6): global, desktop, phone, switch, router, wireless

No IE Smart-port macros

Enterprise (qty 6): global, desktop, phone, switch, router, wireless

Device ManagerEase of use on device web server for

device managementOn device web server for device management

Network ManagementIndustrial Network Director (IND)

Prime Infrastructure / DNA-CPrime Infrastructure/DNA-C

Typical Boot Time 30sec – 2 min,20 sec 5 mins (single switch)

L2 and L3 Images Yes, same hardware Yes, same hardware

Precise Timing

IEEE 1588 PTP

IEEE C37.238-2011 (Power Profile)

YesIEEE 1588, inc. Power Profile level of

accuracy (50ns per hop)Option for GPS and IRIG-B on IE 5000,

including Grand Master with Stratum 3E on board oscillator

No

Industrial Ethernet Switch Characteristics Cont.

© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential

Cisco Industrial Ethernet 1000 Series Switches • 4 models with 5, 6, 8 and 10 Ethernet

ports options

• SFP, Copper, PoE/PoE+ ports

• Extensive Industrial environmental compliance

and certifications

• GUI to troubleshoot, monitor and diagnose

• Improve Network Resiliency: link redundancy

and fast recovery

• Increase Network Security: port security

• Prioritize Critical Traffic: guarantee critical traffic

• Reduce Overall TCOManufacturing CityTransportation

Available since: 07/2016

Small, lightly managed with PnP


IE1000 Hardware Overview

SKU IE1K-Copper IE1K-PoE

Downlinks4 10/100M RJ456 10/100M RJ45

4 10/100M RJ45 (w/PoE)8 10/100M RJ45 (w/PoE)

Uplinks(5port) 1 FE Copper(8port) 2 FE copper

2 GigE Fiber(copper and Fiber SFP)

PoE No PoE/PoE+

Total Ports 5 or 8 6 or 10

Power Input 24 VDC nominal (9–36) 48/54 VDC nominal (44–57)

Size (cm)(5port) W3.81 x H12.7 x D11.5 (8port) W4.5 x H12.7 x D11.5

W4.5 x H12.7 x D13.3

Console port None

Alarm input/output No Yes

Temperature range -20-60C -40-70C

Ingress Protection IP30

Warranty 5yr


IE1000: Cisco Lightly Managed PIDs

IE-1000-4T1T-LM

• 5 Fast Ethernet Copper• No PoE• Single power input• W3.81 x H12.7 x D11.5

IE-1000-6T2T-LM

• 8 Fast Ethernet Copper• No PoE• Single power input• W4.5 x H12.7 x D11.5

IE-1000-P2S-LM

• 2 Gig Fiber SFP uplinks• 4 Fast Ethernet copper with PoE• 120W PoE budget• Redundant power input• W4.5 x H12.7 x D13.3

IE-1000-8P2S-LM

• 2 Gig Fiber SFP uplinks• 8 Fast Ethernet copper with PoE• 180W PoE budget• Redundant power input• W4.5 x H12.7 x D13.3

All PIDs: 5-Year Warranty; IP30


• DIN rail and wall and/or pole mounting

versions available

• IE2000have 6 to 20 ports/IE2000 IP67 have 8

to 24 10/100 Ethernet interfaces with or w/o

2 x GE uplinks

• Ingress Protection 30 and 67 – IP67 options

• PoE/PoE+ and conformal coating options

• Native support of Industrial protocols

(Ethernet/IP, PrOFINET – incl. MRP-)

• Advanced QoS and Security features

Available since: 2011

Small, lightly managed with PnPCisco Industrial Ethernet 2000 Series Switches

Manufacturing CityTransportation

MiningOil & Gas


IE2000 Fixed Port Switches

IE2K – 4 + 2 Ports

• 4 10/100 ports• 2 RJ45 and SFP Uplink• H5.1 x W2.95 x D4.51

IE2K – 8 + 2 Ports

• 8 10/100 ports• 2 Combo (SFP/Copper)• H5.1 x W6.3 x D4.51

(or D5.26 (-E & -N))

IE2K 16 Ports + 4 Ports

• 16 10/100 ports• 4 Combo (SFP/Copper) • Uplinks• H5.1 x W5 x D5.26

• 4, 8, and 16 Fixed Port configuration

• 2 Gig combo ports uplinks with copper, SFP,

POE/PoE+ options, Conformal Coating variant

• Enterprise Software Feature: Support for

DHCP, 802.1x, Security, QOS, 1588, PTP,

NAT, L2 Multicast, and REP ring protocol

• Integrated power supply, Alaarm relay,

Optional SD card for easy replacement

• Industrial protocols capability: Ethernet/IP

& PROFINET


Cisco IE Switching Product Security Feature Support

Security FeatureSupport

Cisco IE2000Cisco IE2000U / IE3000 /

IE3010 / CGS2520Cisco IE3200 /

IE3300 / IE3400Cisco IE4000 / IE4010 / IE5000

LAN Lite (Layer 2)

LAN Base /IP Lite

LAN Base(Layer 2)

IP ServicesNetwork Essentials /Network Advantage

LAN Base (Layer 2)

IP Services(Full Layer 3)

Layer 2 Port Security

802.1x Security Features

IEEE 802.1AE MACsec Roadmap(256 Bit)

TrustSec SXP Roadmap

Dynamic/Downloadable ACLs

TrustSec SGT/SGACL Roadmap

Full Flexible Netflow

Secure Boot

FIPS 140-2 Compliant CGS2520 CGS2520 Roadmap

Apps of IoX Roadmap IE4000 IE4000

Cisco IE2000Cisco IE2000U / IE3000 /

IE3010 / CGS2520Cisco IE3200 /

IE3300 / IE3400Cisco IE4000 / IE4010 / IE5000

Full support No supportLegend:


Cisco CatalystIE3200, 3300, 3400 Rugged Switches

‘*’ – Post FCS

Fixed System

Expandable modular system

Feature-packed modern software for scalable

IoT deployments

• Flexible, resilient, secure Cisco® IOS XE

operating system

• Simplified management, automation, and

visibility IND, Cisco DNA Center, Prime®,

WebUI

• Rich IE features – PRP*, HSR*, MRP*, PTP,

MACSEC*, TSN*, CIP, Profinet*

• Flexible licensing options:

• Network Essentials comes as PIK-PAK

• Cisco DNA Essentials*

• Network Advantage, and Cisco DNA

Advantage (post-FCS)*

Gigabit modular system


IE3x00 platforms at a glance

PositioningLow port count, low power, Network

Essentials features

IE 3000 transitionHigh port count, Cisco DNA Essentials,

or Cisco DNA Advantage features

Advanced features, high port countHigh port count

FCS features

• Layer 2• Fixed: 10 x 1GE ports• PTP, REP, • PoE/PoE+

• Layer 2• Modular – 26 x1GE ports• PTP, REP, Netflow• PoE/PoE+

• Layer 2• Modular –26 x 1GE ports• PTP, REP, Netflow

Post-FCS features

• Profinet, MRP• Macsec• Cisco DNA Essentials

• Layer 3• Profinet, MRP, L2NAT• Macsec• Cisco DNA Essentials, Cisco DNA

Advantage• SDA Extended Node

• Layer 3• Profinet, MRP, HSR, PRP, L2NAT• Macsec, SGT, SGACL• Cisco DNA Essentials, Cisco DNA

Advantage• SDA Extended Node, SDA Fabric

Edge• TSN• Cisco® IOx

IE3200

Fixed Basic

IE3300

Modular Basic

IE3400

Modular Advanced


Systems and modules at a glanceHighly flexible architecture with a wide array of module choices

IE3200fixed system

IE3300, IE3400expandable systems

IEM3300, IEM3400 expansion modules

Note: No support for Expansion modules

• IE3200 copper fixed

• IE3200 PoE + fixed

• IE3300 copper basic modular system

• IE3300 POE+ basic modular system

• IE3400 advanced modular system

2p SFP and 8p Cu 8p Cu 2p Fi + 6p Cu 16p Cu 8p Fi 8p Fi2p Fi + 14p Cu

• IEM-3300 8p copper

• IEM-3300 8p PoE+

• IEM-3400 Adv copper

• IEM-3300 6p copper + 2p fiber mixed

• IEM-3300 16p copper

• IEM-3300 16p PoE+

• IEM-3300 14p copper + 2p fiber mixed

• IEM-3300 8p fiber

• IEM-3400 Advanced 8p fiber

Note: IEM-3400 expansion modules only work with IE3400 base


Cisco Catalyst IE3400 Heavy Duty Switches

‘*’ – Post FCS

Feature-packed modern software for scalable IoT

deployments

• Water and dust resistant

• Very rugged resilient, secure Cisco® IOS XE

operating system

• Simplified management, automation, and visibility

IND, Cisco DNA Center, Prime®, WebUI

• Rich IE features – PRP*, HSR*, MRP*, PTP,

MACSEC*, TSN*, CIP, Profinet*

• Flexible licensing options:

• Network Essentials comes as PIK-PAK

• Cisco DNA Essentials*

• Network Advantage, and Cisco DNA

Advantage (postFCS)*

Heavy Duty Networking


IE-3400H Product family – FCS Aug 2019

IE-3400 products IDs (PIDs) will have either ‘-E’ or ‘-A’ suffix:• Network Essentials “-E” license, or (Network Essentials comes as default License from Mfg.)• Network Advantage “-A” licenseA customer must have a Cisco® Smart Account for “-A”• A customer must have a Cisco® Smart Account for “-A”

Base Product ID # Ports Port Speed

IE-3400H-8FT 8-M12 copper 1/100

IE-3400H-8T 8-M12 copper 10/100/1000

IE-3400H-16FT 16-M12 copper 10/100

IE-3400H-16T 16-M12 copper 10/100/1000

IE-3400H-24FT 24-M12 copper 10/100

IE-3400H-24T 24-M12 copper 10/100/1000


IoT Industrial Switching portfolio

‘*’ –Selected Models

IE 4010IE 4000

10/100M

Featu

re

1G 10G

IE 5000

IE 3010CGS 2520IE2000UIE 2000

• Designed for all industries

• Layer 2 or 3 (IP service)

• 4 10 GE* uplinks• 24 GE downlinks• IEEE1588 PTP

(default and powerprofiles)

• Layer 2 NAT• Up to 12

PoE/PoE+• Dying gasp• Cisco TrustSec

SGT/SGACL• MACSec• FNF • TSN-ready• Stacking*• Conformal

coating*• Iox-ready• MRP, REP, PRP• HSR• Timing interfaces

(IRIG-B, GPS)• Cisco DNA

Essentials/Advantage

• L2 or L3 (IP lite)• Small form factor• IP30, IP67• MRP, REP • Layer 2 NAT• IEEE1588 PTP• Up to 8

PoE/PoE+ ports• Conformal

coating *• Cisco DNA

Essentials

• L2 or L3 (IPservices)

• Small form factor• PRP, REP• IEEE 1588 PTP

(default and power profiles)

• Up to 4PoE/PoE+ ports

• Conformal coating *

• L2 or L3 (IP services)

• 1 RU• 2 GE uplink

ports• 24 FE downlink

ports• REP• 8 PoE/PoE+

ports, 16 SFP, or 24 copper

• IEEE 1588 PTP (default and power profiles) *

• For all industries• Layer 2 or 3

(IP service)• 4 GE uplinks• Up to 20 GE

ports• IEEE1588 PTP

(default andpower profiles)

• Layer 2 NAT• Up to 8

PoE/PoE+• Dying gasp• Cisco TrustSec®

SGT/SGACL• MACSec, FNF• Time-Sensitive

Network (TSN)• IOx• MRP, REP, PRP• HSR• Cisco DNA


• For all industries• Layer 2 or 3

(IP service)• 4 GE uplinks• 28 total GE

ports• IEEE1588 PTP

(default andpower profiles)

• Layer 2 NAT• Up to 12 or 24

PoE/PoE+• Dying gasp• Cisco® TrustSec

SGT/SGACL• MACSec • TSN-ready• Iox-ready• MRP, REP, PRP• HSR• Cisco DNA


Best in class

AggregationAccess

IE3300IE3200

• Layer 2 • 2 GE uplinks• 8 GE downlinks• Up to 8 PoE/PoE+

ports• REP• IEEE1588 PTP• MacsecRoadmap• Profinet, MRP• Cisco DNA

Essentials

• Layer 2• 2 GE uplinks• Up to 24 GE ports• Up to 24

PoE/PoE+ ports• FNF, REP• IEEE1588 PTP• Layer 2 NAT,• MACSecRoadmap• Layer 3• Profinet• MRP• Cisco DNA

Essentials• Cisco DNA

Advantage

IE3400

• Layer 2• 2 GE uplinks• Up to 24 GE ports• FNF, REP• TrustSec®

SGT/SGACL• IEEE1588 PTP• Layer 2 NAT,• MACSecRoadmap• Layer 3• Profinet• MRP, PRP, HSR• IOX• TSN• SDA FE• Cisco DNA Essentials• Cisco DNA

Advantage


Cisco Industrial Network Director

Native industrial protocol support

Plug-and-play day-0 configuration

Dashboard for monitoring alarms, system health, and traffic statistics

APIs for integration with automation systems and security platforms

Network Management made

Simple for OT


Industrial Network Director

• Empower operations team with real-time maps of automation device connectivity for increased plant floor asset visibility

• Simplify troubleshooting for technicians by generating network information in the context of the automation process

• Rapid integration of network information with existing automation applications, tools, and processes through Open APIs

• Deliver common information framework shared by operations and Plant-IT to manage the industrial network

PAC$/$PLCHMI

Machines

IO$&$Sensors

Drives,$motors,$Actuators

Switching

Wireless Routing

Security

Compute

Industrial Network Director

Manage Network(Network Protocols)

Discover AutomationAssets (Industrial Protocols)

Manage Automation

Partial Network

Integration

Automation & ControlApplications

RESTful API

Control Engineer View Operator View IT / Technician View


FeatureHighlights

OT intent driven security workflows with Cisco ISE, and anomaly detection with Cisco Stealthwatch integration

Dynamic topology of Industrial and Network assets, with support for (DLR) Device Level Rings

CIP, PROFINET, Modbus, BACnet, Siemens S7, OPC-UA industrial device discovery

Switch configuration backup, compare, and restore, and switch IOS software upgrade

Switch monitoring and troubleshooting with alarms

Rich APIs for rapid integration with industrial applications

Detailed audit trails to track adds, moves, and changes

Plug-and-play server for zero-touch switch commissioning

Bridge across PLC backplane to discover devices behind it


IO

PLC

DRIVE

CONTROLLER

ISE

pxGrid

IT / Security

Validated extended enterprise designEnabling IT-OT partnership to secure the OT network

Cyber VisionIND

Modbus

CIP

PROFINET

BACNet

Operational Environment

V I S I B I L I T Y

IE Switching

NGFW

Stealthwatch

SGACL Segmentation

Context based Host Groups

C O N T E X T

SGTdACL

C O N T E X T

C O N T E X T

SXP SGT Firewall Rules


Cisco Industrial Ethernet switching portfolioDesigned for industrial IoT

Innovation

Industrialprotocols

Management and automation

IE switching and security

IOxTSN

Industrial Network Director Device Manager

OT IT

Cisco DNACenter

Prime® Infrastructure

IE 1000 IE 2000 IE 3000 IE 3010 IE 5000IE 2000 (IP67) IE 4000 IE 4010IE3x00


$0.4 M $0.6 M

Reduced Energy Costs

$0.4 M $0.6 M

Reduced Maintenance Load

$10.4 M $13.9 M

Increase Machine Availability

$0.9 M $1.2 M

Reduce Labor Costs

$0.5 M $0.6 M

Reduce Scrap Costs

$12.5 M $16.9 MPotential Annual Benefits:

Conservative Estimate

Likely Scenario

Multivariate analysis using customer KPI from CAPEX, OPEX, OEE, sales, maintenance, TCO and more provide benchmarkable ROI and improvement forecast figures.

Benefits Summary: ExampleDelivering Tangible Value


Detect anomalies, block threats, ID

compromise hosts

Secure third-party access with control

and visibility

Reduce risk, design, deploy and respond to

incidents while protecting the business

• IIoT changed type, amount and type of the communication of entities as sender, recipient, or actor

• Communication of machines independently of exposed interfaces, or protocol and payload type

• Unidirectional, bidirectional, or multidirectional communication• Provide data from originators to a consumer, at the right time

and format, securely and scalable

Security - OverviewChallenges

Security

Scalability

Resiliency

Performance

Flexibility

Reusability

Extensible, scalable segmentation to protect

IoT devices

Remote AccessRemote Access Visibility & Analysis Security Service

NGFWISE / TrustSec AnyConnect

AMPCybervision

UmbrellaStealthwatch

ISE / TrustSecCognitive Threat Analysis

DesignRisk Assessment

Incident Response


Security – Cost and ImpactRetrofitting security to an existing architecture is complex and costly

OT/IT Security• IT Security in focus when designing a new architecture• OT Security often added after service, or functionally provided

Costs for IIoT are multifaceted and interdependent• Time to invest at the expense of the point in time of functional availability• Complexity to invest at the expense of operability, maintainability and risk to fail• Manpower to invest at the expense of OPEX• Financial budget in CAPEX and/or OPEX

Security has to embrace all architectural components and must include architectural delineation, monitoring and visibility, data security, device and communication security, secure administrative access and services with deployable components.

Neglecting on security aspects comes at the cost of immense risk for business safety.


SecurityKey Aspects

Environmental IntegrityOperational safety requires control of entities sending/receiving data, entities stopping communication and new entities joining network and communication.

CommunicationSecure communication between entitled participants following the minimum-principle.

Data IntegrityStandards require proof for untampered data (G10, PIPEDA, GDPR, or GxP)

Encryption, checksums for data and virtual sensors for plausibility checks.

Apply Purdue model for segmentation and zoning. Prevent unauthorized access to devices, data exposure and misuse of execution layer by using access profiles to devices and applications.

Apply segmentation and isolation techniques do data.

Operational overview, monitoring and transparency, automated access control provisioning, unique and immutable digital identities, isolation and protection of trustworthy and non-trustworthy compute base

EncryptionSemantic access to data for entitled participants only.

Apply encryption to data in-transit, at-rest and in-memory. Use encrypted network tunnels to communicate to skids and remote entities. FIPS 140-2 defines the security standards, that will be satisfied by encryption and helps to rank and scale an implementation.


1. Develop Plant or Manufacturing Security Policies—Most enterprises have IT Security policies. These policies drive the behavior, processes and awareness of all enterprise network users Plant networks have distinctly different requirements and priorities. Therefore, a specific Plant Security policy should be developed and put into place.

2. Use IT-approved user access and authentication policies and procedures—Access to enterprise and plant resources and services should be monitored and logged. Every user must be a known entity to the organization and use a unique account. Unfortunately, these are typically based on users entering account and passwords and having certificates available. IACS devices are often not capable of any of these and are therefore not authenticated when connected to the network. Thus the following are important.

3. Strong Physical Security of Network Infrastructure – Access to Plant network infrastructure should be limited. Switches are typically installed in locked or hard to reach locations. Unused ports are turned off or even blocked. Specific ports for appropriate personnel are clearly marked and authentication policies are applied to them.

4. Endpoint Hardening – Antivirus applications, regularly deploying security updates and turning of or removing unnecessary applications and services on systems with common operating systems are considered best practices

5. Keep industrial Ethernet protocols at home—Industrial Ethernet network protocols, such as CIP and others, shall be contained to the Manufacturing zone. These protocols tend not to include enough security considerations, such as encryption or authorization, to be opened to generally available networks. They were designed to run in segmented networks where trust is implicit based on tight physical control of the network.

SecurityGeneral Security Guidelines


6. Control the applications—As a best practice, partners and remote engineers should use versions of IACS applications on controlled application servers when accessing the IACS remotely. This suggests creating remote access servers within the Manufacturing zone, on which the appropriate IACS applications are executed. 2012 ODVA Industry Conference 14 ©2012 ODVA, Inc.

7. Don't allow direct traffic—It is recommended that no direct traffic is permitted between the Enterprise zone (including the Internet) and the Manufacturing zone. The plant firewall acts as a proxy between remote users or applications and target IACS applications in the Manufacturing zone. The firewall also strictly polices the traffic into and out of each zone.

8. Create only one path in or out—The path from the DMZ through the lower firewall (or firewall instance) into the Manufacturing zone should be the only path in or out of the Manufacturing zone.

9. Protecting the Interior—Plant networks tend to be stable. With appropriate assistance, the network can be configured to limit traffic flows through the use of access control lists (ACLs).

10.Domains of Trust—Users should segment the network into smaller areas (VLANs) based on function or access requirements. These then form the basis on which to manage traffic flows, drastically simplifying application of additional Security functions.

SecurityGeneral Security Guidelines


Assessment1. Implementation Guidelines, Implementation Recommendations2. Evaluate on current and future capacity requirements, throughput/latency, scaling, growth3. Document requirements for

• VLAN/segmentation, addressing schema and trust marking of apps, connectivity between Cells• Remote Access• Services like SNMP, NTP, Netflow, FactoryTalk, STP, Quality of Service• Minimal invasive asset reconfiguration using Cell Layer 2 NAT• IT, physical and plant security policies

4. Decide on network topology, reassess wiring schema

Hardware5. Order hardware from CCW

Best Practices

Step 01 Pre-Deployment

Configuration Template6. Create versioned device configuration templates

Deployment7. Deploy Cisco Industrial Network Director/Prime Infrastructure, or8. Integrate configuration management and monitoring into existing solutions

Step 02 Deployment

Hardware9. Wiring as per 1.5 decision10. Device installation as per 1.5 decision

Configuration11. Configuration of devices as per 1.3-1.6

Step 03 Management

Supervise12. Diagnose connectivity, configurational changes, usage of the network13. Generate reports and receive alerts

Step 04 Monitoring


1. Develop Plant or Manufacturing Security Policies—Most enterprises have IT Security policies. These policies drive the behavior, processes and awareness of all enterprise network users Plant networks have distinctly different requirements and priorities. Therefore, a specific Plant Security policy should be developed and put into place.

2. Use IT-approved user access and authentication policies and procedures—Access to enterprise and plant resources and services should be monitored and logged. Every user must be a known entity to the organization and use a unique account. Unfortunately, these are typically based on users entering account and passwords and having certificates available. IACS devices are often not capable of any of these and are therefore not authenticated when connected to the network. Thus the following are important.

3. Strong Physical Security of Network Infrastructure – Access to Plant network infrastructure should be limited. Switches are typically installed in locked or hard to reach locations. Unused ports are turned off or even blocked. Specific ports for appropriate personnel are clearly marked and authentication policies are applied to them.

4. Endpoint Hardening – Antivirus applications, regularly deploying security updates and turning of or removing unnecessary applications and services on systems with common operating systems are considered best practices

5. Keep industrial Ethernet protocols at home—Industrial Ethernet network protocols, such as CIP and others, shall be contained to the Manufacturing zone. These protocols tend not to include enough security considerations, such as encryption or authorization, to be opened to generally available networks. They were designed to run in segmented networks where trust is implicit based on tight physical control of the network.

1.1 Implementation General Security Guidelines


6. Control the applications—As a best practice, partners and remote engineers should use versions of IACS applications on controlled application servers when accessing the IACS remotely. This suggests creating remote access servers within the Manufacturing zone, on which the appropriate IACS applications are executed. 2012 ODVA Industry Conference 14 ©2012 ODVA, Inc.

7. Don't allow direct traffic—It is recommended that no direct traffic is permitted between the Enterprise zone (including the Internet) and the Manufacturing zone. The plant firewall acts as a proxy between remote users or applications and target IACS applications in the Manufacturing zone. The firewall also strictly polices the traffic into and out of each zone.

8. Create only one path in or out—The path from the DMZ through the lower firewall (or firewall instance) into the Manufacturing zone should be the only path in or out of the Manufacturing zone.

9. Protecting the Interior—Plant networks tend to be stable. With appropriate assistance, the network can be configured to limit traffic flows through the use of access control lists (ACLs).

10.Domains of Trust—Users should segment the network into smaller areas (VLANs) based on function or access requirements. These then form the basis on which to manage traffic flows, drastically simplifying application of additional Security functions.

1.1 Implementation General Security Guidelines


• Use standard enterprise remote access solutions in the form of client-based, IPsec encryption VPN technology to connect to the enterprise edge and for confidentiality over the Internet. The establishment of a VPN requires RADIUS authentication of the remote person and is typically implemented and managed by the IT organization.

• Limit access of remote partners connecting via IPsec to plant floor DMZ/firewalls using ACLs. Connect to the plant floor DMZ through a secure browser Hypertext Transfer Protocol Secure (HTTPS) only.

• Access a secure browser (HTTPS) portal application running on the DMZ/firewalls. This requires an additional login/authentication.

• Use a Secure Socket Layer (SSL) VPN session between the remote client and the plant DMZ firewall and restrict application usage to a remote terminal session (e.g. Remote Desktop Protocol) over HTTPS.

• Utilize intrusion detection and prevention systems (IPSs/IDSs) on the firewall to inspect traffic to and from the remote access server for attacks and threats, and appropriately stop them. This is important to prevent viruses and other security threats from remote machines from traversing the firewall and impacting the remote access server.

• Allow the remote user to execute, via the terminal session, a selected set of automation and control applications that reside on the remote access server. Application-level login/authentication is required.

• Implement application security that restricts users from the remote access server to a limited set of application functions (such as read-only, non-line-of-site functions).

• Segment the remote access server on a separate VLAN and have all traffic between the remote access server and the manufacturing zone go back through the firewall. Apply intrusion protection and detection services to this traffic to protect the manufacturing zone from attacks, worms, and viruses.

1.1 ImplementationRecommendations


1.2 Understand your application requirements

IACS networks differ significantly from their IT counterparts in their need to support real-time communications, which includes:

• Communicating messages with minimal latency (time delay between message sent and message received)

• Jitter (the variance of the latency), significantly lower than typical Enterprise applications.

• Real-time communications help the IACS become more deterministic.

38

• IACS networks have different real-time communications requirements based on the type of application

• Determining the right access resiliency protocol is based on industrial application requirements

• Following two slides cover application, resiliency and protocol options


1.2 Performance RequirementsIndustrial Automation & Control System Applications

Process Automation Discrete Automation Loss Critical

FunctionInformation Integration,Slower Process Automation

Time-criticalFactory Automation

Multi-axis Motion Control

Comm. Technology

.Net, DCOM, TCP/IP Industrial Protocols, CIP, ProfinetHardware and Software solutions, e.g. CIP Motion, PTP

Period 1 second or longer 1 ms to 100 ms 100 µs to 10 ms

IndustryOil & Gas, chemicals, energy, water

Auto, food and bev, electrical assembly, semiconductor, metals, pharmaceutical

UtilitiesSubset of Discrete automation

ApplicationsPumps, compressors, mixers; monitoring of temperature, pressure, flow

Material handling, filling, labeling, palletizing, packaging; welding, stamping, cutting, metal forming, soldering, sorting

Life/equipment safety, Synchronization of multiple axes: printing presses, wire drawing, web making, picking and placing


1.2 Network Resiliency ProtocolsSelection is Application Driven

Resiliency Protocol Mixed Vendor

RingRedundant

StarNet Conv >250 ms

Net Conv50-100

ms

Net Conv< 0~10 ms

Layer 3 Layer 2

STP (802.1D)

RSTP (802.1w)

MSTP (802.1s)

PVST+

REP

EtherChannel(LACP 802.3ad)

MRP (IEC 62439-2)*

Flex Links

PRP/HSR (IEC 62439)*

DLR (IEC & ODVA)

StackWise

HSRP

VRRP(IETF RFC 3768)

Process and Information

Time Critical

Loss Critical


1.3.1 Segmentation using VLANsRecommendations

• Segment the IACS network into Cell/Area zones, where each zone is a subset of devices that communicate consistently with each other.

• All devices should have an IP address in the same IP subnet and be in the same VLAN It is recommended that Cell/Area zones stay under the Class-C size subnet, therefore less than 250 devices.

• All devices communicating with each other via multicast (I/O) traffic must be in the same VLAN.

• Layer-3 switches or routers are required in order to route traffic between VLANs, which may impact traffic flow.

• Configure the native VLAN to be a dedicated and specific VLAN not already in use (for example as the IACS Cell/Area Zone VLAN). The native VLAN should not be routed to or from, and therefore, it is never enabled on the router or Layer-3 aggregation switch and therefore not reachable outside of network infrastructure devices. No industrial Ethernet traffic should flow in the native VLAN.

• Each VLAN should consist of a single IP subnet.

41


1.3.1 Segmentation using VLANs (continued)Recommendations

• If non-manufacturing traffic (PC and so-on) must exist in the physical topology, it should be on a separate VLAN.

• Configure VLAN Trunking Protocol (VTP) mode as transparent in order to avoid operational error because very few VLANs are used.

• Assign all end-device or host ports a VLAN and set to switchport mode access.

• Do not explicitly use VLAN 1, as it is easily misused and can cause unexpected risks.

• All uplinks are connected as 802.1Q trunks.

• Use an unused VLAN as the native VLAN on all trunk ports.

• Prune all unused VLANs from a trunk.

42


1.3.1 Segmentation using VLANs

• A single flat network can become un-manageable and very hard to mange. A faulty device attached to the network can send un-necessary broadcasts that would create un-necessary traffic.

• It will be very hard to isolate network failures in a flat network.

• An infected device can scan and reach all the devices in the flat network.

• Segmentation is a process of breaking a large network into smaller units.

• Segmentation using VLANs limits where the broad cast traffic is sent.

• The new networks can be added or removed without impacting the current network.

• A network admin will be able to identify and trouble shoot different parts of the network.

43


• Disable all unused ports and put them in an unused VLAN. Any enabled open port provides an access medium into the network.

• Do not use VLAN 1 for anything. VLAN 1 is the default and is enabled on all ports by default; it is a security best practice to configure all the ports on all switches to be associated with VLANs other than VLAN 1.

• To assist with preventing VLAN hopping attacks, whereby an end station can spoof as a switch, configure all user-facing ports as non-trunking. This prevents the port from going into trunking mode unless explicitly configured.

• Force tagging for the native VLAN on trunks and drop untagged frames to assist with preventing VLAN hopping.

• Explicitly configure trunking on infrastructure ports. Explicitly configure only the VLANs required to be extended to other switches.

1.3.1 Segmentation using VLANs (continued)


1.3.1 Enable Classification and MarkingTrust Boundaries

• Cisco QoS allows classification and marking of different applications at the IE Switch

• Classification provides high priority to critical application traffic

• Cisco QoS performance is at wire speed

Machine

Distribution

QoS Marking and Classification


• Dedicated out-of-band network should be deployed throughout the plant including the IDMZ.

• Least privilege access and AAA: Critical in order to secure interactive access to network devices, provides a highly configurable environment that can be tailored based on the needs of the network.

• Configure Infrastructure Access Control Lists: Prevent unauthorized direct access to network devices; infrastructure access control lists (iACLs) prevent access to the network devices.

• Configure secure networking protocols for access to the networking equipment such as SSH in place of telnet, SNMP v3.

• Network system logging should be enabled throughout the architecture.

• Backup network device configuration after initial installation and post modifications.

1.3.1 Enable Management Plane Protection


1.3.1 Enable Control Plane Protection

Layer 2 IE Switches Layer 3 IE Switches

• Control plane policing and protection where possible

• Protect Layer 2 Protocol integrity:

• BPDU Guard

• Root Guard

• Neighbor authentication

• Routing Peer Definition

• Control Plane Policing/Protection


• Shut down any unused ports: Place port in an unused VLAN to protect against accidental activation

• Implement Port Security

• DHCP Snooping: If servers or workstations in the architecture are using DHCP, then DHCP snooping and Dynamic ARP Inspection (DAI) should be considered.

• Traffic Storm Control

1.3.1 Enable Data Plane Protection


Cisco IE switches are in-built with many trustworthy technologies such as:

1.3.1 In-Built Trustworthy Technologies

Cisco Trustworthy Technologies helps prevent attacks such as hardware tampering, un-authorized software loading. In addition, they also provide unique identity and stronger cryptography

Trust Anchor Module (TAM)

SUDI

Secure Storage

Secure Boot


1.3.1 Enable Segmentation using Cisco TrustSec

• Cisco TrustSec enables assignment of Secure Group Tag (SGT)s to IACS devices attached to the IE switch.

• The traffic generated by the IACS device is marked with the SGT and this information is carried through out the network.

• In the diagram, the IACS devices are given different SGT tags

• The network admin can apply policy based on the SGT tags which implies policy enforcement.

Machine

Distribution

SGT10

SGT20SGT30

SGT40


• Cisco Remote Access allows an expert to remotely access the IACS devices securely.

• In addition, this solution provides control to the OT engineer in providing the access.

• A remote access user from the Internet or Enterprise Zone establishes a VPN connection o the VPN gateway.

• Then the remote user connects to a remote desktop server.

• The remote desktop server connects to a terminal server, which hosts all the applications that are required to manage IACS assets.

• The remote user connects to an asset (for example, PLC) from the terminal server.

• The distribution switch has SGAL that allows or disallows communication to an ICAS asset.

• This control is managed by an OT engineer.

1.3.2 Allow secure Remote Access

Machine

Distribution

SGT10SGT20SGT30

SGT40

Remote user AnyConnect

Internet

ISE PSN

pxGRID

Terminal server with IACS assets

management software loaded

IND


1.3.2 Access to plant floor via IDMZ

Permit Secure Remote

Access to Industrial Assets

Permit Data from the

Industrial Zone to

Enterprise Stakeholders

Firewalls(Active/Standby)

MCC

Enterprise ZoneLevels 4-5

IO

Level 3Site Operations

Drive

IndustrialDemilitarized Zone(IDMZ)

Industrial ZoneLevels 0-3

FactoryTalk Client

WGB

WLC (Active)

ISE

WLC (Standby)

LWAP

PACPAC

PAC

Levels 0-2 Cell/Area Zone

Core switches

Distribution switch

Core switches

WLC (Enterprise)

ISE (Enterprise)

Physical or Virtualized Servers• FactoryTalk Application Servers &

Services• Network Services – e.g. DNS,

AD, DHCP, AAA• Call Manager• Storage Array

Remote Access Server VantagePoint

Plant Manager

RemoteAccess

Untrusted

Untrusted

Block

Block

Permit

Remote Desktop Gateway

Permit

WebReports

Web Proxy

Firewall (Inspect Traffic)

Physical or Virtualized Servers• Patch Management• AV Server• Application Mirror• Remote Desktop

Gateway Server

Wide Area Network (WAN)Physical or Virtualized Servers• ERP, Email• Active Directory (AD),

AAA – Radius• Call Manager

Firewall (Inspect Traffic)

Block Untrusted

Access to Industrial

Zone

Block Untrusted Access to Enterprise Zone

Engineer

Access IE2K / IE3X / IE4K


1.3.3 Enable NetFlow Overview

Machine

Distribution

SGT10

SGT20SGT30

SGT40

Flow collector

Management console

• NetFlow is embedded instrumentation within Cisco software to characterize network operation.

• It provides visibility into the data flows through a switch or router. Enabling NetFlow provides a trace of every data conversation in the network without needing any SPAN ports.

• Cisco Stealthwatch uses NetFlow data to detect any malicious, abnormal behavior in the network.


1.3.4 Advantages of Layer 2 NAT

Allows a single device to act as an agent between the Plant (Outside) network and the Machine (Inside) network

• Helps simplify integration of IP address mapping from a machine level IP addresses to the plant network

• Allows Machine Builders to develop standard machines and eliminate the need for unique IP addressing and code modifications

• Allows End Users to more easily integrate machines into their larger plant network without extensive coordination with machine builders

• Provides better maintainability at the machines as they remain standard

• Allows for reuse of IP addresses allowing for more connected devices in a limited address pool.


Machine

IE2K / IE4K

Inside Address 192.168.1.10

Inside VLAN 10

1.3.4 Layer 2 NAT Design ScenariosSingle-Cell, Single VLAN per Switch

Line Controller 10.10.10.30

Inside to Outside NAT Table

Inside Outside

192.168.10 10.10.10.10

Outside to Inside NAT Table

Outside Inside

10.10.10.30 192.168.1.30

Trunk

VLAN10 IE 5K Distribution Switch

Outside


1.3.4 Enable Layer 2 NAT

• Layer 2 NAT is (1:1) a service that allows the assignment of a unique public IP address to an existing private IP address (end device), so that the end device can communicate on both the private and public subnets.

• Layer 2 NAT is a useful feature for a plant operator to use when the IP addresses of the IACS devices in the machine can’t be changed.


1.3.5 Switch Security Recommendations The Layer-2 access switch can play an important role in security as a port of entry to the Manufacturing and Cell/Area zones. Some key considerations when selecting network infrastructure equipment include the following:

Data Plan Protection Control Plane Protection

• Shut down any unused ports: Place port in an unused VLAN to protect against accidental activation

• Implement Port security via MAC address identification or physical barrier to the port

• MAC filtering and address notification

• DHCP Snooping: If server or workstation in the architecture are using DHCP, then DHCP snooping and Dynamic ARP Inspection (DAI) should be considered

• Traffic Storm Control

• Implement QoS trust boundaries to separate trusted and untrusted devices

• Enable control plane policing and protection where possible

• Protect Layer 2 Protocol integrity using BPDU guard and Root guard

01 02


• Dedicated out-of-band network should be deployed throughout the plant including the IDMZ.

• Least privilege access and AAA: Critical in order to secure interactive access to network devices, provides a highly configurable environment that can be tailored based on the needs of the network.

• Configure Infrastructure Access Control Lists: Prevent unauthorized direct access to network devices; infrastructure access control lists (iACLs) prevent access to the network devices.

• Configure secure networking protocols for access to the networking equipment such as SSH in place of telnet, SNMP v3.

• Network system logging should be enabled throughout the architecture.

• Backup network device configuration after initial installation and post modifications.

1.3.5 Switch Security Recommendations

Management Plane03


1.3.5 Advanced Security using NetFlow

• NetFlow is embedded instrumentation within Cisco software to characterize network operation.

• Most Cisco IE switches supports NetFlow

• NetFlow provides visibility into the data flows through a switch or router. Enabling NetFlow provides a trace of every data conversation in the network without needing any SPAN ports.

• Cisco Stealthwatch uses NetFlow data to detect any malicious, abnormal behavior in the network.

Machine

Distribution

SGT10

SGT20SGT30

SGT40

Flow collector

Management console


• Device level segmentation helps an administrator to design and implement Network Access Control (NAC)

• Cisco TrustSec enables assignment of Secure Group Tag (SGT)s to IACS devices attached to the IE switch.

• The traffic generated by the IACS device is marked with the SGT and this information is carried through out the network.

• In this diagram, the IACS devices are given different SGT tags

• A network administrator can apply policies based on the SGT tags which implies policy enforcement using Cisco Identity Services Engine (ISE)

1.3.5 Advanced SecuritySegmentation using Cisco TrustSec

Machine

Distribution

SGT10

SGT20SGT30

SGT40


1.3.5 Cisco IE Switch Trustworthy Technologies

Cisco IE switches are in-built with many trustworthy technologies such as:

Cisco Trustworthy Technologies helps prevent attacks such as hardware tampering, un-authorized software loading. In addition, they also provide unique identity and stronger cryptography

Trust Anchor Module (TAM)

SUDI

Secure Storage

Secure Boot


1.5 IACS Access Topology Options ConsiderationsKey considerations include the following:

Physical layout — The layout of the manufacturing environment is a key driver of topology design. For example, a long conveyor belt system does not easily lend itself to a redundant star configuration, but rather a linear or ring topology.

Availability — Cisco recommends using resilient network topologies (for example, redundant star and ring) over non-redundant topologies. These allow the network to continue to function after an event such as connection loss or switch failure. Although some of these events may still lead to downtime of the IACS, a resilient network topology may reduce that chance and should improve the recovery time

Real-time communications — Latency and jitter are impacted by a large variety of factors, but primary by the amount of traffic and number of hops a packet must make to reach its destination. The amount of traffic in a Layer-2 network is driven by various factors, but the number of nodes is important.

Key guidelines include the following:

• Amount of latency introduced per switch.

• Bandwidth should not consistently exceed 50 percent of the interface capacity on any switch

• Switch CPU should not consistently exceed 50 to 70 percent utilization. Above this level, the chances increase significantly that the switch may not properly process control packets and start behaving abnormally


1.5 Cell/Area Zone Access TopologiesDeployment Options

Linear Ring Redundant Star

Cabling Requirements

Ease of Configuration

Implementation Costs

Bandwidth

Redundancy and Convergence

Disruption During Network Upgrade

Readiness for Network Convergence

Overall in Network TCO and Performance Worst Ok Best

PLC HMI

I/O

Drive

I/O

HMI

HMII/O Drive

Controller


LinksReferenced Name External Link

Cisco INDhttps://www.cisco.com/c/en/us/products/cloud-systems-management/iot-field-network-director/index.html

Cisco IE1000https://www.cisco.com/c/en/us/products/switches/industrial-ethernet-1000-series-switches/index.html

Cisco IE2000https://www.cisco.com/c/en/us/products/switches/industrial-ethernet-2000-series-switches/index.html

Rockwell Stratix https://www.rockwellautomation.com/de_DE/products/industrial-networks-products/overview.page?

Allen Bradley https://ab.rockwellautomation.com/de/

Cisco Top 10 Design Guide: Plant-Wide-Eth

https://www.cisco.com/c/dam/en_us/solutions/industries/docs/manufacturing/top_10_recommendations_plantwide_ethernet_deployments.pdf

Cisco CVD Converged Plant-Wide Eth

https://www.cisco.com/c/en/us/solutions/enterprise/design-zone-manufacturing/landing_ettf.html

Cisco Industrial Automation Design Guide

https://www.cisco.com/c/en/us/td/docs/solutions/Verticals/Industrial_Automation/IA_Horizontal/DG/Industrial-AutomationDG/Industrial-AutomationDG.html

Cisco and Rockwell https://www.cisco.com/c/en/us/solutions/industries/manufacturing/rockwell-automation.html

https://www.cisco.com/c/en/us/products/cloud-systems-management/iot-field-network-director/index.html

https://www.cisco.com/c/en/us/products/switches/industrial-ethernet-1000-series-switches/index.html

https://www.cisco.com/c/en/us/products/switches/industrial-ethernet-2000-series-switches/index.html

https://www.rockwellautomation.com/de_DE/products/industrial-networks-products/overview.page?

https://ab.rockwellautomation.com/de/

https://www.cisco.com/c/dam/en_us/solutions/industries/docs/manufacturing/top_10_recommendations_plantwide_ethernet_deployments.pdf

https://www.cisco.com/c/en/us/solutions/enterprise/design-zone-manufacturing/landing_ettf.html

https://www.cisco.com/c/en/us/td/docs/solutions/Verticals/Industrial_Automation/IA_Horizontal/DG/Industrial-AutomationDG/Industrial-AutomationDG.html

https://www.cisco.com/c/en/us/solutions/industries/manufacturing/rockwell-automation.html


Glossary

Acronym Description

OEE Operational Equipment Efficiency is a measure of how well a manufacturing operation is utilized (facilities, time and material) compared to its full potential, during the periods when it is scheduled to run.

RUL Remaining Useful Lifetime is a prediction of the time at which a system or a component will no longer perform its intended function.

IND Cisco Industrial Network Director (IND) is a network management system.

CIPClean-in-Place systems. A methodology of cleansing complex systems without disassembly.

CVD Cisco Validated Designs (CVD) are design guides for the implementation of Cisco technology for specific use cases.

ACME Inc (Inspired by Qualtech) · 2019-10-30 · Prime Infrastructure / DNA-C Prime...

Documents

Transcript of ACME Inc (Inspired by Qualtech) · 2019-10-30 · Prime Infrastructure / DNA-C Prime...