Industrial Compute€¦ · Cisco DNA fabric IOS-XE programmability Software-defined access...
Transcript of Industrial Compute€¦ · Cisco DNA fabric IOS-XE programmability Software-defined access...
ACME Inc. (Inspired by Lockheed Martin)
Industrial Compute
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Table of Content
Background and current State of Company and Architecture
Challenges for IT, OT, and Business and derived Targets
Project details and Technical Approach
Components for Hardware and Management
IIoT and Security, Guidelines
Best Practices
01.
02.
03.
04.
05.
06.
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Current State of the Company
The Security and Aerospace customer, has it’s majority of business with the U.S. Department of Defense and U.S. federal government agencies. The Security and Aerospace customer operates in four business segments: Aeronautics, Missiles and Fire Control, Rotary and Mission Systems, and Space Systems.
In addition, The Security and Aerospace customer provides military and rotary-wing aircraft to all five branches of the U.S. armed forces along with military services and commercial operators in 40 nations.
The remaining portion of The Security and Aerospace customer business is comprised of international government and commercial sales of products, services and platforms.
Industry: Manufacturing
Focus: Aerospace Defense
Main Business:U.S. Department of Defence Government Agencies
Employees: ~ 100.000
Revenue: ~54 bln USD
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Current Architecture of company
The Security and Aerospace customer operates different machines in a dependent, multi-step and cascaded production. The list of machines –among others - include autoclaves, autodrills, mills, tube benders and pallet shuttle systems being operated isolated with their data remaining non-provisioned.
The Security and Aerospace customer operates different, geographically separated production sites. A site is segmented into zones and production cells.
Machine telemetry remains isolated and is accessible by local operators in the segment or the site only.
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Desired Business GoalsThe North Star
ICS Vendor and Machine Builder
• Equipment Health Management to drive preventative, predictive and automated maintenance of OT components in Strategic-Time
• Production Strategy Management by automatically analyzing ProdOps management data, ProdOpsautomation data and product quality data
• Standard Telemetry acquisition from a variety of machines and other data originators for their utilization and availability, downtime and alert categorization, cost calculation to address OEE optimization
• Quality Monitoring by acquiring inspection data, correlate machine alerts to for instance address RUL
• Anomaly and Crash Recognition by implementing a variety of condition based monitoring mechanisms, enabling correlations and event driven activities.
SI
• Extend your service offerings with the latest customer requirements for secure connectivity, real time data monitoring and cybersecurity event detection.
• Provide subscription based services around the data collected and monitored. Cisco provides a fully scalable solution with capabilities to fully manage your deployments remotely.
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
OT BusinessIT
No notification for deviation from normal operational behavior of machine
Data Availability
No possibility of comparative data analytics and visualization for real time and historic data
Data Availability
No comprehensive protection against malware, intrusion, misuse of execution layer and loss of control on ICS
Security
Machines operated isolated in their cell/segment with no secure, scalable connectivity for data provisioning
Connectivity
Multi-dataset solutions required comprehensive and holistic data access for addressing predictive and prescriptive maintenance and operational equipment efficiency
Data Availability
Challenges Identified with the current customer architecture / technologies used
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Architecture map
Industrial automation control system
Industrial DMZ
Enterprise and external
Industrial automation control room
Network management and security
Industrial networking
Machine control systems
Manufacturing applications
Cisco DNA Center
Scripting Open source
Cisco DNA fabric
IOS-XE programmability
Software-defined access
Industrial networking director
Industrial networking (IoT architecture)
EndpointsCisco on-premises and
partner-hosted HCS
Webex Hybrid Services
Intent-based networking
Intent
Infrastructure
Threat defense for IoT devices/
machines
Cisco security for manufacturing
Cell security
Zone security
Plant security
Multi-cloud
Automation
Corporate data center
Data center infrastructure
Factory industrial data
center
UCS
HyperFlex
• Automotive• High tech
• Consumer packaged goods• Food and beverage
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Machines operated in cells/segments are exposing data using the MTConnect standard.
All machines connect to “Mazak Smartbox”, a connectivity solution that integrates Cisco IE 4000. This devices is the basis of the managed connectivity for each machine.
It additionally allows the lifecycle management and operation of standardized and containerized applications using its virtualization layer IOx.
In this architecture, a MTConnect client application is deployed to the IE4000/smartbox, that manages the data acquisition from every connected machine, the convergence of data, as well as the data provisioning to Cisco Kinetic for further data provisioning.
The IE4000/smartbox is managed using Cisco Field Network Director (FND).
Solution Details and DeliveryKinetic Manufacturing Architecture: CNC Machines
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
New Architecture
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Characteristics MQTT COAP DDS OPC UA Modbus Profinet AMQP MTConnect
NameMessage Queuing Telemtry Transport
Constrained Application Protocol
Data Distribution ServiceOpen Platforms Communication, Unified Architecture
Modbus Process Field NetworkAdvanced Message Queuing Protocol
MTConnect
FocusLightweight protocol to minimize resource allocation.
Specialized embedded P2P protocol for small-resource devices.
Middleware and API standard for data-centric connectivity for distributed systems.
M2M protocol / Modeling Architecture, semantic interoperability
Defacto P2P standard industry protocol for device data (SCADA / PLC / RTU)
(N)RT technical standard bus for data collection and device control.
Protocol standard for asynchronous, message-oriented, reliable
Readonly machine-to-application, REST/XML-based protocol for machine telemetry.
ArchitecturePublish/Subscribe, Broker
RESTPublish/Subscribe, global Dataspace
SOA Client/Server, Publish/Subscribe
Client/Server BusPublish/Subscribe, Broker
Client/Server
Structuring Topics Resources Information Model Register, Type GSD, GSDML Topics Schema
Transport Layer TCP UDP TCP / UDP TCP / UDP TCP, Serial TCP, RT, IRT TCP TCP
Quality of Service
• at most once• at least once• exactly once
• confirmable• nonconfirmable
• data availability• ressource usage• traffic priorization
• dependent on transport protocol (AMQP, DDS)
• best effort
• best effort• at least once• exactly once• at most once
• sequenced and queued client data
Security SSL / TLS DTLS TLS / DTLS / DDS Sec UA-SecConversation TLS SSL / TLS SSL / TLS
Standard ISO / OASIS IETF OMG IEC-Norm IEC-Norm IEC-Norm OASIS MTConnect
Featureset• asynchronous• retention• device status
• discovery• asynchronous
• data centric• decentralized• discovery• data priorization
• discovery• informational models• contextualized data
• slave diagnostics• register model, data
tables• polling
• automation apps• discovery• media redundancy• precision time control
• layered architecture• message routing• extensible
• informational models• asynchronous
Context• Standard Service
Protocol• Cloud Interface
• Utilities• Rail• Traffic Mgmt.
• PLC/SPS• SCADA, RTU• Utilities
• Manufacturing• Process Automation
• Standard Service Protocol
• Cloud Interface
• Asset telemetry• CNC machines• CPS in manufacturing
The data exposed by the connected machines is structured, data typed and transported using the MTConnect standard. This standard is being widely adopted and supports plain machine telemetry, as well as command & control. The matrix below compares functionalities and features of MTConnect to other standards.
Communication Standards
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
MTConnect - Abstract
Equipment: Any data source. In the MTConnect Standard, equipment is defined as any tangible property that is used to equip the operations of a manufacturing facility.
MTConnect Agent: Software that collects data published from one or more piece(s) of equipment, organizes that data in a structured manner, and responds to requests for data from client software systems by providing a structured response.
Client Application: Software that requests data from MTConnect Agents and processes that data in support of manufacturing operations.
MTConnect® is a data and information exchange standard that is based on a data dictionary of terms describing information associated with manufacturing operations. The standard also defines a series of semantic data models that provide a clear and unambiguous representation of how that information relates to a manufacturing operation. The MTConnect Standard has been designed to enhance the data acquisition capabilities from equipment in manufacturing facilities, to expand the use of data driven decision making in manufacturing operations, and to enable software applications and manufacturing equipment to move toward a plug-and-play environment to reduce the cost of integration of manufacturing software systems.
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Cisco Industrial Ethernet switching portfolioDesigned for industrial IoT
Innovation
Industrialprotocols
Management and automation
IE switching and security
IOxTSN
Industrial Network Director Device Manager
OT IT
Cisco DNACenter
Prime® Infrastructure
IE 1000 IE 2000
IE 2000UIE 3000
IE 3010
CGS 2520IE 5000IE 2000 (IP67) IE 4000 IE 4010IE3x00
PROFINET Modbus EtherNet/IP CC-Link
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Cisco Industrial Ethernet 4000 Series Switches
Manufacturing CityTransportation
MiningOil & GasEnergy Utility
• 12 models with up to 20 x GE full non-
blocking
• 4 x GE Combo uplink on all models
• PoE/PoE+ density (up to 8)
• Advanced QoS and Security features
• High Resiliency through mulptiple Gigabit
Ethernet rings, MRP, REP, RPR, Flexlink,
redundant power input, dying gasp
• Trustsec, NFN and Time Sensitive Networking
(TSN) ready
Available since: 12/2014
Full GE and Aggregation
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
240W POE IE-4000-8GT8GP4G
Manufacturing CityTransportation
MiningOil & GasEnergy Utility
• 8 Ports with POE+ (30W)
• Overall POE budget of 240W
• Only IE-4000-8GT8GP4G-E
• Min IOS Ver 15.2(6)E2
• HW Version ID >=V03
Available since: 9/2018
Full POE+
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
240W POE IE-4000-8GT8GP4G
Manufacturing CityTransportation
MiningOil & GasEnergy Utility
Feature-packed modern software for scalable
IoT deployments
• Composed by a main module and expansion
modules, allowing scaling the configuration (up
to 26 Ethernet interfaces) to grow with
customer operational needs
• Ruggedized for Industrial Applications, NEMA
TS-2 and ATEX compliant
• Extended power options, AC and DC
• Advanced QoS and Security features
Flexible Modular System
Available since: 2009
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Cisco CatalystIE3200, 3300, 3400 Rugged Switches
‘*’ – Post FCS
Fixed System
Expandable modular system
Feature-packed modern software for scalable IoT
deployments
• Flexible, resilient, secure Cisco® IOS XE
operating system
• Simplified management, automation, and visibility
IND, Cisco DNA Center, Prime®, WebUI
• Rich IE features – PRP*, HSR*, MRP*, PTP,
MACSEC*, TSN*, CIP, Profinet*
• Flexible licensing options:
• Network Essentials comes as PIK-PAK
• Cisco DNA Essentials*
• Network Advantage, and Cisco DNA
Advantage (post-FCS)*
Gigabit modular system
FCS Feb 2019
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
When do you need IE4000 / IE3400?
IE3400High scale, Future Proof
• 26 GE ports / 16 ports of PoE+
• Roadmap: Industrial Features / TrustSec
• Roadmap: IOx
• Roadmap: Layer 3
IE3400Advanced Platform, Today
• 20 GE ports / 8 ports of PoE+
• Industrial Features / TrustSec
• IOx
• Layer 3
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
IoT Industrial Switching portfolio
‘*’ –Selected Models
IE 4010IE 4000
10/100M
Featu
re
1G 10G
IE 5000
IE 3010CGS 2520IE2000UIE 2000
• Designed for all industries
• Layer 2 or 3 (IP service)
• 4 10 GE* uplinks• 24 GE downlinks• IEEE1588 PTP
(default and powerprofiles)
• Layer 2 NAT• Up to 12
PoE/PoE+• Dying gasp• Cisco TrustSec
SGT/SGACL• MACSec• FNF • TSN-ready• Stacking*• Conformal
coating*• Iox-ready• MRP, REP, PRP• HSR• Timing interfaces
(IRIG-B, GPS)• Cisco DNA
Essentials/Advantage
• L2 or L3 (IP lite)• Small form factor• IP30, IP67• MRP, REP • Layer 2 NAT• IEEE1588 PTP• Up to 8
PoE/PoE+ ports• Conformal
coating *• Cisco DNA
Essentials
• L2 or L3 (IPservices)
• Small form factor• PRP, REP• IEEE 1588 PTP
(default and power profiles)
• Up to 4PoE/PoE+ ports
• Conformal coating *
• L2 or L3 (IP services)
• 1 RU• 2 GE uplink
ports• 24 FE downlink
ports• REP• 8 PoE/PoE+
ports, 16 SFP, or 24 copper
• IEEE 1588 PTP (default and power profiles) *
• For all industries• Layer 2 or 3
(IP service)• 4 GE uplinks• Up to 20 GE
ports• IEEE1588 PTP
(default andpower profiles)
• Layer 2 NAT• Up to 8
PoE/PoE+• Dying gasp• Cisco TrustSec®
SGT/SGACL• MACSec, FNF• Time-Sensitive
Network (TSN)• IOx• MRP, REP, PRP• HSR• Cisco DNA
Essentials/Advantage
• For all industries• Layer 2 or 3
(IP service)• 4 GE uplinks• 28 total GE
ports• IEEE1588 PTP
(default andpower profiles)
• Layer 2 NAT• Up to 12 or 24
PoE/PoE+• Dying gasp• Cisco® TrustSec
SGT/SGACL• MACSec• TSN-ready• Iox-ready• MRP, REP, PRP• HSR• Cisco DNA
Essentials/Advantage
Best in class
AggregationAccess
IE3300IE3200
• Layer 2 • 2 GE uplinks• 8 GE downlinks• Up to 8 PoE/PoE+
ports• REP• IEEE1588 PTP• MacsecRoadmap• Profinet, MRP• Cisco DNA
Essentials
• Layer 2• 2 GE uplinks• Up to 24 GE ports• Up to 24
PoE/PoE+ ports• FNF, REP• IEEE1588 PTP• Layer 2 NAT,• MACSecRoadmap• Layer 3• Profinet• MRP• Cisco DNA
Essentials• Cisco DNA
Advantage
IE3400
• Layer 2• 2 GE uplinks• Up to 24 GE ports• FNF, REP• TrustSec®
SGT/SGACL• IEEE1588 PTP• Layer 2 NAT,• MACSecRoadmap• Layer 3• Profinet• MRP, PRP, HSR• IOX• TSN• SDA FE• Cisco DNA Essentials• Cisco DNA
Advantage
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
What is Field Network Director (FND)?
• Network Management System for FAN and IoT
• Secure zero touch deployment (ZTD)
• Real-time device and endpoint monitoring
• Geographical visualization of assets
• Field device lifecycle management
• API for 3rd party integration
• Scales up to millions of devices
• On-premise
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
FND Functionality
Monitor Maintain
Deploy• Automatic enrollment and provisioning• Secure tunnel provisioning• Secure tunnel provisioning • Zero-touch deployment
Manage• Configuration and network management• Troubleshooting• API for 3rd party integration
• Realtime monitoring & alerts for critical events
• Location tracking & geo fencing• Customizable dashboard
• Over-the-air configuration and firmware management
• Reconfiguration and Field engineer support
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
GUI overview – IOx application management
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Components – Field Area Router (FAR)Network device managed by FND
Supported Devices:
• IXM LoRaWAN Gateway (standalone and virtual mode)
• 800-series (IR807/IR809/IR829/C819)
• CGR1000-series (CGR1120/CGR1240)
• IR1101
• IC3000
• ESR5921
• IE4000(*FND 4.5)
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Cisco Kinetic EFM - Abstract
Key Characteristics:• Reusable microservices for collecting data from, and providing
control over, devices and machines, as well as processing the data prior to delivery to its destination
• Different options for reliable transport of data through the system, encompassing both batch and real-time streaming options
• Flexible mechanisms for integration with IT systems, reporting, and analytics
• Pervasive control paradigm and flow of information back to microservices, devices and machines for management, control, optimization and specific actions
• Open and polyglot system, where third parties can provide devices, processing storage, software modules, analytics, applications, or any combination thereof
Cisco Kinetic is a software designed for connecting to data originators, acquire telemetry, converging on protocol and payload of the data and to provision the data to consumers.
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Cisco Kinetic EFM - Components
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
$0.4 M $0.6 M
Reduced Energy Costs
$0.4 M $0.6 M
Reduced Maintenance Load
$10.4 M $13.9 M
Increase Machine Availability
$0.9 M $1.2 M
Reduce Labor Costs
$0.5 M $0.6 M
Reduce Scrap Costs
$12.5 M $16.9 MPotential Annual Benefits:
Conservative Estimate
Likely Scenario
Multivariate analysis using customer KPI from CAPEX, OPEX, OEE, sales, maintenance, TCO and more provide benchmarkable ROI and improvement forecast figures.
Benefits Summary: ExampleDelivering Tangible Value
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Detect anomalies, block threats, ID
compromise hosts
Secure third-party access with control
and visibility
Reduce risk, design, deploy and respond to
incidents while protecting the business
• IIoT changed type, amount and type of the communication of entities as sender, recipient, or actor
• Communication of machines independently of exposed interfaces, or protocol and payload type
• Unidirectional, bidirectional, or multidirectional communication• Provide data from originators to a consumer, at the right time
and format, securely and scalable
Security - OverviewChallenges
Security
Scalability
Resiliency
Performance
Flexibility
Reusability
Extensible, scalable segmentation to protect
IoT devices
Remote AccessRemote Access Visibility & Analysis Security Service
NGFWISE / TrustSec AnyConnect
AMPCybervision
UmbrellaStealthwatch
ISE / TrustSecCognitive Threat Analysis
DesignRisk Assessment
Incident Response
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Security – Cost and ImpactRetrofitting security to an existing architecture is complex and costly
OT/IT Security• IT Security in focus when designing a new architecture• OT Security often added after service, or functionally provided
Costs for IIoT are multifaceted and interdependent• Time to invest at the expense of the point in time of functional availability• Complexity to invest at the expense of operability, maintainability and risk to fail• Manpower to invest at the expense of OPEX• Financial budget in CAPEX and/or OPEX
Security has to embrace all architectural components and must include architectural delineation, monitoring and visibility, data security, device and communication security, secure administrative access and services with deployable components.
Neglecting on security aspects comes at the cost of immense risk for business safety.
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
SecurityKey Aspects
Environmental IntegrityOperational safety requires control of entities sending/receiving data, entities stopping communication and new entities joining network and communication.
CommunicationSecure communication between entitled participants following the minimum-principle.
Data IntegrityStandards require proof for untampered data (G10, PIPEDA, GDPR, or GxP)
Encryption, checksums for data and virtual sensors for plausibility checks.
Apply Purdue model for segmentation and zoning. Prevent unauthorized access to devices, data exposure and misuse of execution layer by using access profiles to devices and applications.
Apply segmentation and isolation techniques do data.
Operational overview, monitoring and transparency, automated access control provisioning, unique and immutable digital identities, isolation and protection of trustworthy and non-trustworthy compute base
EncryptionSemantic access to data for entitled participants only.
Apply encryption to data in-transit, at-rest and in-memory. Use encrypted network tunnels to communicate to skids and remote entities. FIPS 140-2 defines the security standards, that will be satisfied by encryption and helps to rank and scale an implementation.
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
SecurityRecommendations
• Only physical and environmental security for OT is insufficient
• OT and IT must employ the same security semantics
• NIST 800-5310 for IT and NIST 800-8211, ISA/IEC 6244312 for industrial control systems (ICS) and OT
• Apply a user and entity behavioral analytics (UEBA) to identify deviation of expected and real operation
• Start the lifecycle of installations with many entities with a autogenerated, trustworthy, immutable and non-reusable digital identification
• Deep package inspection (DPI) can become advisable to monitor the traffic sent
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
SecurityGeneral Guidelines
1. Develop Plant or Manufacturing Security Policies—Most enterprises have IT Security policies. These policies drive the behavior, processes and awareness of all enterprise network users Plant networks have distinctly different requirements and priorities. Therefore, a specific Plant Security policy should be developed and put into place.
2. Use IT-approved user access and authentication policies and procedures—Access to enterprise and plant resources and services should be monitored and logged. Every user must be a known entity to the organization and use a unique account. Unfortunately, these are typically based on users entering account and passwords and having certificates available. IACS devices are often not capable of any of these and are therefore not authenticated when connected to the network. Thus the following are important.
3. Strong Physical Security of Network Infrastructure – Access to Plant network infrastructure should be limited. Switches are typically installed in locked or hard to reach locations. Unused ports are turned off or even blocked. Specific ports for appropriate personnel are clearly marked and authentication policies are applied to them.
4. Endpoint Hardening – Antivirus applications, regularly deploying security updates and turning of or removing unnecessary applications and services on systems with common operating systems are considered best practices
5. Keep industrial Ethernet protocols at home—Industrial Ethernet network protocols, such as CIP and others, shall be contained to the Manufacturing zone. These protocols tend not to include enough security considerations, such as encryption or authorization, to be opened to generally available networks. They were designed to run in segmented networks where trust is implicit based on tight physical control of the network.
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
6. Control the applications—As a best practice, partners and remote engineers should use versions of IACS applications on controlled application servers when accessing the IACS remotely. This suggests creating remote access servers within the Manufacturing zone, on which the appropriate IACS applications are executed. 2012 ODVA Industry Conference 14 ©2012 ODVA, Inc.
7. Don't allow direct traffic—It is recommended that no direct traffic is permitted between the Enterprise zone (including the Internet) and the Manufacturing zone. The plant firewall acts as a proxy between remote users or applications and target IACS applications in the Manufacturing zone. The firewall also strictly polices the traffic into and out of each zone.
8. Create only one path in or out—The path from the DMZ through the lower firewall (or firewall instance) into the Manufacturing zone should be the only path in or out of the Manufacturing zone.
9. Protecting the Interior—Plant networks tend to be stable. With appropriate assistance, the network can be configured to limit traffic flows through the use of access control lists (ACLs).
10.Domains of Trust—Users should segment the network into smaller areas (VLANs) based on function or access requirements. These then form the basis on which to manage traffic flows, drastically simplifying application of additional Security functions.
SecurityGeneral Guidelines
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Best Practices Hardware1. Order new IC3000 from CCW with Field Network Director (FND) License
Management2. Order or have existing FND deployment
Switching Uplink3. Order or have existing switching ports for IC3000 mgmt. and data ports
Step 01 Pre-Deployment
Configuration Template4. Provision switch orts to IC3000 accordingly for both mgmt. and data ports5. Create config in FND6. Push config to IC3000 groups
Step 02 Deployment
Management7. Manage IC3000 (reload, upgrade) using FND8. Manage applications deployed on IC3000 using FND
Step 03 Management
Troubleshoot9. Collect device logs using FND or Local Manager10. Collection application logs using FND or Local Manager
Monitor11. Monitor status of device and applications using FND12. View device and application events
Step 04 Monitoring
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Sample Topology
Cisco IC3000 with IOxMTConnect Agent Inside
Cisco IE4000 with IOxMTConnect Agent Inside
Cisco IE2000
Cisco UCS Server
Cisco FND
Cisco IND
CNC
Autoclave
Pipe Bender
MT Connect
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Pre-Deployment
• Order new IC3000 from CCW with Field Network Director (FND) License
• Order or have existing FND deployment
• Order or have existing switching ports for IC3000 mgmt. and data ports
• Connect IC3000 management port to management VLAN on the switch and data ports to appropriate VLANS for data.
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Deployment (Device Configuration)
• Provision switch ports to IC3000 to the VLANs accordingly for both management and data ports
• Import IC3000 serial numbers into FND
• Create DHCP pool on local network with proper option 43 to allow IC3000 to find FND on boot
• ip dhcp pool IC3KNETnetwork 192.168.0.0 255.255.255.0 default-router 192.168.0.50dns-server 192.168.0.15 8.8.8.8 1.1.1.1 option 43 ascii 5A;K4;B2;I192.168.0.175;J9125
• Create groups of IC3000 devices as needed
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Deployment (Device Firmware Upgrade)
Upgrade the firmware of the IC3000 using FND if needed. The upgrade is done on all FND connected IC3000 at once that belong to same group.
The upgrade steps are as follows:
1. Make sure the ADMIN -> Provisioning Settings -> IoT-FND URL point to the FND server by IP or by name if reachable by DNS
2. CONFIG -> Firmware Update -> Images, choose IC3000 from left panel and upload new image
3. CONFIG -> Firmware Update -> Groups, make sure all IC3000 to upgrade below to same group and chose Upload Image and choose the IC3000 image to upload to all devices.
4. CONFIG -> Firmware Update, choose the Group in previous step and click Install Image. This step will install the image downloaded.
Be aware, an upgrade could take 15 min if doing both a Firmware upgrade and IOx upgrade
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Management: IC3000 Device
Create config in FND for each group
• Enable the IC3000 data ports which will be used
• Configure one or more NTP servers for clock synchronization to IC3000
• Push the config to the IC3000 group
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Management: MTConnect Application (Install)Example
Upload MTConnect IOx applications to FND, and install the application to individual or groups of IC3000 as follows:
1. From APPs tab in FND, select Import Apps to first add the app in the FND catalog. The steps below assume an application tar file packaged with IOx SDK
2. Browse for the app file on the local machine and click Upload to store the app on FND
3. From APPs tab in FND, choose app and click Install
4. Select one or more devices, then click Add Selected Devices to install list
5. Click Next to configure the app
6. It is possible to customize a number of features on this screen, but we will only check the networking to make sure we are using in1(bridge) interface in Dynamic mode. Once selected, click REASSIGN NETWORKS to apply the change
7. If asked to Configure VCPUs, select a value from 1-4 and click REASSIGN VCPUto confirm
8. Click Done to complete the install
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Management: MTConnect Application (Uninstall)Example
FND can also do upgrade or uninstall of applications
1. From the APPS tab in FND, choose the application to uninstall and click button to Uninstall
2. Select one or more devices, then click Add Selected Devices to uninstall list
3. Click Done to complete the uninstall
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
• THE MTConnect agent application deployed is built on the open source version 1.4 of the agent published by http://www.mtconnect.org/.
• The application comes pre-configured with a number of agents. Once installed and started, four of those agents automatically come up running. Each agent listens on a specific port (mapped to one machine) and provides a REST interface to northbound applications on another port.
• There are two ways to configure the agents running in a single MTConnect application. First method is to use the application built in Web UI. The second method is to SSH directly to the application. The IP address information of the app can be found in FND by choosing the device, then the App tab where all applications deployed on the device will be listed with their status and IP address information.
• Each configured agent requires two critical files to operate. First is the agent.cfg file which includes IP addresses, port numbers...etc., and second is a machine specific xml file that provides the agent with the schema of the data that will be arriving from the machine on this specific configured port.
Management: MTConnect Application ConfigurationExample
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Below is an example of an agent.cfg file with some inline comments. # name of the machine xml file to be used for this agent. Found in same directory
Devices = ./VMC-3Axis.xml
AllowPut = true
# this is the northbound port to be used by upstream applications
# needing access to the data from this agent via REST API.
Port = 5001
ReconnectInterval = 1000
BufferSize = 17
SchemaVersion = 1.3
Adapters {
VMC-3Axis {
# IP address of the machine/adapter where data is coming to the agent from (can be DNS)
Host = gos.iotspdev.local
# Port on the machine/adapter IP for access to streaming data
Port = 7878
}
}
Files {
schemas {
Path = /home/root/schemas
Location = /schemas/
}
styles {
Path = /home/root/styles
Location = /styles/
}
Favicon {
Path = /home/root/styles/favicon.ico
Location = /favicon.ico
}
}
StreamsStyle {
Location = /styles/Streams.xsl
}
# Logger Configuration
logger_config
{
logging_level = info
# location of log file, currently set to same dir as the agent.cfg
output = file /home/root/data/appdata/agent1/agent.log
}
Management: MTConnect Application Configuration(agent.cfg)
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
• The machine xml file is unique to that machine since it provides the agent with all the data to expect from this machine. The data usually arrives directly from a machine if it has a built in adapter, or from an adapter that sits between the machine and MTConnect application providing the translation.
• Adapter provider will normally probe the machine and generate this xml file to be used by the application.
Management: MTConnect Application Configuration(machine.xml)
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Management: MTConnect Application Scale for IC3000
Below is a sampling of scale validation of the MTConnect application running on IC3000. Testing was done with traffic simulation under controlled environment to scale the number of tags per second and the number of agents within the app that a single device can handle traffic from.
Below are 2 deployment scenarios:
5 Agents: Total 20 Machines
Tags/Sec/Machine Total Tags/Sec Memory (mb) CPU used
14 275 857 40%
30 600 1382 41%
43 870 1388 45%
62 1240 1404 51%
100 2000 1401 59%
3 Agents: Total 12 Machines
Tags/Sec/Machine Total Tags/Sec Memory (mb) CPU used
14 168 840 35%
30 360 1152 39%
43 516 1170 41%
60 720 1176 44%
105 1260 1172 50%
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
• Monitor the IC3000 status using FND Device Tab:
• Reboot or upload logs to send to support as needed
• View applications status and collect logs or restart as needed
Monitoring: (Device and Application)
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Monitoring: IC3000 Device Reset
If an IC3000 needs to be reset (to move to another FND for example or to erase all apps or device configurations), the IC3000 has a multi function reset button to left of the management port depending on how long it is depressed:
10-15 seconds:
• Reboot – A normal reboot of the device equivalent to power cycle
30-35 seconds:
• Config-reset – Erases all the user config, including apps and reboots the device. The device will reboot with the last software image that was running
60-65 seconds:
• Factory-reset – Erases everything and boots up with the factory default image (1.0.1)
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
LinksReferenced Name External Link
MTConnect https://www.mtconnect.org/
Cisco Kinetic https://www.cisco.com/c/en/us/solutions/internet-of-things/iot-kinetic.html
Cisco IOx https://www.cisco.com/c/en/us/products/cloud-systems-management/iox/index.html
Cisco FNDhttps://www.cisco.com/c/en/us/products/cloud-systems-management/iot-field-network-director/index.html
Mazak Smartbox https://www.mazakusa.com/machines/technology/digital-solutions/mazak-smartbox/
Cisco IE4000https://www.cisco.com/c/en/us/products/switches/industrial-ethernet-4000-series-switches/index.html
© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Glossary
Acronym Description
OEE Operational Equipment Efficiency is a measure of how well a manufacturing operation is utilized (facilities, time and material) compared to its full potential, during the periods when it is scheduled to run.
RUL Remaining Useful Lifetime is a prediction of the time at which a system or a component will no longer perform its intended function.
IOx Cisco IOx application environment combines Cisco IOS and the Linux OS for highly secure networking and virtual application operation on Cisco devices.
FND Cisco IoT Field Network Director (FND) is the network management system for FAN deployment at scale.
EFM EFM is the abbreviation for “Edge and Fog Processing Module”, a part of the Cisco Kinetic software stack.
KPI Key peformance indictator. Performance measurement for success and efficiency evaluation.
CAPEX Capital expenditure. Expenses to buy and maintain assets.
OPEX Operational expenditure. Day to day business expenses.
ROI Return on investment. T explains the ration and efficiency between invest and profit
TCO Total cost of ownership. All direct and indirect costs of manufacturing.