Test Tool for Industrial Ethernet Network Performance (June 2009)
-
Upload
jim-gilsinn -
Category
Technology
-
view
191 -
download
0
description
Transcript of Test Tool for Industrial Ethernet Network Performance (June 2009)
Standards
Certification
Education & Training
Publishing
Conferences & Exhibits
Test Tool for Industrial Ethernet Network Performance
Jim Gilsinn & Freemon JohnsonNational Institute of Standards & Technology (NIST)
Manufacturing Engineering Laboratory (MEL)
Author Bio: Jim Gilsinn
• Over 16 years of experience at NIST• Variety of manufacturing projects, including intelligent
unmanned vehicles, welding standards, smart sensors, and nano-manufacturing
• Current research focus is industrial network reliability, including performance and security
• Masters in Electrical Engineering from Johns Hopkins University, specialty in Control Systems; Bachelors in Electrical Engineering from Drexel University, specialties in Robotics and Control Systems
2
Author Bio: Freemon Johnson
• Over 10 years of experience; previously worked at NASA Goddard Space Flight Center and Motorola
• Expertise in communication protocol security, architecture, and embedded real-time systems
• Masters in Computer Science from Johns Hopkins University; Masters in Communication and Information Studies from Rutgers University; Bachelors in Electrical Engineering from New Jersey Institute of Technology
3
Overview
• Background• Performance Testing Methodology• Data Analysis• Test Tool• Future Plans
4
Background
• The Problem– Ethernet & TCP/IP inherently non-deterministic– Users have many questions about industrial Ethernet devices– Automobile Example
• Why NIST?– Aid U.S. businesses and industries, specifically manufacturing– Many companies are global– Industrial Ethernet standards are available worldwide
5
Performance Testing Methodology:Performance Metrics
`
• Publish/subscribe or peer-to-peer communications• Main performance metric: Cyclic frequency
variability/jitter• Real-time EtherNet/IP uses publish/subscribe
– Requested/Accepted Packet Interval (RPI/API)– Measured Packet Interval (MPI)
6
Performance Testing Methodology:Performance Metrics
`
• Command/response or master/slave communications• Main performance metric: Latency
7
Performance Testing Methodology:Testing Methodology
• Overall methodology for testing performance:1. Begin recording network traffic
2. Establish a connection with the device under test (DUT)
3. Begin transmitting background network traffic, based on the particular test conditions
4. Wait for a given amount of time
5. Stop transmitting background network traffic
6. Close the connection with the DUT
7. Stop recording network traffic
8. Analyze the network traffic capture and report the results
• Current test tool only for data analysis & reporting
8
Performance Testing Methodology:Test System
9
Data Analysis: Overview
• Current test tool focuses on cyclic frequency variability/jitter of the MPI
• Capable of analyzing multiple network packet streams from different devices during a single test run
• Conduct a distribution analysis on the MPI results– Determine minimum, maximum, mean, standard deviation,
skewness, and kurtosis of data set
• Produce graphs of MPI results
10
Data Analysis: Example Jitter GraphM
easu
red
Pac
ket
Inte
rval
(s)
Test Time (s)
This ExampleTest Duration ≈ 61 sMean ≈ 2 msMinimum ≈ 1.2 msMaximum ≈ 2.9 ms
11
Data Analysis: Observations
12
Data Analysis: Observations
13
Test Tool: Design
• Geared towards vendors & users– Vendors: develop better products during design process– Users: determine how well devices operate in a real system
• Multi-phase design approach– Analysis → Testing
• Meaningful results– Mathematical foundation– User selectable complexity for results
14
Test Tool: Technical Details
• Free software package– Microsoft Windows-based (Windows 2000 or later)
• Public domain source-code– Visual C# modular design– Commercial graphing package
• Open-source installed package– No special software or hardware needed to run– Integrated with Wireshark for packet analysis
15
Future Plans
• Planned versions of IENetP Test Tool– Data Analysis Tool
– Version 1.x – Basic functionality & test engine– Version 2.x – Additional data analysis capability & metrics– Version 3.x – Additional industrial networks
– Active Testing Tool– Version 4.x – PC-based test tool– Version ?.? – Specialized hardware test tool
16
Summary
• NIST is working with industry partners to develop industrial Ethernet performance metrics, tests, and tools– Determined metrics– Developed basic methodology
• NIST is currently working on a test tool– Designed as a development tool for industrial Ethernet vendors– Focused on data analysis presently
• Some patterns have emerged while analyzing data• Additional functionalities are planned for the test tool
17
Contact Information
• Jim Gilsinn, 301-975-3865, [email protected]• Freemon Johnson, 301-975-3429, [email protected]• Mailing Address
– National Institute of Standards & Technology (NIST)Manufacturing Engineering Laboratory (MEL)100 Bureau Drive, Mailstop 8230Gaithersburg, MD 20899-8230
• Websites– Project: http://ienetp.sourceforge.net– Division: http://www.isd.mel.nist.gov
18