An Architecture and Prototype Implementation for TCP/IP Hardware Support Mirko Benz Dresden...
-
Upload
naomi-hubbard -
Category
Documents
-
view
215 -
download
0
Transcript of An Architecture and Prototype Implementation for TCP/IP Hardware Support Mirko Benz Dresden...
![Page 1: An Architecture and Prototype Implementation for TCP/IP Hardware Support Mirko Benz Dresden University of Technology, Germany TERENA 2001.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649f345503460f94c50b7e/html5/thumbnails/1.jpg)
An Architecture and Prototype Implementation for TCP/IP Hardware Support
Mirko BenzDresden University of Technology, Germany
TERENA 2001
![Page 2: An Architecture and Prototype Implementation for TCP/IP Hardware Support Mirko Benz Dresden University of Technology, Germany TERENA 2001.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649f345503460f94c50b7e/html5/thumbnails/2.jpg)
Required Processing
Routing
Switching
Quality of Service Support
Internet Security Provision
Required Processing Power
Application Complexity1000+ Instructions per Packet
Motivation – Demanding Services
TERENA 2001
![Page 3: An Architecture and Prototype Implementation for TCP/IP Hardware Support Mirko Benz Dresden University of Technology, Germany TERENA 2001.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649f345503460f94c50b7e/html5/thumbnails/3.jpg)
Motivation – MIPS versus Bandwidth Trend
TERENA 2001
Technological Progress / Time
MIPS Performance / Bandwidth
Processor Performance Evolution ~100%/18 month
Available Bandwidth
Hardware Support for Protocol Processing Acceleration
Case Study: TCP
![Page 4: An Architecture and Prototype Implementation for TCP/IP Hardware Support Mirko Benz Dresden University of Technology, Germany TERENA 2001.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649f345503460f94c50b7e/html5/thumbnails/4.jpg)
• Assumptions & Preconditions
- Restriction to Local Area Networks (e.g. Gigabit Ethernet)- High Bandwidth and Low Error Probability- Concentration on Host Implementations
Project Overview
TERENA 2001
Protocol Analysis
TCP/IP Partitioning System
Simulation
Evaluation Optimisation
Efficient OSIntegration
Prototype Variants
Flexible Protocol Engine
Domain Specific
Methodology
![Page 5: An Architecture and Prototype Implementation for TCP/IP Hardware Support Mirko Benz Dresden University of Technology, Germany TERENA 2001.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649f345503460f94c50b7e/html5/thumbnails/5.jpg)
Talk Outline
TCP Protocol Performance Evaluation
TCP Acceleration Approach
System Simulation Environment
Operating System Integration
Hardware Implementation Directions
Myrinet Implementation and Results
Conclusions and Outlook
TERENA 2001
![Page 6: An Architecture and Prototype Implementation for TCP/IP Hardware Support Mirko Benz Dresden University of Technology, Germany TERENA 2001.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649f345503460f94c50b7e/html5/thumbnails/6.jpg)
TCP Protocol Performance Evaluation
• TCP Software Implementation Structure
• Sources of Protocol Processing Overhead- Communication, Synchronisation- Operating System Call Overhead- Copy Operation - Classification: Per-Byte / Per-Packet
• Optimisation Opportunities- Interrupt Suppression- Zero Copy Mechanisms- User Level Networking- Checksum Offloading (e.g. Task Offload)- Extending frame sizes (e.g. Jumbo Frames)
TERENA 2001
Network
Driver
IP
TCP
Socket
Application
![Page 7: An Architecture and Prototype Implementation for TCP/IP Hardware Support Mirko Benz Dresden University of Technology, Germany TERENA 2001.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649f345503460f94c50b7e/html5/thumbnails/7.jpg)
TCP Protocol Performance Evaluation
• Performance TCP versus Myrinet GM:- Throughput 335/967 Mbit/s (TCP/Myrinet)- Latency 81/29 s (TCP/Myrinet)- 100% CPU Utilisation - (RedHat Linux 6.2 / PIII 500 MHz)
TERENA 2001
0
200
400
600
800
1000
0 10000 20000 30000 [byte]
[Mb
it/s
]
GM 1.2
TCP/Myrinet
![Page 8: An Architecture and Prototype Implementation for TCP/IP Hardware Support Mirko Benz Dresden University of Technology, Germany TERENA 2001.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649f345503460f94c50b7e/html5/thumbnails/8.jpg)
Goals?
• Software Implementation as a Foundation• Achieve On Wire Compatibility• Consider Different Target Architectures• Develop Re-Useable Hardware Components• Integration of High Level Tools• System Wide Optimisation• Efficient, Transparent Operating System Integration
Domain SpecificMethodology
Flexible Protocol Engine
TERENA 2001
![Page 9: An Architecture and Prototype Implementation for TCP/IP Hardware Support Mirko Benz Dresden University of Technology, Germany TERENA 2001.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649f345503460f94c50b7e/html5/thumbnails/9.jpg)
TCP Acceleration Approach
• TCP SW Stack Complexity - General Purpose Protocol- Not Designed for High End Networking- Many Interdependent Algorithms - Often Modified, Adapted, Optimised- ~15.000 Lines C
• Approach- TCP Partitioning -> Fast Path Extraction- Hardware Support -> Acceleration - Operating System Bypass
• HW/SW Synchronisation- Initialisation, Termination/Error
• Transparent Integration- Socket Level Switch
Network
Driver
IP
TCP
Socket
Application
PE
TERENA 2001
![Page 10: An Architecture and Prototype Implementation for TCP/IP Hardware Support Mirko Benz Dresden University of Technology, Germany TERENA 2001.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649f345503460f94c50b7e/html5/thumbnails/10.jpg)
Fast Path Protocol Processing
TERENA 2001
TCP Send
Sender Receiver
TCP Recv
Send Ack
Network
Connection Context
TCP Send
TCP Recv
Send Ack
Connection Context
DataAck
• Only for User Data Exchange• No Connection Management• No Error Recovery – Only Detection• Complexity ~10% of SW Stack
![Page 11: An Architecture and Prototype Implementation for TCP/IP Hardware Support Mirko Benz Dresden University of Technology, Germany TERENA 2001.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649f345503460f94c50b7e/html5/thumbnails/11.jpg)
Netserver
Socket
User Mode Linux
Netserver
Socket
User Mode Linux
System Simulation Environment
TERENA 2001
NetworkSimulator
Netperf
Socket
User Mode Linux
CORBA
• Complex Communication System• Real Applications, Operating System (User Mode Linux)• Network Simulation – Error Injection• Fast Path Implementation: Hardware/Software• System Evaluation: Functionality & Performance
ISASimulator
TCP Fast Path SW
VHDLSimulator
TCP Fast Path HW
Evaluation
![Page 12: An Architecture and Prototype Implementation for TCP/IP Hardware Support Mirko Benz Dresden University of Technology, Germany TERENA 2001.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649f345503460f94c50b7e/html5/thumbnails/12.jpg)
Fast Path Hardware Implementation Directions
• Embedded RISC Processor - LEON Sparc 33 MHz, INTEL StrongARM 200 MHz- OS: ucLinux, GNU C Environment
Intelligent Network Adapter (Myrinet)- RISC Core with User/Network Interface, DMA Engines- Control Program Modification, no Operating System
• Network Processor (INTEL IXP1200)- 6 multithreaded microengines - Development: IXP Assembler, Simulator
• Specific Hardware- High Level FPGA Design Flow, XILINX Virtex- SYNOPSYS Protocol Compiler
Software
Hardware
TERENA 2001
![Page 13: An Architecture and Prototype Implementation for TCP/IP Hardware Support Mirko Benz Dresden University of Technology, Germany TERENA 2001.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649f345503460f94c50b7e/html5/thumbnails/13.jpg)
Myrinet Implementation Plattform
TERENA 2001
LOCAL SRAM
LANai 7
HostInterface
PacketInterface
RISCPCI
BridgeDMA
ControllerMyrinet
Link
64 bit64 bit, 33 MHz 1280 Mbit/s
• Technology
- Packet-Communication and Switching Technology
- High-Performance, Highly Reliable
- System-Area Network, Cluster Interconnect
• Intelligent Network Adapter
![Page 14: An Architecture and Prototype Implementation for TCP/IP Hardware Support Mirko Benz Dresden University of Technology, Germany TERENA 2001.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649f345503460f94c50b7e/html5/thumbnails/14.jpg)
TCP Fast Path/Myrinet
• Development Environment
- Host SW GM (message passing), Firmware MCP – open source
- GNU C Suite, no OS, one context only, no Interrupts
•Implementation
- MCP: 4 Event Driven State Machines
- Fast Path Integration within Network Send & Recv Code
- Exploitation of Hardware Support for Checksum Computation
- No specific Optimisations, Some Limitations
TERENA 2001
![Page 15: An Architecture and Prototype Implementation for TCP/IP Hardware Support Mirko Benz Dresden University of Technology, Germany TERENA 2001.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649f345503460f94c50b7e/html5/thumbnails/15.jpg)
0
200
400
600
800
1000
0 10000 20000 30000 [byte]
[Mbi
t/s] GM 1.2
Fast Path
TCP/Myrinet
TCP Fast Path / Myrinet Performance Results
• Performance
- Test Setup: INTEL PIII/500MHz, Myrinet LAN Adapter, Linux OS
- Netperf Benchmark Throughput/Delay
- Throughput Peak: 967, 816, 333 Mbit/s (GM, Fast Path, TCP)
- Delay Minimum: 16.5, 49, 81 s (GM, Fast Path, TCP)
TERENA 2001
![Page 16: An Architecture and Prototype Implementation for TCP/IP Hardware Support Mirko Benz Dresden University of Technology, Germany TERENA 2001.](https://reader036.fdocuments.in/reader036/viewer/2022062517/56649f345503460f94c50b7e/html5/thumbnails/16.jpg)
Summary & Outlook
• Integrated Architecture and Desing Flow for Protocol Processing Acceleration
- TCP Partitioning
- System Simulation Environment
- Integration with existing SW TCP Stack & OS
• Prototype with Promising Performance
• Present Work:
- Fast Path HW Implementation and SoC Integration
Protocol Analysis
TCP/IP Partitioning
System Simulation
Efficient OSIntegration
Prototype Variants
Evaluation Optimisation
Flexible Configurable Protocol Engine
TERENA 2001