Internetworking Over SpaceWire: A Link-Layer Broadcast ... · Space Internetworking Workshop,...
Transcript of Internetworking Over SpaceWire: A Link-Layer Broadcast ... · Space Internetworking Workshop,...
Internetworking Over Internetworking Over SpaceWireSpaceWire::A LinkA Link--Layer Broadcast ServiceLayer Broadcast Service
for Network Stack Supportfor Network Stack Support
Robert Klar, Sandra G. Dykes, Allison Roberts,Chris Mangels, Buddy Walls, Mark A. Johnson, Kristian Persson
Southwest Research Institute
Space Internetworking Workshop, September 2006 Slide 2
Standard Network Stack OnStandard Network Stack On--Board?Board?
Advantages
• Software reuse; mature APIs and implementations.
• Simpler application code.
• Rapid development.
• Lower cost.
• Interoperability.
Standard Network Stack
Integrated Application & Network
Data Link
Physical
Integrated Application & Network
Data Link
Physical
Platform-Specific
Disadvantages
ApplicationSNMP, Web (HTTP), SSH, FTP, SCPS-FP
TransportTCP, UDP, SCPS-TP
NetworkIPv4, IPv6, SCPS-NP
Data LinkSpaceWIre, Ethernet, FireWire, HDLC, WiFi,…
Physical
ApplicationSNMP, Web (HTTP), SSH, FTP, SCPS-FP
TransportTCP, UDP, SCPS-TP
NetworkIPv4, IPv6, SCPS-NP
Data LinkSpaceWIre, Ethernet, FireWire, HDLC, WiFi,…
Physical• Protocol assumptions may not hold
(e.g., TCP assumes delay is related to congestion, IP assumes end-to-end connectivity, …).
• Performance may not be optimal
Space Internetworking Workshop, September 2006 Slide 3
SpaceWire OverviewSpaceWire Overview
• Switched LAN designed for high speed on-board data handling.
• Low power, low cost.
• Scalable
• 2 to 400 Mbps, low latency.
• Backplane or Cables.
• Based on IEEE 1355 and LVDS.
• Wormhole routing
• ESA Specification ECSS-E-50-12A
Onboard Switched Local Area Network (LAN)
CommandComputer
Scientific Instrument
Scientific Instrument
Telemetry System
Onboard Switched Local Area Network (LAN)
CommandComputer
Scientific Instrument
Scientific Instrument
Telemetry System
Onboard Switched Local Area Network (LAN)
CommandComputer
Scientific Instrument
Scientific Instrument
Telemetry System
Space Internetworking Workshop, September 2006 Slide 4
SpaceWire Path AddressingSpaceWire Path Addressing• Header contains output port of each hop (0 .. 32)
• SpaceWire Routers apply header deletion
• Example: N1 sends to N6 <4><2><3><cargo><EOP>
RouterR1
1 32
4
N2
LA 42N1
LA 41
N3
LA 43
N5
LA 130N4
LA 129
N8
LA 42N7
LA 41
N9
LA 43
LA 109
Region 1
LA 165
N6
LA 131
Region 2
RouterR2
1 32
4Router
R31 32
4
RouterR4
1 32
4
RouterR1
1 32
4
RouterR1
1 32
RouterR1
1 321 32
4
N2
LA 42N1
LA 41
N3
LA 43
N5
LA 130N4
LA 129
N8
LA 42N7
LA 41
N9
LA 43
LA 109
Region 1
LA 165
N6
LA 131
Region 2
RouterR2
1 32
4Router
R21 32
RouterR2
1 321 32
4Router
R31 32
4Router
R31 32
RouterR3
1 321 32
4
RouterR4
1 32
4
RouterR4
1 32
RouterR4
1 321 32
4
Space Internetworking Workshop, September 2006 Slide 5
SpaceWire Logical AddressingSpaceWire Logical Addressing• Header contains logical address of destination (32 .. 254)
• Path determined by router forwarding tables.
• Example: N1 sends to N6 <131><cargo><EOP>
RouterR1
1 32
4
N2
LA 42N1
LA 41
N3
LA 43
N5
LA 130N4
LA 129
N8
LA 42N7
LA 41
N9
LA 43
LA 109
Region 1
LA 165
N6
LA 131
Region 2
RouterR2
1 32
4Router
R31 32
4
RouterR4
1 32
4
RouterR1
1 32
4
RouterR1
1 32
RouterR1
1 321 32
4
N2
LA 42N1
LA 41
N3
LA 43
N5
LA 130N4
LA 129
N8
LA 42N7
LA 41
N9
LA 43
LA 109
Region 1
LA 165
N6
LA 131
Region 2
RouterR2
1 32
4Router
R21 32
RouterR2
1 321 32
4Router
R31 32
4Router
R31 32
RouterR3
1 321 32
4
RouterR4
1 32
4
RouterR4
1 32
RouterR4
1 321 32
4
LA Port
131 4--- ----- --
LA Port
131 4--- ----- --
LA Port
131 3--- ----- --
LA Port
131 3--- ----- --
LA Port
131 2--- ----- --
LA Port
131 2--- ----- --
Space Internetworking Workshop, September 2006 Slide 6
SpaceWire Regional Logical AddressingSpaceWire Regional Logical Addressing• Regional addressing used for larger networks.
• Logical addresses may be reused in different regions.
• Routers configured as gateways between regions.
• Example: N1 sends to N9 <165><43><cargo><EOP>
RouterR1
1 32
4
N2
LA 42N1
LA 41
N3
LA 43
N5
LA 130N4
LA 129
N8
LA 42N7
LA 41
N9
LA 43
LA 109
Region 1
LA 165
N6
LA 131
Region 2
RouterR2
1 32
4Router
R31 32
4
RouterR4
1 32
4
RouterR1
1 32
4
RouterR1
1 32
RouterR1
1 321 32
4
N2
LA 42N1
LA 41
N3
LA 43
N5
LA 130N4
LA 129
N8
LA 42N7
LA 41
N9
LA 43
LA 109
Region 1
LA 165
N6
LA 131
Region 2
RouterR2
1 32
4Router
R21 32
RouterR2
1 321 32
4Router
R31 32
4Router
R31 32
RouterR3
1 321 32
4
RouterR4
1 32
4
RouterR4
1 32
RouterR4
1 321 32
4
LA Port
165 4--- ----- --
LA Port
165 4--- ----- --
LA Port
165 3--- ----- --
LA Port
165 3--- ----- --
Header Deletion Applied
LA Port
43 3--- ----- --
LA Port
43 3--- ----- --
Space Internetworking Workshop, September 2006 Slide 7
IP, etc. Over SpaceWireIP, etc. Over SpaceWire
Application
Transport
Network: IP, SCPS-NP
Data Link: SpaceWire
Physical
Application
Transport
Network: IP, SCPS-NP
Data Link: SpaceWire
Physical
• Interest in on-board IP / SCPS-NP, esp. for rapid development programs.
• Need to level the playing field with Ethernet by providing full support for network protocols and standard network management.
• SpaceWire is missing 2 key elements:
• Encapsulation service
• Address Resolution Protocol (ARP)
Space Internetworking Workshop, September 2006 Slide 8
EncapsulationEncapsulation ServiceService
Type Virtual Channel ID
ARP 1
IPv4 2
IPv6 3
CCSDS SCPS-NP 4
CCSDS Source Packet 5
Broadcast 254
1 byte Virtual Channel ID
SpaceWireHeader
EncapsulationHeader Data
SpaceWire cargo
• 1 byte Encapsulation header (Virtual Channel ID).
• Identifies network protocol or software module on destination.
• Enables SpaceWire to support multiple network types.
• Lightweight but sufficient.• SpaceWire Virtual Channel ID: 8 bits
• Ethernet Type field: 16 bits
Space Internetworking Workshop, September 2006 Slide 9
SpaceWire Address ResolutionSpaceWire Address Resolution
• No ARP defined for SpaceWireManual configuration of address resolution tables:− Assign SpaceWire and IP addresses.
− Install table on each node.
Tables are static. Changing network topology requires new tables.
• SpaceWire cannot use standard ARP (RFC 826)
Logical addresses not unique.
No link-layer broadcast for full network.
Solution: Develop these and use standard ARP
Unique SpaceWire Address =Region + Node Logical Address
192.168.0.1 41
IP Address SpaceWireAddress
192.168.0.2 129
192.168.0.3 54
Space Internetworking Workshop, September 2006 Slide 10
Broadcast ServiceBroadcast Service
FeaturesFeatures
• Guarantees loop-free broadcast
• Link-layer implementation, no modification to the network layer (e.g. IP, SCPS-NP)
• Host implementation; no modification to Spacewire routers.
• Driver software implementation, no modification to Spacewire interface hardware.
• Adheres to SpaceWire standard.
Space Internetworking Workshop, September 2006 Slide 11
Broadcast ServiceBroadcast Service
RouterA
SUBNET A
N2LA 42
N2LA 42
N1LA 41
N3LA 43
1 32
SA
4 65Router
C
SUBNET C
N5LA 45
N5LA 45
N4LA 44
N6LA 46
1 32
SC
4 65
RouterB
1 32
4
• Introducing Concept of a SpaceWire Subnet
• Set of nodes attached to one routing switch
• 1 Broadcast Server
Space Internetworking Workshop, September 2006 Slide 12
Broadcast ServiceBroadcast Service
2
Broadcast MessagePayload
SpaceWire header: Port AddressEncapsulation header: Protocol 254 = BroadcastBroadcast header: Type 0
2540
1 byte
• Protocol MessagesType 0: Subnet Broadcast
Space Internetworking Workshop, September 2006 Slide 13
Broadcast ServiceBroadcast Service
2
ARPMessage
SpaceWire header: Port AddressEncapsulation header: Protocol 254 = BroadcastBroadcast header: Type 0
1
Broadcast Message Payload
Encapsulation header: Protocol 1 = ARP
2540
1 byte
• Protocol MessagesType 0: Subnet Broadcast
Space Internetworking Workshop, September 2006 Slide 14
Broadcast ServiceBroadcast Service
45
BroadcastMessagePayload
SpaceWire header: Logical AddressEncapsulation header: Protocol 254 = BroadcastBroadcast header: Type 1
2541
1 byte
• Protocol MessagesType 1: Server Broadcast
Space Internetworking Workshop, September 2006 Slide 15
Broadcast ServiceBroadcast Service
• Protocol OperationNode sends Subnet Broadcast
• Sends Type 0 message to all other ports on local router using path addressing.
• Neighbor Routers configured to discard LA 254
Broadcast Server sends Server Broadcast• On receipt of Type 0 message, extracts the broadcast payload,
encapsulates in Type 1 message and sends to all other Broadcast Servers.
Remote Server sends Subnet Broadcast• On receipt of Type 1 message, extracts the broadcast payload,
encapsulates in Type 0 message and sends to all other ports on local router.
Space Internetworking Workshop, September 2006 Slide 16
Broadcast ServiceBroadcast Service
• Node N3 sends Subnet Broadcast (Type 0)
RouterA
SUBNET A
N2LA 42
N2LA 42
N1LA 41
N3LA 43
1 32
SA
4 65Router
C
SUBNET C
N5LA 45
N5LA 45
N4LA 44
N6LA 46
1 32
SC
4 65
RouterB
1 32
4
Space Internetworking Workshop, September 2006 Slide 17
Broadcast ServiceBroadcast Service
• Broadcast Server sends Server Broadcast (Type 1)
RouterA
SUBNET A
N2LA 42
N2LA 42
N1LA 41
N3LA 43
1 32
SA
4 65Router
C
SUBNET C
N5LA 45
N5LA 45
N4LA 44
N6LA 46
1 32
SC
4 65
RouterB
1 32
4 Type 1Message Sent as
Unicast toAll Servers
Space Internetworking Workshop, September 2006 Slide 18
Broadcast ServiceBroadcast Service
• Remote Server sends Subnet Broadcast (Type 0)
RouterA
SUBNET A
N2LA 42
N2LA 42
N1LA 41
N3LA 43
1 32
SA
4 65Router
C
SUBNET C
N5LA 45
N5LA 45
N4LA 44
N6LA 46
1 32
SC
4 65
RouterB
1 32
4
Space Internetworking Workshop, September 2006 Slide 19
Test NetworkTest Network• SwRI SpaceWire Link Interface Module (SLIM)
Single Channel, full-duplex SpaceWire Link InterfaceFully Compliant CompactPCI target interface3U cPCI form-factor
• Star Dundee Routers (8 port)• 4Links SpaceWire-PCI
SUBNETRouterR2
RouterR2
RouterR1
RouterR1
SUBNET4
N2SLIM
N2SLIM
N1SLIM
N1SLIM
1 32
4
N4SLIM
N4SLIM
1 32
N34Links
N34Links
SUBNETRouterR2
RouterR2
RouterR1
RouterR1
SUBNET4
N2SLIM
N2SLIM
N1SLIM
N1SLIM
1 321 32
4
N4SLIM
N4SLIM
1 321 32
N34Links
N34Links
Space Internetworking Workshop, September 2006 Slide 20
SummarySummary• By adding missing elements, standard network
stacks can be supported with SpaceWire• Encapsulation Service• Address Resolution Protocol (ARP)
• Standard ARP is possible if Broadcast Service supported
• Supports multiple network layer protocols (IPv4, IPv6, SCPS-NP).
Next step: Implement Broadcast and Encapsulation Service in device drivers for SwRI SLIM and COTS interface boards, and test IP and SCPS-NP applications on multi-router SpaceWire test network(SCPS-FP, DHCP, SNMP, HTTP, FTP, etc.).
Spacecraft Onboard LAN
FlightComputer
Scientific Instrument
Scientific Instrument
Telemetry System
Spacecraft Onboard LAN
FlightComputer
Scientific Instrument
Scientific Instrument
Telemetry System
Internet