Panda: MapReduce Framework on GPU’s and CPU’s Hui Li Geoffrey Fox.
DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200...
Transcript of DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200...
DDS Scalability: “One size fits all ??.. ”For OMG RTE-workshop, July 2007
Hans van ‘t Hag, OpenSplice™ DDS product [email protected]
1Agenda
INTO THE MOOD “… A paradigm shift …”
CRITICAL INFRASTRUCTURES “… The Data is the Network © …”
A MODEL “… For DDS applicability …”DDS APPLICABILITY “… One size fits all ?? ...”
DDS USECASE “… Work in Progress ...”
CONCLUSION “… Wait & See ...”
INTO THE MOOD
3
CH
J
G
K
F
B
E
I
A
L
N
U
T
V
R
M
P
Q
S
ZY X
W
D
A B C E F G H I J K L M
N O P Q R S T U V W X Y Z
D
DDS BACKBONE
Into the mood: ‘…Client/server vs. Pub-sub: A mind-shift…’
CRITICAL INFRASTRUCTURES
5© ARMS (Doug Schmidt): R&D Challenge
6Net-centricity: The Data Is The Network ©
© DoD Chief Information Officer, Network and Informat ion Integration, Nov’05
7DDS: a Fielded concept, suitable for mission-critic al systems
CHARACTERISTICS
Many different customers: fielded in over 15 Navies world-wide Many different ships/missions: tens of Ships classes (patrol boats to destroyers)Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/secReal-time and Fault-tolerant: Battle-damage resistant, deterministic, reliable
CHARACTERISTICS
Many different customers: fielded in over 15 Navies world-wide Many different ships/missions: tens of Ships classes (patrol boats to destroyers)Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/secReal-time and Fault-tolerant: Battle-damage resistant, deterministic, reliable
8
SMART-L APAR
IFF LINK 11
KH/SCOUTNAV. RADAR
HELI APPR.RADAR
VESTA
VESTA
VEX/RCPTDLPP
APECS-3ESM / ECM
SRBOC
OFF LINER/S
SIRIUS TDS (2x) POS SYSTEMSHIP’S REF.
IRC
BTSOSD
BTSOSD
COMMS ROOMBRIDGESTAFF ROOM
LARGE SCREEN DISPLAY 2
PIOD /PTSU
PRINTERS
COTS & SIGMA TYPE PROCESSOR
COMBAT INFORMATION CENTER
LARGE SCREENDISPLAY 1
VIDEO INTERFACECABINET
VCR 1/2
DEPARTMENT OFF. / OFF. QUARTERSETC.
WORKSTATION 01 WORKSTATION M
SPARE
FL. DECK
TV CAMERA
TV CAMERA
15 15 15 15
INTEGRATEDMACHINERY
CONTROLSYSTEM COMMS NETWORK
CCC1 CCC2
ATAS
SPHERION+ XBT / XSV
+ SURF. TEMP. REC.HARPOON
TORPEDOWEAPONSYSTEM
MK32 MOD-9(MK46)
GUN 127 MM
30 MM GUNSOFF LINE
GIC-127
MIC-MK41
MIC-MK41
MK41 SM2LAUNCHER GOALKEEPER
CATV BUS
ATM NETWORK
Data-traffic: > 4.000 publications per second over the system-data busPrograms: 2.200 programs allocated over 150 processorsData flows: urgent & non-urgent data (latency), important & les s-important data (priority)
Data-traffic: > 4.000 publications per second over the system-data busPrograms: 2.200 programs allocated over 150 processorsData flows: urgent & non-urgent data (latency), important & les s-important data (priority)
An Example: Naval Frigate Combat System
A MODEL FOR DDS-APPLICABILITY
10SYSTEM SCOPES
Platform-centric systems
Mission Computer
Vehicle Management
Net-centric systems
Net-centric systems-of-systems
REQUIREMENTS
11Information Grids in Net-Centric Operations (NCO)
� NCO: Autonomous processes, based upon ubiquitous information at required QoS� NCO: Autonomous processes, based upon ubiquitous information at required QoS
Planning Grid:CommonOperationalPicture
Command Support(CS)
Awareness Grid:CommonTactical Picture
Command & Control(CC)
Engagement Grid:Weapons Direction
Combat Execution (CE)
12A MODEL FOR DDS APPLICABILITY
PlatformPlatform--centriccentric
““Execution gridExecution grid””
Net(work)-centric
“Awareness grid”
System-of-Systems
“Planning/support grid”
Scale
Realtime
Soft
Firm
Hard
Small Large Ultra-Large
13MAIN CHARACTERISTICS
DETERMINISTICDETERMINISTIC--““ExecutionExecution--gridgrid””
EFFICIENTEFFICIENT--““AwarenessAwareness--gridgrid””
INTEROPERABLEINTEROPERABLE --““Planning/support gridPlanning/support grid””
Soft
Firm
Hard
Small Large Ultra-Large
Scale
Realtime
14EXAMPLES
MilitaryMilitary: AAW fire control: AAW fire control
CivilCivil: Process control: Process control
MilitaryMilitary: Track: Track--ManagementManagement
CivilCivil: SCADA control systems: SCADA control systems
MilitaryMilitary: Mission planning: Mission planning
CivilCivil: Emergency Response: Emergency ResponseSoft
Firm
Hard
Small Large Ultra-Large
Scale
Realtime PlanningPlanning
AwarenessAwareness
ExecutionExecution
15INFORMATION ARTEFACTS (Mil. Example)
Tracking informationTracking information
Timed ordersTimed orders
ObservationsObservations
Repository InformationRepository Information
Multimedia/HypertextMultimedia/Hypertext
DocumentsDocumentsSoft
Firm
Hard
Small Large Ultra-Large
Scale
Realtime
Engagement status
Mission Status
Engagement orders & Cues
Mission Objectives
PlanningPlanning
AwarenessAwareness
ExecutionExecution
16TECHNOLOGIES
LANLAN
Soft
Firm
Hard
Small Large Ultra-Large
WANWAN
WEBWEB
Link/Radio
Internet, satellite
Scale
Realtime
�� STATIC LANSTATIC LAN
�� WS & Embedded HW/OSWS & Embedded HW/OS
�� TIMEDTIMED--IOIO
�� Hierarchical LANHierarchical LAN’’ss
�� Enterprise SMP machinesEnterprise SMP machines
�� AdAd--hoc networkshoc networks
�� COTS HW & SWCOTS HW & SW
�� Databases, GISDatabases, GIS
�� Classification/EncryptionClassification/Encryption
17SYSTEM REQUIREMENTS
�� EndEnd--toto--end Latency & Jitterend Latency & Jitter
�� Safety & FaultSafety & Fault--tolerancetolerance
Soft
Firm
Hard
Small Large Ultra-Large
AwarenessAwareness
PlanningPlanning
Scale
Realtime
ExecutionExecution
�� Throughput performanceThroughput performance
�� Dependability & AvailabilityDependability & Availability
�� SecuritySecurity
�� COTS (DBMS/Web) integrationCOTS (DBMS/Web) integration
Tactical interoperability
COTS interoperability
DDS-APPLICABILITY
19DDS SUITABILITY: FUNCTIONALS (Specification)
�� RELIABILITYRELIABILITY�� TRANSPORT_PRIORITYTRANSPORT_PRIORITY�� OWNERSHIP/LIFELINESSOWNERSHIP/LIFELINESS
Soft
Firm
Hard
Small Large Ultra-Large
EFFICIENCY & AVAILABILITYEFFICIENCY & AVAILABILITY
INTEROPERABILITYINTEROPERABILITY
Scale
Realtime
DETERMINISMDETERMINISM
�� LATENCY_BUDGETLATENCY_BUDGET
�� FT DURABILITYFT DURABILITY
�� IDL/CORBAIDL/CORBA
�� XML ?? XML ??
�� SQL ??SQL ??
Partitions / Domains
IDL���� XML/JSON ??
20DDS SUITABILITY: NON-FUNCTIONALS (Implementation)
�� Info priority based network schedulingInfo priority based network scheduling�� Safety & FaultSafety & Fault--tolerancetolerance�� Footprint & RTOS supportFootprint & RTOS support
Soft
Firm
Hard
Small Large Ultra-Large
Scale
Realtime
�� Latency budget based packingLatency budget based packing
�� Distributed/faultDistributed/fault--tolerant durabilitytolerant durability
�� WebWeb--services gateway ??services gateway ??
�� DBMS gateway ??DBMS gateway ??
Common Information Models
COTS interoperability
INTEROPERABILITYINTEROPERABILITY
DETERMINISMDETERMINISM
EFFICIENCY & AVAILABILITYEFFICIENCY & AVAILABILITY
21DDS SUITABILITY: Standardization Challenges
�� PriorityPriority--bandingbanding
�� Traffic shapingTraffic shaping
Soft
Firm
Hard
Small Large Ultra-Large
Scale
Realtime
�� Shared subscriber cachesShared subscriber caches
�� UserUser--defined storagedefined storage--keyskeys
�� XML based API/topicsXML based API/topics
�� DBMS: topic/table mappingDBMS: topic/table mapping
Corba/RTJava integration
DBMS/Web-serviceintegration
EFFICIENCY/SCALABILITYEFFICIENCY/SCALABILITY
INTEROPERABILITYINTEROPERABILITY
RTRT--DETERMINISMDETERMINISM
Use Case: OpenSplice
…. one size fits all … ???
23USE-CASE: OpenSplice, Addressing the challenges
�� RT PriorityRT Priority--bandingbanding
�� Traffic shaping (size/burst)Traffic shaping (size/burst)
�� InfoInfo--priority prepriority pre--emptionemption
Soft
Firm
Hard
Small Large Ultra-Large
Scale
Realtime
�� DLRL data managementDLRL data management
�� Shared/readShared/read--only cachesonly caches
�� UserUser--defined/Dynamic keysdefined/Dynamic keys
�� XML: topicXML: topic--definition & APIdefinition & API
�� DBMS: topic/table mappingDBMS: topic/table mapping
- (Any) Corba integration: compilable C++ API)- (RT) Java: Pluggable JAVA API on C-core
- Soap-service: Tuner™- DBMS-service: 2-way
ININ--MEMORY (O)DBMS BASED COREMEMORY (O)DBMS BASED CORE
PLUGGABLE (GATEWAY) SERVICESPLUGGABLE (GATEWAY) SERVICES
RT NETWORK_SCHEDULERRT NETWORK_SCHEDULER
24
Enterprise(Windows, Solaris, Linux, AIX)
Embedded(vxWorks, QNX, LynxOS)
Webservicegateway
NetworkScheduler
Fault-tolerantDurability
Minimum Ownership Content-Subscription
Persistence
DCPS
DLRL
DDS API’sC/C++/Java
DynamicXML-based API
Monitor &Control (Tuner)
Logging &Replay
Configuration
SECURITY
InformationModeling
ApplicationModeling
DeploymentModeling
OMG-DDSProfiles
Pluggable-Services
SupportedPlatforms
Middleware-Integration
High-levelServices
UML DDS-DSL
Remote-connect
EclipseFoundation &
Framework
Roundtrip-Engineering
Modeling-Plug-ins
Views &Editors
Development
Deployment
DBMSGatewaysoap
USE-CASE: Creating & Deploying DDS-based systems
COHABITATION
Corba/Java/Dbms
25
Enterprise(Windows, Solaris, Linux, AIX)
Embedded(vxWorks, QNX, LynxOS)
Webservicegateway
NetworkScheduler
Fault-tolerantDurability
Minimum OwnershipContent-
SubscriptionPersistence
DCPS
DLRL
DDS API’sC/C++/Java
DynamicXML-based API
Monitor &Control (Tuner)
Logging &Replay
Configuration
SECURITY
InformationModeling
ApplicationModeling
DeploymentModeling
OMG-DDSProfiles
Pluggable-Services
SupportedPlatforms
Middleware-Integration
High-levelServices
UML DDS-DSL
OpenSplice Tuner &Remote-connect
EclipseFoundation &
Framework
Roundtrip-Engineering
Modeling-Plug-ins
Views &Editors
DBMSGatewaySoap
( Potential )DDS-related
Standardization
UML-profile for DDS(joint submission with SPARX systems)
Dynamic Topics(requirements: DDS-PSIG)
Dynamic DBMS gateway(available, but
IDL/table mapping not standardized)
Webservice gateway(available and used by OpenSplice Tuner™,
but no standard yet)
Security for DDS(node-to-node and end-to-end,
encryption available but no standard yet)
Logging & Replay(following C4I standardization)
Pluggable Discovery(for ultra-large/WAN systems,Several European/US studies)
CONCLUSION: ‘One concept’ fits all, we’re not there yet..
COHABITATION
Corba/Java/Dbms