Distributed Interactive Computing Environment .
-
Upload
bernadette-norris -
Category
Documents
-
view
220 -
download
0
Transcript of Distributed Interactive Computing Environment .
CFD Code 1
CSM Code 1
CCM Code 2
CCM Code 1
Preprocessing
Runtime Analysis
Postprocessing
HPC Codes
Platforms
Tools
Challenge : Stovepiped Applications
Additional Challenges
• Code Setup can be Difficult
• Enormous Amounts of Data
• Heterogeneous Environment
• Security
High PerformanceComputing Resource
User’s Workstation
Graphical UserInterface and Runtime Visualization
Distributed Interactive Computing Environment
Common Distributed Data Model and Format
What is DICE ?
• Heterogeneous DSM *
• Data Organization
• Graphical User Interface
• Visualization
A Distributed Computing Environment Composed of Configurable Mega-Components
User-Ready Computing Environments Customized for Specific Technology Areas
Process Local Address Space
System Shared Memory Disk File
0 10M 30M 50M
Physical Data Storage
Servers
Clients
Network Distributed Global Memory
Put Get
VectorPut VectorGet
Barrier Semaphore
Light Data
• 3D Structured Mesh
• 300 x 200 x 300
• Number of Materials
• Material Names
• Total Residual Mass
Heavy Data
• X, Y, Z Values
• Material Volume Fractions
• Cell Temperatures
Operating System
Distributed Resources
Network Distributed Global Memory ( NDGM )with GSSAPI
Hierarchical Data Format ( HDF5 )Data Format : Number Type, Data Space ...
DICE Object DirectoryObject Oriented Support for Data Model, XML, ...
High PerformanceComputing Code
VisualizationToolkit (vtk)
OpenGL
Mesa
Visual GeneratorsPlots
PlanesSurfaces
Scripting and Graphical User Interface Tools
User Application
File I/OProcess Control
Shared Memory FacilityTCP/IP Sockets
DICE Data Model and Format
Distributed Buffer of Bytes
Structured Data Unstructured Data Groups
Convenience Layer
Network Distributed Global Memory with
GSSAPI
Hierarchical Data Format ( HDF5 )Data Format : Number Type, Data Space ...
DICE Object DirectoryObject Oriented Support for Data Model, XML, ...
FileSystem Serial and Parallel
Global Access to Secondary Storage ( Globus - GASS )
Other ...
Hierarchical Data Format Version 5
• Host Independent Data Format
• Access via API only
• Physical Storage thru Drivers
• Includes Data Compression
• Includes SubSelections
Group
DataSet
Attribute
Group Group
Group
Group
DataSet
DataSet
DataSet
Attribute
Attribute
Attribute
Attribute Not Allowed
eXtensible Markup Language
• Similar to HTML Format
• Human Readable
• Pervasive on the WEB
• Strict Hierarchy with No Inheritance
• Many Parsing Tools Already Exist
< Parent Name=“Top”>
< Child Name=“Left”>
Character Data for Left
< /Child >
<Child Name=“Right” >
Character Data for Right
< /Child >
< /Parent >
Parent
Child
Character Data for Left
Child
Character Data for Right
Document Object Model
In-Memory Data Structures
XML
Parse
Serialize
1
1 1 1
11 1
1
2 2 2 2
2 2
2
2
3
33
3
3
33
3
4
4 4
4
4
4
4
5
55
5
6
6
78
NN
i
j
ki
j
12
3
4
5N
Polyvertex Polyline Triangle Quadrilateral Polygon
Tetrahedron Pyramid Wedge
Hexahedron 2DSMesh3DSMesh
DICE Object Directory Base Topologies
HDF5
NDGM
HPC Code
GASS MPI-I/O . . . .
<Topology Type=“Tetrahedra” NumElements=“3”>
</Topology>
<Geometry>
<Vertices>
<Index Location=“XML” Type=“List”>
1000 1001 21001 21000 ……
</Index>
<Points Type=“XYZ” Location=“HDF” >
NDGM:File1.h5:/Grid1/Geometry/MyPoints
</Points>
</Vertices>
</Geometry>
<Topology Type=“3DSmesh”
Dimensions=“10 20 30”>
</Topology>
<Geometry>
<Points Type=“XYZ” Location=“HDF” >
NDGM:File1.h5:/Grid1/Geometry/MyPoints
<Region Type=“Slab” Location=“XML” >
1 2 10 2 3 20 3 4 30
</Region>
</Points>
</Geometry>
Light Data
100 x 200 x 300
HDF5
File
Domain
Topology
100 Hexahedra
Grid
Geometry XYZPoints.h5
Attribute
NDGM:Pressure.h5
XML
HPC Code
Domain
HDF5
Complexity of Model Defined in Light Data
HPC Code I/O is Natural and Efficient
NDGM GASS
Attribute
NDGM:Temperature.h5
Grid
HDF5
Domain
Topology
100 Hexahedra
Grid
Geometry XYZPoints.h5
Parameter TimeStep = 0 to 99
Attribute
NDGM:&TimeStep;/Pressure.h5
XML
HPC Code
Domain
HDF5 for TimeStep 0
Grid
XML Contains Parameters for Variable Substitution
HDF5 for TimeStep 1
HDF5 for TimeStep 2
Hierarchical Data Format Verision 5 ( HDF5 )
eXtensible Markup Language ( XML )
HPC Codes
Pre-Processing ToolsPost-Processing Tools
Runtime Tools
Distributed Data Hub
Parallel / Distributed Entities
Convenience Layer
DICE Object Directory
DICE Data Reader for EnSight
Data Reader uses DICE Object Directory API
EnSight
DICE Visualization using vtk
Source
Filter Filter
Filter
Visualization “Subnet”
Polygons
HDF5
File
HDF5
NDGM GASSDICE Object
Directory
System Programming LanguageC C++ Fortran77 Fortran90 etc.
SWIG Compiled Object
Tcl Wrapper
Perl Wrapper
Python Wrapper
Java Wrapper
New Scripting Commandvia
Dynamically Loadable Library}
Adding Performance Friendly Functionality
DiceFloat64Array MyData, NewData;
DiceInt64Array MyIndex;
MyData.SetNumberOfElements( 10000 );
MyData.Generate(0.0, .9999 );
MyIndex.SetNumberOfElements( 100 );
MyIndex.Generate( 0, 99 );
MyIndex *= 2;
NewData = MyData[ MyIndex ];
DiceHDF H5Object
H5Object Open NDGM:FileName.h5:/DataSetName
DiceMemory MemoryObject
MemoryObject SetArray [ DiceFloat32Array ]
H5Object Read [ MemoryObject cget -this ]
set NewData [ MemoryObject GetArray ]
DiceExpr $NewData[ where( $NewData > .5 ) ] = .5
C++
Scripting ( Tcl, Perl, etc. )
Allocate 10,000 Doubles
Set Individual Values
Array Operation +, -, *, /
Array Indexing
Object Oriented Scripting Interface
Open HDF5 File, Query Number Type and Shape
Request Automatic Conversion to Float32
MemoryObject has Number Type and Shape
Clip Values to .5
DICE on HPC Resourceusing Off Screen Rendering
SSL Kerberos kinit
• Every URL Request is Intercepted to Verify Session Id
• DICE Runs as a User Process
• HTML FORMS result in Tcl Procedure Invocation
• Graphics Updated via Server-Push ( Netscape only ) or Client-Pull
• Intended for Limited Interfaces from Remote ( Slow Connection ) or Graphically Challenged Client Machines.
DICE Web InterfaceUnder Construction
“Gently Coupling” Codes from Separate Disciplines
Finite Volume Structural Mechanics
Global Monitor
Runtime Visualization
Small Scale Analysis
Injection Flow Analysis
Distributed Data Hub