FEM with Parallel SQL FEM with Parallel SQL Server: A Case StudyServer: A Case Study

Gerd HeberGerd HeberCornell Theory CenterCornell Theory CenterCornell Fracture GroupCornell Fracture Group

Thank YouThank You

Dan Fay (MSR)Dan Fay (MSR) Jim Gray (MSR)Jim Gray (MSR) Todd Needham (MSR)Todd Needham (MSR) Alexander Szalay (JHU)Alexander Szalay (JHU)

OutlineOutline

Parallel SQL ServerParallel SQL Server ContextContext

InfrastructureInfrastructure ApplicationApplication

ExamplesExamples ComplexityComplexity

IssuesIssues ConclusionsConclusions

Parallel SQL ServerParallel SQL Server

HardwareHardware SMPSMP Distributed memoryDistributed memory

SoftwareSoftware Query level parallelismQuery level parallelism

Partitioned views (LDPV)Partitioned views (LDPV) Linked serversLinked servers

Distributed partitioned views (DPV)Distributed partitioned views (DPV)

When to Use DPVWhen to Use DPV

Scale-out (DPV) vs. scale-up (SMP)Scale-out (DPV) vs. scale-up (SMP) Good performance on commodity Good performance on commodity

hardwarehardware Data (and queries) must be suitable for Data (and queries) must be suitable for

partitioningpartitioning Increased application complexityIncreased application complexity

Go to the server with most, or all, of the dataGo to the server with most, or all, of the data

For reliability consider failover For reliability consider failover clusteringclustering

No support for parallel (bulk) insertsNo support for parallel (bulk) inserts

CTC’s Infrastructure TodayCTC’s Infrastructure Today

Domain

Active DirectoryDatabase Servers

Web Servers

Certificate Server

Tape Robot

CaveLogin Nodes

Scheduler

Clusters

File Servers

SCTS

Basic FEM AnalysisBasic FEM Analysis PreprocessingPreprocessing

Topology/Geometry generationTopology/Geometry generation Mesh generationMesh generation Apply boundary conditionsApply boundary conditions Material propertiesMaterial properties

SolutionSolution Equation solvingEquation solving Error analysisError analysis

Post processingPost processing Data analysisData analysis

VisualizationVisualization

How We Used To Do ThingsHow We Used To Do Things

100% file-based100% file-based Monolithic (brittle) codeMonolithic (brittle) code DisconnectedDisconnected No data-sharing, except copyNo data-sharing, except copy Hard to debugHard to debug Plenty of non problem oriented codePlenty of non problem oriented code

Where We Use SQL ServerWhere We Use SQL Server

Data storageData storage AnalysisAnalysis

DebuggingDebugging VisualizationVisualization

ProcessingProcessing Checkpoint / restartCheckpoint / restart Web service state managementWeb service state management Data virtualizationData virtualization

XML repositoryXML repository

InputInput

100 MB - 1 GB (today)100 MB - 1 GB (today) Files ASCII (incl. XML), binaryFiles ASCII (incl. XML), binary Topology, geometry, meshTopology, geometry, mesh Initial / boundary conditionsInitial / boundary conditions Material propertiesMaterial properties

Input may or may not be partitionedInput may or may not be partitioned

OutputOutput Physical fieldsPhysical fields

Temperature (1x double per node)Temperature (1x double per node) Displacement (3x double per node)Displacement (3x double per node) Stress, strainStress, strain (6x double per Gauss point) (6x double per Gauss point)

Tetrahedron: 5 / 11 Gauss pointsTetrahedron: 5 / 11 Gauss points Hexahedron: 27 / 64 Gauss pointsHexahedron: 27 / 64 Gauss points

State variablesState variables Mises plasticity (13x double per Gauss point)Mises plasticity (13x double per Gauss point) Polycrystal plasticity (>= 30x double per GP)Polycrystal plasticity (>= 30x double per GP) ……

Produce 10 - 1000 times the input sizeProduce 10 - 1000 times the input size

10-3 10-6 10-9 m | s

DatasetsDatasets

Pictures provided by Paul Wawrzynek, Cornell Fracture GroupPictures provided by Paul Wawrzynek, Cornell Fracture Group

ExamplesExamples

Cracked

Closed

Visual SQLVisual SQL

Pictures provided by John Emery, Cornell Fracture GroupPictures provided by John Emery, Cornell Fracture Group

DDSimDDSim

ES7000

Spatial SearchSpatial Search Jim Gray et al., Jim Gray et al., There Goes the Neighborhood: There Goes the Neighborhood:

Relational Algebra for Spatial Data SearchRelational Algebra for Spatial Data Search, MSR-TR-, MSR-TR-

2004-322004-32

Web ServicesWeb Services

Adaptive Software ProjectAdaptive Software Project NSF-ITR #0085969: NSF-ITR #0085969:

Adaptive Software for Adaptive Software for Field-driven Field-driven Simulations (09/01/00)Simulations (09/01/00)

Implement a system Implement a system for multi-physics for multi-physics multi-scale adaptive multi-scale adaptive CSE simulationsCSE simulations Computational fracture Computational fracture

mechanicsmechanics Chemically-reacting Chemically-reacting

flow simulationflow simulation Understand principles Understand principles

of implementing of implementing adaptive software adaptive software systemssystems

Adaptivity in CSE SimulationsAdaptivity in CSE Simulations

Application-level adaptivityApplication-level adaptivity Change in modeling / governing equationsChange in modeling / governing equations Example: Elasticity PDE’s vs. molecular-scale Example: Elasticity PDE’s vs. molecular-scale

interactions, symmetryinteractions, symmetry Algorithm-level adaptivityAlgorithm-level adaptivity

Change in solution method for governing Change in solution method for governing equationsequations

Example: Finite-element vs. wavelet basesExample: Finite-element vs. wavelet bases System-level adaptivitySystem-level adaptivity

Response to changing resource availabilityResponse to changing resource availability Example: Processor / link failureExample: Processor / link failure

Test ProblemsTest Problems

(Substantial) Infrastructure(Substantial) Infrastructure

Metadata managementMetadata management State managementState management Event loggingEvent logging Data virtualizationData virtualization AccountingAccounting TransactionsTransactions

YukonYukon

NET CLR integrationNET CLR integration Stored procedures, user-defined Stored procedures, user-defined

functions, and triggers in .NET languages functions, and triggers in .NET languages (and T-SQL)(and T-SQL)

Call unmanaged (unsafe) codeCall unmanaged (unsafe) code User defined aggregates and typesUser defined aggregates and types

Native XML data type (schema support)Native XML data type (schema support) XQuery supportXQuery support Database logic can be invoked as Web Database logic can be invoked as Web

serviceservice

XML RepositoryXML Repository

Polycrystal GenerationPolycrystal Generation

Comments / Issues / WishesComments / Issues / Wishes SQL LibrariesSQL Libraries Better management tools for linked serversBetter management tools for linked servers Embedded SQL renaissanceEmbedded SQL renaissance WSE 2.0 and YukonWSE 2.0 and Yukon WS interface for WMIWS interface for WMI Template(s) for WS state and event Template(s) for WS state and event

managementmanagement O’SOAPO’SOAP

Visual SQLVisual SQL Virtualization not there (yet)Virtualization not there (yet) Data gridsData grids

ConclusionConclusion

““Language shapes the way we Language shapes the way we think, and determines what we think, and determines what we can think about.”can think about.” (B.L. Whorf)(B.L. Whorf)

It’s a slow processIt’s a slow process Most engineers are conservativesMost engineers are conservatives LegacyLegacy

SponsorsSponsors

DARPADARPA IntelIntel MicrosoftMicrosoft Microsoft ResearchMicrosoft Research NASANASA NSFNSF Northrop GrummanNorthrop Grumman Unisys Unisys