CGNSAnd pyCGNS, its Python binding

Poinot MarcONERA - DSNA/ELSA

/ELSA/PRS-02036

poinot@onera.fr

http://www.onera.fr

/ELSA/PRS-02036

Overview

CGNSGoals

History

Actors/ Components

Examples

Future/ ISO process

pyCGNSGoals/ Status

Interoperability architecture

Examples

Proprietary data format and semantics

ISO data format and semantics

/ELSA/PRS-02036

CFD (Computational Fluid Dynamics) General Notation System

CFD standardWe want to use a common data representation for interoperability

• Data semantics• Physical representationGrid creation/ Solvers/ VisualizersPre/Post processing of very large amount of data

We want to ensure our data a long life format• Consistency• Timelessness

Provide a general, portable and extensible standard for storage and retrieval of CFD analysis data.

/ELSA/PRS-02036

Brief history

Started 1994NASA + Boeing to improve technology transfert

NASA contract 1995-1998SIDS document written by S.Allmaras, Boeing

Library implementation (+ICEM +NPARC Alliance)

Steering Committee 1999…Gather people

• 17 members• +500 users• web site

Software and documentation releases

ISO Process• Initiated and supported by Boeing

/ELSA/PRS-02036

Steering Committee

Airbus

Boeing commercial

Boeing Phantom Works

Boeing Space & Communications

CD ADAPCO Group

Fluent, Inc.

ICEM CFD Engineering

Intelligent Light

NASA Ames

NASA Langley

NASA Glenn

NUMECA International

ONERA

Pointwise, Inc.

Pratt & Whitney Aircraft

Rolls-Royce Allison

US Air Force

MissionEnsure the continuation of CGNS

ResponsabilitiesSoftware devt., maintenance, test…

Documentation

Insure a soft. and doc. free distribution

Provide expertise and extensions proposals

Promote acceptance of the standard

OrganizationMain meeting once per year

One chairperson per organization

Governs by consensus

Welcomes participation of all parties, members or not

/ELSA/PRS-02036

Overall architecture

Data semantics and Syntax

CGNSLib

File format

Node API

Specific Sub-trees API

/ELSA/PRS-02036

Components

DocumentationSIDS document

API and users documentation

Technical papers

Libraries/ sourcesMLL and ADF libraries (Fortran +C)

• Binaries• Sources

User tutorial examples

Data file examples

/ELSA/PRS-02036

Data exchange example - An informal view

Fluid solverMesh generator

Communication layer

Post processor

WB234

wing

body

coordX

bodyConnectivity

wingZones

solution#1

pressuredensity drag

/ELSA/PRS-02036

Example - A CGNS tree WB234

wing

coordXSolution#1

density drag

WB234CGNSBase_t

DescriptionWing and body test case 234

Descriptor_t

Date13/03/2002Descriptor_t

Zone_twing

ZoneType_tStructured

GridCoordinates_tCoordinateX

solution#1FlowSolution_t

DataArray_tDensity

UserDefined_tDrag

DimensionalUnits_tDimensionalUnits

DataArray_tdragCoef

/ELSA/PRS-02036

Example - An ADF node WB234

wing

coordXSolution#1

density drag

Name: DensityLabel: DataArray_t DataType: DataTypeDimensions: IndexDimension

Parameters: DataType, IndexDimension, DataSize

Cardinality: 0,N

DimensionValues: DataSize[]Data: [values]

Children: [nodes]

/ELSA/PRS-02036

Development process

CommunityMailing list+ Web site+ Users

Meeting in Reno (AIAA annual conf.)

Teleconferences (every 2 months)

ActorsStarted with ICEM CFD as main developper

Now Intelligent Light also

Members/ users contributions

Means/ processCVS server

Heavy testing/ Insure compatibility/ Check portability

/ELSA/PRS-02036

Extension process

Extending the standardThe SIDS and its implementation has been though extensible from start

Documentation and consensus first

ProcessExtension submission and discussions using mailing lists

Sub-group creation, seek for expertise, document drafting

Proposal for extension, implementation

Some in-progress extensionsPeriodic boundary condition/ Interfaces/ Rotating coordinates

Chemical species/ Multi-Phase and liquid spray/ Gravity

Object-oriented API/ In-memory tree representation

/ELSA/PRS-02036

ISO process

ISO/STEPApplication Protocol 237 - Fluid dynamics

Part 110 - Computational Fluid Dynamics

Part 52 - Mesh-based topology

Part 53 - Numerical analysis

ScheduleApproval as IS in the second half of 2003 for parts 110, 52 and 53

AP 237 second quarter of 2005

Drafts commentsPDF files + CGNS mailing list

See CGNS web site

/ELSA/PRS-02036

Our users

UsersAerospace community

• Industry

• Research (Academy, Govt.)

Software editors

• CAD, mesh generation

• Post-processing tools, viewers

Up to 500 registered users (25 countries)

No declared user in Asia or Japan

Sample usersBoeing

• Adopted CFD data format

• Many benefits

– Common data semantics

– Improved modularity

– Workflow optimization

• Conformance to CGNS for new CFD tools

Pratt & Whitney

• Translation tools between proprietary format and CGNS

• Improve workflow

• Automation of design processes

NASA Langley

• Add CGNS capabalities to CFL3D code

/ELSA/PRS-02036

Future directions

Evolve/ MaintainUse web site to scatter/gather information, examples, tools

Adds extension to meet users needs

Document more and more, tutorial, practical guides

Provide or refer to open source tools

SpreadISO

AIAA recommended practice

Emphasize on the use of CGNS in actual computations

Participate to conferences (including NASA/ESA Aerospace PDE workshop)

Keep source and docs as Open Source

/ELSA/PRS-02036

ONERA

The French aeronautics and space research centerFrom basic research to flight testing

All disciplines involved in aircraft, spacecraft and missile design

DSNAComputational Fluid Dynamics and Aeroacoustics Dept.

ELSA: Software unit in charge of elsA software

elsANew generation CFD solver

• Industrial tool• Framework for research• Gather 20 years of CFD methods and software expertise

Technologies leading edge• OO design and implementation (C++, Python)• Efficiency (scalar/vector, space/time)• High level QA

/ELSA/PRS-02036

CGNS related developments at ONERA

elsAMain I/O data representation

• SIDS compliant• Interoperability / Multi-processing

– In-memory and disk tree low level• Validation resources

pyCGNSA Python binding to CGNS

Used as gluing language for application framework• e.g. translators to CGNS trees

– From/To Tecplot and Aerospace actors data format

Open source

/ELSA/PRS-02036

elsA Software

InteroperabilityMulti-platform/ Multi-applications

• Fluid/ Structures/ Optimization/ Thermics/

pre/post processing know how

TechnologiesOO design/ implementation

• C++ kernel• Fortran for computation leaves• Python user interface

A single representation for I/O• XDT trees + parsers for subset translation from/to

CGNS/ XML/ Python

/ELSA/PRS-02036

Python programming langage

Interpreted and OO script languageEasy to learn

Powerful and extensible• Shared libraries• Package system• Very large scope of applications

Portable• Including NEC, Cray, Fujitsu

Efficient modules for numerical applicationsAddress numerical simulation interoperability concerns

• Contiguous memory arrays• Can point to an already allocated zone• Optimized operations on whole data

/ELSA/PRS-02036

Ease of use example

a=1.4+0.2

b=[1.2, 0.3E-10, 1.4]

import Numeric

c=Numeric.ones([3])*a+b

print "Values :", c

import math

for d in c:

print math.cos(d)

Values : array([2.8, 1.6, 3.])

-0.942222340669

-0.0291995223313

-0.9899924966

/ELSA/PRS-02036

In-memory trees

GoalHave a in-memory representation of trees

• Communicate semantics using trees• Have a contiguous memory zone

– share– transfert

• Use memory buffer based systems– MPI/ SHM/ RPC/

• Fast dump to disk/ recovery

ImplementationADF API unchanged

• New implementation • Thanks the two layers design of CGNS libraries

/ELSA/PRS-02036

Architecture with Python modules

Tree on disk: File system/DBMS

Tree in-memory: Python interpreterRPC

SHMMPI

TCP/IP

/ELSA/PRS-02036

Interoperability example

Two solvers and glueStructure solver

Glue FFT+CGNS tree

Fluid solver

Visualization

Main loopStructure computation

Translator HOST/ CGNS• SPARK parser• Numerical Python (arrays+FFT)• pyCGNS tree

Fluid computation• Full CGNS

Tecplot viewer• Read CGNS

/ELSA/PRS-02036

Semantic/structure & efficiency

Same treeData consistency

"Blind" interoperability

Space/time efficiency

Ease of use

/ELSA/PRS-02036

Archival example

Validation databaseRDBMS+ proprietary tool

• Store CGNS file as binary large object – PostgreSQL blob

• Add attributes

Extracted from tree• Hidden to MLL

RDBMS extra features• Access control• Query• Clever archival server

/ELSA/PRS-02036

Code example - pyCGNS#!/usr/bin/env python# CFD General Notation System - CGNS lib wrapper# ONERA/DSNA/ELSA - poinot@onera.fr# User's Guide to CGNS - C.L.Rumsey et al. examples translation#from CGNS import midlevelfrom CGNS.wrap import *from Numeric import *#

# open CGNS file for read-onlyfile=pyCGNS('grid.cgns',midlevel.MODE_READ)

index_base=1index_zone=1index_flow=1isize=file.zoneread(index_base,index_zone)[3]irmin=[1,1,1]irmax=[isize[0],isize[1],isize[2]]

(loc,name)=file.solinfo(index_base,index_zone,index_flow)

if (loc != midlevel.Vertex): print "Error, GridLocation must be Vertex! Currently:", print midlevel.GridLocation[loc]

# read flow solutionr=file.fieldread(index_base,index_zone,index_flow, 'Density',midlevel.RealSingle,irmin,irmax)p=file.fieldread(index_base,index_zone,index_flow, 'Pressure',midlevel.RealSingle,irmin,irmax)

# close CGNS filedel fileprint "Successfully read flow solution from file 'grid.cgns'"print "For example, r,p(21,17,9)=",r[20,16,8],p[20,16,8]print "Program successful... ending now"

/ELSA/PRS-02036

Parts of XML tree(ADF tree dump using parseTree.py on web site example multi.cgns)

<NODE name='ADF MotherNode' label='Root Node of ADF File'>

<DATA type='MT' />

<DIMENSIONS total='0' />

<NODE name='CGNSLibraryVersion' label='CGNSLibraryVersion_t'>

<DATA type='R4' />

<DIMENSIONS total='1'>

</NODE>

</NODE>

<NODE name='InputFileUsed 4' label='Descriptor_t'>

<DATA type='C1' />

<DIMENSIONS total='19926'>

19926

</DIMENSIONS>

<VALUES>

FILES:

challenge.cgns

...

</VALUES>

</NODE>

<NODE name='CoordinateX' label='DataArray_t'> <DATA type='R8' /> <DIMENSIONS total='6150'> 2 123 25 </DIMENSIONS> <VALUES> </VALUES> </NODE>

/ELSA/PRS-02036

pyCGNS Status

v0.4Binding

• Full ADF wrapper + 60% MLL wrapper• Experimental OOL layer

Python + Numerical Python• Standard package• Documentation+ test suite

In-memory representation• Python/C/C++/Fortran in-memory tree sharing

Tools• SIFT - SIDS instance description check• ParseTree - Tree dump/display as XML tree• StampNode - Add sub-nodes with specific stamps (date, ownership…)

/ELSA/PRS-02036

pyCGNS v1.0 commitments

v1.0 (4Q2002)

Binding• Full ADF+MLL wrapper• Full OOL layer

Documentation• Features 100% documented + tutorial• Design partly documented

Test suite• Complete test suite for OOL/MLL/ADF

Tools• Tree parser + XML I/O with DTD• DBMS interface for blobs• Translators

/ELSA/PRS-02036

CGNS and pyCGNS references summary

CGNShttp://www.cgns.orgAIAA 2002-0752 - CFD General Notation System (CGNS): Status and future directions - S.Legensky,

D.Edwards, R.Bush, D.Poirier, C.Rumsey, R.Cosner, C.Towne

ONERA DSNA/ELSAhttp://www.onera.fr

http://elsa.onera.frAIAA 2002-0108 - An efficient Object-Oriented solution to CFD complexity - L.Cambier, M.Gazaix

pyCGNShttp://elsa.onera.fr/CGNS/releases

Pythonhttp://www.python.org