NEUDOS Kirk Oct 2009 Neutrons

Overview of Neutron

Transport Codes

11th NEUTRON AND ION DOSIMETRY

SYMPOSIUM (NEUDOS-11)

Capetown, South Africa

October 16, 2009

Presented by:

Bernie Kirk

Director, Radiation Safety Information

Computational Center (RSICC)

Nuclear Science and Technology Division

2 Managed by UT-Battellefor the U.S. Department of Energy

SUMMARY

BACKGROUND

STOCHASTIC VERSUS DETERMINISTIC METHODS

MONTE CARLO SOFTWARE

DETERMINISTIC SOFTWARE

CONCLUSION

Science News

ScienceDaily Oct. 1, 2009

Spallation Neutron Source First Of Its Kind To Reach Megawatt Power

— The Department of Energy's Spallation Neutron Source (SNS), already the world's most powerful facility for pulsed neutron scattering science, is now the first pulsed spallation neutron source to break the one-megawatt barrier.

An image made possible by a

phosphorescent coating

colorfully illustrates one

megawatt of power striking

the Spallation Neutron

Source's mercury target.

(Credit: Image courtesy of

DOE/Oak Ridge National

Laboratory)

http://sns.gov/images/sns_aerial_640.jpg

Supercomputers Twenty Years Ago


Stochastic Method (Monte Carlo)

Deterministic Method

Radiation Transport

' '

''',',',',

,,,,4

1,,,,

E

s

tf

ddEErEEr

ErErErSErSEr

Neutron Transport Equation


MONTE CARLO SOFTWARE

Electron/Photon

Transport

EGS5

PENELOPE

EgsNRC

NEUTRON Transport

SCALE – MONACO, MAVRIC, KENO

MCNP/MCNPX

TRIPOLI

VIM

TART

SERPENT

High

Energy

Transport

PHITS

GEANT4

FLUKA

MCNPX

SHIELD

NEUTRON Transport

SCALE - DENOVO

PARTISN

PENTRAN

DETERMINISTIC

SOFTWARE

Non-commercial Software


SCALE

http://www.ornl.gov/sci/scale/

Developed at Oak Ridge National Laboratory

Contact: Brad Rearden, [email protected]

Available:

RSICC http://rsicc.ornl.gov

NEADB http://www.nea.fr/html/databank/welcome.html



mailto:[email protected]

http://rsicc.ornl.gov/

http://www.nea.fr/html/databank/welcome.html


SCALE Standardized Computer Analyses for Licensing Evaluation

• SCALE is a modular code system that uses automated sequences to provide:

– Problem-dependent cross-section processing

– Reactor / lattice physics analysis

– Criticality safety analysis

– Sensitivity/uncertainty analysis

– Radiation shielding analysis

– Spent fuel and HLW characterization

– Advanced 3-D visualization and automated user interface

• Distributed and used worldwide for > 25 years

• Developed at Oak Ridge National Laboratory


D&D

Material processing and fabrication Commercial and

research reactors

Recycling

StorageTransport

SCALE provides mature and

flexible software for nuclear

analysis of nearly any application

within the nuclear fuel cycle

SCALE is used

worldwide (~30

nations) by

regulators,

vendors, utilities,

cask designers,

R&D labs,

safeguard

agencies


MONACO/MAVRIC (SCALE)


Contact: Douglas Peplow, [email protected]



• Neutron/Photon

• general-purpose, fixed source, multigroup Monte Carlo shielding

• multigroup transport methods are inherited from

Monaco’s predecessor, MORSE

Monaco – Multigroup Monte Carlo


• geometry data as sets of quadratic equations -same geometry package as KENO-VI

• Flexible, friendly user input

– Source description is separable:

space, energy, direction

– Region tallies, mesh tallies, point detector tallies

– Integrates fluxes with response functions (dose)

• Variance Reduction capabilities

– Weight windows based on region/energy

– Mesh-based weight windows

• MeshView plotting software

– Plot calculated responses on mesh grid

Monaco – Multigroup Monte Carlo


MAVRIC – Monaco with Automated

Variance Reduction using Importance

Calculations

• Intended for challenging, deep-penetration problems

• CADIS (Consistent Adjoint Driven Importance Sampling)

– Denovo is used to calculate the coarse-mesh adjoint flux for a specific tally

– Creates importance map (space, energy) and biased source

– Monaco is then optimized for that specific tally

• Forward Weighted CADIS

– Denovo estimates forward fluxes, used in the adjoint source

– Helps balance relative uncertainties across multiple tallies or large mesh tallies

14 Managed by UT-Battellefor the U.S. Department of Energy x-distance (cm)

y-d

ista

nce

(cm

)

0 20 40 60 80 100 120 140 1600

20

40

60

80

100

120

140

160

Biased

8.4E-02

3.7E-02

1.6E-02

7.1E-03

3.1E-03

1.4E-03

6.1E-04

2.7E-04

1.2E-04

5.2E-05

2.3E-05

1.0E-05

Example: PWR Ex-Vessel Thermal (10B) Detector Response

Detector

Core

Cavity Pressure vessel

Downcomer

Neutron pads

Baffle plates

Flow channel

Core barrel

Concrete shield

Monte Carlo model Deterministic model

x-dimension (cm)

y-d

ime

nsio

n(c

m)

50 100 150 200 250 300

50

100

150

200

250

300

adjoint

5.00E+30

7.46E+29

1.11E+29

1.66E+28

2.48E+27

3.70E+26

5.51E+25

8.23E+24

1.23E+24

1.83E+23

2.73E+22

4.07E+21

6.08E+20

9.07E+19

1.35E+19

2.02E+18

3.01E+17

4.49E+16

6.70E+15

1.00E+15

Adjoint data

Calculate/apply VR Parameters

CASE

CPU TIME TO

ACHIEVE RE=1%

(h)

SPEEDUP

No VR 8.86E+4 (10.1 yrs) 1

Manual VR 13.6 6500

CADIS VR 1.02 87000

Required ~3 weeks by an experienced MC practitioner using all applicable

MCNP4C VR capabilities

Faster Results

CADIS Methodology for

Automated Variance Reduction


MAVRIC: Dose Rates from Cask Array

Analog calculation:

560 hours, poor resolution in mesh tally

Automated variance reduction: 109 hours,

80% voxels < 5% rel unc

97% voxels < 10% rel unc


KENO (SCALE)


Contact: Sedat Goluoglu, [email protected]



KENO Multigroup Monte Carlo Code

for Criticality Calculations

• KENO was first developed during 1960’s and is internationally accepted tool for criticality safety analysis

• SCALE has two versions

– KENO.Va: restricted geometries, but faster

– KENO-VI: general geometry; rotations; intersections

• Multigroup method runs much faster than continuous energy, but requires self-shielding corrections to XS’s


SCALE 6: Criticality

Safety Enhancements

• Continuous energy capability incorporated in KENO

• ENDF/B-VI and ENDF/B-VII libraries included.

LEU

0.95

0.96

0.97

0.98

0.99

1

1.01

1.02

0 50 100 150 200 250

Benchmark number (arbitrary)

Ke

ff

-2.0

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

2.0

2.5

3.0

(C-B

)/B

%

Calculated Percent difference

CE validation results for LEU benchmarks

CE vs. Multigroup


Explicit 3D neutron

transport model of

application

keff,

EALF,

etc.

neutron flux

continuous-energy

or

multi-group

reaction rates


SCALE: KENO-VI and MAVRIC Coupling

Step 2:

•Use MAVRIC to model particle transport

•Use KENO-VI fission distribution as the

source for MAVRIC

•Calculate detector responses, dose rates

at specific points or mesh tallies of dose

rates

layers of UF4

West wing of the ORCEF,

building 9213Step 1:

•Use KENO-VI to model the

criticality accident and save the

fission distribution as a mesh

source

Dose in rem per 1019 fissionsSpeedups of 3000 to 4500


MCNP/MCNPX

http://mcnp-green.lanl.gov/

Monte Carlo N-Particle

Developed at Los Alamos National Laboratory

Contact: Tim Goorley, [email protected]

Available:RSICC http://rsicc.ornl.gov









MCNP/MCNPX

neutron, photon, electron, or coupled neutron/photon/electrontransport

radiation protection and dosimetry,

radiation shielding, radiography

medical physics,

nuclear criticality safety,

Detector Design and analysis,

nuclear oil well logging,

Accelerator target design

Fission and fusion reactor design

decontamination and decommissioningCalculate

– Flux, Current, Energy or Charge Deposition,

Heating, Reaction

Rates, Response Functions, Radiographs, Mesh

Tallies (E, θ, t bins)

– keff, prompt neutron lifetime, fission

distributions, η, ν, Ē of

neutrons causing fission, neutron balance per

cell and nuclide.


MCNP History

• 50+ years of code development!

– World recognized experts in several fields

• Impressive physics, geometry, tally, variance reduction capabilities

• Modern Teraflop supercomputers can drive Monte Carlo calculations not dreamed possible more than a decade ago

• Now used for design of many systems (criticality example)

1960s: K-effective

1970s: K-effective, detailed assembly power

1980s: K-effective, detailed 2D whole-core

1990s: K-effective, detailed 3D whole-core

2000s: K-effective, detailed 3D whole-core,

depletion, reactor design parameters

1992 200919771940s


MCNP / MCNPX Teams

• MCNP

• Deputy Group Leader: Jeremy Sweezy

• Project Lead: Tim Goorley

• Tom Booth

• Forrest Brown

• Jeff Bull

• Art Forster

• John Hendricks

• Grady Hughes

• Roger Martz

• Stepan Mashnik

• Avneet Sood

• Tony Zukaitis

• MCNPX

• Team Lead: Gregg McKinney

• Project Lead: Laurie Waters

• Joe Durkee

• Jay Elson

• Michael Fensin (N-4)

• John Hendricks (X-3)

• Shannon Holloway (T-2 CINDER)

• Michael James

• Russell Johns

• William Johnson

• Toshihiko Kawano (T-2 CGM)

• Denise Pelowitz


MCNP/X Present

• MCNP5 1.51 and MCNPX 2.6.0

– released through RSICC

– Export Controlled source code (by Department of Energy)

• Active merger effort to produce MCNP6

– ~$5M since 2006

– 3 people / year

– All capabilities of both codes

– Friendly Alpha testing has already begun at LANL

– Anticipated merger by June / July

– Beta available within LANL (Internal milestone in Sept 2009)

– Begin robust V&V for ultimate release to RSICC summer 2010

– Subsequent elimination of duplicate features


New Features 2008

• MCNP5 version 1.51

– Variance Reduction with Pulse Height Tallies

– Temperature dependant (Doppler Broadened) libraries with preprocessor makxsf

– Memory improvements for large lattices

– Long file names & tabs

– Test suites that compare to experiments

– OpenMP threading on all platforms (alone or in combination with MPI)

• MCNPX 2.6.0

– Heavy Ion transport for 2000+ ions (LAQGSM, CEM)

– Burnup / Depletion with full integration of CINDER 90

– Delayed neutron and gammas from CINDER90 decay products

– Muon capture interactions

– Charged ions from neutron capture

– Fission Matrix criticality source convergence acceleration

– Weight windows improvements

– Residual nuclei tally

– Spontaneous photon sources

• Nuclear Data from ENDF/B-VII


Recent Capabilities

Simulated Radiographs

Decay Chains &

their emissionsBackground

Neutron and Photon

Radiation from lat,

long, and elevation


Monte Carlo Applications Tool Kit

(MCATK)

• From scratch Monte Carlo radiation transport code in C++, using Agile-like processes, unit testing, pair programming, modular design

• Driven by desire to utilize GPUs & new hardware (which doesn’t always have a FORTAN 90 compiler), as well as maintain MC expertise in new staff

• Intended to replace portions of MCNP

• Leverages Commercial + Open Source Software

– Eclipse Development Environment

– Unit Test++

– QT

– Doxygen

– Boost

GUI tool to view geometry


MCNP Unstructured Mesh

• track and tally on unstructured mesh

• mesh objects are mix of 4, 5 & 6 sided solids

• Boundaries of solids can be bi-linear.

• Mesh can be added into regular 3-D MCNP geometry

• Already in MCNP6, works in parallel

• Quadratic surfaces under development

• Want to expand to CAE codes other than ABAQUS

• Is currently used for thermo-mechanical analysis

Energy

Deposition


Low Energy Threshold

• Current lower energy limit of photons and electrons is 1 keV

• Not sufficient for smaller geometries desired by medical community

• Also necessitates improved L-M-N shell fluorescence

Millimeter or sub-millimeter

patient –based geometry is

frequent in user community

R&D Topics!


Improved Variance Reduction

• Long time goal of linking together deterministic SN adjoint solutions to feed into foreward Monte Carlo calculations

• Variance Reduction needs reexamination. Many methods were developed to gain information in a few specific locations, not everywhere

• Continuous Energy Adjoint

• Not all pieces work together

MCNP5

PARTISN

Generated ImportancesGeometry

R&D Topics!

Dose from Hiroshima

nuclear weapon

detonation in Times

Square, NY for FEMA.


Criticality Efforts

• Relative Entropy Of Fission Source Distribution

• Dominance Ratio

• Beta-Effective

• Source Shape Effects on Perturbations

• Probability of Initiation

“Stochastic Geometry” for HTGRs: Fuel

kernel displaced randomly within lattice

element each time that neutron enters

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.14

0.16

0.18

0.00 0.10 0.20 0.30 0.40 0.50

Fractional density change

Re

ac

tiv

ity

(k

/kk

')

2 MCNP runs

MCNP 2nd-order

2nd+PS(5-cycle)

2.741cm

6 cm

Perturbed

region

Godiva central perturbation

Exact, 2

MCNP

runs

Standard MCNP,

2nd order perturbation

Modified MCNP,

With source-shape

Effects on perturbation

R&D Topics!


TRIPOLI

Developed at Commissariat à l’Énergie Atomique, Saclay

Contact: Jean-Christophe Trama, (Jean-

[email protected])

O Petit, É Dumonteil, FX Hugot, YK Lee, A Mazzolo, C Diop









A long history at CEA, representing a large Monte Carlo

expertise since the 1960s

The fourth generation of the code,TRIPOLI-4, is a 3D

pointwise Monte Carlo code for radioprotection and

shielding, core physics, criticality and nuclear

instrumentation studies.

TRIPOLI-4 represents more than one and half hundred

person-years of capitalized experience, devoted to

research & development as well as industry.

TRIPOLI


CEA (core physics, radioprotection, criticality studies

through the CRISTAL package)

EDF (core physics, radioprotection)

IRSN (criticality studies through the CRISTAL package)

AREVA NC (criticality studies through the CRISTAL

package)

AREVA TA (core physics, radioprotection)

NURESIM European project : TRIPOLI-4 reference Monte

Carlo tool

TRIPOLI-4: a reference tool for radioprotection and

shielding, core physics, and criticality studies

www.nuresim.com


3D, Monte-Carlo code to compute transport related

quantities (flux, current, reaction rate, keff etc.)

Directly reads pointwise cross sections (pendf files), can

easily mix isotopes from different evaluations

Easy-to-use automated variance reduction tools

especially designed for deep penetration problems

Robust and efficient parallelism capability

(heterogeneous workstation network as well as massive

HPC systems)

Highly qualified 0-20 MeV neutrons and photons nuclear

energy calculations (reactors, plants and labs)

TRIPOLI-4 : main characteristics

CASCAD fuel storage

Reactor simulation


Mainly written in C++

Available on various Unix and Linux

Easy-to-understand key-word based input system,

supplemented by 3D mouse-driven man-machine interface

(Salomé)

Probability table in the URR range from CEA CALENDF

code (also available from NEA)

Externally coupled with depletion code, CFD code

TRIPOLI-4 : some technical features

TERA, supercomputer, © CEA


TRIPOLI-4 directly reads pendf files from various

evaluations

Presently, evaluations from JEF2.2, JEFF3.0 3.1 3.2,

ENDF/B-VI.4 B-VI.8, B-VII.0, JENDL33 may be used

(available from NEADB and RSICC)

TRIPOLI-4 reads PT files from the CEA CALENDF code,

for the Unresolved Resonance Range (URR)

Examples of data libraries already in use are

JEFF3.1 : 381 isotopes + 140 PT in the URR

ENDF/B-VII : 393 isotopes + 252 PT in the URR

JENDL33 : 337 isotopes + 209 PT in the URR

It is possible to easily use a user-defined nuclear data set,

with isotopes coming from different evaluations

TRIPOLI-4 : a focus on nuclear data

management


Variance reduction techniques are necessary to obtain acceptable

Figure of Merit (FOM)

Deep penetration problems are almost impossible to calculate

without them.

Variance reduction techniques are statistical tools to help the user

sample from a modified physics to reach the desired results.

The automated reduction variance built in the INIPOND TRIPOLI-

4 module allows to easily run a deep penetration problem.

TRIPOLI-4 : a focus on automated

variance reduction


Predefined shapes and combinatorial operators :

Reunion, smash, intersection, substraction

Rotation

Repetition of basic pattern

Lattices, Lattice of lattices :

Or definition by surfaces :

Combination of both descriptions is possible.

User friendly interface

2D cut

TRIPOLI-4 : a focus on geometry description



VIM

http://www.vim.anl.gov/

Developed at Argonne National Laboratory

Contact: Roger Blomquist, [email protected]



http://www.vim.anl.gov/






VIM

•Continuous Energy Neutron/Photon Transport Code System for criticality, reactor physics, and shielding

•Geometry options are infinite medium, combinatorial geometry, and hexagonal or rectangular lattices of combinatorial geometry unit cells, and rectangular lattices of cells of assembled plates

•Variance reduction using splitting/Russian roulette, non-terminating absorption with nonanalog weight cutoff energy



VIM Background

•Developed to analyze ANL critical experiments and benchmark ANL diffusion, transport, and spectrum analytical tools

•Nuclear data processed using ANL codes independent of others:

–Not NJOY

–Some other codes use ANL-developed methods (e.g., URR probability tables in MCNP)

•Earlier VIM verification & validation:

–Extensive benchmarking with ZPR/ZPPR criticals and ICSBEP

–Earlier libraries verified by inter-code comparisons with MC2-2 fast reactor spectrum code in great detail, MCNP, etc.



Recent VIM Enhancements

CODE

• FORTRAN 95, with nuclear data in structures

• Eliminated archaic memory management routines

• Improved input diagnostics

• Slicer geometry input color display

• Enhanced precision to reduce tracking errors and differences between platforms

Data Library

• Eliminated intermediate codes previously used to access processed nuclear data libraries

• Default libraries now ASCII, with provenance information in header records

• Increased angular distribution detail and accuracy

• Total fission spectra used, not prompt only

• ENDF/B-VII.0, tested on fast and intermediate systems



ICSBEP HEU-MET Benchmark

Cases: VIM vs. MCNP

003-1

1c

HM

I001

065

061

028

027

022

021

018

015

014

013

012

008

003-1

2

003-1

1d

003-1

1b

003-1

1a

003-1

0g

003-1

0f

003-1

0e

003-1

0d

003-1

0c

003-1

0b

003-1

0a

HM

F001

-100

-50

0

50

100

150

k(V

IM)

- k(M

CN

P)

(pcm

)


TART

http://home.comcast.net/~redcullen1/speed.htm

Developed at Lawrence Livermore National Laboratory

Contact: Dermott E. Cullen, [email protected]



http://home.comcast.net/~redcullen1/speed.htm





Pedigree of TART

•Livermore production code for 40 years

•Livermore’s equivalent of MCNP

•Only recently released outside Livermore

•TART2000

• Coupled Neutron-Photon

• 3-D Combinatorial Geometry

• Time Dependent

• Energy Range: Very Low up to 1 GeV


TART Features

• Very fast

•Uses Latest ENDF/B data

• neutrons & photons

• Runs on ANY computer•UNIX Workstations

•Windows/Linux PC

• Power MAC

•Very user friendly•Only about 15-20 % is the Monte Carlo code

•The remainder is tools to make your job easier


•The TART System is a complete

system

•It helps you prepare and check

input

•Runs your Monte Carlo

calculations

•Source and criticality problems

•Interactive graphics is extensively

used

•In input preparation and

checking

•Overlaying results on your

geometry

•Viewing neutron and photon

data

TART Features


•A well known advantage of Monte

Carlo is its ability to handle

complicated geometry.

•TART tries to optimize this advantage

•Other codes allow first & second

degree surfaces - planes,

spheres, cylinders,…

•TART allows third & fourth

degree surfaces - cubic & quartic

splines, fine and detailed

surfaces, torus

•There is no limit to the detail of

geometry

•Everything is dynamically

dimensioned

•Here’s an example of a

complete seven story building

TART Features

Example of a complete

seven story building


• continuous energy neutrons

• and continuous energy kinematics

• but multigroup cross sections

• 700 groups: 50 per energy decade

• Multiband parameters in all group

• Because sampling continuous energy cross

sections converges too slowly and isn’t

necessary, TART uses self-shielding theory and

the multiband method

TART Features



SERPENT

http://montecarlo.vtt.fi

Developed at VTT Technical Research Centre , Finland

Contact: Jaakko Leppanen, [email protected]



http://montecarlo.vtt.fi/





The Serpent Computer Code

• Serpent - a continuous-energy Monte Carlo reactor physics burnup calculation code

• Developed at VTT Technical Research Centre of Finland since 2004 (until Oct. 2008 under working title ”PSG”)

• Mainly intended for, but not limited to lattice physics calculations:

− Generation of homogenized multi-group constants for deterministic reactor simulator codes

− Fuel cycle studies

− Reactor physics calculations traditionally handled using deterministic lattice transport codes

• Universe-based three-dimensional geometry model – allows the description of practically any 2D or 3D fuel or reactor core configuration

Fig 1. MOX assembly surrounded by UOX

assemblies in a PWR reactor lattice

http://montecarlo.vtt.fi


• Interaction physics:

− Continuous-energy cross sections read from ACE format data libraries and reconstructed on a unionized energy grid

− Neutron interactions modeled using classical collision kinematics and ENDF reaction laws

− Unresolved resonance cross sections sampled from probability tables

• Tracking:

− K-eigenvalue criticality source calculation

− Neutron transport based on the combination of conventional surface tracking and the Woodcock delta-tracking method

• Burnup calculation:

− Fully automated built-in depletion routines

− The Bateman depletion equations solved using the transmutation trajectory analysis (TTA) or the Chebyshev rational approximation method (CRAM)

− Radioactive decay and fission yield data read from standard ENDF format files

Serpent Capabilities


• Capabilities include:

− Homogenized multi-group constants, including cross sections, assembly discontinuity factors, scattering matrices, diffusion coefficients and point-kinetic and effective delayed neutron parameters calculated by default

− Unlimited number of depleted material zones in burnup calculation

− Various user-defined ”detector” (tally) features

• Main limitations:

− Transport simulation limited to neutrons

− K-eigenvalue criticality source calculation, fixed source mode not available

− Memory usage may become a limiting factor in large burnup calculation problems

− Delta-tracking necessitates the use of the collision flux estimator poor efficiency for reaction rates tallies integrated over small cells or regions oflow collision density

Fig 2. The VENUS-2 reactor core

Serpent Limitations


• Parallel calculation using MPI

• Built-in Doppler-broadening preprocessor routine

• B1 fundamental mode calculation and leakage models currently under development

• Geometry models for handling random particle and pebble distributions in HTGR calculations

• Serpent is optimized for performance in lattice physics calculations:

− Efficient geometry routine based on delta-tracking

− Unionized energy grid used for all cross sections minimizes the number of grid search iterations

− Additional tricks to speed up burnup calculation

Full-scale LWR assembly burnup calculations (1

can be completed in about 4 hours on a 2.6 GHz Linux PC cluster using 4 CPU’s in the MPI mode

Fig 3. 15,000 microscopic TRISO particles

randomly dispersed inside a PBMR-type fuel

pebble (explicit particle fuel model)

(1 A standard 17 x 17 PWR fuel assembly with burnable absorber pins.

Materials divided into 65 depletion zones, a total of 42 burnup steps run

with predictor-corrector calculation, 3 million neutron histories per each

transport cycle.

Additional Features of Serpent



DENOVO (SCALE)


Contact: Tom Evans, [email protected]

AvailabilityRSICC http://rsicc.ornl.gov






Denovo

• State of the Art Transport Methods

massively parallel deterministic radiation transport code

–3-D regular grid, Discrete Ordinates (SN)

–Multigroup energy, anisotropic PN scattering

–6 spatial discretization algorithms (linear & tri-linear discontinuous FE, Step-characteristics, theta-weighted diamond, diamond difference + fixup)

• High Performance, Modern, Innovative Solvers

–GMRES, BiCGStab. or Source Iteration options on within-group solves

–DSA-preconditioning (SuperLU/ML-preconditioned CG)

–Transport Two-Grid upscatter acceleration of Gauss-Seidel MG iteration

–Parallel first collision approximation

–Eigenvalue (keff) and fixed-source problem modes

–Krylov solvers provided by Trilinos Library

58


DENOVO APPLICATIONS

Zones Angles Groups State Size

(GB)

Output

(GB)

Time

(m)

103.7M S24/P3 27 568.741 83.457 46.97

1,047.8M S24/P3 27 5,746.180 843.189 79.43

PWR Facility Modeling

Nuclear Energy:

LWR analyses

Fusion:

ITER analyses

285M cell, S24/P3 model of the International Thermonuclear Experimental Reactor (ITER)

59


DENOVO

• Parallel Algorithms

–Koch-Baker-Alcouffe (KBA) wavefront solve

–Domain replicated & decomposed options for parallel first-collision source

–Multi-level decompositions in energy and angle under development

–Parallel I/O for massive problems

• Advanced Visualization and Run-Time Environment

–Python front-end allows high-degree of flexibility in prescribing input/output

–Direct connection to SCALE geometry and data

–HDF5 output directly interfaced with Visit

60


Denovo

• Advanced Methods– GMRES (with DSA preconditioning) within-group

solver

– Transport, Two-Grid accelerated Gauss-Seidel for multi-group

– Koch-Baker-Alcouffe (KBA) parallel domain-decomposition

– Trilinos parallel solver package for highly efficient Krylov solvers and as an interface to the SuperLUdirect solver library

– Parallel first-collision source

– 5 different spatial differencing schemes

– Multiple input front-ends including Python

– High-performance parallel I/O using HDF5

KBA Parallel Sweep

1. Sweep each block starting

in corner of octant.

2. Communicate outgoing

fluxes to neighboring (x,y)

blocks.

3. Continue sweep in z-

direction.

4. MPI communication

across blocks.

5. OpenMP on angles within

block.

Denovo can run on PCs, workstation clusters, supercomputers

Jaguar – 1.64 PF Cray XT: 45,376 Quad-Core Processors, 362 TB memory

Denovo is designed to run

high resolution problems

scaled to thousands of cores.

Weak Scaling

Denovo’s parallel algorithms

are very efficient on cluster-

level platforms as well.

Strong Scaling


Cancer Therapy Planning with Denovo

Denovo generates a model directly from CT

scans:

576,600 voxels

lung/chest scan from UNC medical center

Using this model, Denovo can perform fast

and accurate radiation transport calculations:

S8 angular quadrature

P3 scattering

40 energy groups

Calculation took 438 seconds on a 16

CPU AMD 64 Linux cluster


Therapy Plan

Denovo outputs data that can be read directly by VisIt

inline, real-time visualization

dynamic visualization for modulated therapy planning

2D/3D contours, color plots, etc.

contours at 80, 70, 50, 30, and 10%


Denovo Capabilities for Treatment

Planning

• Coupled photon-electron transport

– Galerkin quadrature + CEPXS cross sections implemented

– in testing and verification phase

• Boltzmann-Fokker-Planck planned for this year

• Photo-neutron library scheduled for testing Fall 09

– estimate neutron full-body doses to personnel and patients

• Coupled proton-neutron-photon physics planned after BFP implementation

– used for proton therapy planning

– occupational neutron dose calculations



PENTRAN

http://www.hswtech.com/

Developed at HSW Technologies LLC, University of Florida

Contact: Ali Haghighat, [email protected] Sjoden, [email protected]

Availability – coming soon to university participants onlyRSICC http://rsicc.ornl.gov

http://www.hswtech.com/





PENTRANTM (Parallel Environment Neutral-particle TRANsport)

Code System

Pre-processing

PENMSH-XP (prepares mesh/material/source distributions &

PENTRAN input file)

PENTRAN (Parallel 3-D, Sn transport code)

Post-processing

PENDATA (prepares tables of flux, source, and material distributions by

processing parallel-partitioned output files)

PENPRL (determines flux values at any arbitrary position by performing

3-D linear interpolation)


• PENTRAN code system was developed by Glenn Sjoden and

Alireza Haghighat in 1996.

• ANSI FORTRAN F77/f90 with MPI library, over 37,000 lines

• Industry standard FIDO input

• Licensed via HSW Technologies LLC

• Solves 3-D Cartesian, multigroup, anisotropic transport problems

• Forward and adjoint mode

• Fixed source, criticality eigenvalue problems

• Parallel processing algorithms

• Hybrid phase-space decomposition in angle, energy, and/or spatial

variables

• Parallel I/O

• Partitioned memory for memory intensive arrays (angular fluxes, etc)

• Builds MPI processor communicators

• Automatic scheduling using a decomposition weighting vector


Numerical formulations

– Adaptive Differencing Strategy

DZ DTW EDI

Upgrade condition: Negative solution w ≥ .96

Diamond Zero Fixup (DZ); Directional Theta-Weighted (DTW); Exponential-Directional Iterative (EDI)

– Fully discontinuous variable meshing

between coarse meshes: Uses a novel

higher order mesh coupling scheme:

Taylor Projection Mesh Coupling (TPMC)

– Acceleration

• Coarse-mesh Rebalance (CMR) techniques; multi-grid (MG);

Combined CMR & MG; Synthetic Even Parity Simplified Sn (EP-SSn);

Preconditioned EP-SSn/Sn


Numerical formulations (cont.)

– Iterative

• Multigroup & One-level Source Iteration (SI)

• Red-Black and Block Jacobi iteration

– Anisotropic scattering with arbitrary order,

– Angular quadrature set:

• Level symmetric (up to S20) with ordinate splitting (OS)

• Pn-Tn (arbitrary order) with OS

– Vacuum, reflective, and albedo boundaries

– Volumetric & planar angular sources

Other versions

– Medical Application:

– PENTRAN-MP (Radiation dose – photon & electron);

– PENTRAN-CRT (Hybrid Sn & Characteristic Ray Trace) (under

testing & evaluation)

– Core physics: PENTRAN with PENBURN (3-D fuel burnup)


Benchmarking: VENUS-3 benchmark facility, Kobayashi 3-D benchmarks,

C5G7 MOX Criticality benchmark, Ganapol TIEL benchmarks

Applications

– BWR Core-Shroud

– Pulsed Gamma Neutron Activation

Analysis (PGNAA) device

– X-Ray room

– Time-of-Flight (TOF)

– Spent fuel storage cask

– UF Training Reactor (UFTR) - Water tank Characterization

– UFTR - Thermal column optimization

– SNM Detection optimization

– Whole body Medical Phantom dosimetry (PENTRAN-MP)

– SiC Power Monitoring Assessment

– LWR 3-D fuel burnup calculation for whole core (PENTRAN with PENBURN)

– Full-core PWR (IBM blueGene/P) (tested on 4096 cores)

– Development of Spent fuel pool monitoring tool

– Cargo Monitoring Assessments

Performance: Achieved high parallel fractions in a range of 96-99%



PARTISN

Developed at Los Alamos National Laboratory

Contact: Randy Baker, [email protected]

Availability:RSICC http://rsicc.ornl.gov







PARTISN

PARallel, TIme-Dependent SN

Evolutionary successor to DANTSYS (ONEDANT, TWODANT,

THREEDANT)

PARTISN 5.97: Time-Dependent

Parallel Neutral Particle Transport Code System

Solves the time-independent or dependent multigroup discrete ordinates

form of the Boltzmann transport equation in several different geometries



PARTISN

Uses MPI (message passing interface)

Designed for UNIX, Linux or Windows systems

solves the transport equation on orthogonal (single level or block-

structured AMR) grids in 1-D (slab, two-angle slab, cylindrical, or

spherical), 2-D (X-Y, R-Z, or R-T) and 3-D (X-Y-Z or R-Z-T) geometries


Supercomputers Today

Million-fold increase in computing and data

capabilities

2004

2006

2007

2008

2009

2011

2015

2018

Cray ―Baker‖8/12-core, dual-

socket SMP 1000 TF (1 PF)

DARPA HPCS20 PF

Future system

100–250 PF

Cray XT4119 TF

Cray XT3 Dual-core

54 TF

Cray XT4 Quad-core

263 TF

Cray X13 TF

Cray XT3Single-core

26 TF

Future system1000 PF(1 EF)

Cray XT4Quad-core

166 TF

Cray ―Baker‖8-core, dual-socket SMP

1379 TF (1.4 PF)

2005

Slide courtesy of Jeff Nichols, Oak Ridge National Laboratory

Cost as low as

$0.13 / gigaflop

for the 40nm GPU

from ATI.

http://en.wikipedia.org/wiki/Graphics_processing_unit



Shielding Aspects of Accelerators and Target Irradiation

Facilities (SATIF-10)

http://indico.cern.ch/conferenceDisplay.py?confId=62629



Computational Medical Physics Working Group

http://cmpwg.ans.org



Monte Carlo 2010 and Supercomputing 2010

http://sna-mc-2010.org/

October 17-20, 2010

Tokyo, Japan







NEUDOS Kirk Oct 2009 Neutrons

Documents

Transcript of NEUDOS Kirk Oct 2009 Neutrons