PDE Discretization and Analysis with libMeshscisoftdays.org/pdf/2016_slides/stogner.pdf · Git...

PECOSPredictive Engineering and Computational Sciences

PDE Discretization and Analysis with libMesh

Roy H. Stogner1

1The University of Texas at Austin

Feb 26, 2016

Roy H. Stogner libMesh Feb 26, 2016 1 / 22

Introduction

libMesh Finite Element Library

• Free, Open source

LGPL2 for core

• 35 Ph.D. theses, 393

papers (58 in 2015)

• ∼ 10 current developers

• 160− 240 current users?

Jan 2010 Jul 2011 Oct 2013 Feb 201650

300libmesh-devellibmesh-users

LibMesh Mailing List Membership Size

Challenges

• Radically different application types

• Widely dispersed core developers

INL, UT-Austin, U.Buffalo, JSC, MIT,

Harvard, Argonne

• OSS, commercial, private applications

Introduction

PetscMetis

Laspack

LibMesh

TumorAngiogenesis

CompressibleNS

• Foundational

(typically optional)

library access via

LibMesh’s “roots”.

• Application

“branches” built off

the library “trunk”.

• Additional

middleware layers

(e.g. Akselos,

GRINS, MOOSE) for

more complex

applications

Introduction

Typical Boundary Value Problem

• Common BVP components:

∂t= F (u) ∈ Ω ⊂ R

G(u) = 0 ∈ Ω

u = uD ∈ ∂ΩD

N(u) = 0 ∈ ∂ΩN

u(x, 0) = u0(x)

• Less common components:

Moving domain Ω(t), Ω(u, t) Multi-dimensional manifolds Self-overlapping, contact Acceleration ∂2u/∂t2

Integro-differential equations

∂ΩD

∂ΩN

Introduction

Mesh Classes

MeshBase

#_dim: unsigned int

#_n_parts: unsigned int

+boundary_info: AutoPtr<BoundaryInfo>

+elements_begin()

+read()

+<<empty>> allgather()

+<<empty>> delete_remote_elements()

Paral lelMesh

+_elements: mapvector<Elem*>

+elements_begin()

+allgather()

+delete_remote_elements()

UnstructuredMesh

+read()

SerialMesh

_elements: std::vector<Elem*>

+elements_begin()

• MeshBase gives node or

element iterators, all or

active, global or local

• SerialMesh or

ParallelMesh manages

synchronized or

distributed data

Introduction

Geometric Element Classes

#_nodes: Node **

#_neighbors: Elem **

#_parent: Elem *

#_children: Elem **

#_*flag: RefinementState

#_p_level: unsigned char

#_subdomain_id: unsigned char

+n_faces,sides,vertices,edges,children(): unsigned int

+centroid(): Point

+hmin,hmax(): Real

NodeElem FaceEdge Cell InfQuad InfCell

Prism Hex Pyramid Tet

Hex8 Hex20 Hex27

DofObject

-_n_systems: unsigned char

-_n_vars: unsigned char *

-_n_comp: unsigned char **

-_dof_ids: unsigned int **

-_id: unsigned int

-_processor_id: unsigned short int

• Abstract interface gives

mesh topology

• Concrete instantiations of

mesh geometry

• Hides element type from

most applications

• Base class data arrays allow

more optimization, inlining

Similar trees:

• Shape functions, quadrature

• Periodic boundary maps

• Reference counts, comms

• Linear algebra, solvers

• Integrators, strategies

Introduction

Poisson Example

• Consider the weak form arising from the Poisson equation,

R(u, v) :=∫

[∇u · ∇v − fv] dx = 0 ∀v ∈ V

Introduction

Poisson Example

• The LibMesh representation of the matrix and rhs assembly is similar

to the mathematical statements.

for (q=0; q<Nq; ++q)

for (i=0; i<Ns; ++i)

Fe(i) += JxW[q]*f(xyz[q])*phi[i][q];

for (j=0; j<Ns; ++j)

Ke(i,j) += JxW[q]*(dphi[j][q]*dphi[i][q]);

Introduction

Poisson Example

F ei =

f(x(ξq))φi(ξq)|J(ξq)|wq

Introduction

Poisson Example

Keij =

∇φj(ξq) · ∇φi(ξq)|J(ξq)|wq

Introduction

Scalability: Hybrid MPI + Threads

Parallel:: interfaces wrap MPI

Library data is distributed, synchronized:

• Simpler, STL-compatible, inlined API

• Works with complex serializable classes

Threads:: interfaces TBB/pthreads/OpenMP

• Sparsity, AMR constraint calculation

• FEMSystem assembly routines

• Asynchronous I/O

• Mesh projections, utility functions

Number of Processor Cores

100 101 102 103100

IdealScaled-Size (Weak) ScalingFixed-Size (Strong) Scaling

0 10 20 30 40 50 60

# of Threads

Speedup

MIC Speedup

2D, no mutex

2D, mutex

3D, no mutex

3D, mutex

• Hybrid parallel apps with no threading or message passing code

• API-independence via libMesh wrappers

Adaptivity

Adaptivity: Error Estimators, Error Indicators

Error decompositions

Subterms on each element K:

• ||u− uh||2H=

K ||u− uh||2H(K) ≤

K |ηK |2

• Q(u)−Q(uh) ≈∑

• |Q(u)−Q(uh)| ≤∑

K |ηK |

Refinement heuristics

• Refinement/coarsening of elements with:

Worst/best fraction sorted by error Error over/under fraction of tolerance Error over/under target mesh size average

• h-vs-p refinement:

a priori singularity identification Behavior vs. cost when h vs. p coarsening?

Adaptivity

Goal-oriented Adaptivity

Refine to reduce solution error when

it influences QoI error:

2 2.5 3 3.5 4 4.5−6.5

−5.5

−4.5

−3.5

−2.5

(Dofs), Degrees of Freedom

Convergence of the interior QoI, ε = 10−8

Uniform Refinements

Flux Jump Error Estimator

Adjoint Error Estimator

• Global error indicator targets

layer alone; QoI temporarily

plateaus

• Rapid convergence from

adjoint-based refinement

Adaptivity

Adjoint-based Error Indicators, Sensitivities

Q(uh)−Q(u) = R(uh, z − zh) +H.O.T.

q′ = Qξ(u; ξ)−Rξ(u, z −Ψ(u; ξ); ξ)

Adjoint Refinement

R(uh, z − zh) ≈∑

RE(uh, zH − zh)

Adjoint Residual∣

∣R(uh, z − zh)

∣≤

∣REu

B(UE ,VE∗)

∣u− uh

∣z − zh

Generalized Adjoint Residual∣

∣R(uh, z − zh)

∣≤

∣zi − zh

∣uj − uh

Collaboration Strategies

Communication

• Face to face, instant messaging, teleconference

• Email lists

libmesh-users@sourceforge.net,

libmesh-devel@sourceforge.net

• Trac/Redmine/Sourceforge issue tracking

• GitHub issues

• Email attachments

• Ticket attachments

• Repository forks!

• Pull requests!

Source Code Control

Git Guidelines

• Strive for “useful nonlinearity.”

• Develop separate feature sets on

separate branches; merge them

back to master when complete.

• Minimize or eliminate periodic or

unnecessary merge commits.

• rebase feature branches on top of

master before merge + push

• Rebasing public branches is badTM.

Complete the shared branch,

branch from the shared branch

locally, rebase from target, then

• http://nvie.com/posts/a-successful-git-branching-model

Development and Testing

Issue, Pull Request Tracking

Tracking API Changes

API versions easily proliferate...

#if PETSC_VERSION_LESS_THAN(3,1,0)

ierr = MatGetSubMatrix(matrix->mat(),

_restrict_to_is, _restrict_to_is_complement,

PETSC_DECIDE, MAT_INITIAL_MATRIX, &submat1);

ierr = MatGetSubMatrix(matrix->mat(),

_restrict_to_is, _restrict_to_is_complement,

MAT_INITIAL_MATRIX, &submat1);

#endif

• Maintain a wide range of external compatibility

Dropped PETSc 2.3.3 (2007) support for libMesh 1.0 (2016) C++11 shims

• Limit libMesh API changes

Signaling API Changes

Development practices

• Old, new APIs overlap

• Easier with C++ function overloading, default arguments

Adding f(a,b) does not preclude keeping f(a) Adding f(a,b=default) can replace f(a)

Runtime warnings

• libmesh experimental() (in-flux APIs)

• libmesh deprecated() (∼1 year, 1-2 releases)

Examples

• OStringStream workaround class

• Parallel:: global functions

Regression Testing

• ∼ 7000 internal assertions in debug mode

• ∼ 60 core example applications

• ∼ 400 unit tests

• Configurations: optional features, index width,

ParallelMesh, solver package, scalar precision,

-np, –n threads

• Middleware, application test suites

Assertions

libmesh_assert(c < _variables.size());

libmesh_assert(s < elem->n_sides());

libmesh_assert((ig >= Ug.first_local_index()) &&

(ig < Ug.last_local_index()));

libmesh_assert(requested_ids[p].size() == ghost_objects_from_proc[p]);

libmesh_assert(obj_procid != DofObject::invalid_processor_id);

libmesh_assert(mesh.is_prepared());

MeshTools::libmesh_assert_valid_node_procids(mesh);

libmesh_assert(neigh->has_children());

libmesh_assert(error_estimator.error_norm.type(var) == H1_SEMINORM ||

error_estimator.error_norm.type(var) == W1_INF_SEMINORM)

libmesh_assert(number_h_refinements > 0 || number_p_refinements > 0);

Assertions

• Trust preconditions/postconditions, not any of your users

Assertions

• Earlier errors are easier errors

Assertions

• Earlier errors are easier errors

• Exceptions! Stack traces! Line numbers!

Manufactured Solution Testing

(Finding unknown unknowns)

Verification of FIN-S hypersonics

• FANS, Spalart-Allmaras

• Derivative:

d(ρ ∗ sa)

dx− sa

• In code:

ρ∗d(ρ ∗ sa)

dx− sa

dx10-5

4 5 6 7 8

Before1567

1567 & 1568O(h2)

Manufactured Analytical Solution Abstraction library:

https://manufactured-solutions.github.io/MASA/

Build Systems

Autotools, Pros and Cons

Autoconf

• Manages feature selection

50+ --enable-foo options

• Portability tests, workarounds

• POSIX shell dependence

Libtool

• DLL management in install

• Broader shared library support

• Easily used via automake

• Trickier in-build debugging

Automake

• dist, check, install

targets

• Out-of-source builds

• Standardized

conventions

• More difficult METHODS

support

• “bootstrap” process

Do users have

autotools? Custom scripts for

libMesh

Build Systems

AcknowledgementsRecent contributors:

• David Andrs

• Paul Bauman

• Vikram Garg

• Derek Gaston

• Dmitry Karpeev

• Benjamin Kirk

• David Knezevic

• Cody Permann

• John Peterson

• Sylvain Vallaghe

Useful resources:

• libMesh: https://libmesh.github.io/

• GRINS: https://grinsfem.github.io/

• MOOSE: https://mooseframework.org/

• MASA: https://manufactured-solutions.github.io/MASA/

Questions?

PDE Discretization and Analysis with libMeshscisoftdays.org/pdf/2016_slides/stogner.pdf · Git...

Documents

Transcript of PDE Discretization and Analysis with libMeshscisoftdays.org/pdf/2016_slides/stogner.pdf · Git...

Simulating Nonlinearity Supplemental issue Summer 2013€¦ · Translate this pageSimulating Nonlinearity Supplemental issue Summer 2013

Lecture 13 - Handling Nonlinearity - Stanford University · Lecture 13 - Handling Nonlinearity • Nonlinearity issues in control practice • Setpoint scheduling/feedforward –

Nonlinearity & Preprocessing

The Sigmoid Nonlinearity in Prepyriform Cortexpapers.nips.cc/...the-sigmoid-nonlinearity-in-prepyriform-cortex.pdf · THE SIGMOID NONLINEARITY IN PREPYRIFORM CORTEX ... with a standard

Nonlinearity in Econometrics

Nonlinearity & chaos

Verification of PDE Discretization Library and Flow ...users.oden.utexas.edu/~roystgnr/short_verification.pdf · I libMesh, FIN-S basic tests now in BuildBot at UT I Ablation, radiation,

Performance Dominik G ö ddeke. 2Overview Motivation and example PDE –Poisson problem –Discretization and data layouts Five points of attack for GPGPU.

On the Intuitiveness of Common Discretization Methodspoloclub.gatech.edu/idea2016/papers/p22-boley.pdf · supervised discretization techniques. 2. Moreover, the discretization should

Chapter 720 Discretization

Computer modeling and simulation of natural phenomena · 2019-01-08 · phenomenon ! PDE! discretization ! numerical solution. From PDEs to virual universe ... I Square lattice (chessboard)

Pde flashmeeting

Packet implementation: discretization

Chapter 8 Nonlinearity

Applied Mathematics: Methods55 functions and accurate, stable discretization of linear PDE. Nevertheless, the control theory methods have significant limitations, including the state-space

LAGA, CNRS, UMR 7539, arXiv:1212.2807v2 …math.univ-paris13.fr Abstract We study a one-dimensional parabolic PDE with degenerate diﬀusion and non-Lipschitz nonlinearity involving

7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

Data Discretization Unification

FEM-BASED DISCRETIZATION-INVARIANT MCMC METHODS …users.ices.utexas.edu/~tanbui/PublishedPapers/FEMBayesian.pdf · fem-based discretization-invariant mcmc methods for pde-constrained

Caim discretization algorithm