1 Institute of Computational Mathematics and Mathematical Geophysics SB RAS, Novosibirsk, Russia

2 Novosibirsk State University, Novosibirsk, Russia3 Siberian Regional Hydrometeorological Research Institute, Novosibirsk, Russia

Sensitivity Operator Inverse Modeling Framework

for Advection-Diffusion-Reaction Models with Heterogeneous Measurement Data

Alexey Penenko 1,2, Vladimir Penenko 1,2, Elena Tsvetova 1, Alexander Gochakov 3, Elza Pyanova 1, and

Viktoriia Konopleva 1,2

vEGU, 2021

E-mail: a.penenko@yandex.ru

•Unified approach to different measurement data including image-type measurement data

(large volume of data with unknown value w.r.t. the considered inverse modelling task):

•Pointwise and time-series concentrations data (in situ data)

•Concentration profiles (aircraft sensing, lidar profiles, etc).

•Satellite images of concentration fields.

Motivation

•The progress in parallel computations technologies: the algorithms should allow natural mapping to the parallel

computation architectures (ensemble algorithms, splitting, decomposition, etc.)

•Unified framework for inverse and data assimilation problems (briefly, inverse modeling problems) for non-

stationary advection-diffusion-reaction models in various applications. E.g.

•Air quality studies (environmental applications)

•Biological problems (morphogen theory & developmental biology)

•The progress in nonlinear ill-posed operator equation solution (different regularization methods, SVD,

convergence theory, etc.) and analysis methods.

2

Solve & Analyze in the same way

Combine various sources of data

Measurement operators

Advection-diffusion-reaction model

( )( )diag ( , , ) = ( , , ) ,ll ll

l l l l lP t t ft

r

− − + + +

yu φ φ y

0= , , = 0,l l t x

The domain

( )( )diag = , ( , ) (0, ],R

l l l l l outt T + n x

[0, ]T T =

= , ( , ) (0, ],D

l l int T xBC:

: ,Q

q

→ φ

Direct problem

operator

advection-diffusion destruction-production

= [qH +( )I φ ] I,

Inverse problem

IC:

rectangular

Given NoiseTo find

• Pointwise concentrations

• 2D images

• vertical profiles,

• etc

Model scale: 0D,1D,2D,3D 1, , cl N= - number of species

{ , , }q r y=

3

H

, :l lP

• Polynomial

• Sigmoid

functions

• Integral

Uncertainty

function

specification

Conventionally used to

evaluate misfit cost

function gradients

(2) (2) (2) (1) (1), , , ,[ , ] [ , ] ( , , , , ) ,yR

K t q qy

+ = + +h H φ h H φδ h φμ δ φr δ φH

Sensitivity relation

( ) = 0,T+Ψ

Lagrange type identity (sensitivity relation)

-divided difference operator

( ) ( ) ( )1 1( ( ) ) ( ( , , ) ) = ,l

l l l ldiag G tt

h

− − − +

2u μ φ φ Ψ

( ) ( ) ( )( )( ) ( ) ( )( ) ( )( ) ( ) ( )( )1 1 1 1( , , ) , , , , , ,G t diag P t P t diag t

= + − 2 2 2 2

φ φφ φ φ φ φ φ φ φ

+ adjoint problem

boundary conditions

(( )) [ ]m mq=φ φ

Sensitivity functions

Adjoint problem: Given find:( ) ( )

, , , 1, 2,m m

m =h φ μ

Linear parametrizations: m m

m

rr = , = , [ ]m m Rm

r

h φ h

Ψ

(2) (1) = −φ φ φ

= , 1, ,m

m

t ll m MH=

=

4

( ) ( ) ( )( ) ( ) ( ) ( )( ) ( )( )1(2) (2) (1) (1)( , , , , ) , ; , ; ,y yK t q q t P t diag= −2 2 1 2 2 1

φ φ φ y ,y φ y ,y φ

Sensitivity Operator Construction

5

Sensitivity operator

Sensitivity relation (Lagrange type identity)

measU U

= (ξ)uGiven functions (functionals)

( )2 1 1 ( )( )[ ] [ ] [ ] [ ]UH

− = − ( ) ( ) (2) ( )φ q φ uq φ q φ q e

( )( )] [ ][ ] [Q

M

− = −(2) (1) (2) (1)(2) (1)qφ q φ q u qu qq

Image (model) to structure operator

[Dimet et al,2015]

Sensitivity operator ( ) ( )[ ][ ]

,U

R

R

MM

→

(2) (

(

1)

2) (1)

q q uq q

z z e

The inverse problem solution for any and satisfy( )

q q U

( ) ( )( )[ ] = [ , ] .PrU U UU

meas

H M H

− − + I φ q q q q q I

Parametric family of quasi-linear

operator equations

Parallel computation w.r.t.

U

To control the data considered and dimensionality

Idea: [Marchuk G. I., On the formulation of certain inverse problems, Dokl. Akad. Nauk SSSR, 156:3 (1964), 503–506], 6

Sensitivity relation based

• Coarse-fine mesh method (Sequential solution refinement with sequential adjoint problems solving) [Hasanov, A.; DuChateau, P. &

Pektas, B. An adjoint problem approach and coarse-fine mesh method for identification of the diffusion coefficient in a linear parabolic equation//

Journal of Inverse and Ill-Posed Problems, 2006 , 14 , 1-29]

• Using sensitivity function aggregates (supports, level sets, etc.): [Pudykiewicz, J. A. Application of adjoint tracer transport equations for

evaluating source parameters Atmospheric Environment, Elsevier BV, 1998, 32, 3039-3050], [Penenko, V. et al. Methods of sensitivity theory and

inverse modeling for estimation of source parameters, Future Generation Computer Systems, Elsevier BV, 2002, 18, 661-671], [Bieringer, P. E. et

al. Automated source term and wind parameter estimation for atmospheric transport and dispersion applications Atmospheric

Environment, Elsevier BV, 2015, 122, 206-219]

• Adjoint function for each measurement datum with the solution of the resulting operator equation [Marchuk G. I., On the

formulation of certain inverse problems, Dokl. Akad. Nauk SSSR, 156:3 (1964), 503–506],

• Passive tracer (linear) case: [Issartel, J.-P. Rebuilding sources of linear tracers after atmospheric concentration measurements //

Atmospheric Chemistry and Physics, Copernicus GmbH, 2003 , 3 , 2111-2125]

• Nonlinear case, pointwise sources: [Mamonov, A. V. & Tsai, Y.-H. R. Point source identification in nonlinear advection-diffusion-

reaction systems Inverse Problems, IOP Publishing, 2013 , 29 , 035009]

Cost function based

• Cost functional gradients with adjoint problem solution (single element ensemble for the discrepancy).

• Gradient computation with adjoint ensemble when adjoint is independent of direct solution [Karchevsky, A., Eurasian journal of

mathematical and computer applications, 2013 , 1 , 4-20]

• Representer method (optimality system decomposition, ensemble generated for discrepancies for each measurement data) [Bennett,

A. F. Inverse Methods in Physical Oceanography (Cambridge Monographs on Mechanics) Cambridge University Press, 1992]

Adjoint problem solution ensemblesin inverse problem algorithms

7

Adjoint ensemble construction

= | [0, ], [0, ], [0, ], ,x x y y t t mesU L ηθ θ θx y t

e• Fourier cos-basis

• Wavelets, curvlets, etc. [Dimet et al.,2015]

«a priori» approach – ensemble for the class of problems (smoothness)

=1

1( , , ) ( , , ) ( , , ), =

= ,

0,

Nc

k

t x i y j

x y t

l

C T t C X x C Y y le

l

2 cos , > 01( , , ) = .

1, = 0

t

C T t TT

«a posteriori» approach – ensemble for the considered problem

• «Adaptive basis»: chose elements of with maximal projections

[Penenko, A. V.; Nikolaev, S. V.; Golushko, S. K.; Romashenko, A. V. & Kirilova, I. A. Numerical Algorithms for Diffusion Coefficient Identification in Problems of Tissue Engineering //

Math. Biol. Bioinf., 2016 , 11 , 426-444 (In Russian)]

• «Informative basis»: use left singular vectors of the operator [ , ]Um

(0) (0)r r

U[ ] ,Pr

Umes

−(0)

ηθ θ θx y t

φ r I e

[Penenko, A.; Zubairova, U.; Mukatova, Z. & Nikolaev, S. Numerical algorithm for morphogen synthesis region identification with indirect image-type measurement data //

Journal of Bioinformatics and Computational Biology, World Scientific Pub Co Pte Lt, 2019 , 17 , 1940002]

(inverse source problem, 2D, components are measured )

Defined by the adjoint problem sources sets

8

Inversion algorithm

( )( ) ( )( ) ( )( ), [ , ] [ ,, ]]Pr [=UU

m

UU

e s

UU

a

H m Hm m − − + + − −

* **qφ q q q qq qI q q q δIq

,

= .Pr,

T T

unknowns

UUT T

mesunknowns

m mm NH I

m m m N

+

+

−

δq φ q

C+

-truncated SVD inversion parametrized by conditional number

, inversesource problem

, inversecoefficient problem

src t x y

unknowns

coeff

L N N NN

N

=

Newton-

Kantorovich

-type update

[ , ]Um m q q

( )unknownsN

Nonlinearity:

sequential increase of

the conditional number

Admissible solutions:

projection regularization

Noise:

discrepancy

principle

Optional monotonicity:

monotonic decrease of the

discrepancy

Theoretical foundations: [Issartel, J.-P., 2003], [Cheverda V.A., Kostin V.I., 1995], [Kaltenbacher et al, 2008],

[Vainikko, Veretennikov, 1986] 9

Adjoint ensemble inverse modeling framework

Inverse Problem Statement(PDE/ODE)

Adjointproblem

Lagrange-type Identity

Family of quasi-linear ill-posed operator

equations

Generalensemble

Inverse problem solution algorithm

Sensitivity function

Finite ensemble

Ill-posed nonlinear

operator equation

methods

# computation

threads &

memory available

Data assimilation mode

Sensitivity operator analysis

measUProcess model

10

«Illumination»functions

Singular spectrum

Measurement

data

specification

Uncertainty function

specification

Information quantity

Projection to the kernel

Heterogeneous Measurement Data

11

Heterogeneous measurement data

12

• «Timeseries»

• «Pointwise»

• «Integral»

• «Snapshot»

0=1

, = ( , ) ( , ) .

Nc T

l lHl

a b a t b t d dt

x x x

( )

( ) ( ) ( )( ) ( ) ( , ), 0, , , ,m m m

m measl

x t t T x l L ( ) ( ) ( ) ( )= ( , , ) ( ) ( );h C T t x x l l

− −

( )

( ) ( ) ( ) ( ) ( )( ) ( ) ( , ), , , ,m m m m m

Tm measl

x t x t l L ( ) ( ) ( ) ( )= ( ) ( ) ( );h x x t t l l

− − −

( )

( ) ( ) ( )( ) ( ) 0( , ) , , ,

T m m m

m measl

x t dt x l L ( ) ( ) ( )= ( ) ( ).h x x l l

− −

( )

( ) ( ) ( )( ) ( ) ( , ), , 0, , ,m m m

m measlx t t l T L x ( ) ( ) ( ) ( ) ( )

= ( , , ) ( , , ) ( ) ( );x yh C X x C Y y t t l l

− −

In variational approach we sum the misfit functions

for different measurements, in the adjoint ensemble-

based approach we join the corresponding

ensembles

A. Penenko, V. Penenko, E. Tsvetova, A. Gochakov, E. Pyanova, V. Konopleva Sensitivity Operator-Based Approach to the Interpretation

of Heterogeneous Air Quality Monitoring Data // submitted

Specific measurement types

13

O3 Monitoring sites locations

(b): time series (red crosses),

pointwise measurements in

space and time (blue circles),

integrals over a time interval

(magenta triangles).

Pointwise (100) Time-series (60)

Integral (5) Snapshot (20x20)

3.5 day-long

summer

scenario,

source-term

uncertainty,

NOx-O3 cycle

NO Emission sources

«Composite»measurements

14

Composite

Concentration field

«сcontinuation» error

Uncertainty (source-term)

identification error

15

Estimating the inverse modeling result prior to solution

Inverse problem solution times measured

in direct problem solution times

Relative error vs computation time

Sensitivity operator analysis

16

( )( ) ( )( )M q q H I A q− = −

( )( ) ( )( )T T T TM MM M q q M MM H I A q+ +

− = −

«Illumination» function characteristics: [ ] T T

l l lE M M MM M+

=

( )1

NT T T T

l ll

M MM M diag M MM M+ +

=

=

Projector to the orthogonalcomplement of the sensitivity

operator kernel

Equivalent to the generalized Sensitivity functions

Issartel, J.-P. Emergence of a tracer source from air concentration measurements, a new strategy

for linear assimilation // Atmospheric Chemistry and Physics, 2005, 5, 249-273

( )T Tdiag M M M M+

Kappel, F. & Munir, M. Generalized sensitivity functions for multiple output systems // Journal of Inverse and Ill-posed Problems, 2017, 25

Pseudo-inverse

( )( )T TM MM M q q+

−

Singular spectrum decay

17

Sensitivity operator analysis«Illumination»

function Reconstructed

solution (~500 iters.)

Projection of the

«Exact»

solutions to the

orthogonal

complement

of the sensitivity

operator kernel

(~1 iter.) Reconstruction error & its

predictions by projection

Singular spectrum

Thank you for your attention!

18

The work has been supported by

Ministry of Science and Higher Education of the Russian Federation Large Scientific Project

No. 075-15-2020-787 (heterogeneous measurement systems)

Penenko A.V. Penenko V.V.

Pianova E.A.Gochakov A.V.

Tsvetova E.A.

Konopleva V.S.

Sensitivity Operator & Adjoint Ensemble References

19

1. Penenko, A. Convergence analysis of the adjoint ensemble method in inverse source problems for advection-diffusion-reaction models with image-

type measurements // Inverse Problems & Imaging (AIMS), 2020, 14, 757-782 doi:10.3934/ipi.2020035

2. Penenko, A. V.; Mukatova, Z. S. & Salimova, A. B. Numerical study of the coefficient identification algorithm based on ensembles of adjoint problem

solutions for a production-destruction model // International Journal of Nonlinear Sciences and Numerical Simulation, accepted doi:

10.1515/ijnsns-2019-0088

3. Penenko, A. V. & Salimova, A. B. Source Identification for the Smoluchowski Equation Using an Ensemble of Adjoint Equation Solutions // Numerical

Analysis and Applications, 2020, 13, 152-164 doi: 10.1134/s1995423920020068

4. Penenko, A.; Zubairova, U.; Mukatova, Z. & Nikolaev, S. Numerical algorithm for morphogen synthesis region identification with indirect image-type

measurement data // Journal of Bioinformatics and Computational Biology, 2019 , 17 , 1940002 doi:10.1142/s021972001940002x

5. V. V. Penenko A. V. Penenko, E. A. Tsvetova and A. V. Gochakov Methods for studying the sensitivity of atmospheric quality models and inverse

problems of geophysical hydrothermodynamics // Journal of Applied Mechanics and Technical Physics, 2019, Vol. 60, No. 2, pp. 392–399 doi:

10.1134/S0021894419020202

6. Penenko, A. V. A Newton–Kantorovich Method in Inverse Source Problems for Production-Destruction Models with Time Series-Type Measurement

Data // Numerical Analysis and Applications, 2019 , 12 , P. 51-69 doi: 10.1134/s1995423919010051

7. Penenko, A. V. Consistent Numerical Schemes for Solving Nonlinear Inverse Source Problems with Gradient-Type Algorithms and Newton–

Kantorovich Methods // Numerical Analysis and Applications, 2018 , 11 , P.73-88 doi:10.1134/s1995423918010081

8. Penenko, A. V.; Nikolaev, S. V.; Golushko, S. K.; Romashenko, A. V. & Kirilova, I. A. Numerical Algorithms for Diffusion Coefficient Identification in

Problems of Tissue Engineering // Math. Biol. Bioinf., 2016 , 11 , 426-444 doi: 10.17537/2016.11.426 (In Russian)

9. Penenko, A. On a solution of the inverse coefficient heatconduction problem with the gradient projection method // Siberian electronic

mathematical reports, 2010, 23 , 178-198. (in Russian)