A Variational Finite Element Method forSource Inversion for Convective-Diffusive Transport?

Volkan Akçelik a, George Birosb, Omar Ghattasa, Kevin R. Longc,Bart van Bloemen Waandersd

aMechanics, Algorithms, and Computing LaboratoryDepartments of Biomedical and Civil & Environmental Engineering

Carnegie Mellon University, Pittsburgh, PA 15213, USA.bCourant Institute for Mathematical Sciences, New York University

New York, NY 10012, USAcSandia National Laboratories, Livermore, CA 94551, USA

dSandia National Laboratories, Albuquerque, NM 87185, USA

Abstract

We consider the inverse problem of determining an arbitrary source in a time-dependentconvective-diffusive transport equation, given a velocity field and pointwise measurementsof the concentration. Applications that give rise to such problems include determinationof groundwater or airborne pollutant sources from measurements of concentrations, andidentification of sources of chemical or biological attacks. To address ill-posedness ofthe problem, we employ Tikhonov and total variation regularization. We present a varia-tional formulation of the first order optimality system, which includes the initial-boundaryvalue state problem, the final-boundary value adjoint problem, and the space-time bound-ary value source problem. We discretize in the space-time volume using Galerkin finiteelements. Several examples demonstrate the influence of the density of the sensor array,the effectiveness of total variation regularization for discontinuous sources, the invertibilityof the source as the transport becomes increasingly convection-dominated, the ability ofthe space-time inversion formulation to track moving sources, and the optimal convergencerate of the finite element approximation.

? Research supported by the Computer Science Research Institute (CSRI) at Sandia Na-tional Laboratories, by the National Science Foundation’s Information Technology Re-search (ITR) program through grant ACI-0121667, and by the Department of Energy’sScientific Discovery Through Advanced Computing (SciDAC) program through the Teras-cale Optimal PDE Simulations (TOPS) Center.

Email addresses:volkan@andrew.cmu.edu (Volkan Akçelik),biros@cs.nyu.edu (George Biros),oghattas@cs.cmu.edu (Omar Ghattas),

Preprint submitted to Elsevier Science 7 January 2003

1 Introduction

Suppose we are given observations of the concentration field of an airborne orwater-borne contaminant, along with a velocity field that convects it and a param-eter that characterizes its diffusion. We are interested in determining the distri-bution and strength of the source of the contaminant. In this article, we considersuch an inverse problem: namely, we wish to determine an arbitrary source func-tion that drives a time-dependent convective-diffusive transport process, given avelocity field and observations of a concentration field at a finite number of pointsthroughout a domain of interest.

Inverse problems of this type are typically ill-posed; the sensors providing the ob-servations are usually coarsely spaced, and we cannot hope to recover componentsof the source function that are more oscillatory than dictated by the spacing of thesensors. A similar idea holds for sampling in time. Therefore, arbitrary oscillatorycomponents will appear as noise in the reconstructed source function; although un-observable by the sensors, these oscillatory components add noise to the predictedconcentration field. To address this source of ill-posedness, we employ Tikhonovregularization, which has the effect of filtering the most oscillatory components ofthe source. However, Tikhonov regularization also smoothes discontinuities, so forsources that have sharp fronts, we instead use total variation regularization, whichpreserves the discontinuities at the expense of giving rise to strong nonlinearities inthe regularization term.

We formulate the transport source inversion problem as an infinite dimensional op-timization problem that is constrained by the time-dependent convection-diffusionequation. The objective function is the squared discrepancy between observed andpredicted concentrations at the sensor locations over time. The optimization vari-able is the source function in space and time. Optimality conditions for the inverseproblem are derived in weak and strong form via calculus of variations. The systemof optimality conditions consists of the initial-boundary value transport problem,the final-boundary value adjoint transport problem, and the space-time boundaryvalue source problem. The weak form leads naturally to a Galerkin space-time finiteelement method that discretizes the space-time volume. This system can be ratherformidable to solve. To implement the numerical solution, we leverage the high-level finite element symbolic toolkitSundance, through which all of the meshing,discretization, element computation, assembly, and solution methods are abstractedaway from the user. We present several numerical examples that examine the influ-ence of the density of the sensor array on the quality of the reconstructed source; theeffectiveness of total variation regularization for discontinuous sources; the invert-ibility of the source as the transport becomes increasingly convection-dominated;

krlong@ca.sandia.gov (Kevin R. Long),bartv@sandia.gov (Bart vanBloemen Waanders).

2

the ability of the space-time inversion formulation to track moving sources, and therate of convergence of the finite element approximation of the reconstructed sourceto the exact solution of the inverse problem.

Source inversion for convective-diffusive transport has a number of environmental,energy, and national security applications. Examples include localizing the sourcesof chemical and biological attacks on public facilities (such as subways or airports),and determining air or groundwater pollutant sources. A first step towards solvingproblems of this type is the development of an appropriate formulation of the condi-tions that characterize optimality, as well as an associated numerical approximationof these conditions; this is the focus of this article. Scalable, fast, robust algebraicsolvers for the optimality conditions are certainly of paramount interest, particu-larly for (near) real-time applications. The subject of solvers is largely beyond ourpresent scope; we comment briefly on some of the issues and provide references toour related work, but defer a full treatment to future articles.

The literature on inverse transport problems is concerned mostly with radiationtransport inverse problems, which are characterized by convective transport. Thisis not surprising, given the importance of applications in computerized tomogra-phy (see for example the survey by Natterer [10]). Recently, interest has begunto emerge in source inversion for atmospheric transport models, specifically esti-mating CO2 surface fluxes from atmospheric observations [6]. While this problemclass is somewhat different from those we consider (in particular unknown bound-ary fluxes, rather than a volume source, are to be estimated), there are neverthelessmany similarities, and the method described here may provide useful for such prob-lems (and vice versa).

The remainder of this article is organized as follows. Section 2 develops the vari-ational formulation and associated space-time Galerkin finite element approxima-tion of the optimality conditions for the transport source inversion problem. Section3 presents results of numerical experiments that study the performance of the in-verse method, while in Section 4 we overview the high-level finite element toolkitSundance, which was used for the numerical implementation. Finally, we offer con-clusions in Section 5.

2 Variational formulation and finite element approximation

In this section, we derive the first-order optimality conditions for the source inver-sion problem. First we state a weak form of the transport equation (to which werefer as theforward, or stateproblem). Then we formulate the inverse problem ofestimating the contaminant source term of the forward problem, given measure-ments of the contaminant concentration at a finite number of distinct points. Next,we derive the system of first-order optimality conditions in weak and strong forms.

3

These are necessary conditions for the solution to the inverse problem. We con-clude the section with a discussion of a space-time finite element approximation ofthese optimality conditions, and comment on algebraic solver issues.

2.1 The forward problem

Given a known velocity fieldv(x; t), the transport equation for the concentrationc(x; t) in a domain is given by the standard convection-diffusion equation

@c

@t� k�c +rc � v + f = 0 in � (0; T );

@c

@n= g on �N � (0; T ); c = 0 on �D � (0; T ); (1)

c = c0 in at t = 0;

wherek > 0 is the molecular diffusion (assumed constant),f(x; t) the sourceterm that drives the concentration in the system,c0(x) the initial concentrationdistribution,�N the portion of the boundary on which Neumann conditionsg(x; t)are prescribed, and�D the portion of the boundary with homogeneous Dirichletconditions. For notational convenience we introduce the function spaceV := fc 2H1() : c = 0 on �Dg and the bilinear forms

a(f1; f2) :=Z

rf1 � rf2 d; b(f1; f2) :=Z

rf1 � v f2 d;

s(f1; f2) :=Z

f1 f2 d; s(f1; f2)� :=Z�

f1f2 d�:

For this setup and for smooth boundaries,c will be in V , whenf is inH�1(). Aweak form of (1) is the following: findc in V such thatc = c0 at t = 0 and

s(@c

@t; ) + k a( ; c) + b(c; )� k s(g; )�N + s(f; ) = 0;

8 2 V ; 8t 2 (0; T ): (2)

2.2 The inverse problem

We are interested in solving the following problem: given observations of the con-centrationfc�jg

Nrj=1 atNr locationsfxjg

Nrj=1 inside, we wish to estimate the source

termf that most closely reproduces the observed concentrations. The inverse prob-lem is formulated as a constrained, regularized least squares parameter estimationproblem:

minf;c

J (c; f) :=1

2

NrXj=1

Z T0

s((c� c�)2; Æ(x� xj)) dt+1

2�R(f); (3)

4

subject to the constraints given by the (weak form of the) convection-diffusionequation (2).

At the outset, we do not assume anya priori information about the structure offin space or time. Upon discretization we will have as many unknowns as the di-mension of the approximation space; for a space-time Galerkin approximation, forexample this will equal the number of spatial grid points multiplied by the num-ber of time steps. If the misfit between predicted and observed concentrations ismeasured throughout the domain, the problem can be viewed as a matching con-trol problem, which is known to have a unique solution (by linearity of constraintsand by strict convexity of the observation functional); see [4, 5] for theory and nu-merical examples for matching flow control problems. However, in most inverseproblems, the observation data is limited. Typically, we have just a finite number(Nr) of observation locations at our disposal, and therefore, by a simple count-ing argument applied to the discretized problem, we see that the inverse problemis underspecified, i.e. limited measurements imply multiple solutions, and henceill-posedness in the sense of Hadamard. Therefore some type ofregularizationisneeded in order to obtain a well-posed problem. For this reason we have includedR(f) in (3) to represent the regularization term. Some possible choices are

R2(f) :=Z T0

s(f; f) dt; (4)

Re(f) :=Z T0

a(f; f) dt+Z T0

s

@f

@t;@f

@t

!dt; (5)

Rt(f) :=Z T0

s (rf;rf)1

2 dt+Z T0

s

@f

@t;@f

@t

! 12

dt: (6)

The first two regularizations are of Tikhonov type:R2 penalizes the space-timeL2 norm, and hence magnitude, off , while Re penalizes theH1 seminorm, andhence rough components, off . Since the sourcef can be discontinuous, and sinceRe will smooth these discontinuities, we have also investigated the functionalRt,which restrictsf to the space of functions with bounded variation in space andtime. Suchtotal variation regularization penalizes oscillatory components whileadmitting discontinuities inf with penalty proportional to the jump (as opposed toan “infinite penalty,” as withRe). The tradeoff is thatRt is nonquadratic and highlynonlinear, creating significant computational difficulties.

The amount of regularization is controlled by the parameter�, and its choice isrelated to the scaling of the spectrum of the observation (the sum of delta functions)and regularization operators. Choosing the proper value of� is a nontrivial task;there are a number of different approaches, and rather than review them we referthe reader to [13, Ch. 7].

5

2.3 Derivation of first-order optimality conditions

We reformulate the original constrained optimization problem as a system of (pos-sibly nonlinear) partial differential equations by introducing an additional function,the Lagrange multiplier� 2 V , and a Lagrangian functional

L(c; �; f) := J (c; f)+Z T0

s(@c

@t; �)+k a(�; c) + b(c; �)� k s(g; �)�N + s(f; �) dt: (7)

Under mild conditions, solutions of (3) correspond to stationary points of (7). Bytaking variations with respect toc, �, andf , we can derive a system of partialdifferential equations that characterize the first order optimality conditions for theoptimization problem (3). These are also known as the Karush-Kuhn-Tucker (KKT)optimality conditions. Below we give details on the derivation.

2.3.1 State equation

Taking variations of the Lagrangian (7) with respect to the Lagrange multipliersand equating the result to zero for all admissible variations, i.e.

@�L(c; �; f) := L(c; �+ ��̂; f)j�=0 = 0; 8�̂ 2 V;

we obtain the weak form of the state problem (2). Integration by parts then yieldsthe strong form initial-boundary-value problem (1).

2.3.2 Adjoint equation

To derive the equation for the Lagrange multipliers, which is known as theadjointequation, we take variations of the Lagrangian (7) with respect toc and equate theresult to zero for all admissible variations. Namely, we take

@cL(c; �; f) :=L(c+ �ĉ; �; f)j�=0 = 0; 8ĉ 2 V;

which gives the weak form of the adjoint equation:

NrXj=1

Z T0

s(c� c�; ĉ Æ(x� xj) ) dt

+Z T0

s(�;@ĉ

@t) + b(ĉ; �) + k a(�; ĉ) dt = 0; 8ĉ 2 V: (8)

By construction, the Lagrange multiplier satisfies homogeneous Dirichlet boundaryconditions on�D. To obtain the remaining conditions on�, we integrate by parts;

6

in time for the termR T0s(�; @ĉ=@t) dt, and in space for the convective and diffusive

terms. From the time integration we have

Z T0

s(�;@ĉ

@t) dt =

Z

�ĉjt=Tt=0 d�Z T0

s(ĉ;@�

@t) dt:

Sinceĉ = 0 at t = 0 but is arbitrary att = T , we obtain thefinal condition� = 0at t = T . In space, using the vector identities

r � ((ĉv)�) = �r � (ĉv) +r� � vĉ = rc � v�+ ĉ(r � v)�;

and the divergence theorem, and assuming that the velocity field is incompressible,i.e.r � v = 0, we arrive at

Z T0

b(ĉ; �) dt =Z T0

�b(�; ĉ) + s(ĉ; � v �n)� dt;

which combined withZ T0

k a(�; ĉ) dt =Z T0

Z

�ĉr � kr� d + s(ĉ; k@�

@n) dt;

results in the mixed condition(kr� + �v) � n = 0 on�N . Therefore, the strongform for the adjoint equation is given by

�@�

@t� k���r� � v =

NrXj=1

(c�j � cj); in � (0; T )

(kr�+ v�) � n = 0; on �N � (0; T ); � = 0 on �D � (0; T ); (9)� = 0 in at t = T:

Notice that this is a final value problem (it has a final, rather than initial, condition),and is linear in the unknownsc and�.

2.3.3 Source equation

Taking variations of the Lagrangian (7) with respect tof , we obtain thesourceequation. Here we present a regularization formulation that has the form of thetotal variation functionalRt in space, but theH1 TikhonovRe functional in time.Let

w(f) :=1

2(rf � rf)1

2

;

then,

@fR(f; f̂) :=Z T0

Z

w(f)rf � rf̂ d dt+Z T0

s(@f

@t;@f̂

@t) dt: (10)

7

Taking variations of the Lagrangian with respect tof we obtain

@fL = �@fR(f; f̂)�Z T0

s(�; f̂) dt = 0; 8f̂ in V: (11)

By carrying out integration by parts in space and time we can derive the boundaryconditions and the strong form of the source equation:

�� (r � w(f)rf +@2f

@t2)� � = 0; in � (0; T )

@f

@n= 0 on �� (0; T );

@f

@t= 0; at t = 0; t = T: (12)

Notice that while the forward and adjoint problems are evolution equations (in factmixed elliptic-hyperbolic), the source problem (12) is an equilibrium problem, infact an elliptic problem, over the space-time domain � (0; T ). The differentialoperator is a(d + 1)-dimensional nonlinear Poisson operator in space-time, andthe problem has Neumann boundary conditions on all boundaries, including thoseassociated with the time direction. Had we chosen the pureH1 Tikhonov regular-izationRe defined in (5), the elliptic operator governing the the source problemwould reduce to the(d + 1)-dimensional Laplacian, and (12) would become a lin-ear PDE problem. Finally, the choice of theL2 regularizerR0 would result in analgebraic source equation.

One final remark: to guarantee solvability of (12), we have to modifyR, sincew(f)is unbounded where the gradient off is zero. Therefore, we modify the regulariza-tion functional by introducing an additional (small) parameter� so that

R�(f ;�) =

Z T0

Z

(rf � rf + �)1

2 d dt+Z T0

s(@f

@t;@f

@t) dt: (13)

The perturbed regularizerR� is now differentiable wherejrf j is zero. The ex-pression for the variation ofR with respect tof , (10), is still valid, but withw(f)replaced by

w�(f ;�) :=1

2(rf � rf + �)1

2

: (14)

The choice of� and� has great effect on the accuracy of the solution and theconditioning of the problem. For a discussion see [1].

2.3.4 KKT optimality conditions

In the derivation of the KKT optimality conditions, we assumed the weak forms tohold a.e. in time, in accordance with a conventional finite-difference type approx-imation in time. However, the system of forward, adjoint, and source equations iselliptically coupled in space-time. Therefore, a natural choice for numerical so-lution is to discretize by space-time finite elements, based on a space-time weak

8

form. This weak form can be stated as: Findf; c; � 2 V � H1(0; T ) such that es-sential boundary conditions are satisfied on the space-time volume (i.e. Dirichletand initial conditions onc, and Dirichlet and final conditions on�) and

Z T0

s(@c

@t; �̂) + k a(�̂; c) + b(c; �̂)� k s(g; �̂)�N + s(f; �̂) dt = 0;

8�̂ 2V �H1(0; t);Z T0

NrXj=1

s(c� c�; ĉ Æ(x� xj) )� s(@�

@t; ĉ) + b(ĉ; �) + k a(�; ĉ) dt = 0; (15)

8ĉ 2V �H1(0; t);Z T0

� s(w�(f)rf;rf̂) + � s(@f

@t;@f̂

@t)� s(�; f̂) dt = 0;

8f̂ 2V �H1(0; t):

Once the space-time weak form (15) is established, finite element approximationfollows in the usual way. Namely, we restrictf; c; � to a family of finite dimen-sional subspaces of the infinite dimensional spaceV �H1(0; T ); let us symbolizethis finite element space bySh. Indicating the finite element approximation by thesubscripth, the problem then becomes:Find fh; ch; �h 2 Sh such that Dirichlet and initial conditions onc and Dirichletand final conditions on� are satisfied on the space-time volume, and

Z T0

s(@ch

@t; �̂h) + k a(�̂h; ch) + b(ch; �̂h)� k s(g; �̂h)�N + s(fh; �̂h) dt = 0;

8�̂h 2Sh;Z T0

NrXj=1

s(ch � c�; ĉh Æ(x� xj) )� s(

@�h

@t; ĉh) + b(ĉh; �h) + k a(�h; ĉh) dt = 0;

(16)

8ĉh 2Sh;Z T0

� s(w�(fh)rfh;rf̂h) + � s(@fh

@t;@f̂h

@t)� s(�h; f̂h) dt = 0;

8f̂h 2Sh:

Our choice of usingH1() approximation forf is somewhat arbitrary; in generalf is a member ofH�1() and anL2 approximation might be more appropriate (forexample we could use piecewise constant approximation, as opposed to piecewiselinear). In this case the resulting linear systems appear to be more ill-conditioned.The best numerical properties are obtained withH1 regularization; this however,can lead to reduced resolution. In addition, we cannot in general expectc; � tobe inH1 in time. But the use ofH1 regularization guarantees thatc; � will besmooth enough in space and time. In practical situations however, we expect thattotal variation will achieve sharper resolution—at the cost of having to solve anonlinear problem.

9

The fact that the optimality system (16) is approximated by a Galerkin finite ele-ment method does not imply that the state or adjoint convection diffusion problemsmust beindividually approximated by Galerkin methods. It means only that thetest space for the state equation must be taken as the trial space for the Lagrangemultiplier, and the test space for the adjoint equation will be the trial space for theconcentration. This allows use of an appropriate Petrov-Galerkin approximationfor the state equation; for example, we could have employed a streamline diffusionmethod, as may be preferred for a convectively-dominated transport problem, andstill maintained consistency with the approximation in (16).

Several of the numerical results presented in Section 3 are for a steady state ap-proximation to the time-dependent optimality system above. In this case, the weakform of the KKT optimality system (assuming zero Neumann boundary conditionson the state problem, i.e.g = 0) is given by:Findf; c; �;2 V such that

k a(�̂; c) + b(c; �̂) + s(f; �̂) = 0; 8�̂ 2 VNrXj=1

(Æ(x� xi)(c� c�)ĉ) + k a(ĉ; �)� b(�; ĉ) = 0; 8ĉ 2 V (17)

� s(w�(f)rf;rf̂)� s(�; f̂) = 0; 8f̂ 2 V:

Finite element approximation of this system follows straightforwardly:Findfh; ch; �h;2 Vh such that

k a(�̂h; ch) + b(ch; �̂h) + s(fh; �̂h) = 0; 8�̂h 2 VhNrXj=1

(Æ(x� xj)(ch � c�)ĉh) + k a(ĉh; �h)� b(�h; ĉh) = 0; 8ĉh 2 Vh (18)

� s(w�(fh)rfh;rf̂h)� s(�h; f̂h) = 0; 8f̂h 2 Vh:

Finally, we note that several of the numerical examples in Section 3 use theH1

Tikhonov regularization (5), rather than the mixed total variation/Tikhonov regular-izer defined by (13). In this case, the time-dependent and steady optimality systems(16) and (18) are still valid, provided we takew� := 1.

2.3.5 Solution Algorithm

The discrete optimality systems (16) and (18) constitute formidable systems tosolve, particularly in the time-dependent case. A full discussion of solver issuesis beyond the scope of this article, and will be presented in a follow-on article.However, several pertinent comments are in order. One way to avoid the large sizeand coupled nature of (16) and (18) is to effect a block-elimination of state andLagrange multiplier variables, and reduce the system to one in just the source un-knowns. We illustrate this idea in the context of the steady-state optimality system

10

(18), but the ideas are similar in the time-dependent case. Given an estimate of thesourcefh, first solve for the concentrationch from the state equation,

k a(�̂h; ch) + b(ch; �̂h) + s(fh; �̂h) = 0; 8�̂h 2 Vh;

then withch known solve the adjoint equation

NrXj=1

(Æ(x� xj)(ch � c�)ĉh) + k a(ĉh; �)h � b(�h; ĉh) = 0; 8ĉh 2 Vh

for �h; and finally solve the “reduced” source equation

� s(w�(fh)rfh;rf̂h)� s(�h; f̂h) = 0; 8f̂h 2 Vh:

to updatefh. This iteration can be accelerated by the conjugate gradient method(the reduced source equation is positive definite). For Tikhonov regularization, thereduced source system is linear, and only linear iterations are required. For totalvariation regularization, the reduced source system is nonlinear, and the inner lineariterations must be embedded within a nonlinear iteration, for example Newton’smethod. This block-elimination is a special case of the reduced space SequentialQuadratic Programming method [11]. Sinceeachlinear iteration of this reducedspace method requires the solution of state and adjoint transport PDEs, the methodcan be very expensive. On the other hand, the advantage is that structure is impartedonto the KKT optimality system (by decomposing into state, adjoint, and sourcesolves); moreover, one can capitalize on available algorithms and software for thestate and adjoint transport equation solves, and (nonlinear) Poisson solvers for thesource equation.

Despite the advantages of this block-elimination, the requirement of solving stateand adjoint systems at each iteration is often onerous, and we prefer to solve the en-tire KKT optimality system (16) or (18) simultaneously. Since these constitute verylarge systems, particularly for 3D or time-dependent problems, iterative solvers andgood preconditioners are essential. We will not discuss these issues in detail, andinstead direct the reader to the detailed descriptions of the inner linear iteration in[2] and the outer nonlinear iteration in [3]. The basic idea is that one can solvein the full space iteratively, but precondition via an approximate version of the re-duced space method, in which state and adjoint operators are replaced by their ownpreconditioners, and the source operator can be approximated by ideas from quasi-Newton optimization theory. At least in the steady-state case, extremely efficientalgorithms result, and one can often obtain the solution to the inverse problem ina small multiple (as little as 4 or 5) of the cost of solving the state equation. Thiswas demonstrated in [2, 3] in the context of boundary optimal control of steadyNavier-Stokes flows. Related algorithms are also discussed in [7].

Finally, regardless of whether we solve the full KKT system simultaneously orwithin the reduced space of source unknowns, in the 3D time-dependent case we

11

are faced with source problems that involve 4D (space plus time) Laplacian op-erators forH1 Tikhonov regularization, and nonlinear 4D Poisson operators fortotal variation regularization. These are certainly challenging problems and requirespecialized solvers, especially on unstructured meshes.

3 Numerical results

In this section we provide sample numerical results to examine the performance ofthe inverse method on synthetic observational data. We provide results for solutionof both the steady state (18) and time-dependent (16) discrete optimality systems,using bothH1 Tikhonov (w� = 1) and total variation (w� given by (14)) regu-larization. Sensors that measure the concentration field are placed throughout thespatial domain on a regular array. We synthesize sensor readings by proposing asourcef �, and then solve the inverse problem to determine how well the source isreconstructed.

In all of the numerical experiments, the discretization in (18) and (16) is effectedvia piecewise linear finite elements in both space and (when applicable) time. Wediscretize the state, adjoint and source variables on the same regular grid consist-ing of linear triangles. Newton’s method is used to solve the nonlinear systems thatarise when total variation regularization is used; a simple backtracking line searchis employed to ensure convergence. The linear systems that are produced either ateach iteration of Newton’s method, or once and for all when Tikhonov regulariza-tion is used, are solved iteratively using the biconjugate gradient stabilized method(BICGSTAB). A level-k incomplete LU factorization (ILUK) is used to constructa preconditioner. We invoke a convergence tolerance of10�10 to terminate both theinner (linear) as well as outer (nonlinear) iterations.

Since we are using a variationally-based Galerkin approximation, it should be pos-sible to obtain a symmetric linear(ized) KKT system. In fact, if the equation blocksare ordered as (adjoint, source, state), and the variable blocks ordered as (state,source, Lagrange multiplier), we do obtain a symmetric discrete system, as can beverified from (18) or (16). However, in this case, the (3,3) block, corresponding tothe coefficient of the Lagrange multiplier in the state equation, is identically zero,and standard ILU preconditioning will fail. Therefore, we reorder the equations as(state, adjoint, source) and the unknowns as (state, Lagrange multiplier, source),which is the order implied by the reduced space method. This places the state, ad-joint, and source operators on the main diagonal blocks, and prevents failure (andindeed leads to reasonable performance) of the ILUK preconditioner. However, thisordering creates a nonsymmetric linear system, which is why we use BICGSTAB.

In all of the examples below, we choose the regularization constant� = 10�5, and(where appropriate) the total variation parameter� = 0:1. The spatial domain is a

12

square of dimension2L � 2L, with L = 8, and for time dependent problems,Tis taken as10. The diffusion constantk is taken as 1. We choose a velocity fieldv given by a Poiseuille flow, i.e.vx(y) = (L + y)(L � y)=(L2); vy = 0. This hasmaximum value of 1, at the centerline, and thus the Peclet number, Pe=2Lv0=k,based on this value is 8. To create a sequence of problems with increasing Pe, wesimply scale up the velocity field.

In the subsections below, we study the dependence of the quality of inversion on thenumber of sensors, the type of regularization, the ratio of convection to diffusion,and the time dependence of the source. We also examine the convergence rate ofthe finite element approximation of the inverse problem to the exact solution.

3.1 Effect of number of sensors

In this subsection we study the influence of the number of sensors on the qual-ity of the reconstructed source. The steady-state problem (18) is solved. Boundaryconditions on the concentrationc are homogeneous Dirichlet on the left boundaryand homogeneous Neumann conditions on the right, top, and bottom sides. Thetarget source functionf �(x; y) consists of two concentrated Gaussians with differ-ent peak magnitudes and location. We discretize the problem on a40 � 40 grid(which means we have 3042 linear triangular elements, 1600 grid points, and a to-tal of 4680 concentration, multiplier, and source unknowns). We use TikhonovH1

regularization, and solve the same problem with three sensor arrays of increasinglyfiner density. The array consists of uniformly-spaced sensors that cover the domain.Figure 1 plots the reconstructed source for arrays of4 � 4, 10 � 10, and40 � 40sensors. The last case, shown at the bottom of the figure, is indistinguishable fromthe the target source, which is why we do not displayf �. Inversion of observationsfrom the4 � 4 sensor array is unable to resolve the location or magnitude of thesecond source, while the first source is greatly smoothed. The10� 10 sensor arrayleads to a much better inversion; the locations and general shape are well-resolved.On the other hand, the inversion is less successful in establishing the magnitudeof the two Gaussians. Finally, inversion with the40 � 40 sensor array leads to areconstruction that is nearly indistinguishable from the exact source. Since sensorsare placed at every grid point, the inverse problem is well-posed in the absence ofthe regularization term, and we could have obtained the exact source by setting theregularization parameter to zero. Since here we are using� = 10�5, the inversionoperator is dominated by the (full-rank) least-squares term, and the influence of theregularization matrix (a discrete Laplacian stiffness matrix) is minimal, leading tosource reconstruction that is almost exact.

13

−4

−2

0

2

4

−4

−2

0

2

4−0.1

−0.05

0

0.05

0.1

0.15

0.2

0.25

−4

−2

0

2

4

−4

−2

0

2

4−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

−4

−2

0

2

4

−4

−2

0

2

4−0.2

0

0.2

0.4

0.6

0.8

1

1.2

Fig. 1. Effect of sensor density on source inversion. Steady transport, TikhonovH1 regu-larization,40 � 40 grid, flow from upper left to lower right. Top left,4 � 4 sensor array;top right,10� 10 sensor array; bottom,40� 40 sensor array.

3.2 Effect of regularization type

The numerical experiments of the previous subsection used a smooth source (a sumof Gaussians), for which Tikhonov regularization performs reliably. In our next setof experiments, we use a nonsmooth target source and compare the effectivenessof Tikhonov and total variation regularization. Boundary conditions and grid sizeare as in Subsection 3.1. The sensors are located on a10 � 10 array. The targetsource function consists of two “point” sources, that actually have constant strengthover a patch of 4 cells (i.e. 8 triangles), and taper linearly to zero in neighboringelements; this is is depicted in the upper left image of Figure 2. The results ofinversion usingH1 Tikhonov regularization are shown in the upper right figure. Asis clear from the figure, Tikhonov does a poor job in reconstructing the source. Bothshapes are severely smoothed, and one can observe substantial oscillations nearthe interface of the right source. In contrast, the Tikhonov regularization shown inFigure 1 on the same sensor array is much more effective at identifying a similarlyconcentrated, but smooth source (a Gaussian). On the other hand, total variationregularization, as shown in the bottom image of the figure, results in a much betterrepresentation of the target source. While the magnitude is still incorrect, the shapesare well-localized, with significantly less smoothing. We remark that total variationregularization is effective at reconstructing discontinuous functions (i.e. inL2), but

14

in this case we are faced with target sources that approach a point source (i.e. inH�1), for which total variation produces an unbounded penalty. We should alsokeep in mind that total variation regularization results in a nonlinear system ofoptimality conditions, which are significantly more expensive to solve than for theTikhonov case.

Fig. 2. Effect of regularization type on source inversion. Steady transport,40 � 40 grid,10 � 10 sensor array, flow from upper left to lower right. Upper left image, target sourcefunction; upper right image, inversion with Tikhonov regularization; bottom image, inver-sion with total variation regularization.

3.3 Effect of Peclet number

In this subsection, we examine the effect of increasing Peclet number on the qual-ity of the inverted source. In particular, we study the ability to invert for both dif-fuse and concentrated sources as the transport becomes increasingly convection-dominated. For these experiments, we solve the steady form of the KKT optimalityconditions (18) and impose homogeneous Dirichlet boundary conditions on theconcentrationc on all boundaries. Figure 3 shows results obtained when the targetsource is taken as a smooth, diffuse function (a Gaussian). The flow is from left toright. We use a160 � 160 grid, and the sensors are located on a6 � 6 uniformly-spaced array. The bottom image in the figure depicts the color contours of the targetsource function. The top three rows show color contours of the concentration field

15

(left column) and source reconstruction (right column) for three different Pecletnumbers, namely 0, 8, and 80. The left column of images illustrates the increasinglyconvection-dominated concentration field. The right column demonstrates that forthis problem the method is able to reconstruct a source that is essentially identi-cal to the target source, independent of the Peclet number. The6 � 6 sensor arrayapparently places a sufficient number of sensors in the concentration field wake toidentify accurately the source field.

A different picture emerges when a highly concentrated Gaussian is chosen asthe target. Figure 4 presents the results. The bottom image displays contours ofthe target source. We compare inversion for Pe 0 (pure diffusion, discretized on a160 � 160 grid) and Pe 800 (strong convection, discretized on a320 � 320 grid).The left column refers to the pure diffusion case, and the right column to the largeconvection case, as can be inferred from the concentration contours shown in thefirst row. The second and third rows show the results of inversion using6 � 6 and21� 21 sensors arrays, respectively. The6� 6 sensor array is unable to resolve thesource, neither for Pe 0 nor 800, as shown in the second row of the figure. However,a21� 21 sensor array is able to accurately locate the source for pure diffusion; thereconstructed source is indistinguishable from the target. The opposite conclusionis reached with the Pe 800. In this case, apparently even a21 � 21 sensor arraydoes not place sufficient sensors in the concentration wake to produce an accurateinversion; the image shows that the recovered source is a highly-smoothed approx-imation of the target.

3.4 Time dependent inversion

In this subsection we demonstrate the ability of the inverse formulation to recon-struct a time-dependent source. Recall that we must solve the space-time-coupledoptimality conditions (16) over the space-time box0 � x � L; 0 � y � L; 0 �t � T . Homogeneous Neumann boundary conditions and zero initial conditions areimposed onc. H1 Tikhonov regularization, in space and time, is used. The three-dimensional space-time volume is discretized on a30� 30 � 30 grid. Sensors areplaced on a10 � 10 � 10 uniform array (i.e. the sensors locations do not changeover time). The target is a concentrated Gaussian that moves in time across thespatial domain from the upper left to the lower right, as shown in Figure 5. Theright column depicts the target source; the four rows correspond to snapshots ofthe source att = 0; 3:3; 6:7; 10, respectively. The left column of the figure displaysfour snapshots (at the same time instants) of the reconstructed source. By compar-ing left and right columns, we see that the reconstruction is excellent inside theinterval0 < t < 10, and deteriorates somewhat near the endpoints of the interval,i.e. t = 0 andt = 10. Actually, considering the rather coarse grid that is used (30grid points or time steps in each direction), even the reconstruction near the initialand final conditions can be considered quite good. It is tempting to expect that the

16

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0

0.001

0.002

0.003

0.004

0.005

0.006

0.007

0.008

0.009

0.01

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Fig. 3. Effect of Peclet on the quality of the inverted source for a smooth, diffuse targetsource. Steady problem, TikhonovH1 regularization, homogeneous Dirichlet boundaryconditions onc, 6�6 sensor array,160�160 grid. Bottom image shows target source. Firstrow corresponds to Pe 0; second, to Pe 10; third, to Pe 100. Left column show contours ofconcentration field; right column shows reconstructed source.

quality of the inversion will worsen steadily over time, since the forward problemevolves from initial conditions. Recall, however, that the source equation (whoselinearized operator determines the character of the inverted source) is a boundaryvalue problem in space-time; there is no reason, therefore, to expect evolutionarybehavior for the inverted source. On the contrary, when the source is near spatialboundaries (i.e. att = 0 andt = 10), we cannot expect that the reconstruction willbe as accurate as when it is in the interior, since there are fewer sensors in or nearits support.

17

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0

1

2

x 10−4

−0.05

0

0.05

0.1

0.15

0.2

0.25

0.3

20 40 60 80 100 120 140 160

20

40

60

80

100

120

140

160−0.05

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

−0.1

0

0.1

0.2

0.3

0.4

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Fig. 4. Effect of Peclet number on the quality of the inverted source, for a concentratedsource. Steady state transport, TikhonovH1 regularization, homogeneous Dirichlet bound-ary conditions onc. Target source is concentrated Gaussian, shown in bottom image. Leftcolumn corresponds to Pe 0 (discretized on160 � 160 grid), right to Pe 800 (discretizedon 320 � 320 grid). Top row shows concentration contours; middle shows reconstructedsources with6�6 sensor array; third row shows reconstructed sources with21�21 sensorarray.

3.5 Convergence Rates

In this subsection we provide a study of the accuracy of the finite element ap-proximation of the optimality system (18). We measure the rate of convergence ofthe finite element approximation of the reconstructed source,fh, to the source thatsolves the inverse problem (17) exactly. Unfortunately, this is notf �, the source

18

0

0.2

0.4

0.6

0

0.5

1

0

0.2

0.4

0.6

0.8

0

0.5

1

00.20.40.60.8

0

0.5

1

0

0.2

0.4

0

0.5

1

Fig. 5. Time dependent source inversion example,H1 Tikhonov regularization, concen-trated moving Gaussian source,30 � 30 � 30 space-time grid,10 � 10 � 10 sensor array.Right column shows color contours of the target source; left column shows reconstructedsource. Rows correspond to four snapshots in time (t = 0; 3:3; 6:7; 10).

that was used to synthesize the sensor observationsc�(xj); the inclusion ofH1

Tikhonov regularization (i.e. the term involving� in (17)) has modified the exactsolution of the optimality system. The reasons is as follows: the null space of thesecond variation of the least squares term (i.e. the first term in (3)) contains oscilla-tory source functions (a coarse spacing of sensors provides sufficient data to recoverjust the smooth components of the source). The inverse operator is thus rank defi-cient, which is why we add the regularization term (the term involving�) in (3).For simplicity, let us assume thatR is chosen to be theH1 Tikhonov regularizer.The second variation of this term is just the Laplacian; therefore, we can interpretthe regularization term as one that penalizes the most oscillatory components offmost strongly (since the largest eigenvalues of the Laplacian are associated with themost oscillatory eigenfunctions). The oscillatory functions lying in the null spaceof the least squares term will now have positive eigenvalues associated with them,the inverse operator (i.e. the second variation ofJ with respect to the reduced spaceof source functionsf ) becomes positive definite, and the inverse problem becomeswell posed. Unfortunately, the Laplacian regularizer also penalizes smooth eigen-functions off , which lie in the range space of the least squares term, albeit thepenalty is much smaller than for the oscillatory components. We have some de-

19

grid kfh � f�k2

16 � 16 1:265 � 10�2

32 � 32 2:163 � 10�3

64 � 64 5:126 � 10�4

128 � 128 1:264 � 10�4

256 � 256 3:160 � 10�5

Table 1Convergence of finite element approximation of reconstructed sourcefh to the exact re-constructed sourcef�. Linear elements, Gaussian source. Convergence rate is optimal.

gree of control through the choice of�, which must be chosen large enough so thatrough components off (those that are poorly-determined by the observations) arefiltered as much as possible, but small enough so that smooth components (thosethat are well determined by the observations) are as unaffected as possible.

To compute the convergence rate of the finite element approximation, however,we require an exact solution of the inverse problem. Therefore, we shall create aregularizer that renders the solution to the inverse problem unique and equal tof �.This can be achieved by choosing the regularizer

R(f) =Z

(rf �rf �) � (rf �rf �) d:

We see that the choicef = f � minimizes both the regularization term and the leastsquares term (becausec = c�) in (3), and thereforef � is the solution to this suitably-regularized inverse problem. Moreover, this choice of regularizer adds only theforcing term�s(rf �h ;rf̂h) to the source equation of the optimality system (17),and does not change the linear operator. This gives us a suitable test of finite elementconvergence rate.

Table 1 lists the error in the finite element approximation off � for a sequence offive increasingly finer meshes. The exact sourcef � is taken as a Gaussian. We usethe standardL2 norm error

kfh � f�

k2

2=Z

(fh � f�)2 d:

The results exhibit a quadratic rate of convergence. Since we are using linear ele-ments, this rate is optimal.

4 Sundance implementation

The numerical examples in this paper were implemented using a general-purpose,high-level, finite element toolkit called Sundance [8]. Sundance accepts a system

20

of coupled PDEs and boundary conditions written in symbolic form that is closeto the notation in which a scientist or engineer would normally write them withpencil and paper. Each function appearing in this symbolic description is anno-tated with a specification of the finite-element basis with which that object willbe discretized. This information, along with a mesh, is then used by Sundance toassemble the implied discrete operators and vectors. This matrix assembly step isfully and transparently parallelized. These capabilities make Sundance a powerfulrapid prototyping and algorithmic research tool. With Sundance’s flexible symbolicinterface, it is straightforward to assemble and solve the discrete KKT systems (16)and (18). This section provides a brief introduction to Sundance.

4.1 Overview of Sundance

Sundance is a collection of C++ classes that can be used as building blocks to as-semble a finite element computation. The PDEs and boundary conditions are writ-ten in terms of symbolic expression objects, orExprs , the subtypes of which canrepresent mathematical concepts such as functions, test functions, and differentialoperators. The problem geometry is given in terms of aMesh object which repre-sents an unstructured mesh. Subdomains of a mesh are represented byCellSetobjects, which are operators that can identify cells on which an equation or bound-ary condition is to be applied. Linear operators, vectors, and solvers are imple-mented using the TSF [9] linear algebra components. Details such as memory man-agement and parallelism are handled transparently.

The guiding design principle behind Sundance is that the user should be able tospecify a finite element problem in terms of a symbolic statement of a variationalform and boundary conditions, augmented by high-level discretization specifierssuch as basis function type and quadrature method. The user should be shieldedfrom all details of low-level bookkeeping of elements, nodes, and degrees of free-dom. A sophisticated user will want fine control over discretization, but that controlshould be done through high-level directives. For example, in Sundance the order-ing of block rows is specified by the order of a list of test functions objects in asingle constructor argument.

We must emphasize that for performance reasons, the high-level objects used forproblem specification are not used for numerical calculations. Rather, they are usedto marshal a set of internal objects that can be used for efficient calculations. Thus,Sundance can provide a high-level interface with a minimal performance loss. Toillustrate basic Sundance objects and their use, below we show code fragments fora forward convection-diffusion problem on a 2D rectangle.

21

4.2 Defining the problem geometry

We begin by reading a mesh and defining boundary surfaces. Sundance has no built-in general-purpose meshing capability; in order to work with 3rd-party meshingcomponents it defines interfaces for mesh generators and mesh readers. Here wecreate a reader object capable of reading a file in Shewchuk’s Triangle [12] format,and read a mesh.

MeshReader reader = new ShewchukMeshReader(‘‘square’’);Mesh mesh = reader.getMesh();

Subdomains such as boundary surfaces or internal regions are identified withCellSetobjects, which can be created in a number of ways. Here, we identify subsets of theboundary that satisfy conditions on the coordinates. The coordinates are expres-sions, orExpr objects, and may be used in any symbolic expression.

/* define coordinate functions for x and y coordinates */Expr x = new CoordExpr(0);Expr y = new CoordExpr(1);

/* define cells sets for each of the four sides of the rectan-gle */CellSet boundary = new BoundaryCellSet();CellSet left = boundary.subset( x == -L );CellSet right = boundary.subset( x == L );CellSet bottom = boundary.subset( y == -L );CellSet top = boundary.subset( y == L );

The CellSet class supports logical operations such as union, intersection, andexclusion. In many problems, numerical or character labels will be associated withnodes or edges, soCellSet has the ability to identify labeled regions.

4.3 Setting up the symbolic problem

The test and unknown functions and differential operators are represented asExprobjects. The test and unknown functions are constructed with arguments giving thetype and order of their bases.

/* create symbolic objects for test and unknown functions */Expr v = new TestFunction(new Lagrange(2));Expr u = new UnknownFunction(new Lagrange(2));

/* create symbolic differential operators */Expr dx = new Derivative(0);Expr dy = new Derivative(1);

22

Expr grad = List(dx, dy);

The velocity field can be written in terms of coordinate functions.

Expr vx = (L+y)*(L-y);Expr vy = 0.0;Expr velocity = List(vx, vy);

Finally, we can use these components to write the weak PDE and boundary condi-tions in symbolic form

/* Write symbolic weak equation */Expr eqn = Integral(-(grad*v)*(grad*u)/peclet

- v*(velocity*grad)*u + v*source);

/* write dirichlet BCs */EssentialBC bc = EssentialBC(left, u*v);

Had we wanted to use a non-default quadrature scheme for any of these terms, wecould have supplied aQuadratureFamily object as an additional argument totheIntegral method or theEssentialBC constructor.

4.4 Solving the problem

We next build aStaticLinearProblem object which is responsible for build-ing the operators implied by the symbolic description and mesh.

StaticLinearProblem prob(mesh, eqn, bc, v, u);

Finally, we can specify a solver and solve the problem

TSFPreconditionerFactory precond = new ILUKPreconditionerFactory(1);TSFLinearSolver solver = new BICGSTABSolver(precond, 1.e-14, 500);

Expr soln = prob.solve(solver);

The solution is returned as a symbolic object; this allows the use of previous solu-tions in subsequent problems, as occurs in time-marching or nonlinear problems.

4.5 Implementation of the source inversion problems

The source inversion problems that are the subject of this paper are more complexthan the simple forward problem shown above, so space does not permit a complete

23

exposition of their code. However, some comments on the Sundance implementa-tion of the source inversion problems are in order:

� The sensor locations are specified in terms ofCellSet objects, and then theintegrations involving sensor measurements are done over theseCellSets .

� With TV regularization, the problem is nonlinear and must be solved by iter-ating solutions of a linearized problem. Sundance provides high-level facilitiesfor automatically linearizing a problem at the symbolic level, or alternatively thelinearized problem can be written by hand using the Sundance symbolic compo-nents.

� As mentioned in Section 2, block row ordering is critical for the effectiveness ofthe ILU preconditioner used here. In a multivariable problem, a Sundance userspecifies row and column ordering through the ordering of the lists of test andunknown functions given to theStaticLinearProblem constructor.

5 Conclusions and future work

In this article we have considered the inverse problem of determining an arbitrarysource function in both steady-state and time-dependent convective-diffusive trans-port, given a velocity field and measurements of a concentration field at a finitenumber of points. To address ill-posedness of the problem, we employed bothH1

Tikhonov and total variation regularization. We presented a variational weak for-mulation of the first order optimality system, which includes the initial-boundaryvalue state problem, the final-boundary value adjoint problem, and the space-timeboundary value source problem. The weak form leads naturally to a Galerkin space-time finite element method that discretizes the space-time volume. A series of nu-merical experiments, implemented using the high-level finite element toolkit Sun-dance, leads us to the following conclusions:

� Sundance is sufficiently rich and flexible to permit rapid and easy implementa-tion of finite element methods for many inverse problems involving PDEs.

� Increasing sensor coverage leads as expected to better identification of sourcestrength and distribution. A uniformly distributed10� 10 sensor array providesquite reasonable reconstruction of the source field for rather concentrated Gaus-sian sources.

� H1 Tikhonov regularization works well for smooth sources. For discontinuoussources, it performs poorly, smearing the source and leading to oscillations in thevicinity of the interface. Total variation regularization results in a much higherquality inversion, but leads to a highly nonlinear inverse problem.

� As the transport becomes increasingly convection-dominated, the ability to re-construct a diffuse source is unhindered, for a sensor array of reasonable density(6� 6).

� For a concentrated (nearly point) source, increasing the Peclet number results in

24

a deterioration in the quality of the reconstructed source.� The space-time variational formulation and associated finite element method is

capable of tracking a moving concentrated source across a domain. However,introduction of the time dimension into the inverse problem results in a need tosolve ad+ 1-dimensional elliptic problem over the space-time volume, wheredis the space dimension. In three space dimensions, this poses a significant com-putational challenge.

� For reconstruction of a smooth source, we observe an optimal convergence rateof the finite element approximation to the exact solution (i.e. we obtain a rate oftwo for linear elements).

We are pursuing applications of the formulation presented in this article to prob-lems of identification and remediation of hazardous contaminants. One setting ofinterest is the control of an HVAC system in a public facility, such as an airport,that has been attacked by releases of airborne chemical or biological agents. Solu-tion of the source inversion problem, given observations from distributed sensors aswell as modeled air flow fields, provides estimates of the source strength and dis-tribution, as well as the distributed concentration field away from the observationpoints. These estimates are fed into an optimal control solver—with both contam-inant transport and Navier-Stokes equations as constraints—that aims to controlboundary velocities at vent locations to reduce the contaminant hazard as rapidlyas possible. We have only briefly mentioned solver issues in this article. Clearly,scalable and robust solvers for the space-time KKT optimality equations are a crit-ical component of an effective system that is able to respond to such threats in nearreal-time, and this constitutes a topic of focused interest for us.

References

[1] V. Akcelik, G. Biros, and O. Ghattas. Parallel multiscale Gauss-Newton-Krylov methods for inverse wave propagation. InProceedings of IEEE/ACMSC2002 Conference, Baltimore, MD, November 2002.

[2] G. Biros and O. Ghattas. Parallel Lagrange-Newton-Krylov-Schurmethods for PDE-constrained optimization. Part I: The Krylov-Schursolver. Technical report, Laboratory for Mechanics, Algorithms,and Computing, Carnegie Mellon University, 2000. Available fromwww.cs.cmu.edu/ �oghattas .

[3] G. Biros and O. Ghattas. Parallel Lagrange-Newton-Krylov-Schur meth-ods for PDE-constrained optimization. Part II: The Lagrange-Newton solver,and its application to optimal control of steady viscous flows. Technical re-port, Laboratory for Mechanics, Algorithms, and Computing, Carnegie Mel-lon University, 2000. Available fromwww.cs.cmu.edu/ �oghattas .

[4] M.D. Gunzburger and S. Manservisi. The velocity tracking problem for

25

Navier-Stokes flows with bounded distributed controls.SIAM Journal onControl and Optimization, 37(6):1913–1945, 1999.

[5] M.D. Gunzburger and S. Manservisi. Analysis and approximations of thevelocity tracking problem for Navier-Stokes flows with distributed control.SIAM Journal on Numerical Analysis, 37(5):1481–1512, 2000.

[6] K.R. Gurney, R.M. Law, A.S. Denning, P.J. Rayner, D. Baker, P. Bousquet,L. Bruhwiler, Y.H. Chen, P. Ciais, S. Fan, I.Y. Fung, M. Gloor, M. Heimann,K. Higuchi, J. John, T. Maki, S. Maksyutov, K. Masarie, P. Peylin, M. Prather,B.C. Pak, J. Randersonand J. Sarmiento, S. Taguchi, T. Takahashi, and C.W.Yuen. Towards robust regional estimates of CO2 sources and sinks usingatmospheric transport models.Nature, 415(6872):626–630, 2002.

[7] E. Haber and U. Ascher. Preconditioned all-at-once methods for large, sparseparameter estimation problems.Inverse Problems, 2001.

[8] K. Long. Sundance: A rapid prototyping toolkit for parallel PDE simula-tion and optimization. In L. T. Biegler, O. Ghattas, M. Heinkenschloss, andB. van Bloemen Waanders, editors,Large-Scale PDE-Constrained Optimiza-tion, Lecture Notes in Computational Science and Engineering. Springer-Verlag, Heidelberg, 2003. Under review by LNCSE series editors.

[9] K. Long and M. A. Heroux. The Trilinos Solver Framework. In O. Marquesand T. Drummond, editors,2002 ACTS Workshop Proceedings, Berkeley, CA,Sept 3–6, 2002, 2002.

[10] F. Natterer. Algorithms in tomography. In I.S. Duff and G.A. Watson, editors,The State of the Art in Numerical Analysis. Clarendon Press, 1997.

[11] J. Nocedal and S.J. Wright.Numerical Optimization. Springer-Verlag, 1999.[12] J.R. Shewchuk. Triangle: Engineering a 2D quality mesh generator and De-

launay triangulator. In M.C. Lin and D. Manocha, editors,Applied Compu-tational Geometry: Towards Geometric Engineering, volume 1148 ofLectureNotes in Computer Science, pages 203–222. Springer-Verlag, May 1996.

[13] C.R. Vogel.Computational Methods for Inverse Problems. SIAM, 2002.

26