J
I
National Research Centerfor Environment & Health
Institute of Biomathematics& Biometry
Mathematical Modelling in Ecology and the Biosciences
Stochastic Simulation andBayesian Inference for Gibbs Fields:
The Software-Package ANTSInFields
Felix FriedrichLeipzig, 6.12.2002
J
I
The Software Package ANTSInFields
ANTSInFields is a Software Package for Simulation and Statistical Inferenceon Gibbs Fields. It is intended for mainly two purposes: To support tea-ching by demonstrating well known sampling and estimation techniquesand for assistance in research.
ANTSInFieldsis available for download from http://www.antsinfields.de
contact: [email protected]
J
I
History
1995–1998:Development of Voyager
Portable and extensible System for simulation and data analysis
1998:Diploma thesis
Parameter estimation on Gibbs fields in the context of statistical ImageAnalysis
Need for implementation of• samplers for various Gibbs Fields (Ising Model and extensions)
• parameter estimators on simulated or external data.
today: ANTSInFields
Software for Simulation of and Statistical Inference on Gibbs Fields
J
I
Aims / Requirements
Aims
• support teaching
• assistance for research
Requirements
• (really) easy to handle
• interactive visualization turned out to be efficient tool for teaching
• flexibility for research, testing new techniques etc.
• extensibility for implementing new samplers etc.
J
I
Realization
• Strongly object oriented concept
allows implementation close to mathematical structureintuitive and self-explaining
easy implementation of interaction and consistent visualization
• modular design for extensibility, reusability
• command languagefor flexibility on intermediate level
ANTSInFields is written in OberonSystem 3 (ETH Zurich, N.Wirth, Gutknecht, H.Marais, E.Zelleret al.)
Oberon is also anOperating System. It runs on bare PC Hardware or as Emulation on Windows,MacOS, Linux,. . . (Portability)
ANTSInFields uses theVoyagerextension (University of Heidelberg, G.Sawitzki, M.Diller, F.Friedrich
et al.)
J
I
Scope
ANTSInFields contains
• Handling and visualization of 1D, 2D and 3D data• Gibbs and Metropolis Hastings Algorithms, Simulated Annealing, Exact
Sampling (CFTP)• Bayesian image reconstruction methods• parameter estimators on Gibbs fields• . . .
ANTSInFields is attached to 2nd Edition of G. Winklers Book‘Image Analyis, Random Fields and Dynamic Monte Carlo Methods’, Sprin-ger Verlag
In progress: Meta compiler (alpha) for easy implementing of new models
Planned: Interface to R ( both command language and procedure calls).
Demonstration: Look and feel, Commands, Panels, Random Numbers
J
I
Bayesian Image Restoration
J
I
Random Fields
Notation
E finite space ofstates {•, •}
S ⊂ Zd finite index set
x = (xs)s∈S ∈ ES configuration
X := ES space of configurations { , . . . , , . . . , , . . . , }
J
I
Gibbs Fields
We considerNeighbour-Gibbs fields
Π(x) =exp(−H(x))∑
y∈X exp(−H(y)),
whereH is of the form
H(x) =∑s∈S
f (xs, x∂(s))
with ∂(t) some neighbourhood oft, t ∈ S.
∂ =
J
I
Ising Model
Easiest nontrivial case: E = {−1, 1}, nearest neighbours and isotropy.
An Ising Model with parameters β, h ∈ R is a Gibbs-Fieldwith energy
H(x) = −β∑s∼t
xsxt + h∑
s
xs
h: global tendency to take value1β: tendency of neighbours to be alike.
J
I
Problems with sampling:
P(X = x) =exp(−H(x))∑
y∈X exp(−H(y))untractable,
but
P(Xt = xt|Xs = xs, s 6= t) =exp(xt(h + β
∑s∂t xs))
cosh(h + β∑
s∂t xs)easy to calculate
Solution→ MCMC techniques like
• Gibbs Sampler
• Metropolis Hastings Algorithm
• Exact Sampling
J
I
Markov Chains
An irreducible Markov Chain with a stationary distribution µ is ergodicand fulfills
(a)P t(i, j)
t→∞−→ µ(j)
(b)fn
n→∞−→ Eµ(f (X))) in L2
ifEµ(f(X)) < ∞,
wherefn =
1
n
∑i=1...n
f(xi)
Demonstration: Reflected Markov Chain
J
I
Algorithm for the Gibbs Sampler:
1. 0 7→ n
2. Samplex(0) from initial distribution, say uniform distribution on X
3. Apply Kt on x(n) for all t ∈ S
i.e. sample from local characteristicsΠ1 in each point
4. copyx(n) to x(n+1)
5. n + 1 7→ n
6. Return to step 3 until close enough toΠ.
Algorithm is a realization of a Markov chain with stationary distribution Π
J
I
A sweep
j j j j j j
j j j j j j
j j j j j j
j j j j j j
j j j j j j
j j j j j j
J
I
Visiting schedule
j j j j j j
j j j j j j
j j j j j j
j j j j j j
j j j j j j
j j j j j j
~
J
I
Visiting schedule
j j j j j j
j j j j j j
j j j j j j
j j j j j j
j j j j j j
j j j j j j
~
~
J
I
Visiting schedule
j j j j j j
j j j j j j
j j j j j j
j j j j j j
j j j j j j
j j j j j j
~
~
~
J
I
Visiting schedule
j j j j j j
j j j j j j
j j j j j j
j j j j j j
j j j j j j
j j j j j j
~
~
~ ~
~
J
I
Visiting schedule
j j j j j j
j j j j j j
j j j j j j
j j j j j j
j j j j j j
j j j j j j
~
~
~ ~
~
~
J
I
Visiting schedule
j j j j j j
j j j j j j
j j j j j j
j j j j j j
j j j j j j
j j j j j j
~
~
~ ~
~
~
~
J
I
Visiting schedule
j j j j j j
j j j j j j
j j j j j j
j j j j j j
j j j j j j
j j j j j j
~
~
~ ~
~
~
~
~ ~
~
~
~
~
~
~
~
~
~~
J
I
Visiting schedule:Whole sweep finished
j j j j j j
j j j j j j
j j j j j j
j j j j j j
j j j j j j
j j j j j j
~
~
~ ~
~
~
~
~ ~
~
~
~
~
~
~
~
~
~~
~
~
~ ~
~
~
~
~
~
~
~
~
~
~~ ~ ~~
~
~ ~
J
I
Demonstrations
• Ising Model, Gibbs Sampler
• Ising Model + Channel noise, MMSE
H(x, y) = −β∑s∼t
xsxt + h∑
s
xs +1
2ln(
p
1− p)∑
s
xsys
• Cooling Schemes, Simulated Annealing, ICM
• Grey-valued “Ising Model” (Potts and others)
H(x) = β∑s∼t
ϕ(xs, xt) + h∑
s
xs
J
I
• Sampling from arbitrary Posterior Distributions
H(y, x) = β∑x∼t
ϕ(xs, xt) + h∑
s
xs +∑
s
ϑ(xs, ys),
• Φ-Model, Texture Synthesis
H(x) =∑
i
βi
∑s∼
it
ϕ(xs, xt) + h∑
s
xs
J
I
.
J
I
Estimating (Hyper-)Parameters
Assume observations onΛ = Λ + ∂Λ
(Conditional) Maximum-Likelihood estimator (MLE)
θn := arg minθ− log(Pθ(Xt = xt, t ∈ Λ|Xs = xs, s ∈ ∂Λ))
= arg minθ
(log ZΛ(x∂Λ)−HΛ(xΛ | x∂Λ))
Problem: ZΛ not computable .Solution: Subsampling method (Younes(88), Winkler(01))
Alternative approach: Estimators regarding only the conditional distri-butions Pθ(Xt = xt|Xs, s ∈ ∂(t)) like:Coding, Maximum-Pseudolikelihood, Minimal least squares, MinimumChi Square estimator etc.
J
I
Coding Estimator(CE)With
Λ+ = {t ∈ Λ | t1 + . . . + td even}the MLE on Λ+ becomes:
θn = arg minθ− log(P(Xt = xt, t ∈ Λ+|Xs = xs, s ∈ ∂Λ+))
= arg minθ− log
∑t∈Λ+
Pθ(Xt = xt|Xs, s ∈ ∂(t)).
Maximum-Pseudolikelihood Estimator (MPLE)Coding Estimator with replacement: Λ+ −→ Λ.
θn = arg minθ− log
∑t∈Λ
Pθ(Xt = xt|Xs, s ∈ ∂(t))
.
Demonstration: Estimating Parameters
Top Related