Statistics in Astronomy
-
Upload
peter-coles -
Category
Science
-
view
3.026 -
download
0
Transcript of Statistics in Astronomy
![Page 1: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/1.jpg)
Statistics in Astronomy
Peter Coles
STFC Summer School, Cardiff 27th August 2015
![Page 2: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/2.jpg)
![Page 3: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/3.jpg)
![Page 4: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/4.jpg)
Lecture 1Probability
![Page 5: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/5.jpg)
“The Essence of Cosmology is Statistics”
George McVittie
![Page 6: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/6.jpg)
![Page 7: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/7.jpg)
![Page 8: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/8.jpg)
![Page 9: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/9.jpg)
![Page 10: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/10.jpg)
![Page 11: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/11.jpg)
![Page 12: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/12.jpg)
![Page 13: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/13.jpg)
![Page 14: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/14.jpg)
![Page 15: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/15.jpg)
![Page 16: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/16.jpg)
![Page 17: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/17.jpg)
![Page 18: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/18.jpg)
![Page 19: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/19.jpg)
1 May 2023
![Page 20: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/20.jpg)
![Page 21: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/21.jpg)
Precision Cosmology
“…as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns -- the ones we don't know we don't know.”
![Page 22: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/22.jpg)
SAY “PRECISION COSMOLOGY” ONE MORE TIME…
![Page 23: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/23.jpg)
“The Essence of Cosmology is Statistics”
George McVittie
![Page 24: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/24.jpg)
Direct versus Inverse Reasoning
Theory (, H0…)
Observations
![Page 25: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/25.jpg)
Fine Tuning• In the standard model of cosmology the
free parameters are fixed by observations• But are these values surprising?• Even microscopic physics seems to have
“unnecessary” features that allow complexity to arise
• Are these coincidences? Are they significant?
• These are matters of probability…
![Page 26: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/26.jpg)
What is a Probability?• It’s a number between 0 (impossible) and 1
(certain)• Probabilities can be manipulated using simple
rules (“sum” for OR and “product” for “AND”).• But what do they mean?• Standard interpretation is frequentist (proportions
in an ensemble)
![Page 27: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/27.jpg)
Bayesian Probability• Probability is a measure of the “strength of
belief” that it is reasonable to hold.• It is the unique way to generalize
deductive logic (Boolean Algebra)• Represents insufficiency of knowledge to
make a statement with certainty• All probabilities are conditional on stated
assumptions or known facts, e.g. P(A|B)• Often called “subjective”, but at least the
subjectivity is on the table!
![Page 28: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/28.jpg)
Balls• Two urns A and B.• A has 999 white balls and 1 black one; B
has 1 white balls and 999 black ones.• P(white| urn A) = .999, etc. • Now shuffle the two urns, and pull out a
ball from one of them. Suppose it is white. What is the probability it came from urn A?
• P(Urn A| white) requires “inverse” reasoning: Bayes’ Theorem
![Page 29: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/29.jpg)
Urn A Urn B
999 white 1 black
999 black 1 white
P(white ball | urn is A)=0.999, etc
![Page 30: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/30.jpg)
Bayes’ Theorem: Inverse reasoning
• Rev. Thomas Bayes (1702-1761)
• Never published any mathematical papers during his lifetime
• The general form of Bayes’ theorem was actually given later (by Laplace).
![Page 31: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/31.jpg)
Bayes’ Theorem
• In the toy example, X is “the urn is A” and Y is “the ball is white”.
• Everything is calculable, and the required posterior probability is 0.999
I)|P(YI)X,|I)P(Y|P(X=I)Y,|P(X
![Page 32: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/32.jpg)
Probable Theories
I)|P(DI)H,|I)P(D|P(H=I)D,|P(H
• Bayes’ Theorem allows us to assign probabilities to hypotheses (H) based on (assumed) knowledge (I), which can be updated when data (D) become available
• P(D|H,I) – likelihood• P(H|I) – prior probability• P(H|D,I) – posterior probability• The best theory is the most probable!
![Page 33: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/33.jpg)
Why does this help?• Rigorous Form of Ockham’s Razor: the hypothesis
with fewest free parameters becomes most probable.
• Can be applied to one-off events (e.g. Big Bang)• It’s mathematically consistent!• It can even make sense of the Anthropic
Principle…
![Page 34: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/34.jpg)
Null Hypotheses• The frequentist approach to statistical
hypothesis testing involves the idea of a null hypothesis H0,which is the model you are prepared to accept unless there is evidence to the contrary.
• Under the null hypothesis one then constructs the sampling distribution of some statistic Q, called f(Q).
• If the measured value of Q is unlikely on the basis of H0 then the null hypothesis is rejected.
![Page 35: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/35.jpg)
Type I and Type II Errors• There are two ways of making an error in this
kind of test.• Type I is to reject the null when it is actually
true. The probability of this happening is called the significance level (or p-value or “size”), usually called . It is usually chosen to be 5% or 1%.
• The other possibility is to fail to reject the null when it is wrong. If the probability of this happening is then (1-) is called the power.
![Page 36: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/36.jpg)
Bayesian Hypothesis TestingTwo of the advantages of this is that it doesn’t put one hypothesis in a special position (the null), and it doesn’t separate estimation and testing.Suppose Dr A has a theory that makes a direct prediction while Professor B has one that has a free parameter, say .Suppose the likelihoods for a given set of data are P(D|A) and P(D|B,)
![Page 37: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/37.jpg)
Occam’s Razor
λ)B,|(DB)|(λdλA)|(D
(B)(A)
λ)B,|(Dλ)(B,dλA)|(D(A)
D)|λ(B,dλD)|(A=
D)|(BD)|(A
PrPrPr
PrPr
PrPrPrPr
PrPr
PrPr
Occam factor
![Page 38: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/38.jpg)
Bayesian estimation
aI)d,aa|xI)p(x|ap(a=K
I),aa|xI)p(x|aKp(a=I),xx|ap(a
mmnm
mnmnm
.......
............
1111
11111
This involves finding the posterior distribution of the parameters given the data and any prior information.
Evidence!
![Page 39: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/39.jpg)
Is there anything wrong with Frequentism?
• The laws for manipulating probabilities are no different
• What is different is the interpretation.• OK to imagine an ensemble, but there is
no need to assert that it is real! (mind projection fallacy)
• The idea of a prior is worrying for many, but is the only way to make this reasoning consistent
![Page 40: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/40.jpg)
Prior and Prejudice• Priors are essential. • You usually know more than you
think..• Flat priors usually don’t make much
sense.• Maximum entropy, etc, give useful
insights within a well-defined theory: “objective Bayesian”
• “Theory” priors are hard to assign, especially when there isn’t a theory…
![Page 41: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/41.jpg)
Why is the Universe (nearly) flat?
• Assume the Universe is one of the Friedman family
• Q: What should we expect, given only this assumption?
• Ω=1 is a fixed point (so is Ω=0)..
• The Universe is walking a tightrope..
![Page 42: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/42.jpg)
a2=8πGρ
3 a2−kc 2
The Friedman ModelsThe simplest relativistic cosmological models are remarkably similar (although the more general ones have additional options…)
a=−4πGρ
3 a
Solutions of these are complicated, except when k=0 (flat Universe). This special case is called the Einstein de Sitter universe.
Notice that
ρ∝1a3
For non-relativistic particles (“dust”)
Curvature
![Page 43: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/43.jpg)
Cosmology by Numbers
c
2
2
ρ=ρ
H==aaa=a=k
kca=a
8ππ3H
38ππG
38ππG0
38ππG
2
222
22
The “Critical Density”
This applies at any time, but we usually take the “present” time. In general,
020
0
3H8ππG Ω=,ρ=ρ,H=H,t=t 000
![Page 44: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/44.jpg)
![Page 45: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/45.jpg)
The Cosmic Tightrope• We know the Universe doesn’t have either
a very large or a very small one, or we wouldn’t be around.
• We exist and this fact is an observation about the Universe
• The most probable value of is therefore very close to unity
• Still leaves the mystery of what trained the Universe to walk the tightrope (inflation?)
![Page 46: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/46.jpg)
Theories
Observations
FrequentistBayesian
![Page 47: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/47.jpg)
“The Essence of Cosmology is Statistics”
George McVittie
![Page 48: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/48.jpg)
“CONCORDANCE”
![Page 49: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/49.jpg)
![Page 50: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/50.jpg)
![Page 51: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/51.jpg)
![Page 52: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/52.jpg)
![Page 53: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/53.jpg)
![Page 54: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/54.jpg)
Cosmology is an exercise in data compression
Cosmology is a massive exercise in data compression...
….but it is worth looking at the information that has been thrown away to check that it makes sense!
![Page 55: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/55.jpg)
1 May 2023
![Page 56: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/56.jpg)
How Weird is the Universe?• The (zero-th order) starting point is
FLRW.• The concordance cosmology is a “first-
order” perturbation to this• In it (and other “first-order” models), the
initial fluctuations were a statistically homogeneous and isotropic Gaussian Random Field (GRF)
• These are the “maximum entropy” initial conditions having “random phases” motivated by inflation.
• Anything else would be weird….
![Page 57: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/57.jpg)
![Page 58: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/58.jpg)
A)!|P(MM)|P(A
Beware the Prosecutor’s Fallacy!
![Page 59: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/59.jpg)
Is there an Elephant in the Room?
![Page 60: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/60.jpg)
Types of CMB Anomalies• Type I – obvious problems with data
(e.g. foregrounds)• Type II – anisotropies (North-South, Axis
of Evil..)• Type III – localized features, e.g. “The
Cold Spot”• Type IV – Something else (even/odd
multipoles, magnetic fields, ?)
![Page 61: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/61.jpg)
![Page 62: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/62.jpg)
“If tortured sufficiently, data will confess to almost
anything”
Fred Menger
![Page 63: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/63.jpg)
![Page 64: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/64.jpg)
![Page 65: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/65.jpg)
Weirdness in PhasesΔT (θ,φ )
T=∑∑ a l,mY lm (θ,φ )
ml,ml,ml, ia=a exp
For a homogeneous and isotropic Gaussian random field (on the sphere) the phases are independent and uniformly distributed. Non-random phases therefore indicate weirdness..
![Page 66: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/66.jpg)
![Page 67: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/67.jpg)
![Page 68: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/68.jpg)
![Page 69: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/69.jpg)
![Page 70: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/70.jpg)
![Page 71: Statistics in Astronomy](https://reader036.fdocuments.in/reader036/viewer/2022070514/588248771a28ab37158b52fd/html5/thumbnails/71.jpg)
Final Points