Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A...
Transcript of Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A...
![Page 1: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/1.jpg)
...
1 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Maximum Entropy and Inductive Logic II
Jürgen Landes
Spring School on Inductive Logic
Canterbury, 20.04.2015 - 21.04.2015
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 2: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/2.jpg)
...
1 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 3: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/3.jpg)
...
2 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
...
Rational subjective BeliefsFinite propositional language LVariables v1, . . . , vn
Sentences of L, SLNo funny business, self-reference, truth predicates,self-fulfilling
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 4: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/4.jpg)
...
2 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
...
Rational subjective BeliefsFinite propositional language LVariables v1, . . . , vn
Sentences of L, SLNo funny business, self-reference, truth predicates,self-fulfilling
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 5: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/5.jpg)
...
2 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
...
Rational subjective BeliefsFinite propositional language LVariables v1, . . . , vn
Sentences of L, SLNo funny business, self-reference, truth predicates,self-fulfilling
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 6: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/6.jpg)
...
2 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
...
Rational subjective BeliefsFinite propositional language LVariables v1, . . . , vn
Sentences of L, SLNo funny business, self-reference, truth predicates,self-fulfilling
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 7: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/7.jpg)
...
2 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
...
Rational subjective BeliefsFinite propositional language LVariables v1, . . . , vn
Sentences of L, SLNo funny business, self-reference, truth predicates,self-fulfilling
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 8: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/8.jpg)
...
3 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Probabilities
P : SL→ [0,1]
Set of probability functions PPossible world, states
ω = v1 ∧ v2 ∧ ¬v3 ∧ . . . ∧ vn1 ∧ ¬vn
P(ϕ) =∑ω∈Ωω|=ϕ
P(ω).
A probability function P ∈ P is uniquely determined by itsvalues on possible worlds, 〈P(ω) : ω ∈ Ω〉.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 9: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/9.jpg)
...
3 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Probabilities
P : SL→ [0,1]
Set of probability functions PPossible world, states
ω = v1 ∧ v2 ∧ ¬v3 ∧ . . . ∧ vn1 ∧ ¬vn
P(ϕ) =∑ω∈Ωω|=ϕ
P(ω).
A probability function P ∈ P is uniquely determined by itsvalues on possible worlds, 〈P(ω) : ω ∈ Ω〉.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 10: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/10.jpg)
...
3 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Probabilities
P : SL→ [0,1]
Set of probability functions PPossible world, states
ω = v1 ∧ v2 ∧ ¬v3 ∧ . . . ∧ vn1 ∧ ¬vn
P(ϕ) =∑ω∈Ωω|=ϕ
P(ω).
A probability function P ∈ P is uniquely determined by itsvalues on possible worlds, 〈P(ω) : ω ∈ Ω〉.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 11: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/11.jpg)
...
3 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Probabilities
P : SL→ [0,1]
Set of probability functions PPossible world, states
ω = v1 ∧ v2 ∧ ¬v3 ∧ . . . ∧ vn1 ∧ ¬vn
P(ϕ) =∑ω∈Ωω|=ϕ
P(ω).
A probability function P ∈ P is uniquely determined by itsvalues on possible worlds, 〈P(ω) : ω ∈ Ω〉.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 12: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/12.jpg)
...
3 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Probabilities
P : SL→ [0,1]
Set of probability functions PPossible world, states
ω = v1 ∧ v2 ∧ ¬v3 ∧ . . . ∧ vn1 ∧ ¬vn
P(ϕ) =∑ω∈Ωω|=ϕ
P(ω).
A probability function P ∈ P is uniquely determined by itsvalues on possible worlds, 〈P(ω) : ω ∈ Ω〉.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 13: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/13.jpg)
...
4 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Inference Processes
Knowledge K leads to E ⊆ P.Formally, an inference process is a map from a set of prob-ability functions (here E) to the set of probability functions.An inference process is a map (or function)Input: EOutput: P+.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 14: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/14.jpg)
...
4 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Inference Processes
Knowledge K leads to E ⊆ P.Formally, an inference process is a map from a set of prob-ability functions (here E) to the set of probability functions.An inference process is a map (or function)Input: EOutput: P+.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 15: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/15.jpg)
...
4 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Inference Processes
Knowledge K leads to E ⊆ P.Formally, an inference process is a map from a set of prob-ability functions (here E) to the set of probability functions.An inference process is a map (or function)Input: EOutput: P+.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 16: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/16.jpg)
...
4 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Inference Processes
Knowledge K leads to E ⊆ P.Formally, an inference process is a map from a set of prob-ability functions (here E) to the set of probability functions.An inference process is a map (or function)Input: EOutput: P+.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 17: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/17.jpg)
...
4 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Inference Processes
Knowledge K leads to E ⊆ P.Formally, an inference process is a map from a set of prob-ability functions (here E) to the set of probability functions.An inference process is a map (or function)Input: EOutput: P+.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 18: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/18.jpg)
...
5 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Obviously right
Adopt the function, P†, which solve this optimisation prob-lem
maximise: −∑ω∈Ω
P(ω) log(P(ω))
subject to: P ∈ E .
Shannon Entropy: H(P) = −∑
ω∈Ω P(ω) log(P(ω)).Maximum Entropy Inference Process (MaxEnt):
P+ = arg supP∈E
H(P)
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 19: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/19.jpg)
...
5 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Obviously right
Adopt the function, P†, which solve this optimisation prob-lem
maximise: −∑ω∈Ω
P(ω) log(P(ω))
subject to: P ∈ E .
Shannon Entropy: H(P) = −∑
ω∈Ω P(ω) log(P(ω)).Maximum Entropy Inference Process (MaxEnt):
P+ = arg supP∈E
H(P)
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 20: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/20.jpg)
...
5 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Obviously right
Adopt the function, P†, which solve this optimisation prob-lem
maximise: −∑ω∈Ω
P(ω) log(P(ω))
subject to: P ∈ E .
Shannon Entropy: H(P) = −∑
ω∈Ω P(ω) log(P(ω)).Maximum Entropy Inference Process (MaxEnt):
P+ = arg supP∈E
H(P)
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 21: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/21.jpg)
...
6 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Desideratum: Internal
P+ ∈ Eif E is non-empty, convex and closed.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 22: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/22.jpg)
...
6 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Desideratum: Internal
P+ ∈ Eif E is non-empty, convex and closed.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 23: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/23.jpg)
...
7 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Desideratum: Open-mindedness
P+(ω) > 0, if there exists a P ∈ E such that P(ω) > 0.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 24: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/24.jpg)
...
8 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Desideratum: Language Invariance
L′ generated by v1, v2, . . . , vn, vn+1
and the same knowledge and the same patient? For allϕ ∈ SL
P ′(ϕ) = P+(ϕ) .
Repeat argument for even larger languages.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 25: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/25.jpg)
...
8 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Desideratum: Language Invariance
L′ generated by v1, v2, . . . , vn, vn+1
and the same knowledge and the same patient? For allϕ ∈ SL
P ′(ϕ) = P+(ϕ) .
Repeat argument for even larger languages.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 26: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/26.jpg)
...
8 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Desideratum: Language Invariance
L′ generated by v1, v2, . . . , vn, vn+1
and the same knowledge and the same patient? For allϕ ∈ SL
P ′(ϕ) = P+(ϕ) .
Repeat argument for even larger languages.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 27: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/27.jpg)
...
9 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 28: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/28.jpg)
...
10 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 29: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/29.jpg)
...
11 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Renaming
Names should not matter.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 30: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/30.jpg)
...
12 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Irrelevance
Knowledge entirely irrelevant to the problem in hand can heignored.Languages: L1 with variables v1, . . . , vs, L2 with variablesvs+1, . . . , vn.2 Bodies of Knowledge: K1 formulated within L1, K2 formu-lated with in L2
For all ϕ ∈ L1 (problem at hand)
IP(v1, . . . , vn,K1)(ϕ) = IP(v1, . . . , vn,K1 ∪ K2)(ϕ) .
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 31: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/31.jpg)
...
12 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Irrelevance
Knowledge entirely irrelevant to the problem in hand can heignored.Languages: L1 with variables v1, . . . , vs, L2 with variablesvs+1, . . . , vn.2 Bodies of Knowledge: K1 formulated within L1, K2 formu-lated with in L2
For all ϕ ∈ L1 (problem at hand)
IP(v1, . . . , vn,K1)(ϕ) = IP(v1, . . . , vn,K1 ∪ K2)(ϕ) .
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 32: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/32.jpg)
...
12 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Irrelevance
Knowledge entirely irrelevant to the problem in hand can heignored.Languages: L1 with variables v1, . . . , vs, L2 with variablesvs+1, . . . , vn.2 Bodies of Knowledge: K1 formulated within L1, K2 formu-lated with in L2
For all ϕ ∈ L1 (problem at hand)
IP(v1, . . . , vn,K1)(ϕ) = IP(v1, . . . , vn,K1 ∪ K2)(ϕ) .
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 33: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/33.jpg)
...
12 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Irrelevance
Knowledge entirely irrelevant to the problem in hand can heignored.Languages: L1 with variables v1, . . . , vs, L2 with variablesvs+1, . . . , vn.2 Bodies of Knowledge: K1 formulated within L1, K2 formu-lated with in L2
For all ϕ ∈ L1 (problem at hand)
IP(v1, . . . , vn,K1)(ϕ) = IP(v1, . . . , vn,K1 ∪ K2)(ϕ) .
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 34: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/34.jpg)
...
13 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
I-AH
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 35: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/35.jpg)
...
14 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Obstinacy
Learning something one already beliefs should not makeany difference.Given two consistent bodies of knowledge, K1,K2 (on thesame language)If IP(K1) (which is a probability function), is consistent withK2, then
IP(K1) = IP(K1 ∪ K2).
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 36: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/36.jpg)
...
14 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Obstinacy
Learning something one already beliefs should not makeany difference.Given two consistent bodies of knowledge, K1,K2 (on thesame language)If IP(K1) (which is a probability function), is consistent withK2, then
IP(K1) = IP(K1 ∪ K2).
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 37: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/37.jpg)
...
14 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Obstinacy
Learning something one already beliefs should not makeany difference.Given two consistent bodies of knowledge, K1,K2 (on thesame language)If IP(K1) (which is a probability function), is consistent withK2, then
IP(K1) = IP(K1 ∪ K2).
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 38: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/38.jpg)
...
14 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Obstinacy
Learning something one already beliefs should not makeany difference.Given two consistent bodies of knowledge, K1,K2 (on thesame language)If IP(K1) (which is a probability function), is consistent withK2, then
IP(K1) = IP(K1 ∪ K2).
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 39: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/39.jpg)
...
14 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Obstinacy
Learning something one already beliefs should not makeany difference.Given two consistent bodies of knowledge, K1,K2 (on thesame language)If IP(K1) (which is a probability function), is consistent withK2, then
IP(K1) = IP(K1 ∪ K2).
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 40: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/40.jpg)
...
14 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Obstinacy
Learning something one already beliefs should not makeany difference.Given two consistent bodies of knowledge, K1,K2 (on thesame language)If IP(K1) (which is a probability function), is consistent withK2, then
IP(K1) = IP(K1 ∪ K2).
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 41: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/41.jpg)
...
15 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Relativisation
Suppose you know the probability of ϕ.That is, for all P ∈ E there exists some c ∈ [0,1] such thatP(ϕ) = c.If ω− |= ϕ, then IP(ω−) should not depend on yourknowledge about the ¬ϕ-worlds.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 42: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/42.jpg)
...
15 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Relativisation
Suppose you know the probability of ϕ.That is, for all P ∈ E there exists some c ∈ [0,1] such thatP(ϕ) = c.If ω− |= ϕ, then IP(ω−) should not depend on yourknowledge about the ¬ϕ-worlds.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 43: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/43.jpg)
...
15 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Relativisation
Suppose you know the probability of ϕ.That is, for all P ∈ E there exists some c ∈ [0,1] such thatP(ϕ) = c.If ω− |= ϕ, then IP(ω−) should not depend on yourknowledge about the ¬ϕ-worlds.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 44: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/44.jpg)
...
16 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Independence
If K does not contain any information which makes v1, v2conditionally dependent on v3, then v1, v2 should be condi-tionally independent given v3.If K = P(v3) = γ,P(v1|v3) = α
γ ,P(v2|v3) = βγ (P(γ) > 0),
then
IP(K )(v1 ∧ v2|v3) =α
γ
β
γ.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 45: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/45.jpg)
...
16 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Independence
If K does not contain any information which makes v1, v2conditionally dependent on v3, then v1, v2 should be condi-tionally independent given v3.If K = P(v3) = γ,P(v1|v3) = α
γ ,P(v2|v3) = βγ (P(γ) > 0),
then
IP(K )(v1 ∧ v2|v3) =α
γ
β
γ.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 46: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/46.jpg)
...
17 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Continuity
If E′ can be obtained from E by moving or deforming E a bit,thenIP(E′) ≈ IP(E).For all sentences ϕ ∈ SL: IP(E′)(ϕ) ≈ IP(E)(ϕ).
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 47: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/47.jpg)
...
17 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Continuity
If E′ can be obtained from E by moving or deforming E a bit,thenIP(E′) ≈ IP(E).For all sentences ϕ ∈ SL: IP(E′)(ϕ) ≈ IP(E)(ϕ).
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 48: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/48.jpg)
...
17 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Continuity
If E′ can be obtained from E by moving or deforming E a bit,thenIP(E′) ≈ IP(E).For all sentences ϕ ∈ SL: IP(E′)(ϕ) ≈ IP(E)(ϕ).
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 49: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/49.jpg)
...
18 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 50: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/50.jpg)
...
19 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Alena & Jeff
TheoremIf E is closed, convex and non-empty, IP satisfies Renaming,Irrelevance, Obstinacy, Relativisation, Independence andContinuity, then IP is MaxEnt.MaxEnt satisfies language invariance and open-mindednessand is internal.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 51: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/51.jpg)
...
20 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 52: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/52.jpg)
...
21 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 53: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/53.jpg)
...
22 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Continuity - Bristolean
One considers the decision problem of setting degrees ofbeliefand wonders which beliefs are best.A utility function u is used to measure the goodness / bad-ness / utility of a belief function.Determine which beliefs have the “best utility”.The usual caveats for decision making apply: Non-causal,act-state independence, etc.Decision theoretic norm still undetermined.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 54: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/54.jpg)
...
22 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Continuity - Bristolean
One considers the decision problem of setting degrees ofbeliefand wonders which beliefs are best.A utility function u is used to measure the goodness / bad-ness / utility of a belief function.Determine which beliefs have the “best utility”.The usual caveats for decision making apply: Non-causal,act-state independence, etc.Decision theoretic norm still undetermined.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 55: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/55.jpg)
...
22 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Continuity - Bristolean
One considers the decision problem of setting degrees ofbeliefand wonders which beliefs are best.A utility function u is used to measure the goodness / bad-ness / utility of a belief function.Determine which beliefs have the “best utility”.The usual caveats for decision making apply: Non-causal,act-state independence, etc.Decision theoretic norm still undetermined.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 56: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/56.jpg)
...
22 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Continuity - Bristolean
One considers the decision problem of setting degrees ofbeliefand wonders which beliefs are best.A utility function u is used to measure the goodness / bad-ness / utility of a belief function.Determine which beliefs have the “best utility”.The usual caveats for decision making apply: Non-causal,act-state independence, etc.Decision theoretic norm still undetermined.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 57: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/57.jpg)
...
22 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Continuity - Bristolean
One considers the decision problem of setting degrees ofbeliefand wonders which beliefs are best.A utility function u is used to measure the goodness / bad-ness / utility of a belief function.Determine which beliefs have the “best utility”.The usual caveats for decision making apply: Non-causal,act-state independence, etc.Decision theoretic norm still undetermined.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 58: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/58.jpg)
...
22 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Continuity - Bristolean
One considers the decision problem of setting degrees ofbeliefand wonders which beliefs are best.A utility function u is used to measure the goodness / bad-ness / utility of a belief function.Determine which beliefs have the “best utility”.The usual caveats for decision making apply: Non-causal,act-state independence, etc.Decision theoretic norm still undetermined.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 59: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/59.jpg)
...
23 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 60: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/60.jpg)
...
24 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Logarithmic Utility
If ω is the true world, then the utility is u(ω,P+) = log(P+(ω)).Expected utility for P ∈ E is:
∑ω∈Ω P(ω) log(P+(ω)).
If E is closed, convex and non-empty, maximise worst caseexpected utility:
arg supP+∈P
infP∈E
∑ω∈Ω
P(ω) log(P+(ω)) = P† .
The latest rage is to understand u as an accuracy measure.Measure closeness to the truth.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 61: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/61.jpg)
...
24 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Logarithmic Utility
If ω is the true world, then the utility is u(ω,P+) = log(P+(ω)).Expected utility for P ∈ E is:
∑ω∈Ω P(ω) log(P+(ω)).
If E is closed, convex and non-empty, maximise worst caseexpected utility:
arg supP+∈P
infP∈E
∑ω∈Ω
P(ω) log(P+(ω)) = P† .
The latest rage is to understand u as an accuracy measure.Measure closeness to the truth.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 62: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/62.jpg)
...
24 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Logarithmic Utility
If ω is the true world, then the utility is u(ω,P+) = log(P+(ω)).Expected utility for P ∈ E is:
∑ω∈Ω P(ω) log(P+(ω)).
If E is closed, convex and non-empty, maximise worst caseexpected utility:
arg supP+∈P
infP∈E
∑ω∈Ω
P(ω) log(P+(ω)) = P† .
The latest rage is to understand u as an accuracy measure.Measure closeness to the truth.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 63: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/63.jpg)
...
24 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Logarithmic Utility
If ω is the true world, then the utility is u(ω,P+) = log(P+(ω)).Expected utility for P ∈ E is:
∑ω∈Ω P(ω) log(P+(ω)).
If E is closed, convex and non-empty, maximise worst caseexpected utility:
arg supP+∈P
infP∈E
∑ω∈Ω
P(ω) log(P+(ω)) = P† .
The latest rage is to understand u as an accuracy measure.Measure closeness to the truth.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 64: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/64.jpg)
...
24 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Logarithmic Utility
If ω is the true world, then the utility is u(ω,P+) = log(P+(ω)).Expected utility for P ∈ E is:
∑ω∈Ω P(ω) log(P+(ω)).
If E is closed, convex and non-empty, maximise worst caseexpected utility:
arg supP+∈P
infP∈E
∑ω∈Ω
P(ω) log(P+(ω)) = P† .
The latest rage is to understand u as an accuracy measure.Measure closeness to the truth.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 65: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/65.jpg)
...
25 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Utility Theory
If you do have a utility function and a Decision TheoreticNorm,then you can show that a particular inference process isoptimal with respect to the above.Justifications in terms of common-sense principles hinge onthe common-sensicality of the principles.Justifications in terms of utility functions appear much moreobjective.However, one has to give a story explaining where theutility function and the DTN come from.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 66: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/66.jpg)
...
25 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Utility Theory
If you do have a utility function and a Decision TheoreticNorm,then you can show that a particular inference process isoptimal with respect to the above.Justifications in terms of common-sense principles hinge onthe common-sensicality of the principles.Justifications in terms of utility functions appear much moreobjective.However, one has to give a story explaining where theutility function and the DTN come from.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 67: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/67.jpg)
...
25 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Utility Theory
If you do have a utility function and a Decision TheoreticNorm,then you can show that a particular inference process isoptimal with respect to the above.Justifications in terms of common-sense principles hinge onthe common-sensicality of the principles.Justifications in terms of utility functions appear much moreobjective.However, one has to give a story explaining where theutility function and the DTN come from.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 68: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/68.jpg)
...
25 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Utility Theory
If you do have a utility function and a Decision TheoreticNorm,then you can show that a particular inference process isoptimal with respect to the above.Justifications in terms of common-sense principles hinge onthe common-sensicality of the principles.Justifications in terms of utility functions appear much moreobjective.However, one has to give a story explaining where theutility function and the DTN come from.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 69: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/69.jpg)
...
25 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Utility Theory
If you do have a utility function and a Decision TheoreticNorm,then you can show that a particular inference process isoptimal with respect to the above.Justifications in terms of common-sense principles hinge onthe common-sensicality of the principles.Justifications in terms of utility functions appear much moreobjective.However, one has to give a story explaining where theutility function and the DTN come from.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 70: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/70.jpg)
...
26 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 71: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/71.jpg)
...
27 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Ensembles, populations
Consider an entire population M with |M| members.K consists of statements of the form32.2% of patients complain of symptom ϕ
|x ∈ M | x complains about ϕ| ≈ 32.2100|M|
Consider all populations M of fixed large size |M| which sat-isfy the above.Then almost all such populations M satisfy
|x ∈ M | x complains about ϕ||M|
≈ P†(ϕ) .
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 72: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/72.jpg)
...
27 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Ensembles, populations
Consider an entire population M with |M| members.K consists of statements of the form32.2% of patients complain of symptom ϕ
|x ∈ M | x complains about ϕ| ≈ 32.2100|M|
Consider all populations M of fixed large size |M| which sat-isfy the above.Then almost all such populations M satisfy
|x ∈ M | x complains about ϕ||M|
≈ P†(ϕ) .
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 73: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/73.jpg)
...
27 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Ensembles, populations
Consider an entire population M with |M| members.K consists of statements of the form32.2% of patients complain of symptom ϕ
|x ∈ M | x complains about ϕ| ≈ 32.2100|M|
Consider all populations M of fixed large size |M| which sat-isfy the above.Then almost all such populations M satisfy
|x ∈ M | x complains about ϕ||M|
≈ P†(ϕ) .
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 74: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/74.jpg)
...
27 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Ensembles, populations
Consider an entire population M with |M| members.K consists of statements of the form32.2% of patients complain of symptom ϕ
|x ∈ M | x complains about ϕ| ≈ 32.2100|M|
Consider all populations M of fixed large size |M| which sat-isfy the above.Then almost all such populations M satisfy
|x ∈ M | x complains about ϕ||M|
≈ P†(ϕ) .
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 75: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/75.jpg)
...
27 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Ensembles, populations
Consider an entire population M with |M| members.K consists of statements of the form32.2% of patients complain of symptom ϕ
|x ∈ M | x complains about ϕ| ≈ 32.2100|M|
Consider all populations M of fixed large size |M| which sat-isfy the above.Then almost all such populations M satisfy
|x ∈ M | x complains about ϕ||M|
≈ P†(ϕ) .
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 76: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/76.jpg)
...
27 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Ensembles, populations
Consider an entire population M with |M| members.K consists of statements of the form32.2% of patients complain of symptom ϕ
|x ∈ M | x complains about ϕ| ≈ 32.2100|M|
Consider all populations M of fixed large size |M| which sat-isfy the above.Then almost all such populations M satisfy
|x ∈ M | x complains about ϕ||M|
≈ P†(ϕ) .
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 77: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/77.jpg)
...
28 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Alena & Jeff 2 – one last justification
TheoremFor all δ > 0 there exists some natural number M0 such that forall sentences ϕ ∈ SL and all fixed sizes of populations|M| ≥ M0 the proportion of populations M of fixed size |M| ≥ M0which satisfy∣∣∣ |x ∈ M | x complains about ϕ|
|M|− P†(ϕ)
∣∣∣ < δ
is greater or equal than 1− δ.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 78: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/78.jpg)
...
29 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 79: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/79.jpg)
...
30 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 80: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/80.jpg)
...
31 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Equivocation, come what may
The least opinionated function isP=(ω) = 1
|Ω| , P= = arg supP∈E H(P).
If P= ∈ E, then P† = P=.Entropy strictly decreases along the rays originating fromP=.Entropy maximiser different from P= face P=.If E = P ∈ P | 0 ≤ P(v1) ≤ 0.5, then P†(v1) = 0.5.Can this be right?
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 81: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/81.jpg)
...
31 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Equivocation, come what may
The least opinionated function isP=(ω) = 1
|Ω| , P= = arg supP∈E H(P).
If P= ∈ E, then P† = P=.Entropy strictly decreases along the rays originating fromP=.Entropy maximiser different from P= face P=.If E = P ∈ P | 0 ≤ P(v1) ≤ 0.5, then P†(v1) = 0.5.Can this be right?
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 82: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/82.jpg)
...
31 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Equivocation, come what may
The least opinionated function isP=(ω) = 1
|Ω| , P= = arg supP∈E H(P).
If P= ∈ E, then P† = P=.Entropy strictly decreases along the rays originating fromP=.Entropy maximiser different from P= face P=.If E = P ∈ P | 0 ≤ P(v1) ≤ 0.5, then P†(v1) = 0.5.Can this be right?
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 83: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/83.jpg)
...
31 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Equivocation, come what may
The least opinionated function isP=(ω) = 1
|Ω| , P= = arg supP∈E H(P).
If P= ∈ E, then P† = P=.Entropy strictly decreases along the rays originating fromP=.Entropy maximiser different from P= face P=.If E = P ∈ P | 0 ≤ P(v1) ≤ 0.5, then P†(v1) = 0.5.Can this be right?
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 84: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/84.jpg)
...
31 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Equivocation, come what may
The least opinionated function isP=(ω) = 1
|Ω| , P= = arg supP∈E H(P).
If P= ∈ E, then P† = P=.Entropy strictly decreases along the rays originating fromP=.Entropy maximiser different from P= face P=.If E = P ∈ P | 0 ≤ P(v1) ≤ 0.5, then P†(v1) = 0.5.Can this be right?
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 85: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/85.jpg)
...
31 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Equivocation, come what may
The least opinionated function isP=(ω) = 1
|Ω| , P= = arg supP∈E H(P).
If P= ∈ E, then P† = P=.Entropy strictly decreases along the rays originating fromP=.Entropy maximiser different from P= face P=.If E = P ∈ P | 0 ≤ P(v1) ≤ 0.5, then P†(v1) = 0.5.Can this be right?
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 86: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/86.jpg)
...
31 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Equivocation, come what may
The least opinionated function isP=(ω) = 1
|Ω| , P= = arg supP∈E H(P).
If P= ∈ E, then P† = P=.Entropy strictly decreases along the rays originating fromP=.Entropy maximiser different from P= face P=.If E = P ∈ P | 0 ≤ P(v1) ≤ 0.5, then P†(v1) = 0.5.Can this be right?
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 87: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/87.jpg)
...
32 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Donkey
If E = P, then P† = P=.But what if your learn this?Can this be right?
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 88: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/88.jpg)
...
32 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Donkey
If E = P, then P† = P=.But what if your learn this?Can this be right?
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 89: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/89.jpg)
...
32 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Donkey
If E = P, then P† = P=.But what if your learn this?Can this be right?
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 90: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/90.jpg)
...
33 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Intuition?!?
Maximising−∑
ω∈Ω P(ω) log(P(ω)) is sooo counter-intuitive.How would you(!) respond to this objection?
If the aim is to reconstruct “rational” human thinking, thenMaxEnt fails; I claim.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 91: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/91.jpg)
...
33 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Intuition?!?
Maximising−∑
ω∈Ω P(ω) log(P(ω)) is sooo counter-intuitive.How would you(!) respond to this objection?
If the aim is to reconstruct “rational” human thinking, thenMaxEnt fails; I claim.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 92: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/92.jpg)
...
33 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Intuition?!?
Maximising−∑
ω∈Ω P(ω) log(P(ω)) is sooo counter-intuitive.How would you(!) respond to this objection?
If the aim is to reconstruct “rational” human thinking, thenMaxEnt fails; I claim.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 93: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/93.jpg)
...
33 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Intuition?!?
Maximising−∑
ω∈Ω P(ω) log(P(ω)) is sooo counter-intuitive.How would you(!) respond to this objection?
If the aim is to reconstruct “rational” human thinking, thenMaxEnt fails; I claim.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 94: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/94.jpg)
...
33 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Intuition?!?
Maximising−∑
ω∈Ω P(ω) log(P(ω)) is sooo counter-intuitive.How would you(!) respond to this objection?
If the aim is to reconstruct “rational” human thinking, thenMaxEnt fails; I claim.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 95: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/95.jpg)
...
33 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Intuition?!?
Maximising−∑
ω∈Ω P(ω) log(P(ω)) is sooo counter-intuitive.How would you(!) respond to this objection?
If the aim is to reconstruct “rational” human thinking, thenMaxEnt fails; I claim.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 96: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/96.jpg)
...
34 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 97: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/97.jpg)
...
35 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Dependence
No knowledge, E = P.Possible worlds: red and blue.P†(red) = 1
2 . Okay.Possible worlds: red and light blue and dark blue.P†(red) = 1
3 . Ohho!This phenomenon is called language dependence.It is quite common.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 98: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/98.jpg)
...
35 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Dependence
No knowledge, E = P.Possible worlds: red and blue.P†(red) = 1
2 . Okay.Possible worlds: red and light blue and dark blue.P†(red) = 1
3 . Ohho!This phenomenon is called language dependence.It is quite common.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 99: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/99.jpg)
...
35 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Dependence
No knowledge, E = P.Possible worlds: red and blue.P†(red) = 1
2 . Okay.Possible worlds: red and light blue and dark blue.P†(red) = 1
3 . Ohho!This phenomenon is called language dependence.It is quite common.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 100: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/100.jpg)
...
35 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Dependence
No knowledge, E = P.Possible worlds: red and blue.P†(red) = 1
2 . Okay.Possible worlds: red and light blue and dark blue.P†(red) = 1
3 . Ohho!This phenomenon is called language dependence.It is quite common.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 101: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/101.jpg)
...
35 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Dependence
No knowledge, E = P.Possible worlds: red and blue.P†(red) = 1
2 . Okay.Possible worlds: red and light blue and dark blue.P†(red) = 1
3 . Ohho!This phenomenon is called language dependence.It is quite common.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 102: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/102.jpg)
...
35 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Dependence
No knowledge, E = P.Possible worlds: red and blue.P†(red) = 1
2 . Okay.Possible worlds: red and light blue and dark blue.P†(red) = 1
3 . Ohho!This phenomenon is called language dependence.It is quite common.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 103: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/103.jpg)
...
35 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Dependence
No knowledge, E = P.Possible worlds: red and blue.P†(red) = 1
2 . Okay.Possible worlds: red and light blue and dark blue.P†(red) = 1
3 . Ohho!This phenomenon is called language dependence.It is quite common.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 104: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/104.jpg)
...
36 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Invariance
L = 〈v1, v2, . . . , vn〉, L′ = 〈v1, v2, . . . , vn, vn+1〉.Knowledge only concerns v1, . . . , vn.ϕ ∈ SL.
IP(L)(ϕ) = IP(L′)(ϕ)
Centre of Mass is not language invariant! MaxEnt is lan-guage invariant.Ay, caramba!
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 105: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/105.jpg)
...
36 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Invariance
L = 〈v1, v2, . . . , vn〉, L′ = 〈v1, v2, . . . , vn, vn+1〉.Knowledge only concerns v1, . . . , vn.ϕ ∈ SL.
IP(L)(ϕ) = IP(L′)(ϕ)
Centre of Mass is not language invariant! MaxEnt is lan-guage invariant.Ay, caramba!
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 106: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/106.jpg)
...
36 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Invariance
L = 〈v1, v2, . . . , vn〉, L′ = 〈v1, v2, . . . , vn, vn+1〉.Knowledge only concerns v1, . . . , vn.ϕ ∈ SL.
IP(L)(ϕ) = IP(L′)(ϕ)
Centre of Mass is not language invariant! MaxEnt is lan-guage invariant.Ay, caramba!
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 107: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/107.jpg)
...
36 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Invariance
L = 〈v1, v2, . . . , vn〉, L′ = 〈v1, v2, . . . , vn, vn+1〉.Knowledge only concerns v1, . . . , vn.ϕ ∈ SL.
IP(L)(ϕ) = IP(L′)(ϕ)
Centre of Mass is not language invariant! MaxEnt is lan-guage invariant.Ay, caramba!
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 108: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/108.jpg)
...
36 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Invariance
L = 〈v1, v2, . . . , vn〉, L′ = 〈v1, v2, . . . , vn, vn+1〉.Knowledge only concerns v1, . . . , vn.ϕ ∈ SL.
IP(L)(ϕ) = IP(L′)(ϕ)
Centre of Mass is not language invariant! MaxEnt is lan-guage invariant.Ay, caramba!
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 109: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/109.jpg)
...
37 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 110: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/110.jpg)
...
38 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
MaxEntW
MaxEntW : An agent ought to equivocate (sufficiently) be-tween the basic possibilities that she can express.Language now contains the chance functions P.Density functions:
C1E := f1 : E→ [0,1] :
∫P∈E
f1(P)P dp ∈ P.
“Likelihood of a chance function”MaxEntW : Pick density with greatest entropy
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 111: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/111.jpg)
...
38 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
MaxEntW
MaxEntW : An agent ought to equivocate (sufficiently) be-tween the basic possibilities that she can express.Language now contains the chance functions P.Density functions:
C1E := f1 : E→ [0,1] :
∫P∈E
f1(P)P dp ∈ P.
“Likelihood of a chance function”MaxEntW : Pick density with greatest entropy
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 112: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/112.jpg)
...
38 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
MaxEntW
MaxEntW : An agent ought to equivocate (sufficiently) be-tween the basic possibilities that she can express.Language now contains the chance functions P.Density functions:
C1E := f1 : E→ [0,1] :
∫P∈E
f1(P)P dp ∈ P.
“Likelihood of a chance function”MaxEntW : Pick density with greatest entropy
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 113: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/113.jpg)
...
38 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
MaxEntW
MaxEntW : An agent ought to equivocate (sufficiently) be-tween the basic possibilities that she can express.Language now contains the chance functions P.Density functions:
C1E := f1 : E→ [0,1] :
∫P∈E
f1(P)P dp ∈ P.
“Likelihood of a chance function”MaxEntW : Pick density with greatest entropy
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 114: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/114.jpg)
...
38 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
MaxEntW
MaxEntW : An agent ought to equivocate (sufficiently) be-tween the basic possibilities that she can express.Language now contains the chance functions P.Density functions:
C1E := f1 : E→ [0,1] :
∫P∈E
f1(P)P dp ∈ P.
“Likelihood of a chance function”MaxEntW : Pick density with greatest entropy
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 115: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/115.jpg)
...
39 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Level One
Entropy: H(f1) := −∫
P∈E f1(P) · log(f1(P))dp.
Let f †1 be the density in C1E with maximal entropy.
Pick a probability function P+ :∫
P∈E f †1 (P)PdP.P+ = PCoM .
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 116: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/116.jpg)
...
39 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Level One
Entropy: H(f1) := −∫
P∈E f1(P) · log(f1(P))dp.
Let f †1 be the density in C1E with maximal entropy.
Pick a probability function P+ :∫
P∈E f †1 (P)PdP.P+ = PCoM .
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 117: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/117.jpg)
...
39 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Level One
Entropy: H(f1) := −∫
P∈E f1(P) · log(f1(P))dp.
Let f †1 be the density in C1E with maximal entropy.
Pick a probability function P+ :∫
P∈E f †1 (P)PdP.P+ = PCoM .
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 118: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/118.jpg)
...
39 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Level One
Entropy: H(f1) := −∫
P∈E f1(P) · log(f1(P))dp.
Let f †1 be the density in C1E with maximal entropy.
Pick a probability function P+ :∫
P∈E f †1 (P)PdP.P+ = PCoM .
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 119: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/119.jpg)
...
40 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Levels
Density functions for agents with richer languages
Cn+1E := fn+1 : E→ [0,1] :
∫fn∈Cn
E
fn+1(fn)fn dfn ∈ CnE.
Entropy: H(fn+1) := −∫
fn∈CnE
fn+1(fn) · log(fn+1(fn))dfn.
Let f †n+1 be the density in Cn+1E with maximal entropy (it is
flat).Pick a n density f +
n :∫
f∈CnE
f †n+1(fn)fndfn.
Eventually this determines some P++ ∈ E.P++ = PCoM .
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 120: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/120.jpg)
...
40 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Levels
Density functions for agents with richer languages
Cn+1E := fn+1 : E→ [0,1] :
∫fn∈Cn
E
fn+1(fn)fn dfn ∈ CnE.
Entropy: H(fn+1) := −∫
fn∈CnE
fn+1(fn) · log(fn+1(fn))dfn.
Let f †n+1 be the density in Cn+1E with maximal entropy (it is
flat).Pick a n density f +
n :∫
f∈CnE
f †n+1(fn)fndfn.
Eventually this determines some P++ ∈ E.P++ = PCoM .
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 121: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/121.jpg)
...
40 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Levels
Density functions for agents with richer languages
Cn+1E := fn+1 : E→ [0,1] :
∫fn∈Cn
E
fn+1(fn)fn dfn ∈ CnE.
Entropy: H(fn+1) := −∫
fn∈CnE
fn+1(fn) · log(fn+1(fn))dfn.
Let f †n+1 be the density in Cn+1E with maximal entropy (it is
flat).Pick a n density f +
n :∫
f∈CnE
f †n+1(fn)fndfn.
Eventually this determines some P++ ∈ E.P++ = PCoM .
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 122: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/122.jpg)
...
40 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Levels
Density functions for agents with richer languages
Cn+1E := fn+1 : E→ [0,1] :
∫fn∈Cn
E
fn+1(fn)fn dfn ∈ CnE.
Entropy: H(fn+1) := −∫
fn∈CnE
fn+1(fn) · log(fn+1(fn))dfn.
Let f †n+1 be the density in Cn+1E with maximal entropy (it is
flat).Pick a n density f +
n :∫
f∈CnE
f †n+1(fn)fndfn.
Eventually this determines some P++ ∈ E.P++ = PCoM .
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 123: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/123.jpg)
...
40 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Levels
Density functions for agents with richer languages
Cn+1E := fn+1 : E→ [0,1] :
∫fn∈Cn
E
fn+1(fn)fn dfn ∈ CnE.
Entropy: H(fn+1) := −∫
fn∈CnE
fn+1(fn) · log(fn+1(fn))dfn.
Let f †n+1 be the density in Cn+1E with maximal entropy (it is
flat).Pick a n density f +
n :∫
f∈CnE
f †n+1(fn)fndfn.
Eventually this determines some P++ ∈ E.P++ = PCoM .
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 124: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/124.jpg)
...
40 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Levels
Density functions for agents with richer languages
Cn+1E := fn+1 : E→ [0,1] :
∫fn∈Cn
E
fn+1(fn)fn dfn ∈ CnE.
Entropy: H(fn+1) := −∫
fn∈CnE
fn+1(fn) · log(fn+1(fn))dfn.
Let f †n+1 be the density in Cn+1E with maximal entropy (it is
flat).Pick a n density f +
n :∫
f∈CnE
f †n+1(fn)fndfn.
Eventually this determines some P++ ∈ E.P++ = PCoM .
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 125: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/125.jpg)
...
41 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Density Invariance
Equivocating over a density level (≥ 1) leads to centre ofmass, regardless of the level.Density InvarianceEquivocating over P gives MaxEnt!The level matters.Ay, ay, caramba!
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 126: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/126.jpg)
...
41 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Density Invariance
Equivocating over a density level (≥ 1) leads to centre ofmass, regardless of the level.Density InvarianceEquivocating over P gives MaxEnt!The level matters.Ay, ay, caramba!
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 127: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/127.jpg)
...
41 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Density Invariance
Equivocating over a density level (≥ 1) leads to centre ofmass, regardless of the level.Density InvarianceEquivocating over P gives MaxEnt!The level matters.Ay, ay, caramba!
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 128: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/128.jpg)
...
41 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Density Invariance
Equivocating over a density level (≥ 1) leads to centre ofmass, regardless of the level.Density InvarianceEquivocating over P gives MaxEnt!The level matters.Ay, ay, caramba!
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 129: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/129.jpg)
...
41 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Density Invariance
Equivocating over a density level (≥ 1) leads to centre ofmass, regardless of the level.Density InvarianceEquivocating over P gives MaxEnt!The level matters.Ay, ay, caramba!
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 130: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/130.jpg)
...
42 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
References I
Csiszár, I. (1991).Why Least Squares and Maximum Entropy? An AxiomaticApproach to Inference for Linear Inverse Problems.The Annals of Statistics, 19(4):2032–2066.
Grünwald, P. D. and Dawid, A. (2004).Game theory, maximum entropy, minimum discrepancyand robust Bayesian decision theory.Annals of Statistics, 32(4):1367–1433.
Jaynes, E. (1957).Information Theory and Statistical Mechanics.Physical Review, 106(4):620–630.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 131: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/131.jpg)
...
43 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
References II
Jaynes, E. T. (2003).Probability Theory: The Logic of Science.Cambridge University Press.
Paris, J. B. (1998).Common Sense and Maximum Entropy.Synthese, 117:75–93.
Paris, J. B. (2006).The Uncertain Reasoner’s Companion: A MathematicalPerspective, volume 39 of Cambridge Tracts in TheoreticalComputer Science.Cambridge University Press, 2 edition.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 132: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/132.jpg)
...
44 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
References III
Paris, J. B. and Vencovská, A. (1989).On the applicability of maximum entropy to inexactreasoning.International Journal of Approximate Reasoning,3(1):1–34.
Paris, J. B. and Vencovská, A. (1990).A note on the inevitability of maximum entropy.International Journal of Approximate Reasoning,4(3):183–223.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 133: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/133.jpg)
...
45 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
References IV
Paris, J. B. and Vencovská, A. (1997).In Defense of the Maximum Entropy Inference Process.International Journal of Approximate Reasoning,17(1):77–103.
Williamson, J. (2010).In Defence of Objective Bayesianism.Oxford University Press.
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 134: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/134.jpg)
...
46 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
That’s it. Thank you! Questions? – Progic Tomorrow
Jürgen Landes Maximum Entropy and Inductive Logic II
![Page 135: Maximum Entropy and Inductive Logic II · 2015. 4. 20. · 1 / 47 Recap The Classical Derivation A Modern Approach Universes Objections, your Honor! Maximum Entropy and Inductive](https://reader033.fdocuments.in/reader033/viewer/2022060902/609eb763d0935e53e272d393/html5/thumbnails/135.jpg)
...
47 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Logarithmic Utility
Intuitive human sensations tend to be logarithmic functionsof the stimulus. – JaynesSavageJon’s L1 – L4
Jürgen Landes Maximum Entropy and Inductive Logic II