Post on 24-Dec-2019
...
1 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Maximum Entropy and Inductive Logic I
Jürgen Landes
Spring School on Inductive Logic
Canterbury, 20.04.2015 - 21.04.2015
Jürgen Landes Maximum Entropy and Inductive Logic I
...
1 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Outline
1 Motivation & ProblemMotivationProblem
2 FormalitiesLanguageUncertaintyKnowledge
3 Inference ProcessesThe GistThe PointRight!
4 Desiderata for Inference Processes
Jürgen Landes Maximum Entropy and Inductive Logic I
...
2 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Outline
1 Motivation & ProblemMotivationProblem
2 FormalitiesLanguageUncertaintyKnowledge
3 Inference ProcessesThe GistThe PointRight!
4 Desiderata for Inference Processes
Jürgen Landes Maximum Entropy and Inductive Logic I
...
3 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Background
Ballpark: Rational BeliefsWhat does an agent rationally belief?As opposed to: When do we say that an agent beliefs X?Imagine: You are the agent.Normative formal approach
Jürgen Landes Maximum Entropy and Inductive Logic I
...
3 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Background
Ballpark: Rational BeliefsWhat does an agent rationally belief?As opposed to: When do we say that an agent beliefs X?Imagine: You are the agent.Normative formal approach
Jürgen Landes Maximum Entropy and Inductive Logic I
...
3 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Background
Ballpark: Rational BeliefsWhat does an agent rationally belief?As opposed to: When do we say that an agent beliefs X?Imagine: You are the agent.Normative formal approach
Jürgen Landes Maximum Entropy and Inductive Logic I
...
3 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Background
Ballpark: Rational BeliefsWhat does an agent rationally belief?As opposed to: When do we say that an agent beliefs X?Imagine: You are the agent.Normative formal approach
Jürgen Landes Maximum Entropy and Inductive Logic I
...
3 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Background
Ballpark: Rational BeliefsWhat does an agent rationally belief?As opposed to: When do we say that an agent beliefs X?Imagine: You are the agent.Normative formal approach
Jürgen Landes Maximum Entropy and Inductive Logic I
...
4 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Outline
1 Motivation & ProblemMotivationProblem
2 FormalitiesLanguageUncertaintyKnowledge
3 Inference ProcessesThe GistThe PointRight!
4 Desiderata for Inference Processes
Jürgen Landes Maximum Entropy and Inductive Logic I
...
5 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Ouch
Patient, migraines, doctorGiven the doctor’s background knowledge,patient complaining of certain symptoms,exhibiting certain traits:what should the doctor believe?Eventually, what should the doctor do?On the basis of imperfect information.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
5 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Ouch
Patient, migraines, doctorGiven the doctor’s background knowledge,patient complaining of certain symptoms,exhibiting certain traits:what should the doctor believe?Eventually, what should the doctor do?On the basis of imperfect information.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
5 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Ouch
Patient, migraines, doctorGiven the doctor’s background knowledge,patient complaining of certain symptoms,exhibiting certain traits:what should the doctor believe?Eventually, what should the doctor do?On the basis of imperfect information.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
5 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Ouch
Patient, migraines, doctorGiven the doctor’s background knowledge,patient complaining of certain symptoms,exhibiting certain traits:what should the doctor believe?Eventually, what should the doctor do?On the basis of imperfect information.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
5 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Ouch
Patient, migraines, doctorGiven the doctor’s background knowledge,patient complaining of certain symptoms,exhibiting certain traits:what should the doctor believe?Eventually, what should the doctor do?On the basis of imperfect information.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
5 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Ouch
Patient, migraines, doctorGiven the doctor’s background knowledge,patient complaining of certain symptoms,exhibiting certain traits:what should the doctor believe?Eventually, what should the doctor do?On the basis of imperfect information.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
5 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Ouch
Patient, migraines, doctorGiven the doctor’s background knowledge,patient complaining of certain symptoms,exhibiting certain traits:what should the doctor believe?Eventually, what should the doctor do?On the basis of imperfect information.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
5 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Ouch
Patient, migraines, doctorGiven the doctor’s background knowledge,patient complaining of certain symptoms,exhibiting certain traits:what should the doctor believe?Eventually, what should the doctor do?On the basis of imperfect information.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
5 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Ouch
Patient, migraines, doctorGiven the doctor’s background knowledge,patient complaining of certain symptoms,exhibiting certain traits:what should the doctor believe?Eventually, what should the doctor do?On the basis of imperfect information.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
5 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Ouch
Patient, migraines, doctorGiven the doctor’s background knowledge,patient complaining of certain symptoms,exhibiting certain traits:what should the doctor believe?Eventually, what should the doctor do?On the basis of imperfect information.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
6 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Carnap’s Principle
Carnap1947: A rational agent ought to take all availableevidence into account when forming beliefs.Reasonable.So, the doctor has to take all background knowledge andthe patient’s individual properties into account.Ignoring some information is a “No no”....Reasonable, ... really?
Jürgen Landes Maximum Entropy and Inductive Logic I
...
6 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Carnap’s Principle
Carnap1947: A rational agent ought to take all availableevidence into account when forming beliefs.Reasonable.So, the doctor has to take all background knowledge andthe patient’s individual properties into account.Ignoring some information is a “No no”....Reasonable, ... really?
Jürgen Landes Maximum Entropy and Inductive Logic I
...
6 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Carnap’s Principle
Carnap1947: A rational agent ought to take all availableevidence into account when forming beliefs.Reasonable.So, the doctor has to take all background knowledge andthe patient’s individual properties into account.Ignoring some information is a “No no”....Reasonable, ... really?
Jürgen Landes Maximum Entropy and Inductive Logic I
...
6 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Carnap’s Principle
Carnap1947: A rational agent ought to take all availableevidence into account when forming beliefs.Reasonable.So, the doctor has to take all background knowledge andthe patient’s individual properties into account.Ignoring some information is a “No no”....Reasonable, ... really?
Jürgen Landes Maximum Entropy and Inductive Logic I
...
6 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Carnap’s Principle
Carnap1947: A rational agent ought to take all availableevidence into account when forming beliefs.Reasonable.So, the doctor has to take all background knowledge andthe patient’s individual properties into account.Ignoring some information is a “No no”....Reasonable, ... really?
Jürgen Landes Maximum Entropy and Inductive Logic I
...
6 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
MotivationProblem
Carnap’s Principle
Carnap1947: A rational agent ought to take all availableevidence into account when forming beliefs.Reasonable.So, the doctor has to take all background knowledge andthe patient’s individual properties into account.Ignoring some information is a “No no”....Reasonable, ... really?
Jürgen Landes Maximum Entropy and Inductive Logic I
...
7 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Outline
1 Motivation & ProblemMotivationProblem
2 FormalitiesLanguageUncertaintyKnowledge
3 Inference ProcessesThe GistThe PointRight!
4 Desiderata for Inference Processes
Jürgen Landes Maximum Entropy and Inductive Logic I
...
8 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Outline
1 Motivation & ProblemMotivationProblem
2 FormalitiesLanguageUncertaintyKnowledge
3 Inference ProcessesThe GistThe PointRight!
4 Desiderata for Inference Processes
Jürgen Landes Maximum Entropy and Inductive Logic I
...
9 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Our Language
Finite propositional language LVariables v1, . . . , vn
Connectives ∧,∨,¬,→,←,↔Sentences of L, SL
No funny business, self-reference, truth predicates,self-fulfilling
Jürgen Landes Maximum Entropy and Inductive Logic I
...
9 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Our Language
Finite propositional language LVariables v1, . . . , vn
Connectives ∧,∨,¬,→,←,↔Sentences of L, SL
No funny business, self-reference, truth predicates,self-fulfilling
Jürgen Landes Maximum Entropy and Inductive Logic I
...
9 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Our Language
Finite propositional language LVariables v1, . . . , vn
Connectives ∧,∨,¬,→,←,↔Sentences of L, SL
No funny business, self-reference, truth predicates,self-fulfilling
Jürgen Landes Maximum Entropy and Inductive Logic I
...
9 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Our Language
Finite propositional language LVariables v1, . . . , vn
Connectives ∧,∨,¬,→,←,↔Sentences of L, SL
No funny business, self-reference, truth predicates,self-fulfilling
Jürgen Landes Maximum Entropy and Inductive Logic I
...
9 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Our Language
Finite propositional language LVariables v1, . . . , vn
Connectives ∧,∨,¬,→,←,↔Sentences of L, SL
No funny business, self-reference, truth predicates,self-fulfilling
Jürgen Landes Maximum Entropy and Inductive Logic I
...
9 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Our Language
Finite propositional language LVariables v1, . . . , vn
Connectives ∧,∨,¬,→,←,↔Sentences of L, SL
No funny business, self-reference, truth predicates,self-fulfilling
Jürgen Landes Maximum Entropy and Inductive Logic I
...
10 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Outline
1 Motivation & ProblemMotivationProblem
2 FormalitiesLanguageUncertaintyKnowledge
3 Inference ProcessesThe GistThe PointRight!
4 Desiderata for Inference Processes
Jürgen Landes Maximum Entropy and Inductive Logic I
...
11 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Probabilities
Probabilistic frameworkProbability function, PSubjective degrees of belief – Dutch BookP : SL→ [0,1]
P(τ) = 1 for all tautologies τ ∈ SL.P(ϕ ∨ θ) = P(ϕ) + P(θ), if |= ¬(ϕ ∧ θ).ϕ |= θ implies P(ϕ) ≤ P(θ).P(ϕ ∨ θ) = P(ϕ) + P(θ)− P(ϕ ∧ θ).Set of probability functions P
Jürgen Landes Maximum Entropy and Inductive Logic I
...
11 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Probabilities
Probabilistic frameworkProbability function, PSubjective degrees of belief – Dutch BookP : SL→ [0,1]
P(τ) = 1 for all tautologies τ ∈ SL.P(ϕ ∨ θ) = P(ϕ) + P(θ), if |= ¬(ϕ ∧ θ).ϕ |= θ implies P(ϕ) ≤ P(θ).P(ϕ ∨ θ) = P(ϕ) + P(θ)− P(ϕ ∧ θ).Set of probability functions P
Jürgen Landes Maximum Entropy and Inductive Logic I
...
11 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Probabilities
Probabilistic frameworkProbability function, PSubjective degrees of belief – Dutch BookP : SL→ [0,1]
P(τ) = 1 for all tautologies τ ∈ SL.P(ϕ ∨ θ) = P(ϕ) + P(θ), if |= ¬(ϕ ∧ θ).ϕ |= θ implies P(ϕ) ≤ P(θ).P(ϕ ∨ θ) = P(ϕ) + P(θ)− P(ϕ ∧ θ).Set of probability functions P
Jürgen Landes Maximum Entropy and Inductive Logic I
...
11 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Probabilities
Probabilistic frameworkProbability function, PSubjective degrees of belief – Dutch BookP : SL→ [0,1]
P(τ) = 1 for all tautologies τ ∈ SL.P(ϕ ∨ θ) = P(ϕ) + P(θ), if |= ¬(ϕ ∧ θ).ϕ |= θ implies P(ϕ) ≤ P(θ).P(ϕ ∨ θ) = P(ϕ) + P(θ)− P(ϕ ∧ θ).Set of probability functions P
Jürgen Landes Maximum Entropy and Inductive Logic I
...
11 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Probabilities
Probabilistic frameworkProbability function, PSubjective degrees of belief – Dutch BookP : SL→ [0,1]
P(τ) = 1 for all tautologies τ ∈ SL.P(ϕ ∨ θ) = P(ϕ) + P(θ), if |= ¬(ϕ ∧ θ).ϕ |= θ implies P(ϕ) ≤ P(θ).P(ϕ ∨ θ) = P(ϕ) + P(θ)− P(ϕ ∧ θ).Set of probability functions P
Jürgen Landes Maximum Entropy and Inductive Logic I
...
11 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Probabilities
Probabilistic frameworkProbability function, PSubjective degrees of belief – Dutch BookP : SL→ [0,1]
P(τ) = 1 for all tautologies τ ∈ SL.P(ϕ ∨ θ) = P(ϕ) + P(θ), if |= ¬(ϕ ∧ θ).ϕ |= θ implies P(ϕ) ≤ P(θ).P(ϕ ∨ θ) = P(ϕ) + P(θ)− P(ϕ ∧ θ).Set of probability functions P
Jürgen Landes Maximum Entropy and Inductive Logic I
...
11 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Probabilities
Probabilistic frameworkProbability function, PSubjective degrees of belief – Dutch BookP : SL→ [0,1]
P(τ) = 1 for all tautologies τ ∈ SL.P(ϕ ∨ θ) = P(ϕ) + P(θ), if |= ¬(ϕ ∧ θ).ϕ |= θ implies P(ϕ) ≤ P(θ).P(ϕ ∨ θ) = P(ϕ) + P(θ)− P(ϕ ∧ θ).Set of probability functions P
Jürgen Landes Maximum Entropy and Inductive Logic I
...
11 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Probabilities
Probabilistic frameworkProbability function, PSubjective degrees of belief – Dutch BookP : SL→ [0,1]
P(τ) = 1 for all tautologies τ ∈ SL.P(ϕ ∨ θ) = P(ϕ) + P(θ), if |= ¬(ϕ ∧ θ).ϕ |= θ implies P(ϕ) ≤ P(θ).P(ϕ ∨ θ) = P(ϕ) + P(θ)− P(ϕ ∧ θ).Set of probability functions P
Jürgen Landes Maximum Entropy and Inductive Logic I
...
11 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Probabilities
Probabilistic frameworkProbability function, PSubjective degrees of belief – Dutch BookP : SL→ [0,1]
P(τ) = 1 for all tautologies τ ∈ SL.P(ϕ ∨ θ) = P(ϕ) + P(θ), if |= ¬(ϕ ∧ θ).ϕ |= θ implies P(ϕ) ≤ P(θ).P(ϕ ∨ θ) = P(ϕ) + P(θ)− P(ϕ ∧ θ).Set of probability functions P
Jürgen Landes Maximum Entropy and Inductive Logic I
...
12 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Worlds
Possible world, states
ω = v1 ∧ v2 ∧ ¬v3 ∧ . . . ∧ vn1 ∧ ¬vn
Set of possible worlds Ω.Proposition F ⊆ Ω.
P(ϕ) =∑ω∈Ωω|=ϕ
P(ω).
A probability function P ∈ P is uniquely determined by itsvalues on possible worlds, 〈P(ω) : ω ∈ Ω〉.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
12 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Worlds
Possible world, states
ω = v1 ∧ v2 ∧ ¬v3 ∧ . . . ∧ vn1 ∧ ¬vn
Set of possible worlds Ω.Proposition F ⊆ Ω.
P(ϕ) =∑ω∈Ωω|=ϕ
P(ω).
A probability function P ∈ P is uniquely determined by itsvalues on possible worlds, 〈P(ω) : ω ∈ Ω〉.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
12 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Worlds
Possible world, states
ω = v1 ∧ v2 ∧ ¬v3 ∧ . . . ∧ vn1 ∧ ¬vn
Set of possible worlds Ω.Proposition F ⊆ Ω.
P(ϕ) =∑ω∈Ωω|=ϕ
P(ω).
A probability function P ∈ P is uniquely determined by itsvalues on possible worlds, 〈P(ω) : ω ∈ Ω〉.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
12 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Worlds
Possible world, states
ω = v1 ∧ v2 ∧ ¬v3 ∧ . . . ∧ vn1 ∧ ¬vn
Set of possible worlds Ω.Proposition F ⊆ Ω.
P(ϕ) =∑ω∈Ωω|=ϕ
P(ω).
A probability function P ∈ P is uniquely determined by itsvalues on possible worlds, 〈P(ω) : ω ∈ Ω〉.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
12 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Worlds
Possible world, states
ω = v1 ∧ v2 ∧ ¬v3 ∧ . . . ∧ vn1 ∧ ¬vn
Set of possible worlds Ω.Proposition F ⊆ Ω.
P(ϕ) =∑ω∈Ωω|=ϕ
P(ω).
A probability function P ∈ P is uniquely determined by itsvalues on possible worlds, 〈P(ω) : ω ∈ Ω〉.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
13 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Outline
1 Motivation & ProblemMotivationProblem
2 FormalitiesLanguageUncertaintyKnowledge
3 Inference ProcessesThe GistThe PointRight!
4 Desiderata for Inference Processes
Jürgen Landes Maximum Entropy and Inductive Logic I
...
14 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Knowledge
Think of P as some set of functions.Spanned by the possible worlds ω.It can be represented by a simplex.Dimension equal to the number of possibleworlds/states.Doctor’s background knowledge KIdentify K with a set E ⊆ P
Jürgen Landes Maximum Entropy and Inductive Logic I
...
14 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Knowledge
Think of P as some set of functions.Spanned by the possible worlds ω.It can be represented by a simplex.Dimension equal to the number of possibleworlds/states.Doctor’s background knowledge KIdentify K with a set E ⊆ P
Jürgen Landes Maximum Entropy and Inductive Logic I
...
14 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Knowledge
Think of P as some set of functions.Spanned by the possible worlds ω.It can be represented by a simplex.Dimension equal to the number of possibleworlds/states.Doctor’s background knowledge KIdentify K with a set E ⊆ P
Jürgen Landes Maximum Entropy and Inductive Logic I
...
14 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Knowledge
Think of P as some set of functions.Spanned by the possible worlds ω.It can be represented by a simplex.Dimension equal to the number of possibleworlds/states.Doctor’s background knowledge KIdentify K with a set E ⊆ P
Jürgen Landes Maximum Entropy and Inductive Logic I
...
14 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Knowledge
Think of P as some set of functions.Spanned by the possible worlds ω.It can be represented by a simplex.Dimension equal to the number of possibleworlds/states.Doctor’s background knowledge KIdentify K with a set E ⊆ P
Jürgen Landes Maximum Entropy and Inductive Logic I
...
14 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Knowledge
Think of P as some set of functions.Spanned by the possible worlds ω.It can be represented by a simplex.Dimension equal to the number of possibleworlds/states.Doctor’s background knowledge KIdentify K with a set E ⊆ P
Jürgen Landes Maximum Entropy and Inductive Logic I
...
14 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Knowledge
Think of P as some set of functions.Spanned by the possible worlds ω.It can be represented by a simplex.Dimension equal to the number of possibleworlds/states.Doctor’s background knowledge KIdentify K with a set E ⊆ P
Jürgen Landes Maximum Entropy and Inductive Logic I
...
14 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Knowledge
Think of P as some set of functions.Spanned by the possible worlds ω.It can be represented by a simplex.Dimension equal to the number of possibleworlds/states.Doctor’s background knowledge KIdentify K with a set E ⊆ P
Jürgen Landes Maximum Entropy and Inductive Logic I
...
14 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Knowledge
Think of P as some set of functions.Spanned by the possible worlds ω.It can be represented by a simplex.Dimension equal to the number of possibleworlds/states.Doctor’s background knowledge KIdentify K with a set E ⊆ P
Jürgen Landes Maximum Entropy and Inductive Logic I
...
15 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Watt’s Assumption
The set E contains all the doctor’s knowledge.If you don’t formalise all knowledge, then you should not besurprised by our answer.Garbage in, garbage out.This course is not about the highly relevant and non-trivialproblem of obtaining E from K .We will assume that E 6= ∅.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
15 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Watt’s Assumption
The set E contains all the doctor’s knowledge.If you don’t formalise all knowledge, then you should not besurprised by our answer.Garbage in, garbage out.This course is not about the highly relevant and non-trivialproblem of obtaining E from K .We will assume that E 6= ∅.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
15 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Watt’s Assumption
The set E contains all the doctor’s knowledge.If you don’t formalise all knowledge, then you should not besurprised by our answer.Garbage in, garbage out.This course is not about the highly relevant and non-trivialproblem of obtaining E from K .We will assume that E 6= ∅.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
15 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Watt’s Assumption
The set E contains all the doctor’s knowledge.If you don’t formalise all knowledge, then you should not besurprised by our answer.Garbage in, garbage out.This course is not about the highly relevant and non-trivialproblem of obtaining E from K .We will assume that E 6= ∅.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
15 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Watt’s Assumption
The set E contains all the doctor’s knowledge.If you don’t formalise all knowledge, then you should not besurprised by our answer.Garbage in, garbage out.This course is not about the highly relevant and non-trivialproblem of obtaining E from K .We will assume that E 6= ∅.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
16 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Example Knowledge
Symptom S1 is a very good indicator of condition C.10% of migraines are triggered by stress.In 23% of cases in which patients complain about symptomS2, migraines will spontaneously cease within half a day.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
16 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Example Knowledge
Symptom S1 is a very good indicator of condition C.10% of migraines are triggered by stress.In 23% of cases in which patients complain about symptomS2, migraines will spontaneously cease within half a day.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
16 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
LanguageUncertaintyKnowledge
Example Knowledge
Symptom S1 is a very good indicator of condition C.10% of migraines are triggered by stress.In 23% of cases in which patients complain about symptomS2, migraines will spontaneously cease within half a day.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
17 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Outline
1 Motivation & ProblemMotivationProblem
2 FormalitiesLanguageUncertaintyKnowledge
3 Inference ProcessesThe GistThe PointRight!
4 Desiderata for Inference Processes
Jürgen Landes Maximum Entropy and Inductive Logic I
...
18 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Outline
1 Motivation & ProblemMotivationProblem
2 FormalitiesLanguageUncertaintyKnowledge
3 Inference ProcessesThe GistThe PointRight!
4 Desiderata for Inference Processes
Jürgen Landes Maximum Entropy and Inductive Logic I
...
19 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Condensing Knowledge
Idea:Input knowledge KOutput belief function P+ ∈ PAdopt this function P+ for decision making.P+ reflects the doctor’s knowledge K .We are not fabricating knowledge out of thin air.We do the best we can, given limited information.Some information is lost in this process.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
19 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Condensing Knowledge
Idea:Input knowledge KOutput belief function P+ ∈ PAdopt this function P+ for decision making.P+ reflects the doctor’s knowledge K .We are not fabricating knowledge out of thin air.We do the best we can, given limited information.Some information is lost in this process.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
19 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Condensing Knowledge
Idea:Input knowledge KOutput belief function P+ ∈ PAdopt this function P+ for decision making.P+ reflects the doctor’s knowledge K .We are not fabricating knowledge out of thin air.We do the best we can, given limited information.Some information is lost in this process.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
19 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Condensing Knowledge
Idea:Input knowledge KOutput belief function P+ ∈ PAdopt this function P+ for decision making.P+ reflects the doctor’s knowledge K .We are not fabricating knowledge out of thin air.We do the best we can, given limited information.Some information is lost in this process.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
19 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Condensing Knowledge
Idea:Input knowledge KOutput belief function P+ ∈ PAdopt this function P+ for decision making.P+ reflects the doctor’s knowledge K .We are not fabricating knowledge out of thin air.We do the best we can, given limited information.Some information is lost in this process.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
19 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Condensing Knowledge
Idea:Input knowledge KOutput belief function P+ ∈ PAdopt this function P+ for decision making.P+ reflects the doctor’s knowledge K .We are not fabricating knowledge out of thin air.We do the best we can, given limited information.Some information is lost in this process.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
19 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Condensing Knowledge
Idea:Input knowledge KOutput belief function P+ ∈ PAdopt this function P+ for decision making.P+ reflects the doctor’s knowledge K .We are not fabricating knowledge out of thin air.We do the best we can, given limited information.Some information is lost in this process.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
19 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Condensing Knowledge
Idea:Input knowledge KOutput belief function P+ ∈ PAdopt this function P+ for decision making.P+ reflects the doctor’s knowledge K .We are not fabricating knowledge out of thin air.We do the best we can, given limited information.Some information is lost in this process.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
20 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Conditionalisation
What about the properties of the patient?Yes, what about them??I have not forgotten about them.Properties ψ1, . . . , ψr
ψ := ψ1 ∧ . . . ∧ ψr
Consider conditional probability (P+(ψ) > 0)
P+(ϕ|ψ) :=P+(ϕ ∧ ψ)
P+(ψ).
Roughly, assume that ψ is true.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
20 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Conditionalisation
What about the properties of the patient?Yes, what about them??I have not forgotten about them.Properties ψ1, . . . , ψr
ψ := ψ1 ∧ . . . ∧ ψr
Consider conditional probability (P+(ψ) > 0)
P+(ϕ|ψ) :=P+(ϕ ∧ ψ)
P+(ψ).
Roughly, assume that ψ is true.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
20 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Conditionalisation
What about the properties of the patient?Yes, what about them??I have not forgotten about them.Properties ψ1, . . . , ψr
ψ := ψ1 ∧ . . . ∧ ψr
Consider conditional probability (P+(ψ) > 0)
P+(ϕ|ψ) :=P+(ϕ ∧ ψ)
P+(ψ).
Roughly, assume that ψ is true.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
20 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Conditionalisation
What about the properties of the patient?Yes, what about them??I have not forgotten about them.Properties ψ1, . . . , ψr
ψ := ψ1 ∧ . . . ∧ ψr
Consider conditional probability (P+(ψ) > 0)
P+(ϕ|ψ) :=P+(ϕ ∧ ψ)
P+(ψ).
Roughly, assume that ψ is true.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
20 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Conditionalisation
What about the properties of the patient?Yes, what about them??I have not forgotten about them.Properties ψ1, . . . , ψr
ψ := ψ1 ∧ . . . ∧ ψr
Consider conditional probability (P+(ψ) > 0)
P+(ϕ|ψ) :=P+(ϕ ∧ ψ)
P+(ψ).
Roughly, assume that ψ is true.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
20 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Conditionalisation
What about the properties of the patient?Yes, what about them??I have not forgotten about them.Properties ψ1, . . . , ψr
ψ := ψ1 ∧ . . . ∧ ψr
Consider conditional probability (P+(ψ) > 0)
P+(ϕ|ψ) :=P+(ϕ ∧ ψ)
P+(ψ).
Roughly, assume that ψ is true.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
20 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Conditionalisation
What about the properties of the patient?Yes, what about them??I have not forgotten about them.Properties ψ1, . . . , ψr
ψ := ψ1 ∧ . . . ∧ ψr
Consider conditional probability (P+(ψ) > 0)
P+(ϕ|ψ) :=P+(ϕ ∧ ψ)
P+(ψ).
Roughly, assume that ψ is true.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
21 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
The General Picture
Formally, an inference process is a map from a set of prob-ability functions (here E) to the set of probability functions.An inference process is a map (or function)Input: EOutput: P+.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
21 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
The General Picture
Formally, an inference process is a map from a set of prob-ability functions (here E) to the set of probability functions.An inference process is a map (or function)Input: EOutput: P+.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
21 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
The General Picture
Formally, an inference process is a map from a set of prob-ability functions (here E) to the set of probability functions.An inference process is a map (or function)Input: EOutput: P+.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
21 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
The General Picture
Formally, an inference process is a map from a set of prob-ability functions (here E) to the set of probability functions.An inference process is a map (or function)Input: EOutput: P+.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
22 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Outline
1 Motivation & ProblemMotivationProblem
2 FormalitiesLanguageUncertaintyKnowledge
3 Inference ProcessesThe GistThe PointRight!
4 Desiderata for Inference Processes
Jürgen Landes Maximum Entropy and Inductive Logic I
...
23 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Inductive Logic
How strongly do uncertain premises entail a conclusion?Uncertain premises ...conclusion ...entail ...
P∗(ϕ1) ∈ I1,P∗(ϕ2) ∈ I2, . . . ,P∗(ϕk ) ∈ Ik︸ ︷︷ ︸Knowledge
|≈︸︷︷︸Entailment
ψ?︸︷︷︸Conclusion
? = P+(ψ), a single real number.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
23 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Inductive Logic
How strongly do uncertain premises entail a conclusion?Uncertain premises ...conclusion ...entail ...
P∗(ϕ1) ∈ I1,P∗(ϕ2) ∈ I2, . . . ,P∗(ϕk ) ∈ Ik︸ ︷︷ ︸Knowledge
|≈︸︷︷︸Entailment
ψ?︸︷︷︸Conclusion
? = P+(ψ), a single real number.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
23 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Inductive Logic
How strongly do uncertain premises entail a conclusion?Uncertain premises ...conclusion ...entail ...
P∗(ϕ1) ∈ I1,P∗(ϕ2) ∈ I2, . . . ,P∗(ϕk ) ∈ Ik︸ ︷︷ ︸Knowledge
|≈︸︷︷︸Entailment
ψ?︸︷︷︸Conclusion
? = P+(ψ), a single real number.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
23 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Inductive Logic
How strongly do uncertain premises entail a conclusion?Uncertain premises ...conclusion ...entail ...
P∗(ϕ1) ∈ I1,P∗(ϕ2) ∈ I2, . . . ,P∗(ϕk ) ∈ Ik︸ ︷︷ ︸Knowledge
|≈︸︷︷︸Entailment
ψ?︸︷︷︸Conclusion
? = P+(ψ), a single real number.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
23 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Inductive Logic
How strongly do uncertain premises entail a conclusion?Uncertain premises ...conclusion ...entail ...
P∗(ϕ1) ∈ I1,P∗(ϕ2) ∈ I2, . . . ,P∗(ϕk ) ∈ Ik︸ ︷︷ ︸Knowledge
|≈︸︷︷︸Entailment
ψ?︸︷︷︸Conclusion
? = P+(ψ), a single real number.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
23 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Inductive Logic
How strongly do uncertain premises entail a conclusion?Uncertain premises ...conclusion ...entail ...
P∗(ϕ1) ∈ I1,P∗(ϕ2) ∈ I2, . . . ,P∗(ϕk ) ∈ Ik︸ ︷︷ ︸Knowledge
|≈︸︷︷︸Entailment
ψ?︸︷︷︸Conclusion
? = P+(ψ), a single real number.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
23 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Inductive Logic
How strongly do uncertain premises entail a conclusion?Uncertain premises ...conclusion ...entail ...
P∗(ϕ1) ∈ I1,P∗(ϕ2) ∈ I2, . . . ,P∗(ϕk ) ∈ Ik︸ ︷︷ ︸Knowledge
|≈︸︷︷︸Entailment
ψ?︸︷︷︸Conclusion
? = P+(ψ), a single real number.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
24 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Outline
1 Motivation & ProblemMotivationProblem
2 FormalitiesLanguageUncertaintyKnowledge
3 Inference ProcessesThe GistThe PointRight!
4 Desiderata for Inference Processes
Jürgen Landes Maximum Entropy and Inductive Logic I
...
25 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Obviously right – Tomorrow!
Adopt the function, P† = P+, which solve this optimisationproblem
maximise: −∑ω∈Ω
P(ω) log(P(ω))
subject to: P ∈ E .
Jürgen Landes Maximum Entropy and Inductive Logic I
...
26 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Intuitively right
Pick a “middling” function in E.How formalise “middling”?
Pick the centre of E!Okay, yes, but how does one work out the centre?
Jürgen Landes Maximum Entropy and Inductive Logic I
...
26 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Intuitively right
Pick a “middling” function in E.How formalise “middling”?
Pick the centre of E!Okay, yes, but how does one work out the centre?
Jürgen Landes Maximum Entropy and Inductive Logic I
...
26 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Intuitively right
Pick a “middling” function in E.How formalise “middling”?
Pick the centre of E!Okay, yes, but how does one work out the centre?
Jürgen Landes Maximum Entropy and Inductive Logic I
...
26 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Intuitively right
Pick a “middling” function in E.How formalise “middling”?
Pick the centre of E!Okay, yes, but how does one work out the centre?
Jürgen Landes Maximum Entropy and Inductive Logic I
...
26 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
Intuitively right
Pick a “middling” function in E.How formalise “middling”?
Pick the centre of E!Okay, yes, but how does one work out the centre?
Jürgen Landes Maximum Entropy and Inductive Logic I
...
27 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
At the tip of a finger
Jürgen Landes Maximum Entropy and Inductive Logic I
...
28 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
The centre
Centre of MassPoint of Balance
P+CoM :=
∫P∈E P dp∫P∈E dp
.
Centre of Mass inference process.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
28 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
The centre
Centre of MassPoint of Balance
P+CoM :=
∫P∈E P dp∫P∈E dp
.
Centre of Mass inference process.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
28 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
The centre
Centre of MassPoint of Balance
P+CoM :=
∫P∈E P dp∫P∈E dp
.
Centre of Mass inference process.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
28 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
The GistThe PointRight!
The centre
Centre of MassPoint of Balance
P+CoM :=
∫P∈E P dp∫P∈E dp
.
Centre of Mass inference process.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
29 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
Outline
1 Motivation & ProblemMotivationProblem
2 FormalitiesLanguageUncertaintyKnowledge
3 Inference ProcessesThe GistThe PointRight!
4 Desiderata for Inference Processes
Jürgen Landes Maximum Entropy and Inductive Logic I
...
30 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
You
....
Jürgen Landes Maximum Entropy and Inductive Logic I
...
31 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
Internal
P+ ∈ E......if E is not emptyconvexand closed.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
31 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
Internal
P+ ∈ E......if E is not emptyconvexand closed.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
31 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
Internal
P+ ∈ E......if E is not emptyconvexand closed.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
31 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
Internal
P+ ∈ E......if E is not emptyconvexand closed.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
31 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
Internal
P+ ∈ E......if E is not emptyconvexand closed.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
31 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
Internal
P+ ∈ E......if E is not emptyconvexand closed.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
32 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
Careful, careful
Open-mindednessDo not rule anything out, which you consider possible.If there exists a probability function P ∈ E with P(ω) > 0,then P+(ω) > 0.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
32 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
Careful, careful
Open-mindednessDo not rule anything out, which you consider possible.If there exists a probability function P ∈ E with P(ω) > 0,then P+(ω) > 0.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
32 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
Careful, careful
Open-mindednessDo not rule anything out, which you consider possible.If there exists a probability function P ∈ E with P(ω) > 0,then P+(ω) > 0.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
33 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
Open-mindedness is Possible
For all such ω, pick an Pω ∈ Pand also pick an rω ∈ (0,1) such that
∑rω = 1.
Then the convex combination∑
rωPω is a probability func-tion. (P is convex.)It is possible to satisfy open-mindedness.If E is convex, then
∑rωPω ∈ E.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
33 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
Open-mindedness is Possible
For all such ω, pick an Pω ∈ Pand also pick an rω ∈ (0,1) such that
∑rω = 1.
Then the convex combination∑
rωPω is a probability func-tion. (P is convex.)It is possible to satisfy open-mindedness.If E is convex, then
∑rωPω ∈ E.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
33 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
Open-mindedness is Possible
For all such ω, pick an Pω ∈ Pand also pick an rω ∈ (0,1) such that
∑rω = 1.
Then the convex combination∑
rωPω is a probability func-tion. (P is convex.)It is possible to satisfy open-mindedness.If E is convex, then
∑rωPω ∈ E.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
33 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
Open-mindedness is Possible
For all such ω, pick an Pω ∈ Pand also pick an rω ∈ (0,1) such that
∑rω = 1.
Then the convex combination∑
rωPω is a probability func-tion. (P is convex.)It is possible to satisfy open-mindedness.If E is convex, then
∑rωPω ∈ E.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
33 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
Open-mindedness is Possible
For all such ω, pick an Pω ∈ Pand also pick an rω ∈ (0,1) such that
∑rω = 1.
Then the convex combination∑
rωPω is a probability func-tion. (P is convex.)It is possible to satisfy open-mindedness.If E is convex, then
∑rωPω ∈ E.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
34 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
Language Invariance
What if we started with v1, v2, . . . , vn, vn+1
the same knowledge and the same patient?Apply the above recipe and obtain P ′.For a sentence in the original language ϕ ∈ SL, pleasecomplete the below
P ′(ϕ) = ?
Shocker: Centre of Mass is not language invariant!
Jürgen Landes Maximum Entropy and Inductive Logic I
...
34 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
Language Invariance
What if we started with v1, v2, . . . , vn, vn+1
the same knowledge and the same patient?Apply the above recipe and obtain P ′.For a sentence in the original language ϕ ∈ SL, pleasecomplete the below
P ′(ϕ) = ?
Shocker: Centre of Mass is not language invariant!
Jürgen Landes Maximum Entropy and Inductive Logic I
...
34 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
Language Invariance
What if we started with v1, v2, . . . , vn, vn+1
the same knowledge and the same patient?Apply the above recipe and obtain P ′.For a sentence in the original language ϕ ∈ SL, pleasecomplete the below
P ′(ϕ) = ?
Shocker: Centre of Mass is not language invariant!
Jürgen Landes Maximum Entropy and Inductive Logic I
...
34 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
Language Invariance
What if we started with v1, v2, . . . , vn, vn+1
the same knowledge and the same patient?Apply the above recipe and obtain P ′.For a sentence in the original language ϕ ∈ SL, pleasecomplete the below
P ′(ϕ) = ?
Shocker: Centre of Mass is not language invariant!
Jürgen Landes Maximum Entropy and Inductive Logic I
...
34 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
Language Invariance
What if we started with v1, v2, . . . , vn, vn+1
the same knowledge and the same patient?Apply the above recipe and obtain P ′.For a sentence in the original language ϕ ∈ SL, pleasecomplete the below
P ′(ϕ) = ?
Shocker: Centre of Mass is not language invariant!
Jürgen Landes Maximum Entropy and Inductive Logic I
...
35 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
References I
Paris, J. B. (2006).The Uncertain Reasoner’s Companion: A MathematicalPerspective, volume 39 of Cambridge Tracts in TheoreticalComputer Science.Cambridge University Press, 2 edition.
Jürgen Landes Maximum Entropy and Inductive Logic I
...
36 / 36
Motivation & ProblemFormalities
Inference ProcessesDesiderata for Inference Processes
That’s it. Thank you! Questions?
Jürgen Landes Maximum Entropy and Inductive Logic I