[email protected] Rennes, February, 22th...
Transcript of [email protected] Rennes, February, 22th...
-
Information fusion
Arnaud [email protected]
Université de Rennes 1 - IRISA, Lannion, France
Rennes, February, 22th 2018
Information fusion, A. Martin 22/02/18
1/36
-
OutlineFusionBeliefApplications
1. Introduction to information fusion
2. Theory of belief functions for information fusion
3. Applications
Information fusion, A. Martin 22/02/18
2/36
-
What is information?FusionBeliefApplications
Information fusion, A. Martin 22/02/18
3/36
(1/12)
: The traffic is dense on the runway, ready for takeoff.
-
What is information?FusionBeliefApplications
Information fusion, A. Martin 22/02/18
3/36
(1/12)
: The traffic is dense on the runway, ready for takeoff.
Imprecise proposition
-
What is information?FusionBeliefApplications
Information fusion, A. Martin 22/02/18
3/36
(1/12)
: The traffic is dense on the runway, ready for takeoff.
Imprecise proposition
: Ten aircrafts were on the runway, ready for takeoff.
-
What is information?FusionBeliefApplications
Information fusion, A. Martin 22/02/18
3/36
(1/12)
: The traffic is dense on the runway, ready for takeoff.
Imprecise proposition
: Ten aircrafts were on the runway, ready for takeoff.
Precise proposition
-
What is information?FusionBeliefApplications
Information fusion, A. Martin 22/02/18
3/36
(1/12)
: The traffic is dense on the runway, ready for takeoff.
Imprecise proposition
: Ten aircrafts were on the runway, ready for takeoff.
Precise proposition
Imprecision is a kind of imperfection of information
-
What is information?FusionBeliefApplications
Information fusion, A. Martin 22/02/18
4/36
(2/12)
: A boeing 777 is at the position x, y, z
-
What is information?FusionBeliefApplications
Information fusion, A. Martin 22/02/18
4/36
(2/12)
: A boeing 777 is at the position x, y, z
Bad weatherUncertain proposition
-
What is information?FusionBeliefApplications
Information fusion, A. Martin 22/02/18
4/36
(2/12)
: A boeing 777 is at the position x, y, z
Bad weatherUncertain proposition
Good weatherCertain proposition
-
What is information?FusionBeliefApplications
Information fusion, A. Martin 22/02/18
4/36
(2/12)
: A boeing 777 is at the position x, y, z
Bad weatherUncertain proposition
Good weatherCertain proposition
Uncertainty is another kind of imperfection of information
-
How are the sources?FusionBeliefApplications
Information fusion, A. Martin 22/02/18
5/36
(3/12)
: I am sure that is a fighter plane.A
: That is maybe a A380B
FUSION
-
How are the sources?FusionBeliefApplications
Information fusion, A. Martin 22/02/18
5/36
(3/12)
: I am sure that is a fighter plane.A
: That is maybe a A380B
Conflict of information sources hasto be solved
FUSION
-
How are the sources?FusionBeliefApplications
Information fusion, A. Martin 22/02/18
5/36
(3/12)
: I am sure that is a fighter plane.A
: That is maybe a A380B
FUSION
A is often mistakenB is never wrong
-
How are the sources?FusionBeliefApplications
Information fusion, A. Martin 22/02/18
5/36
(3/12)
: I am sure that is a fighter plane.A
: That is maybe a A380B
FUSION
: That is maybe a A380A is often mistakenB is never wrong
-
How are the sources?FusionBeliefApplications
Information fusion, A. Martin 22/02/18
5/36
(3/12)
: I am sure that is a fighter plane.A
: That is maybe a A380B
FUSION
A is often mistakenB is never wrong
Reliability of information sourceshas to be considered
-
Interest of information fusionFusionBeliefApplications
Information fusion, A. Martin 22/02/18
6/36
(4/12)
Assume all the sources are completly reliable giving totally certain andprecise information
FUSION
-
Interest of information fusionFusionBeliefApplications
Information fusion, A. Martin 22/02/18
6/36
(4/12)
Assume all the sources are completly reliable giving totally certain andprecise information
: This is an A380
: This is an A380
: This is an A380
FUSION
-
Interest of information fusionFusionBeliefApplications
Information fusion, A. Martin 22/02/18
6/36
(4/12)
Assume all the sources are completly reliable giving totally certain andprecise information
: This is an A380
: This is an A380
: This is an A380
: This is an A380FUSION
-
ImperfectionsFusionBeliefApplications
Imperfections on information
I Uncertainty: Degree of conformity to the reality
I Imprecision: quantitative default of knowledge on theinformation contain
I Incompleteness: lack of information
Sources of information
I Reliability: to give an indisputable information
I Independence: Sources are not linked (assumption oftenmade)
I Conflicting sources: Sources give information in contradiction
Information fusion, A. Martin 22/02/18
7/36
(5/12)
-
Information fusionFusionBeliefApplications
Goal: To combine information coming from many imperfectsources in order to improve the decision making taking intoaccount of imprecisions and uncertaintiesTo combine information coming from imperfect sources leadsfatally to conflict
Three actions are possible face to imperfections:
1. We can try to suppress its
2. We can tolerate its and so we need robust algorithms againstthese imperfections
3. We can model its
To model imperfections: uncertainty theories:Probability theory (Bayesian approach) or possibility theory orthe theory of belief functions
Information fusion, A. Martin 22/02/18
8/36
(6/12)
-
Fusion architecture for classifiers fusionFusionBeliefApplications
s sources S1, S2, ..., Ss that must take a decision on anobservation x in a set of n classes x ∈ Ω = {ω1, ω2, . . . , ωn}classes
ω1 . . . ωi . . . ωnS1...Sj...Ss
M11 (x) . . . M
1i (x) . . . M
1n(x)
.... . .
.... . .
...
M j1 (x) . . . Mji (x) . . . M
jn(x)
.... . .
.... . .
...M s1 (x) . . . M
si (x) . . . M
sn(x)
(1)
Information fusion, A. Martin 22/02/18
9/36
(7/12)
-
Steps of information fusionFusionBeliefApplications
Problems
I How to combine the information coming from all the sources?
I How to take the decision?
Under-problems depending on the application
I How to model information? i.e. choice of a formalism
I How to estimate the model parameters?
4 steps
1. Modeling
2. Estimation
3. Combination
4. Decision
Information fusion, A. Martin 22/02/18
10/36
(8/12)
-
Voting processFusionBeliefApplications
Modeling: Indicator functions
M ji (x) =
{1 if Sj : x ∈ ωi0 otherwise
Estimation: αij : reliability of a source for a given classCombination:
MEi (x) =s∑j=1
αijMji (x)
Decision:
x ∈ ωk if MEk (x) = maxiMEi (x) ≥ c.s+ b(x)x ∈ ωn+1 otherwise i.e. no decisionc ∈ [0, 1], b(x) function of MEk (x)
Information fusion, A. Martin 22/02/18
11/36
(9/12)
-
Voting process: A resultFusionBeliefApplications
I AssumptionsI Statistical independence of sourcesI Same probability of success pI s odd and > 2 (same kind of result if s is even)
I ThenWith PR the probability of success after fusion by votingprocess
I If p > 0.5, PR tends toward 1 with sI If p < 0.5, PR tends toward 0 with sI If p = 0.5, PR = 0.5 for all s
Behind this result: The majority should have right for fusion.
Information fusion, A. Martin 22/02/18
12/36
(10/12)
-
Probability theory for information fusionFusionBeliefApplications
Modeling: A probability is a positive and additive measure, p isdefined on a σ-algebra of Ω = {ω1, ω2, . . . , ωn} and takes values in[0,1].
It verifies: p(∅) = 0, p(Ω) = 1,∑X∈Ω
p(X) = 1
Estimation: Choice of the distribution, and/or estimation ofparametersCombination: Bayes rule
p(x ∈ ωi/S1, . . . , Ss) =p(S1, . . . , Ss/x ∈ ωi)p(x ∈ ωi)
p(S1, . . . , Ss)(2)
Independence assumption must of the time necessaryDecision: a posteriori maximum, likelihood maximum, meanmaximum, etc.
Information fusion, A. Martin 22/02/18
13/36
(11/12)
-
Limits of the theory of probabilitiesFusionBeliefApplications
I Difficulties to model the absence of knowledge (ex: Sirius)
I Constraint on the classes (exhaustive and exclusive)
I Constraint on the measures (additivity)
Example of Smets on the matter of additivity
If one symptom f (for fiver) is always true when a patient get aillness A (flu) (p(f |A) = 1), and if we observe this symptom f ,then the probability of the patient having A increases (becausep(A|f) = p(A)/p(f) so p(A|f) ≥ p(A)).The additivity constraint require then that the probability of thepatient having not A decreases:p(A|f) = 1− p(A|f) so p(A|f) ≤ p(A)While there is no reason if the symptom f can be also observe insome other diseases.
Information fusion, A. Martin 22/02/18
14/36
(12/12)
-
OutlineFusionBeliefApplications
1. Introduction to information fusion
2. Theory of belief functions for information fusion
3. Applications
Information fusion, A. Martin 22/02/18
15/36
-
Bases on Belief functionsFusionBeliefApplications
I Use of functions defined on sub-sets instead of singletons suchas probabilities
I Discernment frame: Ω = {ω1, . . . , ωn}, with ωi are exclusiveand exhaustive classes
I Power set: 2Ω = {∅, {ω1}, {ω2}, {ω1 ∪ ω2}, . . . ,Ω}.I Several functions in one to one correspondence model
uncertainty and imprecision: mass functions, belief functions,plausilibity functions
I Extension of 2Ω to DΩ, hyper power set in order to model theconflicts
I DΩ closed set by union and intersectionoperators
I DΩr : reduced set with constraints(ω2 ∩ ω3 ≡ ∅)
Information fusion, A. Martin 22/02/18
16/36
Modeling (1/13)
-
Mass functionsFusionBeliefApplications
I The basic belief functions (bba or mass functions) are definedon 2Ω and take values in [0, 1]
I Normalization condition:∑X∈2Ω
m(X) = 1
I A focal element is an element X of 2Ω such as m(X) > 0
I Closed world: m(∅) = 0I We note mj the mass function of the source Sj
Information fusion, A. Martin 22/02/18
17/36
Modeling (2/13)
-
Mass functionsFusionBeliefApplications
Special cases:
I If only focal elements are ωi then mj is a probability
I mj(Ω) = 1: total ignorance of SjI categorical mass function: mj(X) = 1 (noted mX): Sj has
an imprecise knowledge
I mj(ωi) = 1: Sj has a precise knowledge
I simple mass functions Xw:mj(X) = w and mj(Ω) = 1− w: Sj has an uncertain andimprecise knowledge
Information fusion, A. Martin 22/02/18
18/36
Modeling (3/13)
-
DiscountingFusionBeliefApplications
From (Shafer, 1976):mαj (X) = αjmj(X), ∀X ∈ 2Ωmαj (Ω) = 1− αj(1−mj(Ω))
αj ∈ [0, 1] discounting coefficient can be seen as the reliability ofthe source SjIf αj = 0 the source are completely unreliable, all the mass istransfered on Ω, the total ignorance
The discounting process increases the intervals [belj ,plj ] (and soreduces the global conflict in the conjunctive combination rule)
Information fusion, A. Martin 22/02/18
19/36
Modeling (4/13)
-
Go back on fusion architectureFusionBeliefApplications
s sources S1, S2, ..., Ss that must take a decision on anobservation x in a set of n classes x ∈ Ω = {ω1, ω2, . . . , ωn}classes
ω1 . . . ωi . . . ωnS1...Sj...Ss
m11(x) . . . m
1i (x) . . . m
1n(x)
.... . .
.... . .
...
mj1(x) . . . mji (x) . . . m
jn(x)
.... . .
.... . .
...ms1(x) . . . m
si (x) . . . m
sn(x)
Information fusion, A. Martin 22/02/18
20/36
Estimation (5/13)
-
Distance based model (Denœux 1995)FusionBeliefApplications
Only ωi and Ω are focal elements, n ∗ s sources (experts)I Prototypes case (xi center of ωi). For the observation x
mij(ωi) = αij exp[−γijd2(x,xi)]mij(Ω) = 1− αij exp[−γijd2(x,xi)]
I 0 ≤ αij ≤ 1: discounting coefficient and γij > 0, areparameters to play on the quantity of ignorance and on theform of the mass functions
I The distance allows to give a mass to x higher according tothe proximity to ωi
I belief k-nn: we consider the k-nearest neighbors instead to xiI Then we combine the bbas
Information fusion, A. Martin 22/02/18
21/36
Estimation (6/13)
-
Probabilistic based modelFusionBeliefApplications
I 2 models proposed by Appriou according to three axioms:
1. the n ∗ s couples [mji , αij ] are distinct information sourceswhere focal elements are: ωi, ωi and Ω
2. If mji (ωi) = 0 and the information is valid (αij = 1) then it iscertain that ωi is not true.
3. Conformity to the Bayesian approach (case where p(Sj |ωj) isexactly the reality (αij = 1) for all i, j) and all the a prioriprobabilities p(ωi) are known)
I Need to estimate p(Sj |ωi)
Information fusion, A. Martin 22/02/18
22/36
Estimation (7/13)
-
Probabilistic based modelFusionBeliefApplications
Model 1:
mij(ωi) =αijRjp(Sj |ωj)1 +Rjp(Sj |ωj)
mij(ωi) =αij
1 +Rjp(Sj |ωj)mij(Ω) = 1− αij
with Rj ≥ 0 a normalization factor.Model 2:
mij(ωi) = 0
mij(ωi) = αij(1−Rjp(Sj |ωj))mij(Ω) = 1− αij(1−Rjp(Sj |ωj)
with Rj ∈ [0, (maxSj ,i(p(Sj |ωj)))−1]
Information fusion, A. Martin 22/02/18
23/36
Estimation (8/13)
-
Probabilistic vs DistanceFusionBeliefApplications
Difficulties:
I Appriou: learning the probabilities p(Sj |ωj)I Denœux: choice of the distance d(x,xi)
Easiness:
I p(Sj |ωj) easier to estimate on decisions with the confusionmatrix of the classifiers
I d(x,xi) easier to choose on the numeric outputs of classifiers(ex.: Euclidean distance)
Information fusion, A. Martin 22/02/18
24/36
Estimation (9/13)
-
Conjunctive rulesFusionBeliefApplications
I Assume: two cognitively independent and reliable sources S1and S2.
I The conjunctive rule is given for m1 and m2 bbas of S1 andS2, for all X ∈ 2Ω, with X 6= ∅ by:
mConj(X) =∑
Y1∩Y2=Xm1(Y1)m2(Y2) (3)
∅ ω1 ω2 ω3 Ωm1 0 0.5 0.1 0 0.4
m2 0 0.2 0 0.5 0.3
m 0.32 0.33 0.03 0.2 0.12
Information fusion, A. Martin 22/02/18
25/36
Combination (10/13)
-
Dempster’s ruleFusionBeliefApplications
I Dempster’s rule:
mD(X) =1
1− κmConj(X) (4)
where κ =∑
A∩B=∅
m1(A)m2(B) is generally called conflict or
global conflict. That is the sum of the partial conflicts.
I That is not a conflict measure (Liu, 2006).
I Conjunctive rules are not idempotent
Information fusion, A. Martin 22/02/18
26/36
Combination (11/13)
-
Decision on ΩFusionBeliefApplications
I In general the decision is made on Ω and not on 2Ω
I Pessimist: maxω∈Ω bel(ω)I Optimist: maxω∈Ω pl(ω)I Compromise: maxω∈Ω betP (ω)
Pignistic probability:
betP(ω) =∑
Y ∈2Ω,ω∩Y 6=∅
1
|Y |m(Y )
1−m(∅)(5)
Information fusion, A. Martin 22/02/18
27/36
Decision (12/13)
-
Decision on the unionsFusionBeliefApplications
I Decision on 2Ω
I The decision functions fd (belief, plausibility, pignisticprobability, etc.) increase by inclusion (Appriou, 2014)
A = argmaxX∈2Ω
(md(X)fd(X)) ,
where
md(X) =
(KdλX|X|r
)with r ∈ [0, 1], is a weighted factor of the wanted precision ofthe decision:r = 1: singleton,r = 0: ignorance
Information fusion, A. Martin 22/02/18
28/36
Decision (13/13)
-
OutlineFusionBeliefApplications
1. Introduction to information fusion
2. Theory of belief functions for information fusion
3. Applications
Information fusion, A. Martin 22/02/18
29/36
-
Druid scientific challenges and goalsFusionBeliefApplications
Goals
1. To propose well-founded models for social networks andcrowdsourcing management
2. To develop theories for user qualification, in terms ofreliability, data provenance, skills, motivation, independence
3. To implement scalable systems that are proof-of-concepts ofthese models
Information fusion, A. Martin 22/02/18
30/36
(1/6)
-
Druid scientific challenges and goalsFusionBeliefApplications
Goals
1. To propose well-founded models for social networks andcrowdsourcing management
Example
I Declarative, graph-based languages for social network analyticsI Declarative, data oriented workflows for complex crowdsourcing
2. To develop theories for user qualification, in terms ofreliability, data provenance, skills, motivation, independence
3. To implement scalable systems that are proof-of-concepts ofthese models
Information fusion, A. Martin 22/02/18
30/36
(1/6)
-
Druid scientific challenges and goalsFusionBeliefApplications
Goals
1. To propose well-founded models for social networks andcrowdsourcing management
2. To develop theories for user qualification, in terms ofreliability, data provenance, skills, motivation, independence
3. To implement scalable systems that are proof-of-concepts ofthese models
Information fusion, A. Martin 22/02/18
30/36
(1/6)
-
Druid scientific challenges and goalsFusionBeliefApplications
Goals
1. To propose well-founded models for social networks andcrowdsourcing management
2. To develop theories for user qualification, in terms ofreliability, data provenance, skills, motivation, independence
Example
I Crowdsourcing quality optimization: consider the right usersI Information fusion from disagreeing users
3. To implement scalable systems that are proof-of-concepts ofthese models
Information fusion, A. Martin 22/02/18
30/36
(1/6)
-
Druid scientific challenges and goalsFusionBeliefApplications
Goals
1. To propose well-founded models for social networks andcrowdsourcing management
2. To develop theories for user qualification, in terms ofreliability, data provenance, skills, motivation, independence
3. To implement scalable systems that are proof-of-concepts ofthese models
Information fusion, A. Martin 22/02/18
30/36
(1/6)
-
Druid scientific challenges and goalsFusionBeliefApplications
Goals
1. To propose well-founded models for social networks andcrowdsourcing management
2. To develop theories for user qualification, in terms ofreliability, data provenance, skills, motivation, independence
3. To implement scalable systems that are proof-of-concepts ofthese models
Example
I Real-time visualization of social network analyticsI Optimized execution platforms for the crowd
Information fusion, A. Martin 22/02/18
30/36
(1/6)
-
Druid scientific foundationsFusionBeliefApplications
Users qualification through information fusion
I Combining of information coming from many users
I Taking into account reliability of users
I Managing the conflict
I Modeling of user information
I Computing user qualification
Based on uncertainty theories
Information fusion, A. Martin 22/02/18
31/36
(2/6)
-
Example 1: Influencers identificationFusionBeliefApplications
Problem: Given a social network, find a set of influencers that areable to trigger a large cascade.
Information fusion, A. Martin 22/02/18
32/36
(3/6)
-
Example 1: Influencers identificationFusionBeliefApplications
Problem: Given a social network, find a set of influencers that areable to trigger a large cascade.
Information fusion, A. Martin 22/02/18
32/36
(3/6)
-
Example 1: Influencers identificationFusionBeliefApplications
Problem: Given a social network, find a set of influencers that areable to trigger a large cascade.
Information fusion, A. Martin 22/02/18
32/36
(3/6)
-
Example 1: Influencers identificationFusionBeliefApplications
Solution: Influencers on Twitter (Jendoubi, et al, 2016, 2017)I Define an influence measure based on belief functions by:
I Ω = {I, P} I for influencer, P for passiveI Calculate belief weights on each edge (u, v)I Integrate opinion of tweetI Combine the mass functions
I Compute influence maximisation by CELF algorithm(Leskovec et al. 2007)
Information fusion, A. Martin 22/02/18
33/36
(4/6)
-
Example 1: Influencers identificationFusionBeliefApplications
Solution: Influencers on Twitter (Jendoubi, et al, 2016, 2017)I Define an influence measure based on belief functions by:
I Ω = {I, P} I for influencer, P for passiveI Calculate belief weights on each edge (u, v)
from numbers of common neighbours, number of tweets whereu mentions v, number of tweets where v retweets from u
I Integrate opinion of tweetI Combine the mass functions
I Compute influence maximisation by CELF algorithm(Leskovec et al. 2007)
Information fusion, A. Martin 22/02/18
33/36
(4/6)
-
Example 1: Influencers identificationFusionBeliefApplications
Solution: Influencers on Twitter (Jendoubi, et al, 2016, 2017)I Define an influence measure based on belief functions by:
I Ω = {I, P} I for influencer, P for passiveI Calculate belief weights on each edge (u, v)I Integrate opinion of tweet
I Give a label to each word in the tweet using Stanford POSTagger with the model GATE Twitter part-of-speech tagger,
I Use the SentiWordNet dictionary to get the polarity of eachword in the tweet
I Build a belief function on Θ = {Pos,Neg,Neut}I Combine the mass functions
I Compute influence maximisation by CELF algorithm(Leskovec et al. 2007)
Information fusion, A. Martin 22/02/18
33/36
(4/6)
-
Example 2: Community detection: SELPFusionBeliefApplications
Initialize the bba of each node in the network
Select a node nx in VU, find its rx direct neighbors and construct rx bbas
Calculate the fused bba of node nx
Output the bba of each node
Input the labeled nodes in the graph
the maximum of mass assignment of nx is larger than a given threshold
Yes
No
move node nx from set VU to set VL
There is no node in VU
Yes
No
Semi-supervised Evidential Label Propagation algorithm (Zhou et al., 2018)
Information fusion, A. Martin 22/02/18
34/36
(5/6)
-
Example 2: Community detection: SELPFusionBeliefApplications
Example on Karate Club network
1
2
3
4
5
6
78
9
10
11
12
13
14
15
16
17
18
19
20
2122
23
24
25
26
27
28
29
30
31
32
33
34
Initialization
●
●
Labeled data in ω1Labeled data in ω2Labeled as noisy dataunlabeled data
Information fusion, A. Martin 22/02/18
35/36
(6/6)
-
Example 2: Community detection: SELPFusionBeliefApplications
Example on Karate Club network
1
2
3
4
5
6
78
9
10
11
12
13
14
15
16
17
18
19
20
2122
23
24
25
26
27
28
29
30
31
32
33
34
Iteration 1
●
●
Labeled data in ω1Labeled data in ω2Labeled as noisy dataunlabeled data
Information fusion, A. Martin 22/02/18
35/36
(6/6)
-
Example 2: Community detection: SELPFusionBeliefApplications
Example on Karate Club network
1
2
3
4
5
6
78
9
10
11
12
13
14
15
16
17
18
19
20
2122
23
24
25
26
27
28
29
30
31
32
33
34
Iteration 2
●
●
Labeled data in ω1Labeled data in ω2Labeled as noisy dataunlabeled data
Information fusion, A. Martin 22/02/18
35/36
(6/6)
-
Example 2: Community detection: SELPFusionBeliefApplications
Example on Karate Club network
1
2
3
4
5
6
78
9
10
11
12
13
14
15
16
17
18
19
20
2122
23
24
25
26
27
28
29
30
31
32
33
34
Iteration 3
●
●
Labeled data in ω1Labeled data in ω2Labeled as noisy dataunlabeled data
Information fusion, A. Martin 22/02/18
35/36
(6/6)
-
Example 2: Community detection: SELPFusionBeliefApplications
Example on Karate Club network
1
2
3
4
5
6
78
9
10
11
12
13
14
15
16
17
18
19
20
2122
23
24
25
26
27
28
29
30
31
32
33
34
Iteration 4
●
●
Labeled data in ω1Labeled data in ω2Labeled as noisy dataunlabeled data
Information fusion, A. Martin 22/02/18
35/36
(6/6)
-
Example 2: Community detection: SELPFusionBeliefApplications
Example on Karate Club network
1
2
3
4
5
6
78
9
10
11
12
13
14
15
16
17
18
19
20
2122
23
24
25
26
27
28
29
30
31
32
33
34
Iteration 5
●
●
Labeled data in ω1Labeled data in ω2Labeled as noisy dataunlabeled data
Information fusion, A. Martin 22/02/18
35/36
(6/6)
-
To endFusionBeliefApplications
The theory of belief functions provides an appropriate frameworkfor information fusion.
Druid team
I Social networks
I Crowdsourcing applications
I Sensor fusion
http://www-druid.irisa.fr/
Information fusion, A. Martin 22/02/18
36/36
FusionBeliefApplications