The irresistible inequality of Milne
Adam [email protected]
Department of Applied Analysis and Computational Mathematics,Eotvos Lorand University, Budapest
CIA2016Hajduszoboszlo, September 1, 2016
An inspirational quotation
Vladimir Arnold(1937–2010)
“Mathematics is a part of physics. Physics is an experimentalscience, a part of natural science. Mathematics is the part ofphysics where experiments are cheap.”
On teaching mathematics (Paris, March 7, 1997)
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 1 / 17
A brief lesson on circuits
Resistors connected in series:
I
U1
I
U2
I I
Un
IR1 R2 Rn
+ −
U
• Kirchhoff’s voltage law (Konigsberg, 1845):
U = U1 + U2 + . . . + Un,
• Ohm’s law (1827): current = voltage / resistance
=⇒ RtotalI = R1I + R2I + . . . + RnI
=⇒ Rtotal = R1 + R2 + . . . + Rn
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 2 / 17
A brief lesson on circuits
Resistors connected in parallel:
−
+U
I
I I1
R1
I1
U1
I2
R2
I2
U2
In
Rn
In
Un
• Kirchhoff’s current law (Konigsberg, 1845):
I = I1 + I2 + · · ·+ In.
• Ohm’s law (1827): current = voltage / resistance
=⇒ U
Rtotal= U
R1+ U
R2+ · · ·+ U
Rn
=⇒ 1Rtotal
= 1R1
+ 1R2
+ · · ·+ 1Rn
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 3 / 17
A brief lesson on circuits
Rayleigh’s monotonicity principle:
resistance of any part increases =⇒ total resistance does not decrease
• John William Strutt, 3rd Baron Rayleigh (1842–1919):On the Theory of Resonance (1871), On the Theory of Sound (1877)
• James Clerk Maxwell (1831–1879):A Treatise on Electricity and Magnetism (1873)
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 4 / 17
Circuits and means
Monotonicity Principle =⇒ Arithmetic Mean ≥ Harmonic Mean
R1
R2
R2
R1
Ropentotal = 1
1R1+R2
+ 1R1+R2
= R1 + R22
R1
R2
R2
R1⇐⇒
R1
R2 R1
R2
Rclosedtotal = 1
1R1
+ 1R2
+ 11
R1+ 1
R2
= 21
R1+ 1
R2
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 5 / 17
Circuits and means
Monotonicity Principle =⇒ Arithmetic Mean ≥ Harmonic Mean
R1
R2
R2
R1
Ropentotal = 1
1R1+R2
+ 1R1+R2
= R1 + R22
R1
R2
R2
R1⇐⇒
R1
R2 R1
R2
Rclosedtotal = 1
1R1
+ 1R2
+ 11
R1+ 1
R2
= 21
R1+ 1
R2
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 5 / 17
Circuits and means
Monotonicity Principle =⇒ Arithmetic Mean ≥ Harmonic Mean
R1
R2
R2
R1
Ropentotal = 1
1R1+R2
+ 1R1+R2
= R1 + R22
R1
R2
R2
R1⇐⇒
R1
R2 R1
R2
Rclosedtotal = 1
1R1
+ 1R2
+ 11
R1+ 1
R2
= 21
R1+ 1
R2
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 5 / 17
Circuits and means
Monotonicity Principle =⇒ Arithmetic Mean ≥ Harmonic Mean
R1
R2
R2
R1
Ropentotal = 1
1R1+R2
+ 1R1+R2
= R1 + R22
R1
R2
R2
R1
⇐⇒
R1
R2 R1
R2
Rclosedtotal = 1
1R1
+ 1R2
+ 11
R1+ 1
R2
= 21
R1+ 1
R2
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 5 / 17
Circuits and means
Monotonicity Principle =⇒ Arithmetic Mean ≥ Harmonic Mean
R1
R2
R2
R1
Ropentotal = 1
1R1+R2
+ 1R1+R2
= R1 + R22
R1
R2
R2
R1⇐⇒
R1
R2 R1
R2
Rclosedtotal = 1
1R1
+ 1R2
+ 11
R1+ 1
R2
= 21
R1+ 1
R2
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 5 / 17
Circuits and means
Monotonicity Principle =⇒ Arithmetic Mean ≥ Harmonic Mean
R1
R2
R2
R1
Ropentotal = 1
1R1+R2
+ 1R1+R2
= R1 + R22
R1
R2
R2
R1⇐⇒
R1
R2 R1
R2
Rclosedtotal = 1
1R1
+ 1R2
+ 11
R1+ 1
R2
= 21
R1+ 1
R2
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 5 / 17
Circuits and means
Monotonicity Principle =⇒ Arithmetic Mean ≥ Harmonic Mean
R1
R2
R2
R1
Ropentotal = 1
1R1+R2
+ 1R1+R2
= R1 + R22
R1
R2
R2
R1⇐⇒
R1
R2 R1
R2
Rclosedtotal = 1
1R1
+ 1R2
+ 11
R1+ 1
R2
= 21
R1+ 1
R2
>
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 5 / 17
Circuits and means
Monotonicity Principle =⇒ Arithmetic ≥ Geometric ≥ Harmonic
R3
R1
R2
R2
R1
R3 =∞ =⇒ Ropentotal = R1 + R2
2
R3 =√
R1R2 =⇒ Rbetweentotal =
√R1R2
R3 = 0 =⇒ Rclosedtotal = 2
1R1
+ 1R2
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 6 / 17
Circuits and means
Monotonicity Principle =⇒ Arithmetic ≥ Geometric ≥ Harmonic
R3
R1
R2
R2
R1
R3 =∞ =⇒ Ropentotal = R1 + R2
2
R3 =√
R1R2 =⇒ Rbetweentotal =
√R1R2
>
R3 = 0 =⇒ Rclosedtotal = 2
1R1
+ 1R2
>
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 6 / 17
Circuits and inequalities
A generalization:a1
b1
a2
b2
a3
b3
an
bn
Ropentotal = (a1 + . . . + an)(b1 + . . . + bn)
a1 + . . . + an + b1 + . . . + bn
a1
b1
a2
b2
a3
b3
an
bn
Rclosedtotal = a1b1
a1 + b1+ . . . + anbn
an + bn
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 7 / 17
Circuits and inequalities
A generalization:a1
b1
a2
b2
a3
b3
an
bn
Ropentotal = (a1 + . . . + an)(b1 + . . . + bn)
a1 + . . . + an + b1 + . . . + bn
a1
b1
a2
b2
a3
b3
an
bn
Rclosedtotal = a1b1
a1 + b1+ . . . + anbn
an + bn
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 7 / 17
Circuits and inequalities
A generalization:a1
b1
a2
b2
a3
b3
an
bn
Ropentotal = (a1 + . . . + an)(b1 + . . . + bn)
a1 + . . . + an + b1 + . . . + bn
a1
b1
a2
b2
a3
b3
an
bn
Rclosedtotal = a1b1
a1 + b1+ . . . + anbn
an + bn
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 7 / 17
Circuits and inequalities
A generalization:a1
b1
a2
b2
a3
b3
an
bn
Ropentotal = (a1 + . . . + an)(b1 + . . . + bn)
a1 + . . . + an + b1 + . . . + bn
a1
b1
a2
b2
a3
b3
an
bn
Rclosedtotal = a1b1
a1 + b1+ . . . + anbn
an + bn
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 7 / 17
Circuits and inequalities
A generalization:a1
b1
a2
b2
a3
b3
an
bn
Ropentotal = (a1 + . . . + an)(b1 + . . . + bn)
a1 + . . . + an + b1 + . . . + bn
a1
b1
a2
b2
a3
b3
an
bn
Rclosedtotal = a1b1
a1 + b1+ . . . + anbn
an + bn
>
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 7 / 17
Let us switch to Mathematics
Theorem (Milne’s inequality)
ai, bi > 0 (i = 1, . . . , n) =⇒n∑
i=1
aibi
ai + bi≤
n∑i=1
ai ·n∑
i=1bi
n∑i=1
(ai + bi)
Proof:n∑
i=1ai ·
n∑i=1
bi −n∑
i=1(ai + bi) ·
n∑i=1
aibi
ai + bi=
∑1≤i<j≤n
(aibj − ajbi)2
(ai + bi)(aj + bj)
Edward Arthur Milne (1896–1950):
• British astrophysicist and mathematician
• in 1925 he proved the integral version of the inequality
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 8 / 17
Let us switch to Mathematics
Theorem (Milne’s inequality)
ai, bi > 0 (i = 1, . . . , n) =⇒n∑
i=1
aibi
ai + bi≤
n∑i=1
ai ·n∑
i=1bi
n∑i=1
(ai + bi)
Proof:n∑
i=1ai ·
n∑i=1
bi −n∑
i=1(ai + bi) ·
n∑i=1
aibi
ai + bi=
∑1≤i<j≤n
(aibj − ajbi)2
(ai + bi)(aj + bj)
Edward Arthur Milne (1896–1950):
• British astrophysicist and mathematician
• in 1925 he proved the integral version of the inequality
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 8 / 17
Let us switch to Mathematics
Theorem (Milne’s inequality)
ai, bi > 0 (i = 1, . . . , n) =⇒n∑
i=1
aibi
ai + bi≤
n∑i=1
ai ·n∑
i=1bi
n∑i=1
(ai + bi)
Proof:n∑
i=1ai ·
n∑i=1
bi −n∑
i=1(ai + bi) ·
n∑i=1
aibi
ai + bi=
∑1≤i<j≤n
(aibj − ajbi)2
(ai + bi)(aj + bj)
Edward Arthur Milne (1896–1950):
• British astrophysicist and mathematician
• in 1925 he proved the integral version of the inequality
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 8 / 17
Circuits and inequalities
Brief history:
• Alfred Lehman (1931–2006): SIAM Review Problem Section, 1960.nonlinear Ohm’s law U = R · I−1/p =⇒ Minkowski’s inequality
• William Niles Anderson, Richard James Duffin (1909–1996)Series and parallel addition of matrices, 1968.
• appeared in many books, popular papers:
– P. G. Doyle and J. L. Snell’s book: Random Walks and ElectricNetworks, 1984.
– Mark Levi’s book: The Mathematical Mechanic: Using PhysicalReasoning to Solve Problems, 2009.
– Alfred Witkowski’s paper in the Mathematics Magazine, 2014.– Adam Besenyei’s paper in the Hungarian Mathematical Journal
for Secondary Schools (KoMaL), 2016.
• Zoltan Bertalan: A ≥ G ≥ H, KoMaL, Problem P. 4813., 2016.
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 9 / 17
Entropy
Physics
• Rudolph Clausius (1822–1888) in 1865 in classical thermodynamics
• Josiah Willard Gibbs (1839–1903), Ludwig Boltzmann (1844–1906)
Information Theory
• Claude Shannon (1916–2001) in 1948 in his paper at Bell Telephone:A Mathematical Theory of Communication
Definition (Shannon entropy)For a discrete random variable X with possible values of outcome{x1, . . . , xn} and probability distribution p(x) = P (X = x):
H(X) = −n∑
i=1p(xi) log p(xi)
with the convention 0 log 0 = 0. (The letter H is the Greek capital eta.)
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 10 / 17
Entropy
What is entropy?
• Greek: “a turning towards”
• measures the “uncertainty” of the random variable
• John von Neumann to Shannon in 1949:
“You should call it entropy, for two reasons. In the first placeyour uncertainty function has been used in statistical mechanicsunder that name, so it already has a name. In the second place,and more important, nobody knows what entropy really is, so ina debate you will always have the advantage.”
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 11 / 17
Subadditivity of Shannon entropy
Joint entropy: for a pair (X, Y ) of discrete random variables
H(X, Y ) = −n∑
i=1
m∑j=1
p(xi, yj) log p(xi, yj).
Theorem1. Subadditivity of entropy:
H(X, Y ) ≤ H(X) + H(Y ).(Joint entropy not greater than the sum of the individual entropies.)
2. Strong subadditivity of entropy:H(X, Y, Z) + H(Y ) ≤ H(X, Y ) + H(Y, Z).
(Conditional entropies, X = “past”, Y = “present”, Z = “future”.)
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 12 / 17
q-entropy
DefinitionFor a discrete random variable X and q 6= 1 the q-entropy is defined as
Sq(X) = −n∑
i=1p(xi) logq p(xi) = 1−
∑ni=1 p(xi)q
q − 1 ,
where the q-logarithm function is
logq x = xq−1 − 1q − 1 (q 6= 1).
History:
• Zoltan Daroczy (1938–):Generalized information functions, 1970.
• Constantino Tsallis (1943–):A possible generalization of Boltzmann–Gibbs statistics, 1988.
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 13 / 17
Strong subadditivity of Tsallis entropy
Theorem (Daroczy, 1970)If q > 1, then the q-entropy is subadditive:
Sq(X, Y ) ≤ Sq(X) + Sq(Y ).Equality holds if and only if q = 1 and X, Y are independent.
A special case:
(x+y)q +(z+v)q +(x+z)q +(y+v)q ≤ xq +yq +zq +vq +(x+y+z+v)q.
A similar inequality: proposed in Amer. Math. Monthly, 2014.
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 14 / 17
Strong subadditivity of Tsallis entropy
Theorem (Shigeru Furuichi, 2004)If q > 1, then the q-entropy is strongly subadditive:
Sq(X, Y, Z) + Sq(Y ) ≤ Sq(X, Y ) + Sq(Y, Z).
Reformulation: notation pijk := p(xi, yj , zk), p−j− = p(yj) etc.r∑
k=1
m∑j=1
n∑i=1
pijk
(logq pijk + logq p−j− − logq pij− − logq p−jk
)≥ 0.
Theorem (B.–Petz, 2013)For q > 1 and fixed 1 ≤ j ≤ n, 1 ≤ k ≤ r the following inequality holds:
n∑i=1
pijk
(logq pijk + logq p−j− − logq pij− − logq p−jk
)≥ 0.
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 15 / 17
Partial strong subadditivity of Tsallis entropy
Proof: The inequality is equivalent ton∑
i=1pijk
(pq−1
ij− − pq−1ijk
)≤ p−jk
(pq−1
−j− − pq−1−jk
).
With the notation ai = pijk, bi = pij− − pijk (i = 1, . . . , n) andA =
∑ni=1 ai = p−jk, B =
∑ni=1 bi = p−j− it reduces to
n∑i=1
ai
((ai + bi)q−1 − aq−1
i
)≤ A
((A + B)q−1 −Aq−1
),
or equivalently
(q − 1)∫ 1
0
n∑i=1
aibi(ai + tbi)q−2 dt ≤ (q − 1)∫ 1
0AB(A + tB)q−2 dt,
where by Milne’s inequalityn∑
i=1aitbi(ai+tbi)q−1 ≤
n∑i=1
ai · tbi
ai + tbi(A+tB)(A+tB)q−2 ≤ tAB(A+tB)q−2.
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 16 / 17
References
Adam Besenyei, Denes Petz, Partial subadditivity of entropies, LinearAlgebra Appl., 439 (2013), 3297–3305.
Z. Daroczi, General information functions, Information and Control,16 (1970), 36–51.
S. Furuichi, Information theoretical properties of Tsallis entropies,J.Math.Phys., 47 (2006), 023302.
E. A. Milne, Note on Rosseland’s integral for the stellar absorptioncoefficient, Mon. Not. R. Astron. Soc., 85 (1925) 979–984. http://mnras.oxfordjournals.org/content/85/9/979.full.pdf
A. Witkowski, Proof Without Words: An Electrical Proof of theAM-HM Inequality, Math. Mag., 87 (2014), 275.http://www.jstor.org/stable/10.4169/math.mag.87.4.275
Adam BESENYEI (ELTE) Milne’s inequality CIA2016, August, 2016 17 / 17
Thank you for your attention!