A short note on soft-plus polynomials

Tomonari MASADA @ Nagasaki University

February 3, 2016

Consider a set V of two binary variables {x1, x2} and the function of the form

ϕ : {0, 1}V → R;x 7→ log(1 + exp(w⊤x+ c)) . (1)

As is explained in [1], this corresponds to the free energy added by one hidden binary variable interactingpairwise with each of the two visible binary variable x1 and x2.

All possible values obtained by ϕ are

log(1 + exp(c)) for (x1, x2) = (0, 0),

log(1 + exp(w1 + c)) for (x1, x2) = (1, 0),

log(1 + exp(w2 + c)) for (x1, x2) = (0, 1), and

log(1 + exp(w1 + w2 + c)) for (x1, x2) = (1, 1). (2)

Therefore, the function ϕ can be rewritten as follows based on the fact that x1 and x2 are binary:

ϕ(x1, x2) = log(1 + exp(c))(1− x1)(1− x2)

+ log(1 + exp(w1 + c))x1(1− x2)

+ log(1 + exp(w2 + c))(1− x1)x2

+ log(1 + exp(w1 + w2 + c))x1x2 . (3)

That is, ϕ is a polynomial. The coefficients of the monomials can be given explicitly as below:

ϕ(x1, x2) ={log(1 + exp(c))− log(1 + exp(w1 + c))− log(1 + exp(w2 + c)) + log(1 + exp(w1 + w2 + c))}x1x2

+ {− log(1 + exp(c)) + log(1 + exp(w1 + c))}x1

+ {− log(1 + exp(c)) + log(1 + exp(w2 + c))}x2

+ log(1 + exp(c)) . (4)

The following formula gives the coefficients:

KB(w, c) =∑C⊆B

(−1)|B\C| log

(1 + exp

(∑i∈C

wi + c))

, B ∈ 2V . (5)

For example, when B = {x1, x2},

K{x1,x2}(w, c) =(−1)|{x1,x2}\∅| log

(1 + exp

(∑i∈∅

wi + c))

+ (−1)|{x1,x2}\{x1}| log

(1 + exp

( ∑i∈{x1}

wi + c))

+ (−1)|{x1,x2}\{x2}| log

(1 + exp

( ∑i∈{x2}

wi + c))

+ (−1)|{x1,x2}\{x1,x2}| log

(1 + exp

( ∑i∈{x1,x2}

wi + c))

= log(1 + exp(c))− log(1 + exp(w1 + c))− log(1 + exp(w2 + c)) + log(1 + exp(w1 + w2 + c)) .(6)

A similar discussion can be made for the case where we have more than two binary variables. See [1].

1

References

[1] Guido Montufar, Johannes Rauh. Hierarchical Models as Marginals of Hierarchical Models. 2015.arXiv:1508.03606.

2