Supplementary Materials for Bounded Memory Folk

Theorem

Mehmet Barlo

Sabancı University

Guilherme Carmona

University of Surrey

Hamid Sabourian

University of Cambridge

September 17, 2015

This document presents supplementary materials for our paper “Bounded Memory Folk

Theorem”. While the first section concerns those related to the bounded memory Folk

Theorem with pure strategies, the second involves those associated with the one employing

mixed (behavioral) strategies.

A Theorem 1 (pure Folk Theorem)

A.1 Auxiliary results

Claims A.1 to A.15 establish results needed to show that the strategy given in (28) is well-

defined and subgame perfect.

The first two claims establish some properties of the paths π(0), . . . , π(n).

Claim A.1 For all i ∈ {0, . . . , n} and d, d′ ∈ N with d �= d′:

−B(1− δ(n+5)(T+1)) + δ(n+5)(T+1)Vd(π(d′)) > B(1− δn+6) + δn+6+TVd(π

(d)), (A.1)

−B(1− δK) + δKVd(π(i)) > (1− δn+6)B + δn+6+TVd(π

(d)), (A.2)

−(1− δn+5)B + δn+5+(n+4)TVd(π(d)) > (1− δn+5)B + δ(n+5)(T+1)Vd(π

(d)). (A.3)

Proof. First, by (25) and the definitions of {xj}nj=0, {yj}nj=0, {π(j)}nj=0, ξ, ζ, and ζ ′, we have

1

the following two conditions:

Vd(π(d)) > xd

d − ξ > ydd − 2ξ > ζ, (A.4)

Vd(π(d′)) > xd′

d − ξ > yd′

d − 2ξ > ydd + ζ ′ − 2ξ > Vd(π(d)) + ζ ′ − 4ξ = Vd(π

(d)) + ζ.(A.5)

By (24), we have δ(n+5)(T+1)ζ > B(2 − 2δ(n+5)(T+1) − δn+6(1 − δT )). Then (A.1) follows

immediately from (A.5).

Consider inequality (A.2). Since K < T , (A.1) implies (A.2) when d, i ∈ N and d �= i.

Therefore, to demonstrate (A.2), it suffices to consider two cases: i = 0 and d = i ∈ N .

By (A.5) and (24), we have (Vd(π(0)) − Vd(π

(d)))δK > B(2 − δK − δn+6). This together

with K < T and (A.4) implies that (A.2) holds when i = 0.

By (A.4) and (24), Vd(π(d))(δK − δn+6+T ) > ζ(δK − δn+6+T ) > B(2 − δK − δn+6). This

implies that (A.2) holds when d = i ∈ N .

Finally, consider (A.3). By (A.4) and (24), Vd(π(d))δn+5+(n+4)T (1−δT ) > ζδn+5+(n+4)T (1−

δT ) > 2B(1− δn+5). This implies that (A.3) holds.

Claim A.2 For all i ∈ {0, . . . , n}, the following hold:

1. All actions a �= s′ are played for t ≥ n+ i+ 2 consecutive periods in π(i).

2. Suppose that, for some t ∈ N, (π(i),t, . . . , π(i),t+l+1) = ((s′; 2), (s; l)) and l > 0. Then

either π(i),t+l+2 = s or π(i),t+l+2 = s′ and l = n+ i+ 2.

3. Suppose that, for some t ∈ N, π(i),t ∈ D(s′) and π(i),t+1 ∈ D(s). Then π(i),t = s′ and

π(i),t+1 = s. Furthermore, either (i) π(i),t−1 = s′ and t = 2 + βK or (ii) π(i),t−1 = s

and t = n+ i+ 5 + βK for some β ∈ N0.

4. Suppose that, for some t ∈ N, π(i),t ∈ D(s) and π(i),t+1 ∈ D(s′). Then π(i),t+1 = s′.

Proof. This follows immediately from the ordering of A and the definition of π(i).

Claim A.3 If h ∈ H(i),k1,a for some i ∈ {0, . . . , n} and k ∈ {n+ i+ 5, . . . ,M}, then T k′(h) �∈

Σd,k′ for all d ∈ N and k′ ∈ N such that d+ 4 ≤ k′ ≤ k.

Proof. Suppose otherwise; then there exists h ∈ H(i),k1,a such that T k′(h) ∈ Σd,k′ for some

d ∈ N and k′ ∈ N with d + 4 ≤ k′ ≤ k. Let T k(h) = (a1, . . . , ak). Since T k′(h) ∈ Σd,k′ , we

have ak−k′+2 ∈ D(s′), ak−k′+3 ∈ D(s) and ak−k′+d+4 ∈ D(s′). Therefore, there is an action,

namely ak−k′+3, which is different from s′ and is played at most d+1 consecutive periods in

T k(h). Since h ∈ H(i),k1,a , this contradicts Claim A.2.1.

2

Next, for all τ ∈ N and i ∈ {0, . . . , n}, define

Λi,τ ={h ∈ H : h = (at)τt=1 such that at = π(i),t if t ≤ n+ i+ 5

}.

Claim A.4 If h ∈ H(i),k1,a for some i ∈ {0, . . . , n} and k ∈ {n + i + 5, . . . ,M} and T k′(h) ∈

Λi′,k′ for some i′ ∈ {0, . . . , n} and n+ i′ + 5 ≤ k′ ≤ k, then i = i′ and k = k′ + βK for some

β ∈ N0.

Proof. Since h ∈ H(i),k1,a and T k′(h) ∈ Λi′,k′ , (π(i),t, . . . , π(i),t+n+i′+4) = ((s′; 2), (s;n+i′+2), s′)

where t = k + 1 − k′. It then follows from Claim A.2.2 that n + i′ + 2 = n + i + 2 and so

i = i′. Hence, by Claim A.2.3, t+ 1 = 2 + βK for some β ∈ N0 and so k = k′ + βK.

Claim A.5 Suppose h ∈ Hk,d3 ∪Hk,d,0

4 for some k ∈ {1, . . . ,M} and d ∈ N . Then T k′(h) �∈Λi,k′ ∪ Σd′,k′ for all i ∈ {0, . . . , n}, d′ ∈ N and k′ ∈ N such that 3 ≤ k′ < k.

Proof. Suppose otherwise; then T k′(h) ∈ Λi,k′ ∪ Σd′,k′ for some i ∈ {0, . . . , n}, d′ ∈ N and

k′ ∈ N such that 3 ≤ k′ < k. Let T k(h) = (a1, . . . , ak). Since T k′(h) ∈ Λi,k′ ∪ Σd′,k′ and

h ∈ Hk,d3 ∪ Hk,d,0

4 it follows, respectively, that ak−k′+1, ak−k′+2 ∈ Dd′(s′) and a3, . . . , ad+3 ∈

D(s). Consequently, k − k′ + 1 ≥ d + 4. This means that ak−k′+2−d = ak−k′+3

−d = md−d.

Also, by appealing again to T k′(h) ∈ Λi,k′ ∪ Σd′,k′ , one has ak−k′+3 ∈ Dd′(s). Therefore, by

ak−k′+2 ∈ Dd′(s′), we have that s−d,d′ = s′−d,d′ ; a contradiction.

Claim A.6 If h ∈ Hk,d,r4 for some k ∈ {1, . . . ,M} and d ∈ N and r ∈ {1, . . . , n + d + 4}

and T k′(h) ∈ Λi,k′ for some i ∈ {0, . . . , n} and 3 ≤ k′ < k, then k′ = r and k′ < n+ i+ 5.

Proof. Let T k(h) = (a1, . . . , ak). By T k′(h) ∈ Λi,k′ , ak−k′+1 = ak−k′+2 = s′ and, by

h ∈ Hk,d,r4 , at ∈ D(s) for all t ∈ {3, . . . , d + 3} ∪ {k − r + 3, . . . , k}. It then follows that

k − k′ + 1 ∈ {2, d+ 4, . . . , k − r + 2}.Note, however, that it cannot be that k−k′+1 = k− r+2; otherwise, ak−r+3 = s′. Also,

k − k′ + 1 = k − r − 1 is not possible because otherwise, by T k′(h) ∈ Λi,k′ , ak−k′+3 = s and,

by h ∈ Hk,d,r4 , ak−k′+3 = ak−r+1 = s′. Furthermore, k− k′+1 = k− r is not possible because

otherwise, by T k′(h) ∈ Λi,k′ , ak−k′+3 = s and, by h ∈ Hk,d,r4 , ak−r+2 = s′.

By Claim A.5, it also cannot be that k − k′ + 1 ≤ k − r − 2. The reasoning for this is

as follows. Since h ∈ Hk,d,r4 and T k′(h) ∈ Λi,k′ , it follows, respectively, that Br(h) ∈ Hk−r,d,0

4

and T k′−r(Br(h)) ∈ Λi,k′−r. But then by Claim A.5 it must be that k′ − r < 3.

Hence, it follows from all the above that k − k′ + 1 = k − r + 1, i.e. k′ = r.

Finally, k′ < n + i + 5 because otherwise, by T k′(h) ∈ Λi,k′ , ak−k′+n+i+5 = s′ and, by

h ∈ Hk,d,r4 , ak−r+n+i+5 = s.

3

Claim A.7 If h ∈ Hk,d,r4 for some k ∈ {1, . . . ,M} and d ∈ N and r ∈ {1, . . . , n + d + 4},

then T k′(h) �∈ Σd′,k′ for all d′ ∈ N and k′ ∈ N such that d′ + 4 ≤ k′ < k.

Proof. Suppose otherwise. Let T k(h) = (a1, . . . , ak). Since h ∈ Hk,d,r4 and T k′(h) ∈ Σd′,k′ ,

it follows, respectively, that Br(h) ∈ Hk−r,d,04 and T k′−r(Br(h)) ∈ Σd′,k′−r if k′ > r. But then

by Claim A.5 it must be that k′ − r < 3. Therefore, k − k′ + d′ + 4 > k − r + 2. Hence,

k−k′+d′+4 = k− r+ t for some t > 2. But this is a contradiction since, by T k′(h) ∈ Σd′,k′ ,

ak−k′+d′+4 ∈ D(s′) and, by h ∈ Hk,d,r4 , ak−r+t = s.

Claim A.8 If h ∈ Hk,d,τ5 for some k ∈ {1, . . . ,M} and d ∈ N and τ ∈ {0, . . . , d + 3}, then

T k′(h) �∈ Σd′,k′ for all d′ ∈ N and k′ ∈ N such that d′ + 4 ≤ k′ < k.

Proof. Suppose not. Then there exist h = (a1, . . . , at) ∈ Hk,d,τ5 such that T k′(h) ∈ Σd′,k′

for the parameters given in the statement of the claim. We now derive a contradiction by

considering six different possibilities. Before doing so, we let h = Bτ+1(h) and point out

that, by h ∈ Hk,d,τ5 , T k(h) ∈ Σi,k for some i ∈ N .

Case 1: k′ − (τ + 1) ≥ d′ + 4. Then, T k′−(τ+1)(h) ∈ Σd′,k′−(τ+1) and h ∈ Hk−(τ+1),i3 ∪(

∪n+i+4r=0 H

k−(τ+1),i,r4

). But this contradicts either Claim A.5 or Claim A.7.

Case 2: k′ − (τ + 1) < d′ + 1. Then, by T k′(h) ∈ Σd′,k′ , at−k′+d′+4 ∈ D(s′) and, by

h ∈ Hk,d,τ5 and t− k′ + d′ + 4 > t− τ + 2, at−k′+d′+4 ∈ D(s); a contradiction.

Case 3: k′ − (τ + 1) = d′ + 1. Then, by T k′(h) ∈ Σd′,k′ , at−k′+d′+3 ∈ D(s) and, by

h ∈ Hk,d,τ5 and t− k′ + d′ + 3 = t− τ + 1, at−k′+d′+3 ∈ D(s′); a contradiction.

Case 4: k′− (τ + 1) equals either d′ + 2 or d′ + 3 and h ∈ Hk−(τ+1),i3 ∪H

k−(τ+1),i,04 . Then

3 ≤ k′ − (τ + 1) < k − (τ + 1) and T k′−(τ+1)(h) ∈ Σd′,k′−(τ+1). This contradicts Claim A.5.

Case 5: k′ − (τ + 1) = d′ + 3 and h ∈ Hk−(τ+1),i,r4 with r > 0. By T k′(h) ∈ Σd′,k′ , (i)

at−k′+2 ∈ D(s′), (ii) at−k′+l ∈ D(s) for all 3 ≤ l ≤ d′ + 3 and (iii) at−k′+d′+4 ∈ D(s′). By

h ∈ Hk,d,τ5 and h ∈ H

k−(τ+1),i,r4 , (iv) at−τ ∈ Dd(π

(i),r+1) and (v) at−τ−r′ = π(i),r−r′+1 for all

1 ≤ r′ ≤ r.

By r > 0 and t−k′+d′+4 = t− τ , (iii) and (iv) imply that either r = 1 or r = n+ i+4.

If r = 1, by (v), at−τ−1 = π(i),1 = s′. This together with t− τ −1 = t−k′+d′+3 contradicts

(ii). If r = n+ i+ 4, then, by (v) and n+ i− d′ + 3 ≥ 3, at−τ−d′−2 = π(i),n+i−d′+3 = s. This

together with t− τ − d′ − 2 = t− k′ + 2 contradicts (i).

4

Case 6: k′ − (τ + 1) = d′ + 2 and h ∈ Hk−(τ+1),i,r4 with r > 0. Then, by (ii) and

t− τ = t− k′ + d′ + 3, at−τ ∈ Dd′(s). Therefore, by (iv), at−τ ∈ Dd(s) and d = d′.

Next, we show that r = d+2. If r > d+2, then, by (v), at−τ−d−1 = π(i),r−d = s. But this is

a contradiction because t−k′+2 = t−τ−d−1 and, by (i), at−k′+2 ∈ Dd(s′). If r < d+2, then,

by (ii), at−k′+d−r+4 ∈ D(s). But this is a contradiction because t−τ−r+1 = t−k′+d−r+4

and, by (v), at−τ−r+1 = π(i),2 = s′.

By h ∈ Hk−(τ+1),i,r4 and r = d+2, we have T d+2(h) = ((s′; 2), (s; d)). Since at−τ ∈ Dd(s),

it then follows from the definition ofHk,d,τ5 that τ = 0. But this contradicts k′−(τ+1) = d′+2

and k′ ≥ d′ + 4.

Claim A.9 If T k(h) ∈ Σd,k ∩ Σd′,k for some d, d′ ∈ N and k ∈ {min{d, d′} + 4, . . . ,M},then d = d′.

Proof. Suppose not; assume that d > d′. Let T k(h) = (a1, . . . , ak). Since T k(h) ∈ Σd,k,

ad′+4 ∈ Dd(s) and, by T k(h) ∈ Σd′,k, ad

′+4 ∈ Dd′(s′). But this is a contradiction.

Claim A.10 Let h ∈ H \∪5l=1Hl and a ∈ A. Then one of the following conditions hold: (a)

h · a �∈ ∪5l=1Hl, (b) h · a ∈ H

(i),n+i+51,a for some i ∈ {0, . . . , n} and (c) T d+4(h · a) ∈ Σd,d+4 for

some d ∈ N . Furthermore, if a ∈ D(s), then h · a �∈ ∪5l=1Hl.

Proof. Suppose that h · a does not satisfy (a)–(c). Then there are six cases to consider. (i)

h · a ∈ H(i),k1,a for some i and k > n+ i+ 5: then h ∈ H1,a; a contradiction. (ii) h · a ∈ H1,b:

then h ∈ H1,b; a contradiction. (iii) h · a ∈ H2: then h ∈ H1 ∪ H2; a contradiction. (iv)

h ·a ∈ H3 and T d+4(h ·a) �∈ Σd,d+4 for all d ∈ N : then h ∈ H3; a contradiction. (v) h ·a ∈ H4:

then h ∈ H3 ∪H4; a contradiction. (vi) h · a ∈ H5: then h ∈ H3 ∪H4 ∪H5; a contradiction.

Furthermore, if a ∈ D(s), by the definition of H(i),n+i+51,a and Σd,d+4, (b) and (c) cannot

hold. Therefore, (a) must hold, i.e. h · a �∈ ∪5l=1Hl.

Claim A.11 If h ∈ H(i),M1,a for some i ∈ {0, . . . , n}, then h ∈ H

(i),k1,a for some k < M .

Proof. Since h ∈ H(i),M1,a , TM(h) = (π(i),1, . . . , π(i),M). Also, by T > K and by the choice of

M , M > K + n+ i+ 5. Therefore, h ∈ H(i),M−K1,a .

Claim A.12 If h ∈ HM,d,τ2 for some d ∈ N and τ ∈ {0, . . . , d+3}, then h ∈ Hk,d,τ

2 for some

k < M .

5

Proof. Since h ∈ HM,d,τ2 , TM(h) = h · a · h where h, a, and h are as in the definition of

HM,d,τ2 . In particular, h ∈ H

(i),k′1,a ∪ H

(0),k′1,b for some i ∈ {0, . . . , n} and k′ ≤ M − (τ + 1).

Therefore, h ∈ Hk′+τ+1,d,τ2 .

If k′ + τ + 1 < M , the claim follows. If k′ + τ + 1 = M , then h = (π(i),1, . . . , π(i),M−(τ+1))

for some i ∈ {0, . . . , n}. Also, by T > K and by the choice of M , M ≥ (n+4)+(2n+5)+K.

Therefore, h ∈ H(i),M−(τ+1)−K1,a and so h ∈ HM−K,d,τ

2 .

Claim A.13 If h ∈ Hk,d3 for some k ∈ {1, . . . ,M} and d ∈ N , then k < M .

Proof. Let h and h be such that T k(h) = h · h and satisfy the conditions in the definition

of Hk,d3 . Then k = �(h) + �(h) < d+ 4 + (θ(h) + 1)T ≤ n+ 4 + (n+ 5)T < M .

Claim A.14 If h ∈ Hk,d,r4 for some k ∈ {1, . . . ,M} and d ∈ N and r ∈ {0, . . . , n+ d+ 4},

then k < M .

Proof. Let h, h and h be such that T k(h) = h · h · h and satisfy the conditions in the

definition of Hk,d,r4 . Then k ≤ �(h) + �(h) + �(h) = d + 4 + (θ(h) + 1)T + n + d + 4 ≤

n+ 4 + (n+ 5)T + 2n+ 4 < M .

Claim A.15 If h ∈ Hk,d,τ5 for some k ∈ {1, . . . ,M} and d ∈ N and τ ∈ {0, . . . , d+3}, then

k < M .

Proof. Let h, a and h be such that T k(h) = h·a·h and satisfy the conditions in the definition

of Hk,d,τ5 . By the proof of Claim A.13 and Claim A.14, �(h) ≤ n + 4 + (n + 5)T + 2n + 4.

Therefore, k = �(h) + �(a · h) ≤ [n+ 4 + (n+ 5)T + 2n+ 4] + 1 + d+ 3 < M .

A.2 f is well-defined

Now we show that f ∈ F p, described in (28), is well-defined.

Claim A.16 If h ∈ H(i),k1,a ∩ H

(i′),k′1,a for some i, i′ ∈ {0, . . . , n} and k ∈ {n + i + 5, . . . ,M}

and k′ ∈ {n+ i′ + 5, . . . ,M}, then i = i′ and k = k′ + βK for some β ∈ Z.

Proof. It follows immediately by Claim A.4.

Claim A.17 For all i ∈ {0, . . . , n} and k ∈ {n + i + 5, . . . ,M} and k′ ∈ {0, . . . , n + 4},H

(i),k1,a ∩H

(0),k′1,b = ∅.

Proof. If h ∈ H(0),k′1,b , then �(h) < n + 5, whereas if h ∈ H

(i),k1,a , then �(h) ≥ n + i + 5 for

some i ∈ {0, . . . , n}. Hence, H(i),k1,a ∩H

(0),k′1,b = ∅.

6

Claim A.18 For all i ∈ {0, . . . , n} and k ∈ {n + i + 5, . . . ,M} and k′ ∈ {1, . . . ,M} and

d ∈ N and τ ∈ {0, . . . , d+ 3}, H(i),k1,a ∩Hk′,d,τ

2 = ∅.Proof. Suppose not; then there exist h = (a1, . . . , at) ∈ H

(i),k1,a ∩ Hk′,d,τ

2 for some i, k, k′,

d and τ as described in the claim. Since h ∈ H(i),k1,a , then π(i),r = at−k+r for all 1 ≤ r ≤ k.

Also since h ∈ Hk′,d,τ2 , then T k′(h) = h · at−τ · h where h ∈ Σd,τ , at−τ ∈ Dd(π

(i′),k′−τ ) and

h ∈ H(i′),k′−(τ+1)1 for some i′ ∈ {0, . . . , n} satisfying either k′ − (τ + 1) ≥ n+ i′ + 5 or i′ = 0

and k′ = t and k′ − (τ + 1) < n+ 5. Next, consider each of these possibilities separately.

Case 1: h ∈ H(0),k′−(τ+1)1,b , k′ = t and k′−(τ+1) < n+5. In this case ak

′−k+1 = ak′−k+2 =

s′, h = ((s′; 2), s, . . . , s) and �(h) = k′ − (τ + 1). Thus, since τ ≤ d + 3 and k ≥ n + i + 5,

it must be that k′ − k + 1 < k′ − τ and so k′ = k. But then at−τ ∈ Dd(s) ∪ Dd(s′) and

at−τ = π(i),t−τ ∈ {s, s′}, a contradiction.

Case 2: h ∈ H(i′),k′−(τ+1)1,a and k′ − (τ + 1) ≥ n+ i′ + 5. We consider four subcases.

Subcase 1: k ≥ n+ i+5+ τ +1. Let h = Bτ+1(h). Since k− (τ +1) ≥ n+ i+5, it follows

that h ∈ H(i),k−(τ+1)1,a . Also, since T k′−(τ+1)(h) = h, h ∈ H

(i′),k′−(τ+1)1,a . Then, by Claim A.16,

i = i′ and k − (τ + 1) = k′ − (τ + 1) + βK for some β ∈ Z. This together with a = π(i),k−τ

and a ∈ Dd(π(i′),k′−τ ) implies that π(i),k′−τ = π(i),k−τ ∈ Dd(π

(i),k′−τ ), a contradiction.

Subcase 2: k = n + i + 5 + τ . By h ∈ H(i),k1,a , at−τ−1 = s and at−τ = s′. Also, by

h ∈ Hk′,d,τ2 , at−τ−1 = π(i′),k′−τ−1 and at−τ ∈ Dd(π

(i′),k′−τ ). Hence, it follows from at−τ−1 = s

and at−τ = s′, respectively, that π(i′),k′−τ−1 = s and π(i′),k′−τ ∈ Dd(s′). But this contradicts

Claim A.2.4.

Subcase 3: k = n + i + 4 + τ . First, we show that k′ ≥ k. Suppose otherwise. Since

h ∈ H(i′),k′−(τ+1)1,a , T k′(h) ∈ Λi′,k′ . By Claim A.4, this together with h ∈ H

(i),k1,a implies that

k = k′ + βK for some β ∈ N. Also, by the supposition that k < n + i + 5 + τ + 1 and

k′ − (τ + 1) ≥ n+ i′ + 5, we have k − k′ ≤ n. Since n < K, we have a contradiction.

By h ∈ H(i),k1,a , (at−τ−(n+i+3), . . . , at−τ ) = ((s′; 2), (s;n + i + 2)). Also, by h ∈ Hk′,d,τ

2 ,

(at−τ−(n+i+3), . . . , at−τ ) = (π(i′),k′−τ−(n+i+3), . . . , π(i′),k′−τ−1, at−τ ). Hence, it follows from

at−τ = s that π(i′),k′−τ ∈ Dd(s). But this contradicts Claim A.2.2.

Subcase 4: k < n+ i+ 4 + τ . By k ≥ n+ i+ 5 and τ ≤ d+ 3, it follows that τ > 0 and

k − τ ≥ 2. Hence, by h ∈ Hk′,d,τ2 , at−τ+1 ∈ D(s′) and, by h ∈ H

(i),k1,a and k − τ < n + i + 4,

at−τ+1 = s; but this is a contradiction.

7

Claim A.19 For all k ∈ {0, . . . , n+4} and k′ ∈ {1, . . . ,M} and d ∈ N and τ ∈ {0, . . . , d+3}, H(0),k

1,b ∩Hk′,d,τ2 = ∅.

Proof. Suppose otherwise; then there exists h ∈ H(0),k1,b ∩ Hk′,τ,d

2 . This means that h =

(π(0),1, . . . , π(0),k) and T k′(h) = h · a · h satisfying the remaining conditions in the definition

of Hk′,d,τ2 . Therefore, for some k < n+ 5, h · a ∈ H

(0),k+11,b . But this is a contradiction as, by

h ∈ Hk′,d,τ2 , a ∈ D(π(0),k+1).

Claim A.20 If h ∈ Hk,d,τ2 ∩ Hk′,d′,τ ′

2 for some k, k′ ∈ {1, . . . ,M} and d, d′ ∈ N and τ ∈{0, . . . , d + 3} and τ ′ ∈ {0, . . . , d′ + 3}, then τ = τ ′ and d = d′ and k = k′ + βK for some

β ∈ Z.

Proof. Let h = (a1, . . . , at). First, we establish that τ = τ ′. Suppose, without loss

of generality, that τ < τ ′. Define h = Bτ+1(h) = (a1, . . . , at−(τ+1)) and note that h ∈(∪n

i=0H(i),k−(τ+1)1

)∩H

k′−(τ+1),d′,τ ′−(τ+1)2 . But this contradicts Claim A.18 or Claim A.19.

By τ = τ ′ and the definition of Hk,d,τ2 and Hk′,d′,τ ′

2 , we have that h ∈ H(i),k−(τ+1)1 and

h ∈ H(i′),k′−(τ ′+1)1 for some i, i′ ∈ {0, . . . , n}. It then follows from Claim A.16 and Claim

A.17 that k = k′ + βK for some β ∈ Z and i = i′. Also, by h ∈ Hk,d,τ2 , at−τ = (ad, π

(i),k−τ−d )

and, by h ∈ Hk′,d′,τ ′2 , at−τ ′ = (ad′ , π

(i′),k′−τ ′−d′ ). Since i = i′, k = k′ + βK and τ = τ ′, then

(ad, π(i),k−τ−d ) = (ad′ , π

(i),k−τ−d′ ) and so d = d′.

Claim A.21 For all i ∈ {0, . . . , n} and k ∈ {n + i + 5, . . . ,M} and k′ ∈ {1, . . . ,M} and

d ∈ N , H(i),k1,a ∩Hk′,d

3 = ∅.Proof. Suppose that h ∈ H

(i),k1,a ∩Hk′,d

3 . Since Hk′,d3 ⊆ Σd,k′ and, by h ∈ Hk′,d

3 , k′ ≥ d+4, we

have a contradiction to Claim A.3 when k ≥ k′. Also, since H(i),k1,a ⊆ Λi,k and k ≥ n+ i+ 5,

we have a contradiction to Claim A.5 when k < k′.

Claim A.22 For all k ∈ {0, . . . , n+ 4} and k′ ∈ {1, . . . ,M} and d ∈ N , H(0),k1,b ∩Hk′,d

3 = ∅.Proof. Suppose there exists h = (a1, . . . , at) ∈ H

(0),k1,b ∩ Hk′,d

3 . By h ∈ Hk′,d3 , at−k′+d+4 ∈

D(s′) and, by h ∈ H(0),k1,b , ar = s for all r > 2. But this is a contradiction as it implies

d+ 4 ≤ t− k′ + d+ 4 ≤ 2.

Claim A.23 For all k, k′ ∈ {1, . . . ,M} and d, d′ ∈ N and τ ∈ {0, . . . , d+3}, Hk,d,τ2 ∩Hk′,d′

3 =

∅.

8

Proof. Suppose that h = (a1, . . . , at) ∈ Hk,d,τ2 ∩Hk′,d′

3 . There are two possibilities.

Case 1: k′ > k. First, note that, for some i ∈ {0, . . . , n}, Bτ+1(h) ∈ H(i),k−(τ+1)1,a with

k− (τ +1) ≥ n+ i+5; otherwise, Bτ+1(h) ∈ H(0),k−(τ+1)1,b and �(h) = k < k′; a contradiction.

It follows from Bτ+1(h) ∈ H(i),k−(τ+1)1,a , for some i ∈ {0, . . . , n}, and k−(τ+1) ≥ n+ i+5,

that T k(h) ∈ Λi,k. But this contradicts Claim A.5 because h ∈ Hk′,d′3 and k′ > k ≥ n+ i+5.

Case 2: k ≥ k′. There are two possibilities.

Subcase 1: k′ − τ > d′ + 4. In this case, Bτ+1(h) belongs to H(i),k−(τ+1)1 , for some

i ∈ {0, . . . , n}, and to Hk′−(τ+1),d′3 . But this contradicts Claim A.21 or A.22.

Subcase 2: k′− τ ≤ d′ +4. Since h ∈ Hk′,d′3 , we have (i) at−k′+r ∈ Dd′(s

′) for r = 1, 2, (ii)

at−k′+r ∈ Dd′(s) for r = 3, . . . , d′ + 3 and (iii) at−k′+d′+4 ∈ Dd′(s′). When k′ − τ = d′ + 4,

by (i), (ii) and Bτ+1(h) ∈ H(i),k−(τ+1)1 for some i ∈ {0, . . . , n}, Claim A.2.3 implies that

(at−τ−d′−3, . . . , at−τ−1) = ((s′; 2), (s; d′ + 1)) and at−τ ∈ Dd(s). But the latter contradicts

(iii). Therefore, it must be that k′ − τ < d′ + 4.

By h ∈ Hk,d,τ2 , it must be that (iv) at−τ+r ∈ Dd(s

′) for all r = 1, 2 and (v) at−τ+r ∈ Dd(s)

for all r ≥ 3. But then by t − k′ + d′ + 4 > t − τ , (iii) and (v), it must be that either

t − k′ + d′ + 4 = t − τ + 1 or t − k′ + d′ + 4 = t − τ + 2. The latter, however, cannot

hold because, by (ii), at−k′+d′+3 ∈ D(s) and, by (iv), at−τ+1 ∈ D(s′). Therefore, assume the

former. When k′− τ = d′ +3, by (i), (ii) and Bτ+1(h) ∈ H(i),k−(τ+1)1 for some i ∈ {0, . . . , n},

Claim A.2.3 implies that (at−k′+1, . . . , at−k′+d′+2) = ((s′; 2), (s; d′)) and at−k′+d′+3 ∈ Dd(s).

But the latter together with (ii) implies d = d′. Hence, by part (4) of the definition of Hk,d,τ2 ,

τ = 0. Thus, k′ < d′ + 4; but this contradicts h ∈ Hk′,d′3 .

Claim A.24 If h ∈ Hk,d3 ∩ Hk′,d′

3 for some k, k′ ∈ {1, . . . ,M} and d, d′ ∈ N , then k = k′

and d = d′.

Proof. First we show that k = k′. Suppose otherwise and assume, without loss of generality,

that k > k′. By h ∈ Hk′,d′3 , we have T k′(h) ∈ Σd′,k′ and k > k′ ≥ d′ + 4. But this contradicts

Claim A.5 because h ∈ Hk,d3 .

To show that d = d′, by h ∈ Hk,d3 ∩ Hk,d′

3 , we have T k(h) ∈ Σd,k ∩ Σd′,k, k ≥ d + 4 and

k ≥ d′ + 4. Hence, by Claim A.9, d = d′.

Claim A.25 For all i ∈ {0, . . . , n} and k ∈ {n + i + 5, . . . ,M} and k′ ∈ {1, . . . ,M} and

d ∈ N and r ∈ {0, . . . , n+ d+ 4}, H(i),k1,a ∩Hk′,d,r

4 = ∅.

9

Proof. Suppose that h ∈ H(i),k1,a ∩Hk′,d,r

4 . Then T k(h) ∈ Λi,k, k ≥ n + i + 5, T k′(h) ∈ Σd,k′

and k′ ≥ d + 4. But these contradict Claim A.3 if k ≥ k′ and Claim A.5 and Claim A.6 if

k < k′.

Claim A.26 For all k ∈ {0, . . . , n+4} and k′ ∈ {1, . . . ,M} and d ∈ N and r ∈ {0, . . . , n+d+ 4}, H(0),k

1,b ∩Hk′,d,r4 = ∅.

Proof. If h ∈ Hk′,d,r4 , then �(h) > T . If h ∈ H

(0),k1,b , then �(h) < n+ 5 < T by (23).

Claim A.27 For all k, k′ ∈ {1, . . . ,M} and d, d′ ∈ N and r ∈ {0, . . . , n + d′ + 4} and

τ ∈ {0, . . . , d+ 3}, Hk,d,τ2 ∩Hk′,d′,r

4 = ∅.Proof. Suppose that h ∈ Hk,d,τ

2 ∩Hk′,d′,r4 . By h ∈ Hk,d,τ

2 , Bτ+1(h) ∈ H(i),k−(τ+1)1 . Also, since

h ∈ Hk′,d′,r4 and T > τ + 1, then Bτ+1(h) belongs to H

k′−(τ+1),d′,r−(τ+1)4 if r − (τ + 1) ≥ 0 or

to Hk′−(τ+1),d′3 otherwise. But by Claim A.21, Claim A.22, Claim A.25 or Claim A.26, this

is a contradiction.

Claim A.28 For all k, k′ ∈ {1, . . . ,M} and d, d′ ∈ N and r ∈ {0, . . . , n + d′ + 4}, Hk,d3 ∩

Hk′,d′,r4 = ∅.

Proof. Suppose that h ∈ Hk,d3 ∩ Hk′,d′,r

4 . Assume first that k = k′. By Claim A.9, this

implies d = d′. Since h ∈ Hk,d3 , it follows that k < (θ(T k(h)) + 1)T + d + 4 and, since

h ∈ Hk′,d′,r4 , k = k′ and d = d′, it follows that k ≥ (θ(T k(h)) + 1)T + d + 4. But this is a

contradiction.

Suppose next that k > k′. Then, by h ∈ Hk′,d′,r4 , T k′(h) ∈ Σd′,k′ with d′ + 4 ≤ k′ < k.

But this together with h ∈ Hk,d3 contradicts Claim A.5.

Finally, suppose that k′ > k. Then, by h ∈ Hk,d3 , T k(h) ∈ Σd,k with d+ 4 ≤ k < k′. But

this together with h ∈ Hk′,d′,r4 contradicts Claim A.5 and Claim A.7.

Claim A.29 If h ∈ Hk,d,r4 ∩Hk′,d′,r′

4 for some k, k′ ∈ {1, . . . ,M} and d, d′ ∈ N and r, r′ ∈ N0,

then k = k′ and d = d′ and r = r′.

Proof. To show that k = k′, suppose, without loss of generality, that k > k′. Then, by

h ∈ Hk′,d′,r′4 , T k′(h) ∈ Σd′,k′ with d′ + 4 ≤ k′ < k. But this together with h ∈ Hk,d,r

4

contradict Claim A.5 and Claim A.7. Hence, k = k′ and, by Claim A.9, d = d′. Letting

θ = θ(T k(h)) = θ(T k′(h)), we have that k = d+4+(θ+1)T +r and k′ = d′+4+(θ+1)T +r′.

Since d = d′ and k = k′, it follows that r = r′.

10

Claim A.30 For all i ∈ {0, . . . , n} and k ∈ {n + i + 5, . . . ,M} and k′ ∈ {1, . . . ,M} and

d ∈ N and τ ∈ {0, . . . , d+ 3}, H(i),k1,a ∩Hk′,d,τ

5 = ∅.Proof. Suppose that h = (a1, . . . , at) ∈ H

(i),k1,a ∩Hk′,d,τ

5 . Then h ∈ H(i),k1,a , T k′(h) ∈ Σd,k′ and

k′ ≥ d+ 4. Therefore, k′ > k; otherwise we would contradict Claim A.3.

Consider next the case k′ > k and let h = Bτ+1(h). Then h ∈ Hk′−(τ+1),d′3 ∪H

k′−(τ+1),d′,r4

for some d′ ∈ N and 0 ≤ r ≤ n + d′ + 4. Also, by h ∈ H(i),k1,a , T k−(τ+1)(h) ∈ Λi,k−(τ+1).

Furthermore, since 0 ≤ τ ≤ d+3 ≤ n+3, we have that k− (τ +1) ≥ 1. Therefore, by Claim

A.5 and Claim A.6, one of the following must hold: (1) k − (τ + 1) = 1, (2) k − (τ + 1) = 2

and (3) Bτ+1(h) ∈ Hk′−(τ+1),d′,r4 , r > 0, k − (τ + 1) = r and 3 ≤ k − (τ + 1) < n+ i+ 5.

Case (1) implies that t− τ +1 = t− k+3 and case (2) implies that t− τ +1 = t− k+4.

Since T k(h) ∈ H(i),k1,a , we have at−k+3 = at−k+4 = s, and so, at−τ+1 = s in both cases. Since

h ∈ Hk′,d,τ5 , we also have T τ (h) ∈ Σd,τ . But this implies at−τ+1 ∈ Dd(s

′); a contradiction.

In case (3), we have that t− τ ≤ t− k + n+ i+ 5. This together with h ∈ H(i),k1,a implies

that at−τ ∈ {s, s′}. Since h ∈ Hk′,d,τ5 , Bτ+1(h) ∈ H

k′−(τ+1),d′,r4 and π(d′),r+1 ∈ {s, s′}, it must

be that at−τ ∈ Dd(s) ∪ Dd(s′); a contradiction.

Claim A.31 For all k ∈ {0, . . . , n+4} and k′ ∈ {1, . . . ,M} and d ∈ N and τ ∈ {0, . . . , d+3}, H(0),k

1,b ∩Hk′,d,τ5 = ∅.

Proof. Suppose h = (a1, . . . , ak) ∈ H(0),k1,b ∩Hk′,d,τ

5 . By h ∈ Hk′,d,τ5 , ak−k′+1, ak−k′+i+4 ∈ D(s′)

for some i ∈ N and, by h ∈ H(0),k1,b , h = ((s′; 2), (s; k − 2)); a contradiction.

Claim A.32 For all k, k′ ∈ {1, . . . ,M} and d, d′ ∈ N and τ ∈ {0, . . . , d + 3} and τ ′ ∈{0, . . . , d′ + 3}, Hk,d,τ

2 ∩Hk′,d′,τ ′5 = ∅.

Proof. Suppose that h ∈ Hk,d,τ2 ∩ Hk′,d′,τ ′

5 . Assume first that τ ≤ τ ′. Then Bτ+1(h) ∈ H1

and Bτ+1(h) ∈ H3 ∪H4 ∪H5. But this contradicts Claims A.21, A.22, A.25, A.26, A.30 or

A.31. If τ > τ ′, then Bτ ′+1(h) ∈ H2 and Bτ ′+1(h) ∈ H3 ∪ H4. But this contradicts Claim

A.23 or Claim A.27.

Claim A.33 For all k, k′ ∈ {1, . . . ,M} and d, d′ ∈ N and τ ∈ {0, . . . , d′ + 3}, Hk,d3 ∩

Hk′,d′,τ5 = ∅.

Proof. Suppose that h = (a1, . . . , at) ∈ Hk,d3 ∩ Hk′,d′,τ

5 . By h ∈ Hk,d3 , T k(h) ∈ Σd,k with

k ≥ d + 4; by h ∈ Hk′,d′,τ5 , T k′(h) ∈ Σi′,k′ with k′ ≥ i′ + 4 for some i′ ∈ N . By appealing to

Claims A.5, A.8 and A.9, it then follows that k = k′ and d = i′.

11

By h ∈ Hk′,d′,τ5 and the last two equalities, k − (τ + 1) ≥ d + 4. But then, by h ∈ Hk,d

3 ,

we have Bτ+1(h) ∈ Hk−(τ+1),d3 . This together with h ∈ Hk′,d′,τ

5 implies at−τ ∈ Dd′(md) and

d′ �= d. But since h ∈ Hk,d3 and k − (τ + 1) ≥ d+ 4, at−τ ∈ Dd(m

d); a contradiction.

Claim A.34 For all k, k′ ∈ {1, . . . ,M} and d, d′ ∈ N and r ∈ {0, . . . , n + d + 4} and

τ ∈ {0, . . . , d′ + 3}, Hk,d,r4 ∩Hk′,d′,τ

5 = ∅.Proof. Suppose that h = (a1, . . . , at) ∈ Hk,d,r

4 ∩ Hk′,d′,τ5 . By h ∈ Hk,d,r

4 , T k(h) ∈ Σd,k with

k ≥ d + 4; by h ∈ Hk′,d′,τ5 , T k′(h) ∈ Σi′,k′ with k′ ≥ i′ + 4 for some i′ ∈ N . By appealing to

Claim A.7 and Claim A.8, it then follows that k = k′.

Suppose r ≥ τ + 1. By h ∈ Hk,d,r4 , Bτ+1(h) ∈ H

k−(τ+1),d,r−(τ+1)4 . This implies that

at−τ = π(d),r−τ and, by h ∈ Hk′,d′,τ5 and Claim A.28 and Claim A.29, at−τ ∈ Dd′(π

(d),r−τ ); a

contradiction.

Finally, suppose r < τ + 1. By h ∈ Hk,d,r4 , Bτ+1(h) ∈ H

k−(τ+1),d3 . This implies that

at−τ ∈ Dd(md) and, by h ∈ Hk′,d′,τ

5 and Claim A.24 and ClaimA.28, at−τ ∈ Dd′(md) and

d′ �= d; a contradiction.

Claim A.35 If h ∈ Hk,d,τ5 ∩ Hk′,d′,τ ′

5 for some k, k′ ∈ {1, . . . ,M} and d, d′ ∈ N and τ ∈{0, . . . , d+ 3} and τ ′ ∈ {0, . . . , d′ + 3}, then k = k′, τ = τ ′ and d = d′.

Proof. Suppose h ∈ Hk,d,τ5 ∩Hk′,d′,τ ′

5 . First, note that τ = τ ′. Otherwise, say τ < τ ′; then, by

h ∈ Hk′,d′,τ ′5 , Bτ+1(h) ∈ H5 and, by h ∈ Hk,d,τ

5 , Bτ+1(h) ∈ H3 ∪H4. This contradicts Claim

A.33 or Claim A.34. Second, note that Bτ+1(h) ∈ ∪i∈N(H

k−(τ+1),i3 ∪

(∪rH

k−(τ+1),i,r4

))and

Bτ ′+1(h) ∈ ∪i∈N(H

k′−(τ ′+1),i3 ∪

(∪rH

k′−(τ ′+1),i,r4

)). Since τ = τ ′, by Claim A.24, Claim A.28

and A.29, k = k′. Finally, it follows immediately from the definitions of Hk,d,τ5 and Hk′,d′,τ ′

5 ,

k = k′ and τ = τ ′ that d = d′.

Claim A.36 If h ∈ Hd,τ6 ∩ Hd′,τ ′

6 for some d, d′ ∈ N and τ ∈ {0, . . . , d + 3} and τ ′ ∈{0, . . . , d′ + 3}, then d = d′.

Proof. Let h = (a1, . . . , at) ∈ Hd,τ6 ∩ Hd′,τ ′

6 . We may assume, without loss of generality,

that τ ≥ τ ′. Then, by h ∈ Hd′,τ ′6 , at−τ ′ ∈ Dd′(s) ∪ Dd′(s

′) and, by τ ≥ τ ′ and h ∈ Hd,τ6 ,

at−τ ′ ∈ Dd(s) ∪Dd(s′). Thus, d = d′.

12

A.3 Proofs of Claims 9 – 19

Proof of Claim 9. By Claim A.11, we may assume that k < M . Hence, f(h) = π(i),k+1

and h · f(h) ∈ H(i),k+11,a . Thus, by induction, π(f |h) = (π(i),k+1, π(i),k+2, . . .).

Proof of Claim 10. If k < n + 4, then h · f(h) ∈ H(0),k+11,b . If k = n + 4, then h · f(h) ∈

H(0),n+51,a . Thus, by Claim 9, π(f |h) = (π(0),k+1, π(0),k+2, . . .).

Proof of Claim 11. If r = n + d + 4, then h · f(h) ∈ H(d),n+d+51,a , thus, π(f |h) =

(π(d),r+1, π(d),r+2, . . .) by Claim 9. If r < n + d + 4, then f(h) = π(d),r+1. Therefore,

h · f(h) ∈ Hk+1,d,r+14 . Furthermore, by Claim A.14, k < M . Hence, by induction, π(f |h) =

(π(d),r+1, π(d),r+2, . . .).

Proof of Claim 12. Since f(h) = md, we have h·f(h) ∈ Hk+1,d,04 if k−(d+4) = (θ+1)T−1

and h · f(h) ∈ Hk+1,d3 if k− (d+4) < (θ+1)T − 1. Also, by Claim A.13, k < M . Therefore,

by Claim 11, the result follows by induction.

Proof of Claim 13. We establish this claim by considering the different possible cases.

Case 1: One of the following conditions hold: (a) τ = d + 3, (b) τ = 0, T d+3(h) =

((s′; 2), (s; d), a) and a ∈ Dd(s), and (c) h ∈ Hd,06 and T d+3(h) ∈ Σd,d+3. In this case, it

follows that f(h) = s′ and a ∈ Dd(s′) and h · a ∈ Hd+4,d

3 . Thus, by Claim 12, π(f |h · a) =π(d)(θ(a), t(a)) where t(a) = d+ 5 and θ(a) = θ(T d+3(h) · a). Furthermore, if a ∈ Dd(f(h)),

then θ(T d+3(h) · a) = θ(T d+3(h) · f(h)) + 1. Hence, θ(f(h)) < θ(a).

Case 2: h ∈ Hk,d,τ2 ∪Hk,d,τ

5 and none of the conditions (a)–(c) hold. In this case, f(h) =

π(d),τ+1 and h·a ∈ Hk+1,d,τ+12 ∪Hk+1,d,τ+1

5 . Thus, by Claims A.12 and A.15, f(h·a) = π(d),τ+2.

Then, by induction, it follows that f(h·a·π(d),τ+2) = π(d),τ+3, . . . , f(h·a·π(d),τ+2·· · ··π(d),d+2) =

π(d),d+3. Therefore, by appealing to Case 1, we have π(f |h · a) = π(d)(θ(a), t(a)) where

t(a) = τ + 2 and θ(a) = θ(T τ (h) · (a, π(d),τ+2, . . . , π(d),d+4)). It also follows from the latter

that θ(a) = θ(f(h)) + 1 if a ∈ Dd(f(h)).

Case 3: h ∈ Hd,τ6 and none of the conditions (a)–(c) hold. For any history h′ ∈ H, define

v(h′) = max{t ∈ {0, . . . , d + 3} : h′ ∈ Hd,t6 } and let τ ′ = v(h). Then (i) h ∈ Hd,τ ′

6 , (ii)

T 1(h) ∈ Dd(s) ∪ Dd(s′) if τ ′ = 0, (iii) T 1(h) ∈ Dd(s

′) if τ ′ = 1 and (iv) f(h) = π(d),τ ′+1. To

complete the proof in this case, we first establish two subclaims.

Subclaim 1: h · a ∈ Hd,τ ′+16 . Suppose not. By (iv), h · a ∈ Σd,τ ′+1, hence, h · a ∈ ∪5

l=1Hl.

13

Also, if τ ′ ≥ 2, then a ∈ D(s). Therefore, by Claim A.10, τ ′ = 0 or 1 and either h · a ∈H

(i),n+i+51,a for some i ∈ {0, . . . , n} or T d+4(h · a) ∈ Σd,d+4 for some d ∈ N .

If h · a ∈ H(i),n+i+51,a , then T 1(h) = s, contradicting (ii) and (iii). If T d+4(h · a) ∈ Σd,d+4,

then T 1(h) ∈ Dd(s). Therefore, by (iii), τ ′ = 0. Then, by (ii), it follows that T 1(h) ∈ Dd(s)

and d = d. This implies that T d+3(h) ∈ Σd,d+3. Thus, by (i) and τ ′ = 0, condition (c) is

satisfied. But this contradicts our supposition.

Subclaim 2: v(h · a) = τ ′ + 1. Since h · a ∈ Hd,τ ′+16 , v(h · a) ≥ τ ′ + 1 > 0. Therefore,

h ∈ Σd,v(h·a)−1. Hence, τ ′ ≥ v(h · a)− 1 and thus v(h · a) = τ ′ + 1.

It follows from the above two subclaims that f(h · a) = π(d),τ ′+2. Then, by induction, it

follows that f(h·a·π(d),τ ′+2) = π(d),τ ′+3, . . . , f(h·a·π(d),τ ′+2 ·· · ··π(d),d+2) = π(d),d+3. Therefore,

by appealing to Case 1, we have π(f |h · a) = π(d)(θ(a), t(a)), where t(a) = τ ′ + 2 and

θ(a) = θ(T τ ′(h) · (a, π(d),τ ′+2, . . . , π(d),d+4)). Furthermore, from the latter, θ(a) = θ(f(h)) + 1

for all a ∈ Dd′(f(h)).

Proof of Claim 14. If h �∈ ∪n+3k=2H

k7 and T d+3(h) ∈ Σd,d+3 for some d ∈ N , then f(h) = s′

and T d+4(h · f(h)) ∈ Σd,d+4. Hence, h · f(h) ∈ Hd+4,d3 and the conclusion follows from Claim

12.

If h �∈ ∪n+3k=2H

k7 and T n+i+4(h) = ((s′; 2), (s;n + i + 2)) for some i ∈ {0, . . . , n}, then

f(h) = s′ and T n+i+5(h · f(h)) = ((s′; 2), (s;n + i + 2), s′). Hence, h · f(h) ∈ H(i),n+i+51,a and

the conclusion follows from Claim 9. For the remainder of the proof, therefore assume that

the following holds:

if h �∈ ∪n+3k=2H

k7 , then T n+i+4(h) �= ((s′; 2), (s;n+ i+ 2)),

for all i ∈ {0, . . . , n}, and T d+3(h) �∈ Σd,d+3, for all d ∈ N.(A.6)

Next, for any history h′ ∈ H, define v(h′) = max{t ∈ {0, . . . , n + 4} : T t(h′) ∈ H(0),t1,b }

and let k′ = v(h). If k′ = n + 4, then f(h) = s′ and h · f(h) ∈ H(0),n+51,a , and the conclusion

follows from Claim 9.

If k′ ∈ {0, . . . , n+ 3}, then the claim follows by induction if it is the case that h · f(h) ∈Hk′+1

7 and v(h · f(h)) = k′ + 1. We show that the latter is indeed the case in several steps.

Step 1: h · f(h) �∈ H6. Otherwise, h · f(h) ∈ Hd,τ6 for some d ∈ N and τ ∈ {0, . . . , d+ 3}.

Then if τ > 0, T τ−1(h) ∈ Σd,τ−1; but this is a contradiction as this implies that h ∈ H6.

14

If τ = 0, then by the definition of H06 , f(h) = T 1(h · f(h)) ∈ D(s) ∪ D(s′). But this is a

contradiction because, by h ∈ Hk′7 , f(h) ∈ {s, s′}.

Step 2: h · f(h) �∈ ∪5l=1Hl. When k′ ≥ 2, then f(h) = s. Therefore, the claim in

this step follows immediately from Claim A.10. Next suppose that k′ ∈ {0, 1} and that

h ·f(h) ∈ ∪5l=1Hl. Then Claim A.10 implies that either h ∈ H

(i),n+i+41,a for some i ∈ {0, . . . , n}

or T d+3(h) ∈ Σd,d+3 for some d ∈ N . But this contradicts our supposition in (A.6).

Step 3: T k′+1(h · f(h)) ∈ H(0),k′+11,b and v(h · f(h)) = k′ + 1. By h ∈ Hk′

7 , f(h) = π(0),k′+1

and so T k′+1(h · f(h)) ∈ H(0),k′+11,b . Hence, v(h · f(h)) ≥ k′ + 1. If it were the case that

v(h · f(h)) > k′ + 1, then T v(h·f(h))−1(h) ∈ H(0),v(h·f(h))−11,b , implying v(h) ≥ v(h · f(h)) − 1.

Since k′ = v(h) and v(h · f(h)) > k′ + 1, this is a contradiction.

Proof of Claim 15. By assumption, h ∈ H(i),k1,a ∪H(0),l

1,b for some i, k and l, and a ∈ Dd(f(h)).

Thus, h · a ∈ Hk+1,d,02 . Also, by Claim 13, π(f |h · a · f(h · a)) = π(d)(θ, t) for some θ and t.

Proof of Claim 16. We first argue that it is sufficient to show that either h · a ∈ Hd+4,d3

or h · a �∈ ∪5l=1Hl. If the former holds, then π(f |h · a) = π(d)(θ, d + 5) and the claim follows

from Claim 12. If the latter holds, then this together with f(h) ∈ {s, s′} and a ∈ Dd(f(h))

implies that h · a ∈ Hd,06 . Then the claim follows from Claim 13.

We next establish that either h · a ∈ Hd+4,d3 or h · a �∈ ∪5

l=1Hl. Suppose not; then we

derive a contradiction for each of the different possible cases as follows.

Case 1: T d+4(h · a) ∈ Σd,d+4 for some d ∈ N . Then a ∈ Dd(s′), which together with

a ∈ Dd(s) ∪ Dd(s′) implies that d = d. Hence, h · a ∈ Hd+4,d

3 ; a contradiction.

Case 2: h · a ∈ H(i),n+i+51,a for some i ∈ {0, . . . , n}. Then a = s′ and this contradicts

a ∈ Dd(s) ∪ Dd(s′).

Case 3: h ∈ Hk,d′,τ2 ∪Hk,d′,τ

5 and h·a ∈ H(i),k′1,a ∪H1,b∪H k,d,0

2 ∪H k,d,05 for some i ∈ {0, . . . , n}

and k′ > n + i + 5 and k ∈ {1, . . . ,M} and d ∈ N . Then, by the latter, h ∈ H1 ∪H3 ∪H4

and, by Claims A.18, A.19, A.23, A.27, A.30, A.31, A.33 and A.34, this contradicts the

supposition that h ∈ Hk,d′,τ2 ∪Hk,d′,τ

5 .

Case 4: h ∈ Hk,d′,τ2 ∪ Hk,d′,τ

5 and h · a ∈ H k,d,τ2 ∪ H k,d,τ

5 for some k ∈ {1, . . . ,M} and

d ∈ N and τ ∈ {1, . . . , d + 3}. By the latter, a ∈ Dd(s) ∪ Dd(s′). Since we also have

a ∈ Dd(s) ∪ Dd(s′), it follows that d = d. This together with h · a ∈ H k,d,τ

2 ∪H k,d,τ5 implies

that h ∈ H k−1,d,τ−12 ∪ H k−1,d,τ−1

5 . Since h ∈ Hk,d′,τ2 ∪ Hk,d′,τ

5 , Claims A.20, A.32 and A.35

15

imply that τ = τ − 1 and k = k − 1 + βK for some β ∈ Z. Thus, a = f(h), contradicting

the supposition that a ∈ Dd(f(h)).

Case 5: h ∈ Hk,d′,τ2 ∪Hk,d′,τ

5 and h · a ∈ H3 and T d+4(h · a) �∈ Σd,d+4 for all d ∈ N . Then

h ∈ H3 and, by Claims A.23 and A.33, this contradicts h ∈ Hk,d′,τ2 ∪Hk,d′,τ

5 .

Case 6: h ∈ Hk,d′,τ2 ∪ Hk,d′,τ

5 and h · a ∈ H4. Then h ∈ H3 ∪ H4 and, by Claims A.23,

A.27, A.33 and A.34, this contradicts the supposition that h ∈ Hk,d′,τ2 ∪Hk,d′,τ

5 .

Case 7: h ∈ Hd′,τ6 and h · a �∈ H

(i),n+i+51,a for all i ∈ {0, . . . , n} and T d+4(h · a) �∈ Σd,d+4 for

all d ∈ N . Then h �∈ ∪5l=1Hl and, by Claim A.10, we have h · a �∈ ∪5

l=1Hl; a contradiction.

Proof of Claim 17. If d = d′, then h · a ∈ Hk+1,d,04 if k + 1 − (d + 4) = (θ + 1)T and

h · a ∈ Hk+1,d3 , otherwise. Thus, in this case, the result follows from Claim 11 and Claim 12,

respectively.

If d �= d′, then h · a ∈ Hk+1,d,05 . Thus, by Claim 13, π(f |h · a · f(h · a)) = π(d)(θ, t) for

some θ ∈ {0, . . . , d+ 4} and t ∈ {1, . . . , d+ 5}.Proof of Claim 18. We have that h · a ∈ Hk+1,0,d

5 . Thus, by Claim 13, π(f |h · a ·f(h · a)) =π(d)(θ, t) for some θ ∈ {0, . . . , d+ 4} and t ∈ {1, . . . , d+ 5}.Proof of Claim 19. If h �∈ ∪n+3

k=2Hk7 and T d+3(h) ∈ Σd,d+3, then f(h) = s′ and a ∈ Dd(s

′)

and T d+4(h · a) ∈ Σd,d+4. Hence, h · a ∈ Hd+4,d3 and the conclusion follows from Claim 12.

For the remainder of the proof, therefore, assume that the following holds:

If h �∈ ∪n+3k=2H

k7 , then T d+3(h) �∈ Σd,d+3. (A.7)

Then we will show that h · a ∈ Hd,06 which, by Claim 13, establishes the conclusion of the

claim. We prove the former in two steps.

Step 1: h · a ∈ Σd,0. Since h ∈ Hk7 , f(h) ∈ {s, s′}. Therefore, a ∈ Dd(s) ∪ Dd(s

′).

Step 2: h · a �∈ ∪5l=1Hl. Suppose otherwise. Then, by Claim A.10, a �∈ D(s). Hence, by

h ∈ Hk7 , (i) a ∈ Dd(s

′) and (ii) h �∈ ∪n+3k=2H

k7 . Furthermore, Claim A.10 implies that either

h · a ∈ H(i),n+i+51,a for some i ∈ {0, . . . , n} or T d′+4(h · a) ∈ Σd′,d′+4 for some d′ ∈ N . If h · a ∈

H(i),n+i+51,a for some i ∈ {0, . . . , n}, then a = s′, a contradiction to (i). If T d′+4(h · a) ∈ Σd′,d′+4

for some d′ ∈ N , then a ∈ Dd′(s′) which, by (i), implies that d = d′. Thus, T d+3(h) ∈ Σd,d+3.

But this together with (ii) contradicts our supposition in (A.7).

16

B Theorem 2 (mixed Folk Theorem)

In this section, we provide the proofs of Lemmas 1 and 3.

B.1 Proof of Lemma 1

Proof of Lemma 1. In order to economize on notation, we shall omit the dependence of

Σd(.), S(.), λ(.), λi(.),Φdi (.), α

di (.) on h, Q, δ or η whenever the meaning is clear.

Fix any ζ > 0 and 0 < ε1 < 1. Let Q ∈ N be such that Q > γ and

1

ζQ/(Q+1)(Q+ 1)<

ε12B

.

Moreover, let η > 0 be such that (n− 1)B|A|2η < ε1/2.

We next fix player d ∈ N and consider any δ ∈ (0, 1), t ∈ N and h = (a1, . . . , at) ∈ Ht

such that δt ≥ ζ and αdi (h) = 1 for all i �= d . For ease of exposition we reorder the players

so that d = n. Then for every a ∈ A and every i �= n:

|λ(a)− λi(a−i)μni (ai)| < η

1− δt

1− δ

Fix a = (a1, a2, . . . , an) ∈ A. In particular, we have that∣∣∣∣∣λ(a)− ∑b1∈A1

λ(b1, a2, . . . , an)μn1 (a1)

∣∣∣∣∣ < η1− δt

1− δ

Since, for every b1 ∈ A1,∣∣∣∣∣λ(b1, a−1)−∑b2∈A2

λ(b1, b2, a3, . . . , an)μn2 (a2)

∣∣∣∣∣ < η1− δt

1− δ,

we obtain∣∣∣∣∣∣∑b1∈A1

λ(b1, a−1)μn1 (a1)−

∑(b1,b2)∈A1×A2

λ(b1, b2, a3, . . . , an)μn1 (a1)μ

n2 (a2)

∣∣∣∣∣∣ ≤ μn1 (a1)|A1|η1− δt

1− δ< |A|η1− δt

1− δ.

Hence,∣∣∣∣∣∣λ(a)−∑

(b1,b2)∈A1×A2

λ(b1, b2, a3, . . . , an)μn1 (a1)μ

n2 (a2)

∣∣∣∣∣∣ < (1 + |A|)η1− δt

1− δ< 2|A|η1− δt

1− δ.

17

Repeating the same procedure n− 1 times implies that∣∣∣∣∣∣λ(a)−∑

(b1,...,bn−1)∈A−n

λ(b1, b2, . . . , bn−1, an)n−1∏j=1

μnj (aj)

∣∣∣∣∣∣ < (n− 1)|A|η1− δt

1− δ.

Hence,

|λ(a)−∑(b1,...,bn−1)∈A−n

λ(b1, b2, . . . , bn−1, an)∏n−1

j=1 μnj (aj)|∑

k �∈S δk−1

<η(1− δt)|A|

(1− δ)∑

k �∈S δk−1

. (B.1)

Next, define for each an ∈ An,

rn(an) =

∑b−n∈A−n

λ(b−n, an)∑k �∈S δ

k−1.

It follows from the definition of rn and (B.1) that rn ∈ Δ(An) and, for all a ∈ A,

λ(a)∑k �∈S δ

k−1< rn(an)

n−1∏j=1

μnj (aj) +

η(1− δt)|A|(1− δ)

∑k �∈S δ

k−1.

Hence, because μn−n is a minmax profile,∑

a∈A λ(a)un(a)∑k �∈S δ

k−1< un(rn, μ

n−n) +

η(1− δt)|A|2B(1− δ)

∑k �∈S δ

k−1<

ε12

1− δt

1− δ

1∑k �∈S δ

k−1. (B.2)

Then it follows from (B.2) that

1− δ

1− δt

t∑k=1

δk−1un(ak) =

1− δ

1− δt

{∑k∈S

δk−1un(ak) +

∑a∈A

λ(a)un(a)

}

≤ 1− δ

1− δt

{B∑k∈S

δk−1 +ε12

1− δt

1− δ

}

≤ 1− δ

1− δt

{B1− δ|S|

1− δ+

ε12

1− δt

1− δ

}= B

1− δ|S|

1− δt+

ε12.

Hence, to complete the proof, it remains to be shown that B 1−δ|S|1−δt

< ε12. Note that |S|

t

is bounded by 1Q+1

. Letting f(x) = x1/(Q+1) and g(x) = x, and using the generalized mean

value theorem, we have that, for some b ∈ (δt, 1),

1− δ|S|

1− δt≤ 1− δ

tQ+1

1− δt=

f(1)− f(δt)

g(1)− g(δt)=

f ′(b)g′(b)

=1

bQ

Q+1 (Q+ 1).

Since δt ≥ ζ, it then follows that 1−δ|S|1−δt

≤ 1

ζQ

Q+1 (Q+1)< ε1

2B. This concludes the proof.

18

B.2 Proof of Lemma 3

In what follows, we present the proof of Lemma 3.

B.2.1 Auxiliary results

We first prove two lemmas.

Lemma B.1 For all i, d ∈ N , with i �= d,

Pfd,T

({h ∈ HT : Φi(d, h) ≥ η}

)< 2ε2.

Proof. Suppose not. Then for some i, d ∈ N , with i �= d,

Pfd,T

({h ∈ HT : Φi(d, h) ≥ η}

)≥ 2ε2.

Suppose that in the calculation described by (a)-(d) in section B.2.2 of the main text, player

i plays strategy μdi at every history h ∈ ∪T−1

t=0 Ht and all other players play fd−i. Then the

expected payoff of i is given by

V ∗i = (1− δ)

∑h=(a1,...,aT )∈HT

P(μdi ,f

d−i),T

(h)

(T∑

k=1

δk−1ui(ak) + δTwi(d, h)

).

Given the definition of fd, we have that

V di (H0) ≥ V ∗

i . (B.3)

Furthermore, by Lemma 2 and (47) and (48),

V ∗i ≥ −B(1− δT ) + δT (u′

i + ρ(1− ε2)− 2ξ).

By (47) and (48),

V di (H0) ≤ B(1− δT ) + δT (u′

i + ρ(1− 2ε2) + 2ξ).

Hence, by (51),

V di (H0)− V ∗

i ≤ 2B(1− δT ) + 4ξδT − ρε2δT < 0.

But this contradicts (B.3).

19

Lemma B.2 For all d ∈ N ,

(1− δ)∑

h=(a1,...,aT )∈HT

Pfd,T (h)T∑

k=1

δk−1ud(ak) ≥ −4ξδT .

Proof. Suppose not. Then for some d ∈ N ,

(1− δ)∑

h=(a1,...,aT )∈HT

Pfd,T (h)T∑

k=1

δk−1ud(ak) < −4ξδT .

Consider strategy f ′d for player d defined by setting, for all h ∈ ∪T−1

t=0 Ht, f′d(h) to be such that

it solves maxad∈Adud(ad, f

d−d(h)). Since maxad∈Ad

ud(ad, fd−d(h)) ≥ vi = 0, it follows that

(1− δ)∑

h=(a1,...,aT )∈HT

P(f ′d,f

d−d),T

(h)T∑

k=1

δk−1ud(ak) ≥ 0.

Letting Vd be player d’s expected discounted payoff from (f ′d, f

d−d), by (47) and (48),

Vd − V dd (H0) =

(1− δ)∑h∈HT

P(f ′d,f

d−d),T

(h)T∑

k=1

δk−1ud(ak)− (1− δ)

∑h∈HT

Pfd,T (h)T∑

k=1

δk−1ud(ak)

+δT

⎧⎨⎩ ∑h∈HT

P(f ′d,f

d−d),T

(h)wd(d, h)−∑h∈HT

Pfd,T (h)wd(d, h)

⎫⎬⎭> 4ξδT − 4ξδT = 0.

But this contradicts Vd ≤ V dd (H0).

Next, we provide some results that are similar to those for the pure strategy case in

Appendix A of the main text and Section A of the supplementary materials; for the sake of

completeness we will describe them fully.

Claim B.1 For all ω ∈ {0, . . . , ω}, the following hold:

1. All actions a �= s′ are played for t ≥ n+ ω +Q+ 2 consecutive periods in π(ω).

2. Suppose that, for some t ∈ N, (π(ω),t, . . . , π(ω),t+l+1) = ((s′; 2), (s; l)) and l > 0. Then

either π(ω),t+l+2 = s or π(ω),t+l+2 = s′ and l = n+ ω +Q+ 2.

3. Suppose that, for some t ∈ N, π(ω),t ∈ D(s′) and π(ω),t+1 ∈ D(s). Then π(ω),t = s′ and

π(ω),t+1 = s. Furthermore, either (i) π(ω),t−1 = s′ and t = 2 + βK or (ii) π(ω),t−1 = s

and t = n+ ω +Q+ 5 + βK for some β ∈ N0.

20

4. Suppose that, for some t ∈ N, π(ω),t ∈ D(s) and π(ω),t+1 ∈ D(s′). Then π(ω),t+1 = s′.

Proof. This follows immediately from the ordering of A and the definition of π(ω).

Claim B.2 If h ∈ H(ω),k1,a for some ω ∈ {0, . . . , ω} and k ∈ {n + ω + Q + 5, . . . ,M}, then

T k′(h) �∈ Σd,k′ for all d ∈ N and k′ ∈ N such that d+Q+ 4 ≤ k′ ≤ k.

Proof. Suppose otherwise; then there exists h ∈ H(ω),k1,a such that T k′(h) ∈ Σd,k′ for some

d ∈ N and k′ ∈ N with d +Q + 4 ≤ k′ ≤ k. Let T k(h) = (a1, . . . , ak). Since T k′(h) ∈ Σd,k′ ,

we have ak−k′+2 ∈ D(s′), ak−k′+3 ∈ D(s) and ak−k′+d+Q+4 ∈ D(s′). Therefore, there is an

action, namely ak−k′+3, which is different from s′ and is played at most d+Q+1 consecutive

periods in T k(h). Since h ∈ H(i),k1,a , this contradicts Claim B.1.1.

Next, for all τ ∈ N and ω ∈ {0, . . . , ω}, define

Λω,τ ={h ∈ H : h = (at)τt=1 such that at = π(ω),t if t ≤ n+ ω +Q+ 5

}.

Claim B.3 If h ∈ H(ω),k1,a for some ω ∈ {0, . . . , ω} and k ∈ {n + ω + Q + 5, . . . ,M}and

T k′(h) ∈ Λω′,k′ for some ω′ ∈ {0, . . . , ω} and n + ω′ + Q + 5 ≤ k′ ≤ k, then ω = ω′ and

k = k′ + βK for some β ∈ N0.

Proof. Since h ∈ H(ω),k1,a and T k′(h) ∈ Λω′,k′ , for some t = k + 1 − k′ it must be that

(π(ω),t, . . . , π(ω),t+n+ω′+Q+4) = ((s′; 2), (s;n+ω′+Q+2), s′). By Claim B.1.2, n+ω′+Q+2 =

n+ ω +Q+ 2 and so ω = ω′. Hence, by Claim B.1.3, t+ 1 = 2 + βK for some β ∈ N0 and

so k = k′ + βK.

Claim B.4 Suppose that, for some k ∈ {1, . . . ,M} and d ∈ N , ω ∈ Wd and h ∈ Hk,d3,a ∪

Hk,d3,b ∪Hk,ω,0

4 . If T k′(h) ∈ Λω′,k′ ∪Σd′,k′ for some ω′ ∈ {0, . . . , ω} and d′ ∈ N and k′ ∈ N such

that Q + 2 ≤ k′ < k, then k′ = Q + 2 and h ∈ Hk,d3,b ∪Hk,ω,0

4 . Hence, T k′(h) �∈ Λω′,k′ ∪ Σd′,k′

for all ω′ ∈ {0, . . . , ω} and d′ ∈ N and k′ ∈ N such that Q+ 3 ≤ k′ < k.

Proof. Suppose that T k′(h) ∈ Λω′,k′ ∪Σd′,k′ for some ω′ ∈ {0, . . . , ω} and d′ ∈ N and k′ ∈ N

such that Q+2 ≤ k′ < k. Let T k(h) = (a1, . . . , ak). First, note that by T k′(h) ∈ Λω′,k′∪Σd′,k′ ,

ak−k′+τ ∈⎧⎨⎩ Dd′(s

′) if τ = 1, 2

Dd′(s) if τ = 3, . . . , Q+ 4.(B.4)

Since, by h ∈ Hk,d3,a ∪ Hk,d

3,b ∪ Hk,ω,04 , it must be that a3, . . . , ad+Q+3 ∈ D(s), it follows that

k − k′ + 1 ≥ d+Q+ 4. We next consider three cases regarding k − k′ + 1.

21

Case 1: k − k′ + 1 = d + Q + 4 + (γ + 1)l + τ for some l ∈ {0, . . . , d + Q + 3} and

some τ = 0, . . . , γ − 2. Since h ∈ Hk,d3,a ∪Hk,d

3,b ∪Hk,ω,04 and γ ≥ 3, in this case we must have

ak−k′+2−d = ak−k′+3

−d ∈ {md−d, a

(d)−d}. Also, by (B.4), ak−k′+2 ∈ Dd′(s

′) and ak−k′+3 ∈ Dd′(s). But

this a contradiction.

Case 2: k− k′ +1 = d+Q+4+ (γ +1)l+ γ− 1 for some l ∈ {0, . . . , d+Q+3}. In this

case, ak−k′+3−d = s′−d by h ∈ Hk,d

3,a ∪Hk,d3,b ∪Hk,ω,0

4 . But this contradicts (B.4).

Case 3: k − k′ + 1 = d + Q + 4 + (γ + 1)l + γ for some l ∈ {0, . . . , d + Q + 2}. Since

k′ ≥ Q+ 2 > γ + 2, ak−k′+γ+3 ∈ Dd(s′) by h ∈ Hk,d

3,a ∪Hk,d3,b ∪Hk,ω,0

4 . Also, by (B.4), it must

be that ak−k′+τ ∈ Dd′(s) for all τ = 3, . . . , Q+3. Since Q > γ we then have a contradiction.

It follows from the above three cases that k − k′ + 1 ≥ (d + Q + 4)(γ + 2) − 1. Since

k′ ≥ Q+2 > γ+2 we have h �∈ Hk,d3,a . We next complete the proof by showing that k′ = Q+2.

Suppose otherwise; hence, k′ ≥ Q + 3. If k − k′ + 1 = (d + Q + 3)(γ + 2) − 1, then

ak−k′+3 ∈ Dd(s′) by h ∈ Hk,d

3,b ∪Hk,ω,04 ; but this contradicts (B.4). So suppose that k−k′+1 >

(d+Q+3)(γ +2)− 1. Then, since we have ak−k′+τ ∈ Dd′(s) for all Q+2 ≥ τ ≥ 3 by (B.4),

we obtain a contradiction to (B.4) because ak−k′+Q+3 ∈ Dd(s′) due to h ∈ Hk,d

3,b ∪Hk,ω,04 .

Claim B.5 If h ∈ Hk,ω,r4 for some k ∈ {1, . . . ,M} and ω ∈ {0, . . . , ω} and r ∈ {1, . . . , n +

ω + Q + 4} and T k′(h) ∈ Λω′,k′ for some ω′ ∈ {0, . . . , ω} and Q + 3 ≤ k′ < k, then k′ = r

and k′ < n+ ω′ +Q+ 5.

Proof. Let T k(h) = (a1, . . . , ak) and d ∈ N be such that ω ∈ Wd. By T k′(h) ∈ Λω′,k′ ,

ak−k′+1 = ak−k′+2 = s′; by h ∈ Hk,ω,r4 , at ∈ D(s) for all t ∈ {3, . . . , d + Q + 3} ∪ {k − r +

3, . . . , k}. It then follows that k − k′ + 1 ∈ {2, d+Q+ 4, . . . , k − r + 2}.By Claim B.4, it must be that k − k′ + 1 > k − r − Q − 2. The reasoning for this is as

follows. Since h ∈ Hk,ω,r4 and T k′(h) ∈ Λω′,k′ , it follows, respectively, that Br(h) ∈ Hk−r,ω,0

4

and T k′−r(Br(h)) ∈ Λω′,k′−r. But then by Claim B.4 it must be that k′ − r < Q+ 3.

Note, however, that it cannot be that k− k′ + 1 ∈ {k− r−Q− 1, . . . , k− r}; otherwise,ak−r+2 = s by T k′(h) ∈ Λω′,k′ and k′ ≥ Q + 3, and ak−r+2 = s′ by h ∈ Hk,ω,r

4 . Furthermore,

it cannot be that k − k′ + 1 = k − r + 2; otherwise, ak−r+3 = s′ by T k′(h) ∈ Λω′,k′ and

ak−r+3 = s by h ∈ Hk,ω,r4 . Hence, k − k′ + 1 = k − r + 1, i.e. k′ = r.

Finally, k′ < n + ω′ + Q + 5 because otherwise, by T k′(h) ∈ Λω′,k′ , ak−k′+n+ω′+Q+5 = s′

and, by h ∈ Hk,ω,r4 and k′ = r, ak−r+n+ω+Q+5 = s.

22

Claim B.6 If h ∈ Hk,ω,r4 for some k ∈ {1, . . . ,M} and ω ∈ {0, . . . , ω} and r ∈ {1, . . . , n +

ω +Q+ 4}, then T k′(h) �∈ Σd′,k′ for all d′ ∈ N and k′ ∈ N such that d′ +Q+ 4 ≤ k′ < k.

Proof. Suppose otherwise. Let T k(h) = (a1, . . . , ak) and d ∈ N be such that ω ∈ Wd.

Since h ∈ Hk,ω,r4 and T k′(h) ∈ Σd′,k′ , it follows, respectively, that Br(h) ∈ Hk−r,ω,0

4 and

T k′−r(Br(h)) ∈ Σd′,k′−r if k′ > r. Then, by Claim B.4, k′ − r < Q + 3. Therefore, k − k′ +

d′ +Q+ 4 > k − r + 2. Hence k − k′ + d′ +Q+ 4 = k − r + t for some t > 2. But this is a

contradiction: by T k′(h) ∈ Σd′,k′ , ak−k′+d′+Q+4 ∈ D(s′) and, by h ∈ Hk,ω,r4 , ak−r+t = s.

Claim B.7 If h ∈ Hk,d,τ5 for some k ∈ {1, . . . ,M} and d ∈ N and τ ∈ {0, . . . , d + Q + 3},

then T k′(h) �∈ Σd′,k′ for all d′ ∈ N and k′ ∈ N such that d′ +Q+ 4 ≤ k′ < k.

Proof. Suppose not. Then there exist h = (a1, . . . , at) ∈ Hk,d,τ5 such that T k′(h) ∈ Σd′,k′ for

the parameters given in the statement of the claim. Note that, by h ∈ Hk,d,τ5 , T k(h) ∈ Σi,k

for some i ∈ N . We now derive a contradiction by considering five different possibilities.

Before doing so, let h = Bτ+1(h).

Case 1: k′ − (τ + 1) ≥ d′ +Q+ 4. Then T k′−(τ+1)(h) ∈ Σd′,k′−(τ+1) and h ∈ Hk−(τ+1),i3,a ∪

Hk−(τ+1),i3,b ∪H

k−(τ+1),ω,r4 for some ω ∈ {0, . . . , ω} and r ∈ {0, . . . , n + ω + Q + 4}. But this

contradicts either Claim B.4 or Claim B.6.

Case 2: k′− (τ +1) < d′ +Q+1. Then, by T k′(h) ∈ Σd′,k′ , at−k′+d′+Q+4 ∈ D(s′) and, by

h ∈ Hk,d,τ5 and t− k′ + d′ +Q+ 4 > t− τ + 2, at−k′+d′+Q+4 ∈ D(s); a contradiction.

Case 3: k′ − (τ + 1) = d′ +Q+ 1. Then, by T k′(h) ∈ Σd′,k′ , at−k′+d′+Q+3 ∈ D(s) and, by

h ∈ Hk,d,τ5 and t− k′ + d′ +Q+ 3 = t− τ + 1, at−k′+d′+Q+3 ∈ D(s′); a contradiction.

Case 4: h ∈ Hk−(τ+1),i3,a ∪ H

k−(τ+1),i3,b ∪ H

k−(τ+1),ω,04 for some ω ∈ {0, . . . , ω} and either

k′ − (τ + 1) = d′ + Q + 2 or d′ + Q + 3. Then Q + 3 ≤ k′ − (τ + 1) < k − (τ + 1) and

T k′−(τ+1)(h) ∈ Σd′,k′−(τ+1). This contradicts Claim B.4.

Case 5: h ∈ Hk−(τ+1),ω,r4 with ω ∈ {0, . . . , ω} and r > 0 and either k′−(τ+1) = d′+Q+3

or k′ − (τ + 1) = d′ + Q + 2. By T k′(h) ∈ Σd′,k′ , (i) at−k′+2 ∈ D(s′), (ii) at−k′+l ∈ D(s) for

all 3 ≤ l ≤ d′ + Q + 3 and (iii) at−k′+d′+Q+4 ∈ D(s′). By h ∈ Hk,d,τ5 and h ∈ H

k−(τ+1),ω,r4 ,

(iv) at−τ ∈ Dd(π(ω),r+1) and (v) at−τ−r′ = π(ω),r−r′+1 for all 1 ≤ r′ ≤ r. Next consider two

subcases.

Subcase 5A: k′ − (τ + 1) = d′ + Q + 3. By r > 0 and t − k′ + d′ + Q + 4 = t − τ , (iii)

and (iv) imply that either r = 1 or r = n+ ω +Q+ 4. If r = 1, by (v), at−τ−1 = π(ω),1 = s′.

23

This together with t − τ − 1 = t − k′ + d′ + Q + 3 contradicts (ii). If r = n + ω + Q + 4,

then, by (v) and n + ω − d′ + 3 ≥ 3, at−τ−d′−Q−2 = π(ω),n+ω−d′+3 = s. This together with

t− τ − d′ −Q− 2 = t− k′ + 2 contradicts (i).

Subcase 5B: k′ − (τ + 1) = d′ + Q + 2. Then by (ii) and t − τ = t − k′ + d′ + Q + 3,

at−τ ∈ Dd′(s). Therefore, by (iv), at−τ ∈ Dd(s) and d = d′.

Next, we show that r = d + Q + 2. If r > d + Q + 2, then, by (v), at−τ−d−Q−1 =

π(ω),r−d−Q = s. But this is a contradiction because t − k′ + 2 = t − τ − d − Q − 1 and, by

(i), at−k′+2 ∈ Dd(s′). If r < d + Q + 2, then, by (ii), at−k′+d−r+Q+4 ∈ D(s). But this is a

contradiction because t− τ −r+1 = t−k′+d−r+Q+4 and, by (v), at−τ−r+1 = π(ω),2 = s′.

By h ∈ Hk−(τ+1),ω,r4 and r = d + Q + 2, we have T d+Q+2(h) = ((s′; 2), (s; d + Q)). Since

at−τ ∈ Dd(s), it then follows from the definition of Hk,d,τ5 that τ = 0. But this contradicts

k′ − (τ + 1) = d′ +Q+ 2 and k′ ≥ d′ +Q+ 4.

Claim B.8 If T k(h) ∈ Σd,k ∩Σd′,k for some d, d′ ∈ N and k ∈ {min{d, d′}+Q+4, . . . ,M},then d = d′.

Proof. Suppose not; assume that d > d′. Let T k(h) = (a1, . . . , ak). Since T k(h) ∈ Σd,k,

ad′+Q+4 ∈ Dd(s) and, since T k(h) ∈ Σd′,k, ad

′+Q+4 ∈ Dd′(s′). But this is a contradiction.

Claim B.9 Let h ∈ H \ ∪5l=1Hl and a ∈ A. Then one of the following conditions hold: (a)

h · a �∈ ∪5l=1Hl, (b) h · a ∈ H

(ω),n+ω+Q+51,a for some ω ∈ {0, . . . , ω} and (c) T d+Q+4(h · a) ∈

Σd,d+Q+4 for some d ∈ N . Furthermore, if a ∈ D(s), then h · a �∈ ∪5l=1Hl.

Proof. Suppose that h · a does not satisfy (a)–(c). Then there are six cases to consider.

(i) h · a ∈ H(ω),k1,a for some ω and k > n + ω + Q + 5: then h ∈ H1,a; a contradiction. (ii)

h·a ∈ H1,b: then h ∈ H1,b; a contradiction. (iii) h·a ∈ H2: then h ∈ H1∪H2; a contradiction.

(iv) h · a ∈ H3 and T d+Q+4(h · a) �∈ Σd,d+Q+4 for all d ∈ N : then h ∈ H3; a contradiction.

(v) h · a ∈ H4: then h ∈ H3 ∪H4; a contradiction. (vi) h · a ∈ H5: then h ∈ H3 ∪H4 ∪H5; a

contradiction.

Furthermore, if a ∈ D(s), by the definition of H(ω),n+ω+Q+51,a and Σd,d+Q+4, (b) and (c)

cannot hold. Therefore, (a) must hold, i.e. h · a �∈ ∪5l=1Hl.

Claim B.10 If h ∈ H(ω),M1,a for some ω ∈ {0, . . . , ω}, then h ∈ H

(ω),k1,a for some k < M .

Proof. Since h ∈ H(ω),M1,a , TM(h) = (π(ω),1, . . . , π(ω),M). Also, by T > K and by the choice

of M , M > K + n+ ω +Q+ 5. Therefore, h ∈ H(ω),M−K1,a .

24

Claim B.11 If h ∈ HM,d,τ2 for some d ∈ N and τ ∈ {0, . . . , d+Q+ 3}, then h ∈ Hk,d,τ

2 for

some k < M .

Proof. Since h ∈ HM,d,τ2 , TM(h) = h · a · h, where h, a, and h are as in the definition of

HM,d,τ2 . In particular, h ∈ H

(ω),k′1,a ∪ H

(0),k′1,b for some ω ∈ {0, . . . , ω} and k′ ≤ M − (τ + 1).

Therefore, h ∈ Hk′+τ+1,d,τ2 .

If k′+ τ +1 < M , the claim follows. If k′+ τ +1 = M , then h = (π(ω),1, . . . , π(ω),M−(τ+1))

for some ω ∈ {0, . . . , ω}. Also, by T > K and by the choice of M , M ≥ (n+Q+ 4) + (n+

ω +Q+ 5) +K. Therefore, h ∈ H(ω),M−(τ+1)−K1,a , hence, h ∈ HM−K,d,τ

2 .

Claim B.12 If h ∈ Hk,d3,a ∪Hk,d

3,b for some k ∈ {1, . . . ,M} and d ∈ N , then k < M .

Proof. We have that k ≤ (γ + 2)(d+Q+ 4) + T < M .

Claim B.13 If h ∈ Hk,ω,r4 for some k ∈ {1, . . . ,M} and ω ∈ {0, . . . , ω} and r ∈ {0, . . . , n+

ω +Q+ 4}, then k < M .

Proof. We have that k ≤ (γ + 2)(n+Q+ 4) + T + n+ ω +Q+ 4 < M .

Claim B.14 If h ∈ Hk,d,τ5 for some k ∈ {1, . . . ,M} and d ∈ N and τ ∈ {0, . . . , d+Q+ 3},

then k < M .

Proof. Let h, a and h be such that T k(h) = h·a·h and satisfy the conditions in the definition

of Hk,d,τ5 . By the proof of Claims B.12 and B.13, �(h) < (γ+2)(n+Q+4)+T +n+ ω+Q+4.

Therefore, k = �(h) + �(a · h) ≤ [(γ + 2)(n+Q+ 4) + T + n+ ω +Q+ 4] + 1 + n+Q+ 3 =

(γ + 3)(n+Q+ 4) + T + n+ ω +Q+ 4 ≤M .

B.2.2 f is well-defined

Now we show that f is well-defined.

Claim B.15 If h ∈ H(ω),k1,a ∩ H

(ω′),k′1,a for some ω, ω′ ∈ {0, . . . , ω} and k ∈ {n + ω + Q +

5, . . . ,M} and k′ ∈ {n+ ω′ +Q+ 5, . . . ,M}, then ω = ω′ and k = k′ + βK for some β ∈ Z.

Proof. It follows immediately by Claim B.3.

Claim B.16 For all ω ∈ {0, . . . , ω} and k ∈ {n + ω + Q + 5, . . . ,M} and k′ ∈ {0, . . . , n +

Q+ 4}, H(ω),k1,a ∩H

(0),k′1,b = ∅.

Proof. If h ∈ H(0),k′1,b , then �(h) < n+Q+5; whereas if h ∈ H

(ω),k1,a , then �(h) ≥ n+ω+Q+5.

Hence, H(ω),k1,a ∩H

(0),k′1,b = ∅.

25

Claim B.17 For all ω ∈ {0, . . . , ω} and k ∈ {n + ω + Q + 5, . . . ,M} and k′ ∈ {1, . . . ,M}and d ∈ N and τ ∈ {0, . . . , d+Q+ 3}, H(ω),k

1,a ∩Hk′,d,τ2 = ∅.

Proof. Suppose not; then there exist h = (a1, . . . , at) ∈ H(ω),k1,a ∩ Hk′,d,τ

2 for some ω, k, k′,

d and τ as described in the claim. Since h ∈ H(ω),k1,a , then π(ω),r = at−k+r for all 1 ≤ r ≤ k.

Also since h ∈ Hk′,d,τ2 , then T k′(h) = h · at−τ · h where h ∈ Σd,τ , at−τ ∈ Dd(π

(ω′),k′−τ ) and

h ∈ H(ω′),k′−(τ+1)1 for some ω′ ∈ {0, . . . , ω} satisfying either k′ − (τ + 1) ≥ n+ ω′ +Q+ 5 or

ω′ = 0, k′ = t and k′ − (τ + 1) < n+Q+ 5. We consider each of these separately.

Case 1: h ∈ H(0),k′−(τ+1)1,b , k′ = t and k′ − (τ + 1) < n + Q + 5. In this case ak

′−k+1 =

ak′−k+2 = s′, h = ((s′; 2), s, . . . , s) and �(h) = k′ − (τ + 1). Thus, since τ ≤ d + Q + 3

and k ≥ n + ω + Q + 5, it must be that k′ − k + 1 < k′ − τ and so k′ = k. But then

at−τ ∈ Dd(s) ∪ Dd(s′) and at−τ = π(ω),t−τ ∈ {s, s′}, a contradiction.

Case 2: h ∈ H(ω′),k′−(τ+1)1,a and k′ − (τ + 1) ≥ n+ ω′ +Q+ 5. We consider four subcases.

Subcase 1: k ≥ n+ω+Q+5+τ +1. Let h = Bτ+1(h). Since k− (τ +1) ≥ n+ω+Q+5,

it follows that h ∈ H(ω),k−(τ+1)1,a . Also, since T k′−(τ+1)(h) = h, h ∈ H

(ω′),k′−(τ+1)1,a . Then, by

Claim B.15, ω = ω′ and k − (τ + 1) = k′ − (τ + 1) + βK for some β ∈ Z. This together

with a = π(ω),k−τ and a ∈ Dd(π(ω′),k′−τ ) implies that π(ω),k′−τ = π(ω),k−τ ∈ Dd(π

(ω),k′−τ ), a

contradiction.

Subcase 2: k = n + ω + Q + 5 + τ . By h ∈ H(ω),k1,a , at−τ−1 = s and at−τ = s′. Also, by

h ∈ Hk′,d,τ2 , at−τ−1 = π(ω′),k′−τ−1 and at−τ ∈ Dd(π

(ω′),k′−τ ). Hence, it follows from at−τ−1 = s

and at−τ = s′, respectively, that π(ω′),k′−τ−1 = s and π(ω′),k′−τ ∈ Dd(s′). But this contradicts

Claim B.1.4.

Subcase 3: k = n+ ω+Q+4+ τ . First, we show that k′ ≥ k. Suppose otherwise. Since

h ∈ H(ω′),k′−(τ+1)1,a , then T k′(h) ∈ Λω′,k′ . By Claim B.3, this together with h ∈ H

(ω),k1,a implies

that k = k′ + βK for some β ∈ N. Also, by the supposition that k < n+ ω +Q+ 5 + τ + 1

and k′− (τ +1) ≥ n+ω′+Q+5, we have k−k′ ≤ n. Since n < K, we have a contradiction.

By h ∈ H(ω),k1,a , (at−τ−(n+Q+3), . . . , at−τ ) = ((s′; 2), (s;n + ω + Q + 2)); by h ∈ Hk′,d,τ

2 ,

(at−τ−(n+ω+Q+3), . . . , at−τ ) = (π(ω′),k′−τ−(n+ω+Q+3), . . . , π(ω′),k′−τ−1, at−τ ). Hence, it follows

from at−τ = s that π(ω′),k′−τ ∈ Dd(s). But this contradicts Claim B.1.2.

Subcase 4: k < n+ ω +Q+ 4 + τ . By k ≥ n+ ω +Q+ 5 and τ ≤ d+Q+ 3, it follows

that τ > 0 and k − τ ≥ 2. Hence, by h ∈ Hk′,d,τ2 , at−τ+1 ∈ D(s′) and, by h ∈ H

(ω),k1,a and

26

k − τ < n+ ω +Q+ 4, at−τ+1 = s; a contradiction.

Claim B.18 For all k ∈ {0, . . . , n + Q + 4} and k′ ∈ {1, . . . ,M} and d ∈ N and τ ∈{0, . . . , d+Q+ 3}, H(0),k

1,b ∩Hk′,d,τ2 = ∅.

Proof. Suppose otherwise; then there exists h ∈ H(0),k1,b ∩ Hk′,τ,d

2 . This means that h =

(π(0),1, . . . , π(0),k) and T k′(h) = h · a · h satisfying the remaining conditions in the definition

of Hk′,d,τ2 . Therefore, for some k < n+Q+ 5, h · a ∈ H

(0),k+11,b . But this is a contradiction as

a ∈ D(π(0),k+1) due to h ∈ Hk′,d,τ2 .

Claim B.19 If h ∈ Hk,d,τ2 ∩ Hk′,d′,τ ′

2 for some k, k′ ∈ {1, . . . ,M} and d, d′ ∈ N and τ ∈{0, . . . , d+Q+3} and τ ′ ∈ {0, . . . , d′ +Q+3}, then τ = τ ′ and d = d′ and k = k′ + βK for

some β ∈ Z.

Proof. Let h = (a1, . . . , at). First, we establish that τ = τ ′. Suppose, without loss

of generality, that τ < τ ′. Define h = Bτ+1(h) = (a1, . . . , at−(τ+1)) and note that h ∈(∪ω

i=0H(ω),k−(τ+1)1

)∩H

k′−(τ+1),d′,τ ′−(τ+1)2 . But this contradicts Claims B.17 or B.18.

By τ = τ ′ and the definition of Hk,d,τ2 and Hk′,d′,τ ′

2 , we have that h ∈ H(ω),k−(τ+1)1 and

h ∈ H(ω′),k′−(τ ′+1)1 for some ω, ω′ ∈ {0, . . . , ω}. It then follows from Claims B.15 and B.16

that k = k′ + βK for some β ∈ Z and ω = ω′. Also, by h ∈ Hk,d,τ2 , at−τ = (ad, π

(ω),k−τ−d )

and, by h ∈ Hk′,d′,τ ′2 , at−τ ′ = (ad′ , π

(ω′),k′−τ ′−d′ ). Since ω = ω′, k = k′ + βK and τ = τ ′, then

(ad, π(ω),k−τ−d ) = (ad′ , π

(ω),k−τ−d′ ) and so d = d′.

Claim B.20 For all ω ∈ {0, . . . , ω} and k ∈ {n + ω + Q + 5, . . . ,M} and k′ ∈ {1, . . . ,M}and d ∈ N , H

(ω),k1,a ∩Hk′,d

3 = ∅ where Hk′,d3 = (Hk′,d

3,a ∪Hk′,d3,b ).

Proof. Suppose that h ∈ H(ω),k1,a ∩ Hk′,d

3 . Since Hk′,d3 ⊆ Σd,k′ and k′ ≥ d + Q + 4 (due to

h ∈ Hk′,d3 ), we have a contradiction to Claim B.2 when k ≥ k′. Also, since H(ω),k

1,a ⊆ Λω,k and

k ≥ n+ ω +Q+ 5, we have a contradiction to Claim B.4 when k < k′.

Claim B.21 For all k ∈ {0, . . . , n+Q+4} and k′ ∈ {1, . . . ,M} and d ∈ N , H(0),k1,b ∩Hk′,d

3 =

∅.Proof. Suppose there exists h = (a1, . . . , at) ∈ H

(0),k1,b ∩Hk′,d

3 . By h ∈ Hk′,d3 , at−k′+d+Q+4 ∈

D(s′) and, by h ∈ H(0),k1,b , ar = s for all r > 2. But this is a contradiction as it implies

d+Q+ 4 ≤ t− k′ + d+Q+ 4 ≤ 2.

Claim B.22 For all k, k′ ∈ {1, . . . ,M} and d, d′ ∈ N and τ ∈ {0, . . . , d +Q + 3}, Hk,d,τ2 ∩

Hk′,d′3 = ∅.

27

Proof. Suppose that h = (a1, . . . , at) ∈ Hk,d,τ2 ∩Hk′,d′

3 . There are two possibilities.

Case 1: k′ > k. First, note that, for some ω ∈ {0, . . . , ω}, Bτ+1(h) ∈ H(ω),k−(τ+1)1,a with

k − (τ + 1) ≥ n + ω + Q + 5; otherwise, Bτ+1(h) ∈ H(0),k−(τ+1)1,b and �(h) = k < k′; a

contradiction.

It follows from Bτ+1(h) ∈ H(ω),k−(τ+1)1,a that, for some ω ∈ {0, . . . , ω} and k−(τ+1) ≥ n+

ω+Q+5, T k(h) ∈ Λω,k contradicting Claim B.4 because h ∈ Hk′,d′3 and k′ > k ≥ n+ω+Q+5.

Case 2: k ≥ k′. There are two possibilities.

Subcase 1: k′ − τ > d′ + Q + 4. In this case, Bτ+1(h) belongs to H(ω),k−(τ+1)1 , for some

ω ∈ {0, . . . , ω}, and to Hk′−(τ+1),d′3 . But this contradicts Claim B.20 or B.21.

Subcase 2: k′ − τ ≤ d′ + Q + 4. Since h ∈ Hk′,d′3 , we have (i) at−k′+r ∈ Dd′(s

′) for

r = 1, 2, (ii) at−k′+r ∈ Dd′(s) for r = 3, . . . , d′+Q+3, and (iii) at−k′+d′+Q+4 ∈ Dd′(s′). When

k′ − τ = d′ + Q + 4, by (i), (ii) and Bτ+1(h) ∈ H(ω),k−(τ+1)1 for some ω ∈ {0, . . . , ω}, Claim

B.1.3 implies that (at−τ−d′−Q−3, . . . , at−τ−1) = ((s′; 2), (s; d′ +Q+1)) and at−τ ∈ Dd(s). But

the latter contradicts (iii). Therefore, it must be that k′ − τ < d′ +Q+ 4.

By h ∈ Hk,d,τ2 , it must be that (iv) at−τ+r ∈ Dd(s

′) for all r = 1, 2 and (v) at−τ+r ∈ Dd(s)

for all r ≥ 3. But then by t − k′ + d′ + Q + 4 > t − τ , (iii) and (v), it must be that either

t − k′ + d′ + Q + 4 = t − τ + 1 or t − k′ + d′ + Q + 4 = t − τ + 2. The latter, however,

cannot hold because, by (ii), at−k′+d′+Q+3 ∈ D(s) and, by (iv), at−τ+1 ∈ D(s′). Therefore,

assume the former. This together with (i) and (ii) implies Bτ+1(h) ∈ H(ω),k−(τ+1)1 for some

ω ∈ {0, . . . , ω} and Claim B.1.3 imply that (at−k′+1, . . . , at−k′+d′+Q+2) = ((s′; 2), (s; d′ +Q))

and at−k′+d′+Q+3 ∈ Dd(s). But the latter together with (ii) implies d = d′. Hence, by part

(4) of the definition of Hk,d,τ2 , τ = 0. Thus, k′ < d′ +Q+ 4; but this contradicts h ∈ Hk′,d′

3 .

Claim B.23 If h ∈ Hk,d3 ∩ Hk′,d′

3 for some k, k′ ∈ {1, . . . ,M} and d, d′ ∈ N , then k = k′

and d = d′.

Proof. First we show that k = k′. Suppose otherwise and assume, without loss of generality,

that k > k′. By h ∈ Hk′,d′3 , we have T k′(h) ∈ Σd′,k′ and k > k′ ≥ d′ + Q + 4. But this

contradicts Claim B.4 because h ∈ Hk,d3 .

To show that d = d′, by h ∈ Hk,d3 ∩Hk,d′

3 , we have T k(h) ∈ Σd,k∩Σd′,k where k ≥ d+Q+4

and k ≥ d′ +Q+ 4. Hence, by Claim B.8, d = d′.

28

Claim B.24 For all ω, ω′ ∈ {0, . . . , ω} and k ∈ {n+ω+Q+5, . . . ,M} and k′ ∈ {1, . . . ,M}and r ∈ {0, . . . , n+ ω′ +Q+ 4}, H(ω),k

1,a ∩Hk′,ω′,r4 = ∅.

Proof. Suppose that h ∈ H(ω),k1,a ∩ Hk′,ω′,r

4 and let d ∈ N be such that ω′ ∈ Wd. Then

T k(h) ∈ Λω,k where k ≥ n+ω+Q+5 and T k′(h) ∈ Σd,k′ and k′ ≥ d+Q+4; a contradiction

to Claim B.2 if k ≥ k′ and to Claims B.4 and B.5 if k < k′.

Claim B.25 For all k ∈ {0, . . . , n + Q + 4} and k′ ∈ {1, . . . ,M} and ω ∈ {0, . . . , ω} and

r ∈ {0, . . . , n+ ω +Q+ 4}, H(0),k1,b ∩Hk′,ω,r

4 = ∅.Proof. Suppose there exists h = (a1, . . . , at) ∈ H

(0),k1,b ∩Hk′,ω,r

4 . By h ∈ Hk′,ω,r4 , at−k′+d+Q+4 ∈

D(s′) and, by h ∈ H(0),k1,b , aτ = s for all τ > 2. But this is a contradiction as it implies

d+Q+ 4 ≤ t− k′ + d+Q+ 4 ≤ 2.

Claim B.26 For all k, k′ ∈ {1, . . . ,M} and d ∈ N and ω ∈ {0, . . . , ω} and r ∈ {0, . . . , n +

ω +Q+ 4} and τ ∈ {0, . . . , d+Q+ 3}, Hk,d,τ2 ∩Hk′,ω,r

4 = ∅.Proof. Suppose that h ∈ Hk,d,τ

2 ∩Hk′,ω,r4 and let d′ ∈ N be such that ω ∈ Wd′ . By h ∈ Hk,d,τ

2 ,

Bτ+1(h) ∈ H(ω′),k−(τ+1)1 for some ω′ ∈ {0, . . . , ω}. And since h ∈ Hk′,ω,r

4 and T > τ + 1 (by

(50)) Bτ+1(h) belongs to Hk′−(τ+1),ω,r−(τ+1)4 if r − (τ + 1) ≥ 0 or to H

k′−(τ+1),d′3 otherwise.

But, by Claims B.20, B.21, B.24 or B.25, this is a contradiction.

Claim B.27 For all k, k′ ∈ {1, . . . ,M} and d ∈ N and ω ∈ {0, . . . , ω} and r ∈ {0, . . . , n +

ω +Q+ 4}, Hk,d3 ∩Hk′,ω,r

4 = ∅.Proof. Suppose that h ∈ Hk,d

3 ∩ Hk′,ω,r4 and let d′ ∈ N be such that ω ∈ Wd′ . Assume

first that k = k′. By Claim B.8, this implies d = d′. Since h ∈ Hk,d3 , it follows that

k < (γ + 2)(d + Q + 4) + T and, since h ∈ Hk′,ω,r4 with k = k′ and d = d′, it follows that

k ≥ (γ + 2)(d+Q+ 4) + T . But this is a contradiction.

Suppose next that k > k′. Then, by h ∈ Hk′,ω,r4 , T k′(h) ∈ Σd′,k′ with d′ +Q+4 ≤ k′ < k.

But this together with h ∈ Hk,d3 contradicts Claim B.4.

Finally, suppose that k′ > k. Then, by h ∈ Hk,d3 , T k(h) ∈ Σd,k with d+Q+ 4 ≤ k < k′.

But this together with h ∈ Hk′,ω,r4 contradicts Claims B.4 and B.6.

Claim B.28 If h ∈ Hk,ω,r4 ∩Hk′,ω′,r′

4 for some k, k′ ∈ {1, . . . ,M} and ω, ω′ ∈ {0, . . . , ω} and

r, r′ ∈ N0, then k = k′, ω = ω′ and r = r′.

29

Proof. Let d ∈ N be such that ω ∈ Wd and d′ ∈ N be such that ω′ ∈ Wd′ . To show that

k = k′, without loss of generality, suppose that k > k′. Then, by h ∈ Hk′,ω′,r′4 , T k′(h) ∈ Σd′,k′

with d′ + Q + 4 ≤ k′ < k. But this together with h ∈ Hk,ω,r4 contradicts Claims B.4 and

B.6. Hence, k = k′ and, by Claim B.8, d = d′. Thus, r = k − ((γ + 2)(d + Q + 4) + T ) =

k′ − ((γ + 2)(d′ + Q + 4) + T ) = r′. Furthermore, letting h = Br(T k(h)) = Br′(T k′(h)), it

follows that ω = ω(d, T T (h)) = ω(d′, T T (h)) = ω′.

Claim B.29 For all ω ∈ {0, . . . , ω} and k ∈ {n + ω + Q + 5, . . . ,M} and k′ ∈ {1, . . . ,M}and d ∈ N and τ ∈ {0, . . . , d+Q+ 3}, H(ω),k

1,a ∩Hk′,d,τ5 = ∅.

Proof. Suppose that h = (a1, . . . , at) ∈ H(ω),k1,a ∩Hk′,d,τ

5 . Then h ∈ H(ω),k1,a , T k′(h) ∈ Σd,k′ and

k′ ≥ d+Q+ 4. Therefore, k′ > k; otherwise we would contradict Claim B.2.

Consider next the case k′ > k and let h = Bτ+1(h). Then h ∈ Hk′−(τ+1),d′3 ∪H

k′−(τ+1),ω′,r4

for some d′ ∈ N and ω′ ∈ {0, . . . , ω} and 0 ≤ r ≤ n + ω + Q + 4. Also, by h ∈ H(ω),k1,a ,

T k−(τ+1)(h) ∈ Λω,k−(τ+1). Furthermore, since 0 ≤ τ ≤ d +Q + 3 ≤ n +Q + 3, we have that

k − (τ + 1) ≥ 1. Therefore, by Claims B.4 and B.5, one of the following must hold: (1)

1 ≤ k− (τ +1) ≤ Q+2 or (2) h ∈ Hk′−(τ+1),ω′,r4 and Q+3 ≤ k− (τ +1) = r < n+ω+Q+5.

Case (1) implies that t − τ + 1 ∈ {t − k + 3, . . . , t − k + Q + 4}. By T k(h) ∈ H(ω),k1,a ,

at−τ+1 = s; and, by h ∈ Hk′,d,τ5 , T τ (h) ∈ Σd,τ . These imply at−τ+1 ∈ Dd(s

′); a contradiction.

In case (2), t − τ ≤ t − k + n + ω + Q + 5. This together with h ∈ H(ω),k1,a implies that

at−τ ∈ {s, s′}. Since h ∈ Hk′,d,τ5 and h ∈ H

k′−(τ+1),ω′,r4 and π(ω′),r+1 ∈ {s, s′}, it must be that

at−τ ∈ Dd(s) ∪ Dd(s′); a contradiction.

Claim B.30 For all k ∈ {0, . . . , n + Q + 4} and k′ ∈ {1, . . . ,M} and d ∈ N and τ ∈{0, . . . , d+Q+ 3}, H(0),k

1,b ∩Hk′,d,τ5 = ∅.

Proof. Suppose h = (a1, . . . , ak) ∈ H(0),k1,b ∩Hk′,d,τ

5 . By h ∈ Hk′,d,τ5 , we have ak−k′+1, at−k′+i+Q+4 ∈

D(s′) for some i ∈ N ; and, by h ∈ H(0),k1,b , h = ((s′; 2), (s; k − 2)); a contradiction.

Claim B.31 For all k, k′ ∈ {1, . . . ,M}, d, d′ ∈ N , τ ∈ {0, . . . , d + Q + 3} and τ ′ ∈{0, . . . , d′ +Q+ 3}, Hk,d,τ

2 ∩Hk′,d′,τ ′5 = ∅.

Proof. Suppose that h ∈ Hk,d,τ2 ∩ Hk′,d′,τ ′

5 . Assume first that τ ≤ τ ′. Then Bτ+1(h) ∈ H1

and Bτ+1(h) ∈ H3 ∪H4 ∪H5. But this contradicts Claims B.20, B.21, B.24, B.25, B.29 or

B.30. If τ > τ ′, then Bτ ′+1(h) ∈ H2 and Bτ ′+1 ∈ H3∪H4 contradicting Claim B.22 or Claim

B.26.

30

Claim B.32 For all k, k′ ∈ {1, . . . ,M} and d, d′ ∈ N and τ ∈ {0, . . . , d′ + Q + 3}, Hk,d3 ∩

Hk′,d′,τ5 = ∅.

Proof. Suppose that h = (a1, . . . , at) ∈ Hk,d3 ∩ Hk′,d′,τ

5 . By h ∈ Hk,d3 , T k(h) ∈ Σd,k with

k ≥ d + Q + 4; by h ∈ Hk′,d′,τ5 , T k′(h) ∈ Σi′,k′ with k′ ≥ i′ + Q + 4 for some i′ ∈ N . By

appealing to Claims B.4, B.7 and B.8, it then follows that k = k′ and d = i′.

By h ∈ Hk′,d′,τ5 and the last two equalities, k−(τ+1) ≥ d+Q+4. But then, by h ∈ Hk,d

3 ,

Bτ+1(h) ∈ Hk−(τ+1),d3 . This and h ∈ Hk′,d′,τ

5 imply at−τ ∈ Dd′(f(Bτ+1(h))) and d′ �= d. Since

h ∈ Hk,d3 and k − (τ + 1) ≥ d+Q+ 4, at−τ ∈ Dd(f(B

τ+1(h))); a contradiction.

Claim B.33 For all k, k′ ∈ {1, . . . ,M} and d′ ∈ N and ω ∈ {0, . . . , ω} and r ∈ {0, . . . , n+

ω +Q+ 4} and τ ∈ {0, . . . , d′ +Q+ 3}, Hk,ω,r4 ∩Hk′,d′,τ

5 = ∅.Proof. Suppose that h = (a1, . . . , at) ∈ Hk,ω,r

4 ∩Hk′,d′,τ5 . By h ∈ Hk,ω,r

4 , there is some d ∈ N

such that T k(h) ∈ Σd,k with k ≥ d+Q+4; by h ∈ Hk′,d′,τ5 , T k′(h) ∈ Σi′,k′ with k′ ≥ i′+Q+4

for some i′ ∈ N . So by Claims B.6 and B.7, k = k′.

Suppose r ≥ τ +1. By h ∈ Hk,ω,r4 , Bτ+1(h) ∈ H

k−(τ+1),ω,r−(τ+1)4 implying at−τ = π(ω),r−τ .

By h ∈ Hk′,d′,τ5 and Claims B.27 and B.28, at−τ ∈ Dd′(π

(ω),r−τ ). But this is a contradiction.

Finally, suppose r < τ+1. By h ∈ Hk,ω,r4 , Bτ+1(h) ∈ H

k−(τ+1),d3 . This implies that at−τ ∈

Dd(f(Bτ+1(h))). Also, by h ∈ Hk′,d′,τ

5 and Claims B.23 and B.27, at−τ ∈ Dd′(f(Bτ+1(h)))

and d′ �= d. But this is a contradiction.

Claim B.34 If h ∈ Hk,d,τ5 ∩ Hk′,d′,τ ′

5 for some k, k′ ∈ {1, . . . ,M} and d, d′ ∈ N and τ ∈{0, . . . , d+Q+ 3} and τ ′ ∈ {0, . . . , d′ +Q+ 2}, then k = k′ and τ = τ ′ and d = d′.

Proof. Suppose h ∈ Hk,d,τ5 ∩Hk′,d′,τ ′

5 . First, note that τ = τ ′. Otherwise, say τ < τ ′; then, by

h ∈ Hk,d,τ5 , Bτ+1(h) ∈ H5 and, by h ∈ Hk′,d′,τ ′

5 , Bτ+1(h) ∈ H3 ∪H4. This contradicts Claim

B.32 or Claim B.33. Second, note that Bτ+1(h) ∈(∪i∈NH

k−(τ+1),i3

)∪(∪ω,rH

k−(τ+1),ω,r4

)and Bτ ′+1(h) ∈

(∪i∈NH

k′−(τ+1),i′3

)∪(∪ω,rH

k′−(τ ′+1),ω,r4

). Since τ = τ ′, by Claims B.23, B.27

and B.28, k = k′. Finally, it follows immediately from the definitions of Hk,d,τ5 and Hk′,d′,τ ′

5 ,

k = k′ and τ = τ ′ that d = d′.

Claim B.35 If h ∈ Hd,τ6 ∩ Hd′,τ ′

6 for some d, d′ ∈ N and τ ∈ {0, . . . , d + Q + 3} and

τ ′ ∈ {0, . . . , d′ +Q+ 3}, then d = d′.

Proof. Let h = (a1, . . . , at) ∈ Hd,τ6 ∩ Hd′,τ ′

6 . We may assume, without loss of generality,

that τ ≥ τ ′. Then, by h ∈ Hd′,τ ′6 , at−τ ′ ∈ Dd′(s) ∪ Dd′(s

′); and, by τ ≥ τ ′ and h ∈ Hd,τ6 ,

at−τ ′ ∈ Dd(s) ∪Dd(s′). Thus, d = d′.

31

B.2.3 Outcome paths induced by f and by one-shot deviations from f

Claim B.36 If h ∈ H(ω),k1,a for some ω ∈ {0, . . . , ω} and k ∈ {n + ω + Q + 5, . . . ,M}, then

π(f |h) = (π(ω),k+1, π(ω),k+2, . . .).

Proof. By Claim B.10, we may assume that k < M . Hence, f(h) = π(ω),k+1 and h · f(h) ∈H

(ω),k+11,a . Thus, by induction, π(f |h) = (π(ω),k+1, π(ω),k+2, . . .).

Claim B.37 If h ∈ H(0),k1,b for some k ∈ {0, . . . , n+Q+4}, then π(f |h) = (π(0),k+1, π(0),k+2, . . .).

Proof. If k < n+Q+4, then h ·f(h) ∈ H(0),k+11,b . If k = n+Q+4, then h ·f(h) ∈ H

(0),n+Q+51,a .

Thus, by induction and Claim B.36, π(f |h) = (π(0),k+1, π(0),k+2, . . .).

Claim B.38 If h ∈ Hk,ω,r4 for some k ∈ {1, . . . ,M}, ω ∈ {1, . . . , ω} and r ∈ {0, . . . , n+ω+

Q+ 4}, then π(f |h) = (π(ω),r+1, π(ω),r+2, . . .).

Proof. If r = n+ω+Q+4, then h·f(h) ∈ H(ω),n+ω+Q+51,a . Thus, π(f |h) = (π(ω),r+1, π(ω),r+2, . . .)

by Claim B.36. If r < n+ ω +Q+ 4, then f(h) = π(ω),r+1. Therefore, h · f(h) ∈ Hk+1,ω,r+14 .

Furthermore, by Claim B.13, k < M . Hence, by induction, π(f |h) = (π(ω),r+1, π(ω),r+2, . . .).

Claim B.39 If h ∈ Hk,d3,b for some d ∈ N and k ∈ {1, . . . ,M} and T k−(γ+2)(d+Q+4)(h) = h,

then for any h′ ∈ HT−k+(γ+2)(d+Q+4) the strategy f |h induces the infinite path

h′ · (π(ω(d,h·h′)),1, π(ω(d,h·h′)),2, . . .)

with probability

Pfd|h,T−k+(γ+2)(d+Q+4)(h′).

Furthermore, if k = (γ+2)(d+Q+4), then Pfd|h,T ({h′ ∈ HT : uω(d,h′)i = u′

i+ρ}) > 1−2ε2

for all i �= d.

Proof. If TQ+1(h) �∈ ∪l∈N Σl,Q+1 for all l ∈ N then, for any a ∈ A, we have h · a ∈H

k+1,ω(d,TT (h·a)),04 if k−(γ+2)(d+Q+4) = T −1 and h ·a ∈ Hk+1,d

3,b if k−(γ+2)(d+Q+4) <

T − 1. Similarly, if TQ+1(h) ∈ Σl,Q+1 for some l ∈ N , then h · f(h) ∈ Hk+1,ω(d,TT (h·f(h))),04 if

k − (γ + 2)(d + Q + 4) = T − 1 and h · f(h) ∈ Hk+1,d3,b if k − (γ + 2)(d + Q + 4) < T − 1.

Also, by Claim B.12, k < M . Therefore, by Claim B.38, the first part of the claim follows

by induction.

Furthermore, since {h′ ∈ HT : uω(d,h′)i = u′

i + ρ} = {h′ ∈ HT : Φi(d, h′) < η}, Lemma B.1

implies that if k = (γ + 2)(d +Q + 4), then Pfd|h,T ({h′ ∈ HT : uω(d,h′)i = u′

i + ρ}) > 1− 2ε2

for all i �= d.

32

Claim B.40 Suppose that h ∈ Hk,d3,a for some d ∈ N and k ∈ {1, . . . ,M} and let

h′ = (a(1); γ), (s∗d, s′−d), . . . , (a(d+Q+ 4); γ), (s∗d, s

′−d)),

where, for all l = 1, . . . , d+Q+ 4,

a(l) =

⎧⎨⎩ a(d) if al ∈ {s′, s},md if al �∈ {s′, s}.

Then strategy f |h induces T (γ+2)(d+Q+4)−k(h′) in the first (γ + 2)(d + Q + 4) − k periods.

Furthermore, h · T (γ+2)(d+Q+4)−k(h′) ∈ Hd,(γ+2)(d+Q+4)3,b .

Proof. If k = (γ + 2)(d + Q + 4) − 1, then f(h) = (s∗d, s′−d) and h · f(h) ∈ Hk+1,d

3,b . Hence,

the conclusion follows. If k < (γ + 2)(d +Q + 4)− 1, then h · f(h) ∈ Hk+1,d3,a and the result

follows by induction and the definition of f .

Next, for any d ∈ N , let

(a(d),1, . . . , a(d),d+Q+4) = (s′, s′, (s; d+Q+ 1), s′).

Claim B.41 Let h ∈ Hk,d,τ2 ∪ Hk,d,τ

5 ∪ Hd,τ6 for some k ∈ {1, . . . ,M} and d ∈ N and

τ ∈ {0, . . . , d + Q + 3}. Then for all a ∈ Dd(f(h)), we have either h · a ∈ Hd+Q+4,d3,a or

h · a · (π1(f |h · a), . . . , πt(f |h · a)) ∈ Hd+Q+4,d3,a for some t ∈ {1, . . . , d+Q+ 3}.

Proof. We establish this claim by considering the different possible cases.

Case 1: One of the following conditions hold: (a) τ = d+Q+3, (b) τ = 0, T d+Q+3(h) =

((s′; 2), (s; d + Q), a) and a ∈ Dd(s), and (c) h ∈ Hd,06 and T d+Q+3(h) ∈ Σd,d+Q+3. Since

f(h) = s′ it must be that a ∈ Dd(s′) and h · a ∈ Hd+Q+4,d

3,a .

Case 2: h ∈ Hk,d,τ2 ∪Hk,d,τ

5 and none of the conditions (a)–(c) in case 1 hold. In this case,

let t = d+Q+ 3− τ . Indeed, we have that h · a ∈ Hk+1,d,τ+12 ∪Hk+1,d,τ+1

5 . Thus, by Claims

B.11 and B.14, f(h · a) = a(d),τ+2. Then, by induction, it follows that f(h · a · a(d),τ+2) =

a(d),τ+3, . . . , f(h · a · (a(d),τ+2, . . . , a(d),d+Q+3)) = a(d),d+Q+4.

Case 3: h ∈ Hd,τ6 and none of the conditions (a)–(c) in case 1 hold. For any history

h′ ∈ H, define v(h′) = max{t ∈ {0, . . . , d+Q+3} : T t(h′) ∈ Hd,t6 } and let τ ′ = v(h). In this

case, let t = d + Q + 3 − τ ′. Since (a) in case 1 does not hold, τ ′ < d + Q + 3 . Moreover,

(i) h ∈ Hd,τ ′6 , (ii) T 1(h) ∈ Dd(s) ∪ Dd(s

′) if τ ′ = 0, (iii) T 1(h) ∈ Dd(s′) if τ ′ = 1 and (iv)

h · a ∈ Σd,τ ′+1. To complete the proof in this case, we first establish two subclaims.

33

Subclaim 1: h · a ∈ Hd,τ ′+16 . Suppose not. By (iv), h · a ∈ ∪5

l=1Hl. Also, if τ ′ ≥ 2, then

a ∈ D(s) (recall that τ ′ < d + Q + 3). Therefore, by Claim B.9, τ ′ = 0 or 1 and either

h · a ∈ H(ω),n+ω+Q+51,a for some ω ∈ {0, . . . , ω} or T d+Q+4(h · a) ∈ Σd,d+Q+4 for some d ∈ N .

If h · a ∈ H(ω),n+ω+Q+51,a , then T 1(h) = s, contradicting (ii) and (iii). If T d+Q+4(h · a) ∈

Σd,d+Q+4, then T 1(h) ∈ Dd(s). Therefore, by (iii), τ ′ = 0. Then, by (ii), it follows that

T 1(h) ∈ Dd(s) and d = d. This implies that T d+Q+3(h) ∈ Σd,d+Q+3. Thus, by (i) and τ ′ = 0,

condition (c) is satisfied. But this contradicts our supposition.

Subclaim 2: v(h · a) = τ ′ + 1. Since h · a ∈ Hd,τ ′+16 , v(h · a) ≥ τ ′ + 1 > 0. Therefore,

h ∈ Σd,v(h·a)−1. Hence, τ ′ ≥ v(h · a)− 1 and thus v(h · a) = τ ′ + 1.

It follows from the above two subclaims that f(h · a) = a(d),τ′+2. Then, by induction, it

follows that f(h · a · a(d),τ ′+2) = a(d),τ′+3, . . . , f(h · a · (a(d),τ ′+2, · · · , a(d),d+Q+3) = a(d),d+Q+4.

Claim B.42 Let h ∈ Hk7 for some k ∈ {0, . . . , n+Q+4}. If h �∈ ∪n+Q+3

k=2 Hk7 and T d+Q+3(h) ∈

Σd,d+Q+3 for some d ∈ N , then f(h) = s′ and h · f(h) ∈ Hd+Q+4,d3,a . Otherwise, π(f |h) =(

π(ω),k′+1, π(ω),k′+2, . . .)for some ω ∈ {0, . . . , ω} and k′ ∈ {0, . . . , n+ ω +Q+ 4}.

Proof. First note that if h �∈ ∪n+Q+3k=2 Hk

7 then f(h) = s′. Thus, if T d+Q+3(h) ∈ Σd,d+Q+3

for some d ∈ N we have T d+Q+4(h · f(h)) ∈ Σd,d+Q+4; hence, h · f(h) ∈ Hd+Q+4,d3,a . Also,

if T n+ω+Q+4(h) = ((s′; 2), (s;n + ω + Q + 2)) for some ω ∈ {0, . . . , ω} then, by f(h) = s′,

T n+ω+Q+5(h · f(h)) = ((s′; 2), (s;n+ ω+Q+2), s′); thus, h · f(h) ∈ H(ω),n+ω+Q+51,a ; but then,

by Claim B.36, π(f |h) = (π(ω),k′+1, π(ω),k′+2, . . .

). Therefore, for the remainder of the proof,

assume that the following holds:

if h �∈ ∪n+Q+3k=2 Hk

7 , then T n+ω+Q+4(h) �= ((s′; 2), (s;n+ ω +Q+ 2)),

for all ω ∈ {0, . . . , ω}, and T d+Q+3(h) �∈ Σd,d+Q+3, for all d ∈ N.(B.5)

Next, for any history h′ ∈ H, define v(h′) = max{t ∈ {0, . . . , n+Q+4} : T t(h′) ∈ H(0),t1,b }

and let k′ = v(h). If k′ = n + Q + 4, then f(h) = s′ and h · f(h) ∈ H(0),n+Q+51,a , and the

conclusion follows from Claim B.36.

If k′ ∈ {0, . . . , n + Q + 3}, then the claim follows by induction if it is the case that

h · f(h) ∈ Hk′+17 and v(h · f(h)) = k′ +1. Next, we complete the proof by showing in several

steps that these two conditions indeed hold.

Step 1: h·f(h) �∈ H6. Otherwise, h·f(h) ∈ Hd,τ6 for some d ∈ N and τ ∈ {0, . . . , d+Q+3}.

Then if τ > 0, T τ−1(h) ∈ Σd,τ−1; but this is a contradiction as this implies that h ∈ H6.

34

If τ = 0, then by the definition of H06 , f(h) = T 1(h · f(h)) ∈ D(s) ∪ D(s′). But this is a

contradiction because, by h ∈ Hk′7 , f(h) ∈ {s, s′}.

Step 2: h · f(h) �∈ ∪5l=1Hl. When k′ ≥ 2, then f(h) = s. Therefore, the claim in this

step follows immediately from Claim B.9. Next suppose that k′ ∈ {0, 1} and that h · f(h) ∈∪5l=1Hl. Then Claim B.9 implies that either h ∈ H

(ω),n+ω+Q+41,a for some ω ∈ {0, . . . , ω} or

T d+Q+3(h) ∈ Σd,d+Q+3 for some d ∈ N . But this contradicts our supposition in (B.5).

Step 3: T k′+1(h · f(h)) ∈ H(0),k′+11,b and v(h · f(h)) = k′ + 1. By h ∈ Hk′

7 , f(h) = π(0),k′+1

and so T k′+1(h · f(h)) ∈ H(0),k′+11,b . Hence, v(h · f(h)) ≥ k′ + 1. If it were the case that

v(h · f(h)) > k′ + 1, then T v(h·f(h))−1(h) ∈ H(0),v(h·f(h))−11,b , implying v(h) ≥ v(h · f(h)) − 1.

Since k′ = v(h) and v(h · f(h)) > k′ + 1, this is a contradiction.

Claim B.43 Let h ∈ H1 and a ∈ Dd(f(h)) for some d ∈ N . Then there exists t ∈ {1, . . . , d+Q+ 4} such that h · a · (π1(f |h · a), . . . , πt(f |h · a)) ∈ Hd+Q+4,d

3,a .

Proof. By assumption, h ∈ H(ω),k′1,a ∪ H

(0),l1,b for some ω and k′ and l with l < M and (by

Claim B.10) k′ < M . Thus, h · a ∈ Hk,d,02 with either k = k′ + 1 or k = l + 1. The result

then follows by Claim B.41.

Claim B.44 Let h ∈ Hk,d′,τ ′2 ∪ Hk,d′,τ ′

5 ∪ Hd′,τ ′6 for some k ∈ {1, . . . ,M} and d′ ∈ N and

τ ′ ∈ {0, . . . , d′ + 3}. Let d �= d′ and a ∈ Dd(f(h)). Then either h · a ∈ Hd+Q+4,d3,a or there

exists t ∈ {1, . . . , d+Q+ 4} such that h · a · (π1(f |h · a), . . . , πt(f |h · a)) ∈ Hd+Q+4,d3,a .

Proof. We first argue that it is sufficient to show that either h·a ∈ Hd+Q+4,d3 or h·a �∈ ∪5

l=1Hl.

Indeed, if the latter holds, then this together with f(h) ∈ {s, s′} and a ∈ Dd(f(h)) implies

that h · a ∈ Hd,06 . Then the claim follows from Claim B.41.

We next establish that either h · a ∈ Hd+Q+4,d3 or h · a �∈ ∪5

l=1Hl. Suppose not; then we

derive a contradiction for the different possible cases as follows.

Case 1: T d+Q+4(h · a) ∈ Σd,d+Q+4 for some d ∈ N . Then a ∈ Dd(s′) which together with

a ∈ Dd(s) ∪ Dd(s′) implies that d = d. Hence, h · a ∈ Hd+Q+3,d

3 ; a contradiction.

Case 2: h · a ∈ H(ω),n+ω+Q+51,a for some ω ∈ {0, . . . , ω}. Then a = s′ and this contradicts

a ∈ Dd(s) ∪ Dd(s′).

Case 3: h ∈ Hk,d′,τ2 ∪ Hk,d′,τ

5 and h · a ∈ H(ω),k′1,a ∪ H1,b ∪ H k,d,0

2 ∪ H k,d,05 for some ω ∈

{0, . . . , ω} and k′ > n + ω + Q + 5 and k ∈ {1, . . . ,M} and d ∈ N . Then, by the latter,

35

h ∈ H1 ∪ H3 ∪ H4. By Claims B.17, B.18, B.22, B.26, B.29, B.30, B.32 and B.33, this

contradicts the supposition that h ∈ Hk,d′,τ2 ∪Hk,d′,τ

5 .

Case 4: h ∈ Hk,d′,τ2 ∪ Hk,d′,τ

5 and h · a ∈ H k,d,τ2 ∪ H k,d,τ

5 for some k ∈ {1, . . . ,M} and

d ∈ N and τ ∈ {1, . . . , d + Q + 3}. By the latter, a ∈ Dd(s) ∪ Dd(s′). Since we also have

a ∈ Dd(s) ∪ Dd(s′) it follows that d = d. This together with h · a ∈ H k,d,τ

2 ∪ H k,d,τ5 implies

that h ∈ H k−1,d,τ−12 ∪ H k−1,d,τ−1

5 which, by Claims B.19, B.31 and B.34, contradicts the

supposition that h ∈ Hk,d′,τ2 ∪Hk,d′,τ

5 .

Case 5: h ∈ Hk,d′,τ2 ∪Hk,d′,τ

5 and h · a ∈ H3 and T d+Q+4(h · a) �∈ Σd,d+Q+4 for all d ∈ N .

Then h ∈ H3 and, by Claims B.22 and B.32, this contradicts h ∈ Hk,d′,τ2 ∪Hk,d′,τ

5 .

Case 6: h ∈ Hk,d′,τ2 ∪ Hk,d′,τ

5 and h · a ∈ H4. Then h ∈ H3 ∪ H4 and, by Claims B.22,

B.26, B.32 and B.33, this contradicts the supposition that h ∈ Hk,d′,τ2 ∪Hk,d′,τ

5 .

Case 7: h ∈ Hd′,τ6 and h · a �∈ H

(ω),n+ω+Q+51,a for all i ∈ {0, . . . , n} and T d+Q+4(h · a) �∈

Σd,d+Q+4 for all d ∈ N . Then h �∈ ∪5l=1Hl and, by Claim B.9, we have h · a �∈ ∪5

l=1Hl; a

contradiction.

Claim B.45 Let h ∈ Hk,d′3,a for some k ∈ {1, . . . ,M} and d′ ∈ N , and a ∈ Dd(f(h)) with

d �= d′. Then there exists t ∈ {1, . . . , d+Q+4} such that h · a · (π1(f |h · a), . . . , πt(f |h · a)) ∈Hd+Q+4,d

3,a .

Proof. The claim follows by Claim B.41 since h · a ∈ Hk+1,d,05 .

Claim B.46 Let h ∈ Hk,d′3,b for some k ∈ {1, . . . ,M} and d′ ∈ N , TQ+1(h) ∈ ∪l∈N Σl,Q+1,

and d �= d′ and a ∈ Dd(f(h)). Then there exists t ∈ {1, . . . , d + Q + 4} such that h · a ·(π1(f |h · a), . . . , πt(f |h · a)) ∈ Hd+Q+4,d

3,a .

Proof. The claim follows by Claim B.41 since h · a ∈ Hk+1,d,05 .

Claim B.47 Let h ∈ Hk,ω,r4 for some k ∈ {1, . . . ,M} and ω ∈ {1, . . . , ω} and r ∈ {0, . . . , n+

ω +Q+ 4}, and let a ∈ Dd(f(h)) for some d ∈ N . Then there exists t ∈ {1, . . . , d+Q+ 4}such that h · a · (π1(f |h · a), . . . , πt(f |h · a)) ∈ Hd+Q+4,d

3,a .

Proof. The claim follows by Claim B.41 since h · a ∈ Hk+1,d,05 .

Claim B.48 Let h ∈ Hk7 for some k ∈ {0, . . . , n+Q+4} and a ∈ Dd(f(h)) for some d ∈ N .

If h �∈ ∪n+Q+3k=2 Hk

7 and T d+Q+3(h) ∈ Σd,d+Q+3, then h · a ∈ Hd+Q+4,d3,a . Otherwise, there exists

t ∈ {1, . . . , d+Q+ 4} such that h · a · (π1(f |h · a), . . . , πt(f |h · a)) ∈ Hd+Q+4,d3,a .

36

Proof. If h �∈ ∪n+Q+3k=2 Hk

7 and T d+Q+3(h) ∈ Σd,d+Q+3, then f(h) = s′ and a ∈ Dd(s′) and

T d+Q+4(h · a) ∈ Σd,d+Q+4. Hence, h · a ∈ Hd+Q+4,d3,a .

For the remainder of the proof, therefore assume that the following holds:

If h �∈ ∪n+Q+3k=2 Hk

7 , then T d+Q+3(h) �∈ Σd,d+Q+3. (B.6)

Then we will show that h · a ∈ Hd,06 which, by Claim B.41, establishes the conclusion of the

claim. We prove the former in two steps.

Step 1: h · a ∈ Σd,0. Since h ∈ Hk7 , f(h) ∈ {s, s′}. Therefore, a ∈ Dd(s) ∪ Dd(s

′).

Step 2: h · a �∈ ∪5l=1Hl. Suppose otherwise. Then, by Claim B.9, a �∈ D(s). Hence,

by h ∈ Hk7 , (i) a ∈ Dd(s

′) and (ii) h �∈ ∪n+Q+3k=2 Hk

7 . Furthermore, Claim B.9 implies that

either h · a ∈ H(ω),n+ω+Q+51,a for some ω ∈ {0, . . . , ω} or T d′+Q+4(h · a) ∈ Σd′,d′+Q+4 for some

d′ ∈ N . If h · a ∈ H(ω),n+ω+Q+51,a for some ω ∈ {0, . . . , ω}, then a = s′; a contradiction to (i).

If T d′+Q+4(h · a) ∈ Σd′,d′+Q+4 for some d′ ∈ N , then a ∈ Dd′(s′) which, by (i), implies that

d = d′. Thus, T d+Q+3(h) ∈ Σd,d+Q+3. But this together with (ii) contradicts our supposition

in (B.6).

B.2.4 f is subgame perfect and π(f) = π(0)

By Claim B.36 and Claim B.37, π(f) = π(0). To show that f is subgame perfect, we next

establish the following for all h ∈ H:

Ud(f |h) ≥ (1− δ)ud(a) + δUd(f |h · a) for all d ∈ N and a ∈ Dd(f(h)). (B.7)

By construction, (B.7) holds in the following cases:

1. h ∈ Hk,d3,a for some k ∈ {1, . . . ,M},

2. h ∈ Hk,d3,b for some k ∈ {1, . . . ,M} and TQ+1(h) ∈ ∪l∈N Σl,Q+1, and

3. h ∈ Hk,d′3,b for some k ∈ {1, . . . ,M} and d′ ∈ N , and TQ+1(h) �∈ ∪l∈N Σl,Q+1 for all

l ∈ N .

In the first case, the future play is independent of player d’s current action and such

current action is a static best-reply against the action of the other players. In the second

and third cases, player d’s current action is optimal since by the construction of fd.

37

In what follows, we fix h and a ∈ Dd(f(h)) and consider the remaining cases.

Case 1: Either h ∈ H1∪H4 or the following holds: h ∈ Hk7 for some k ∈ {0, . . . , n+Q+4}

and either h ∈ ∪n+Q+3k=2 Hk

7 or T d′+Q+3(h) �∈ Σd′,d′+Q+3 for all d′ ∈ N . In this case, by Claims

B.36, B.37, B.38 and B.42, π(f |h) = (π(ω),k, . . .) for some ω ∈ {0, . . . , ω} and k ≤ M .

Therefore, the left-hand side of (B.7) is greater or equal to

−B(1− δK−k+1)+ δK−k+1Vd(π(ω)) ≥ −B(1− δK)+ δKVd(π

(ω)) > −B(1− δK)+ δK(u′d− 2ξ).

By Claims B.43, B.47 and B.48, h · a · (π1(f |h · a), . . . , πt(f |h · a)) ∈ Hd+Q+4,d3,a for some

t ∈ {0, . . . , d+Q+ 3}. Therefore, by Claims B.40 and B.39 and Lemma 1 and Lemma B.1,

the right-hand side of (B.7) is less than or equal to

(1− δt+1+(γ+1)(d+Q+4))B + δt+1+(γ+1)(d+Q+4)(1− δT )ε+ δt+1+(γ+1)(d+Q+4)+T (u′d + 2ξ) ≤

(1− δ(γ+2)(n+Q+4)+1)B + δ(γ+2)(n+Q+4)+1(1− δT )ε+ δ(γ+2)(n+Q+4)+1+T (u′d + 2ξ).

Thus, by (53), (B.7) holds.

Case 2: One of the following conditions holds for some d′ ∈ N with d �= d′: (i) h ∈ Hk,d′3,a

for some k ∈ {1, . . . ,M}, (ii) h ∈ Hk,d′3,b for some k ∈ {1, . . . ,M} and TQ+1(h) ∈ ∪l∈N Σl,Q+1,

(iii) h ∈ Hk,d′,τ2 ∪Hk,d′,τ

5 ∪Hd′,τ6 for some k ∈ {1, . . . ,M} and τ ∈ {0, . . . , d′+Q+3}, and (iv)

h ∈ Hk7 for some k ∈ {0, . . . , n + Q + 4} and h �∈ ∪n+Q+3

k=2 Hk7 and T d′+Q+3(h) ∈ Σd′,d′+Q+3.

Claims B.40, B.39, B.41 and B.42 can be applied to cases (i), (ii), (iii) and (iv), respectively;

and we obtain that the left-hand side of (B.7) is greater or equal to

−(1− δ(γ+2)(n+Q+4)+T )B + δ(γ+2)(n+Q+4)+T (u′d + (1− 2ε2)ρ− 2ξ).

The right-hand side of (B.7) is, as in Case 1, less than or equal to

(1− δ(γ+2)(n+Q+4)+1)B + δ(γ+2)(n+Q+4)+1(1− δT )ε+ δ(γ+2)(n+Q+4)+1+T (u′d + 2ξ).

Thus, by (54), (B.7) holds.

Case 3: Either h ∈ Hk,d,τ2 ∪Hk,d,τ

5 ∪Hd,τ6 for some k ∈ {1, . . . ,M} and τ ∈ {0, . . . , d +

Q + 3} or h ∈ Hk7 for some k ∈ {0, . . . , n + Q + 4}, h �∈ ∪n+Q+3

k=2 Hk7 and T d+Q+3(h) ∈

Σd,d+Q+3. By Claim B.41 (when h ∈ Hk,d,τ2 ∪ Hk,d,τ

5 ∪ Hd,τ6 for some k ∈ {1, . . . ,M} and

38

τ ∈ {0, . . . , d+Q+3}) and Claims B.42 and B.48 (in the remaining case), we have that the

difference between the left and the right-hand sides of (B.7) is greater or equal to

−(1− δ)2B + δ(γ+2)(n+Q+4)(1− δγ)(ud(a(d))− ud(m

d)).

Indeed, if the deviation takes place in the tth action of player d’s signalling phase, then the tth

sequence of (γ+1) actions in the phase between the signalling phase and the minmax phase

is ((md; γ), (s∗d, s′−d)), whereas it is ((a

(d); γ), (s∗d, s′−d)) if no deviation occurs. Furthermore,

these are the only differences that occur in future play. Furthermore, this different sequence

of actions can start no later than (γ+2)(n+Q+4) periods after the deviation. Thus, (B.7)

holds if

δ(γ+2)(n+Q+3) (1− δγ)(ud(a(d))− ud(m

d))

(1− δ)2B> 1;

hence, by (52), (B.7) holds.

This concludes the proof of Lemma 3.

39