Instructor: Alan Ritter

Perceptron

Instructor: Alan Ritter

Many Slides from Luke Zettemoyer and Ian Murray

PerceptronWho'needs'probabiliFes?'

•  Previously:'model'data'with'distribuFons'

•  Joint:'P(X,Y)'–  e.g.'Naïve'Bayes'

•  CondiFonal:'P(Y|X)'–  e.g.'LogisFc'Regression'

•  But'wait,'why'probabiliFes?'

•  Lets'try'to'be'errorWdriven!'

mpg cylinders displacementhorsepower weight acceleration modelyear maker

good 4 97 75 2265 18.2 77 asiabad 6 199 90 2648 15 70 americabad 4 121 110 2600 12.8 77 europebad 8 350 175 4100 13 73 americabad 6 198 95 3102 16.5 74 americabad 4 108 94 2379 16.5 73 asiabad 4 113 95 2228 14 71 asiabad 8 302 139 3570 12.8 78 america: : : : : : : :: : : : : : : :: : : : : : : :good 4 120 79 2625 18.6 82 americabad 8 455 225 4425 10 70 americagood 4 107 86 2464 15.5 76 europebad 5 131 103 2830 15.9 78 europe

Linear'Classifiers'

•  Inputs'are'feature'values'•  Each'feature'has'a'weight'•  Sum'is'the'acFvaFon'

•  If'the'acFvaFon'is:'–  PosiFve,'output'class%1%–  NegaFve,'output'class%2' Σ"

x1

x2

x3

w1

w2

w3 >0?

activationw(x) =X

i

wixi = w · x

A two-dimensional space

4 6 8 10

4

6

8

10

width / cm

heig

ht /

cm

Oranges

Lemons

Stars and Galaxies

http://www-wfau.roe.ac.uk/sss/

Minor elliptical axis (y) against Majorelliptical axis (x) for stars (red) andgalaxies (blue). (Amos Storkey)

Linear classifier

4 6 8 10

4

6

8

10

Straight cut dividing input space into two

The weight vector

Define positive class region: w>x + b > 0

w1x1 + w2x2 + w3x3 + . . . b > 0

Pd wdxd + b > 0

We will set b = 0:w>x > 0

Learning the weights

A ‘perceptron’ learning rule:y sgn(w>x)w w + (y � y)x

% Matlab/Octave code

old ww = []; ww = zeros(size(xx,2), 1);

while ~isequal(ww, old ww)

old ww = ww;

for ii = 1:N

pred = sign(xx(ii, :)*ww);

ww = ww + (yy(ii) - pred)*xx(ii, :)’;

end

end

−1 −0.5 0 0.5 1−1

−0.5

0

0.5

1

−1 −0.5 0 0.5 1−1

−0.5

0

0.5

1

−1 −0.5 0 0.5 1−1

−0.5

0

0.5

1

−1 −0.5 0 0.5 1−1

−0.5

0

0.5

1

Pattern Recognition and Machine Learning C. M. Bishop (2006)

Example:'Spam'•  Imagine'3'features'(spam'is'�posiFve�'class):'

–  free'(number'of'occurrences'of'�free�)'–  money'(occurrences'of'�money�)'–  BIAS'(intercept,'always'has'value'1)'

BIAS : -3 free : 4 money : 2 ...

BIAS : 1 free : 1 money : 1 ...

�free money�

w!x'>'0'"'SPAM!!!'

x2

x1

•  For't=1..T,'i=1..n:'–  ''–  if'y'≠'yi'y = sign(w · xi)

w = w + yixi

x1# x2# y#3' 2' 1'

W2' 2' W1'

W2' W3' 1'

IniFal:'•  w'='[0,0]'t=1,i=1'•  [0,0]![3,2]'='0,'sign(0)=W1'•  w'='[0,0]'+'[3,2]'='[3,2]'t=1,i=2'•  [3,2]![W2,2]=W2,'sign(W2)=W1't=1,i=3'•  [3,2]![W2,W3]=W12,'sign(W12)=W1'•  w'='[3,2]'+'[W2,W3]'='[1,W1]'t=2,i=1'•  [1,W1]![3,2]=1,'sign(1)=1't=2,i=2'•  [1,W1]![W2,2]=W4,'sign(W4)=W1't=2,i=3'•  [1,W1]![W2,W3]=1,'sign(1)=1''Converged!!!'•  y=w1x1+w2x2'#'y=x1+-x2'•  So,'at'y=0'#'x2=x1 '

x2=x1

MulFclass'Decision'Rule'

•  If'we'have'more'than'two'classes:'–  Have'a'weight'vector'for'each'class:'

–  Calculate'an'acFvaFon'for'each'class'

'

–  Highest'acFvaFon'wins'

activationw(x, y) = wy · x

y⇤ = argmaxy

(activationw(x, y))

w2 · x

w1 · x

w3 · x

Example: y is {1,2,3} •  We are fitting three planes: w1,

w2, w3 •  Predict i when wi ! x is highest

Example'

BIAS : -2 win : 4 game : 4 vote : 0 the : 0 ...

BIAS : 1 win : 2 game : 0 vote : 4 the : 0 ...


�win the vote�


x

x · wSPORTS = x · wPOLITICS = x · wTECH =2 7 2

POLITICS wins!!!

The'MulFWclass'Perceptron'Alg.'•  Start'with'zero'weights'•  For't=1..T,'i=1..n'(T'Fmes'over'data)'

–  Classify'with'current'weights'

'–  If'correct'(y=yi),'no'change!'$  If wrong: subtract features from

weights for predicted class and add them to weights for correct class

y = argmaxy

wy · xi

wy = wy � xi

wy = wy � xi

wy = wy � xi

wyi = wyi + xi

xi

wyi

wyi = wyi + xi

wyi + xi

wy � xi

Interpreting Perceptron Weights

f(x) = sign(X

d

wdxd + b)

@

@x7(X

d

wdxd + b) =?

Interpreting Perceptron Weights

f(x) = sign(X

d

wdxd + b)

@

@x7(X

d

wdxd + b) =?w7

Questions

» Is the perceptron guaranteed to converge? » Under what circumstances? » What does it converge to? » How long does it take?

Questions


How about a dataset with zero features?

Questions


How about a dataset with zero features?

Linearly Separable -> Perceptron Guaranteed to Converge

Not Linearly Separable -> Never Converges

Linearly'Separable'(binary'case)'•  The'data'is'linearly'separable'with'margin%γ,'if:'

•  For yt=1

•  For yt=-1 �

�

�

9w.8t.yt(w · xt) � � > 0

w · xt � �

w · xt ��

Linearly'Separable'(binary'case)'•  The'data'is'linearly'separable'with'margin%γ,'if:'

•  For yt=1

•  For yt=-1 �

�

�

9w.8t.yt(w · xt) � � > 0

w · xt � �

w · xt ��

Activation (abs. val.)

Mistake'Bound'for'Perceptron'•  Assume'data'is'separable'with'margin%γ:'

•  Also'assume'there'is'a'number'R'such'that:'

•  Theorem:'The'number'of'mistakes'(parameter'updates)'made'by'the'perceptron'is'bounded:'

mistakes R2

�2

kxk2 =

sX

i

x2i

9w⇤ s.t. kw⇤k2 = 1 and 8t.yt(w⇤ · xt) � �

8t.kxtk2 R

Perceptron'Convergence'(by'InducFon)'•  Let'wk'be'the'weights'a^er'the'kWth'update'(mistake),'we'will'

show'that:'

•  Therefore:'

•  Because'R'and'γ are'fixed'constants'that'do'not'change'as'you'learn,'there'are'a'finite'number'of'updates!'

•  Proof'does'each'bound'separately'(next'two'slides)'

k R2

�2

k2�2 kwkk22 kR2

Lower'bound'•  Remember'our'margin'assumpFon:'

•  Now,'by'the'definiFon'of'the'perceptron'update,'for'kWth'mistake'on'tWth'training'example:''

'•  So,'by'inducFon'with'w0=0,'for'all'k:'

'


wk+1 · w⇤ = (wk + ytxt) · w⇤

Perceptron'update: w = w + ytxt

= wk · w⇤ + yt(w⇤ · xt)� wk · w⇤ + �

k� wk · w⇤

kwkk2k2�2 kwkk22 kR2

Because:

and kw⇤k2 = 1wk · w⇤ kwkk2 ⇥ kw⇤k2

Lower'bound'•  Remember'our'margin'assumpFon:'

•  Now,'by'the'definiFon'of'the'perceptron'update,'for'kWth'mistake'on'tWth'training'example:''

'•  So,'by'inducFon'with'w0=0,'for'all'k:'

'


wk+1 · w⇤ = (wk + ytxt) · w⇤

Perceptron'update: w = w + ytxt

= wk · w⇤ + yt(w⇤ · xt)� wk · w⇤ + �

k� wk · w⇤

kwkk2k2�2 kwkk22 kR2

Because:

and kw⇤k2 = 1wk · w⇤ kwkk2 ⇥ kw⇤k2

Cauchy-Schwartz Inequality

Upper'Bound'•  By'the'definiFon'of'the'Perceptron'update,'for'kWth'mistake'

on'tWth'training'example:''

•  So,'by'inducFon'with'w0=0'have,'for'all'k:'

'

k2�2 kwkk22 kR2

< 0 because'Perceptron'made'error'(yt'has'different'sign'than'xt!wt)

kwk+1k22 = kwk + ytxtk22= kwkk22 + (yt)2kxtk22 + 2ytxt · wk

kwkk22 +R2

R2 because and (yt)2 = 1 kxtk2 R

8t.kxtk2 RData'AssumpFon: Perceptron'update:

w = w + ytxt

Perceptron'Convergence'(by'InducFon)'•  Let'wk'be'the'weights'a^er'the'kWth'update'(mistake),'we'will'

show'that:'

•  Therefore:'

•  Because'R'and'γ are'fixed'constants'that'do'not'change'as'you'learn,'there'are'a'finite'number'of'updates!1'

•  If'there'is'a'linear'separator,'Perceptron'will'find'it!!!'•  Proof'does'each'bound'separately'(last'two'slides)'

k R2

�2

k2�2 kwkk22 kR2

From'LogisFc'Regression'

to'the'Perceptron:''

2'easy'steps!'•  LogisFc'Regression:'(in'vector'notaFon):'y'is'{0,1}'

•  Perceptron:'when'y'is'{0,1}:'

•  sign0(x)'='+1'if'x>0'and'0'otherwise''Differences? •  Drop the Σj over training examples: online vs.

batch learning •  Drop the dist’n: probabilistic vs. error driven

learning

w = w + ⌘X

j

[yj � P (yj |xj , w)]xj

w = w + [yj � sign0(w · xj)]xj

Perceptron'update'when'y'is'{W1,1}:

w = w + yjxj

ProperFes'of'Perceptrons'•  Separability:'some'parameters'get'the'

training'set'perfectly'correct'•  Convergence:'if'the'training'is'

separable,'perceptron'will'eventually'converge'(binary'case)'

•  Mistake'Bound:'the'maximum'number'of'mistakes'(binary'case)'related'to'the'margin'or'degree'of'separability'

Separable

Non-Separable

mistakes R2

�2

Problems with the Perceptron $  Noise: if the data isn’t

separable, weights might thrash $  Averaging weight vectors over time

can help (averaged perceptron)

$  Mediocre generalization: finds a “barely” separating solution

$  Overtraining: test / held-out accuracy usually rises, then falls $  Overtraining is a kind of overfitting

Voted Perceptron

» Problem: » Later training examples are counted

more than earlier ones » Example:

» 10,000 training examples » After first 100, no more updates until the

10,000th example » (could ruin the weight vector that did

well on most of the data)

Voted Perceptron

» Would like weight vectors that “survive” to get more say

» Voting! » As the perceptron runs, remember how

long each hyperplane survives » Prediction:

y = sign

KX

k=1

c(k)sign(w(k) · x+ b(k))

!

Voted Perceptron

» Would like weight vectors that “survive” to get more say

» Voting! » As the perceptron runs, remember how

long each hyperplane survives » Prediction:

y = sign

KX

k=1

c(k)sign(w(k) · x+ b(k))

!

Too slow / too much memory!

Averaged Perceptron (much more practical)

y = sign

KX

k=1

c(k)(w(k) · x+ b(k))

!

y = sign

1

PKk=1 c

(k)

KX

k=1

c(k)w(k)

!x+

1PK

k=1 c(k)

KX

k=1

c(k)b(k)!

<latexit sha1_base64="7VTdIS0pzFXHTdgbGWNpaikKEtE=">AAACynichVFda9swFJW9buu8r2x73B4uC4OUQbDLYH0plO5l0D500LSFOA2yIjsismyk665G6G2/af9lP6ZQJfVY0xZ2Qehwzv3SUVZLYTCO/wTho43HT55uPouev3j56nXvzdsTUzWa8RGrZKXPMmq4FIqPUKDkZ7XmtMwkP80W35b66QXXRlTqGNuaT0paKJELRtFT097vdE7Rtg52IUV+idaIQjlIJc9xAGmuKbOJs6lpyqld7Cbu/ADYuR0stty/rNviXxV+dneqRTHHrdWcSwef4b9dH2Cz9WYw7fXjYbwKuA+SDvRJF0fT3lU6q1hTcoVMUmPGSVzjxFKNgknuorQxvKZsQQs+9lDRkpuJXfnr4JNnZpBX2h+FsGJvV1haGtOWmc8sKc7NXW1JPqSNG8x3JlaoukGu2M2gvJGAFSw/C2ZCc4ay9YAyLfyuwObUu4f+S6O1MTOz3G3tIbYucr+0i6LI+5Xcdec+ONkeJvEw+fGlv7ffObdJ3pOPZEAS8pXske/kiIwICz4E+8FBcBgehjpsQ3uTGgZdzTuyFuGva2eT3GQ=</latexit><latexit sha1_base64="7VTdIS0pzFXHTdgbGWNpaikKEtE=">AAACynichVFda9swFJW9buu8r2x73B4uC4OUQbDLYH0plO5l0D500LSFOA2yIjsismyk665G6G2/af9lP6ZQJfVY0xZ2Qehwzv3SUVZLYTCO/wTho43HT55uPouev3j56nXvzdsTUzWa8RGrZKXPMmq4FIqPUKDkZ7XmtMwkP80W35b66QXXRlTqGNuaT0paKJELRtFT097vdE7Rtg52IUV+idaIQjlIJc9xAGmuKbOJs6lpyqld7Cbu/ADYuR0stty/rNviXxV+dneqRTHHrdWcSwef4b9dH2Cz9WYw7fXjYbwKuA+SDvRJF0fT3lU6q1hTcoVMUmPGSVzjxFKNgknuorQxvKZsQQs+9lDRkpuJXfnr4JNnZpBX2h+FsGJvV1haGtOWmc8sKc7NXW1JPqSNG8x3JlaoukGu2M2gvJGAFSw/C2ZCc4ay9YAyLfyuwObUu4f+S6O1MTOz3G3tIbYucr+0i6LI+5Xcdec+ONkeJvEw+fGlv7ffObdJ3pOPZEAS8pXske/kiIwICz4E+8FBcBgehjpsQ3uTGgZdzTuyFuGva2eT3GQ=</latexit><latexit sha1_base64="7VTdIS0pzFXHTdgbGWNpaikKEtE=">AAACynichVFda9swFJW9buu8r2x73B4uC4OUQbDLYH0plO5l0D500LSFOA2yIjsismyk665G6G2/af9lP6ZQJfVY0xZ2Qehwzv3SUVZLYTCO/wTho43HT55uPouev3j56nXvzdsTUzWa8RGrZKXPMmq4FIqPUKDkZ7XmtMwkP80W35b66QXXRlTqGNuaT0paKJELRtFT097vdE7Rtg52IUV+idaIQjlIJc9xAGmuKbOJs6lpyqld7Cbu/ADYuR0stty/rNviXxV+dneqRTHHrdWcSwef4b9dH2Cz9WYw7fXjYbwKuA+SDvRJF0fT3lU6q1hTcoVMUmPGSVzjxFKNgknuorQxvKZsQQs+9lDRkpuJXfnr4JNnZpBX2h+FsGJvV1haGtOWmc8sKc7NXW1JPqSNG8x3JlaoukGu2M2gvJGAFSw/C2ZCc4ay9YAyLfyuwObUu4f+S6O1MTOz3G3tIbYucr+0i6LI+5Xcdec+ONkeJvEw+fGlv7ffObdJ3pOPZEAS8pXske/kiIwICz4E+8FBcBgehjpsQ3uTGgZdzTuyFuGva2eT3GQ=</latexit><latexit sha1_base64="rIc4elZHT3oXILhfcJEssMOX7as=">AAACBHicbVDLSgMxFL1TX3WsWtdugkVwVWbc6FJw47KCfUAdSiaTaUMzyZDcEcrQH3Dbr3En/oUfI5hpu7CtBy4czkm45544l8JiEHx7tb39g8Oj+rF/0vBPz86bjZ7VhWG8y7TUZhBTy6VQvIsCJR/khtMslrwfTx8rv//GjRVaveAs51FGx0qkglF0UmfUbAXtYAmyS8I1acEao+bPa6JZkXGFTFJrh2GQY1RSg4JJPvdfC8tzyqZ0zIeOKppxG5XLmHNy7ZSEpNq4UUiW6t8fJc2snWWxe5lRnNhtrxL/84YFpvdRKVReIFdstSgtJEFNqptJIgxnKGeOUGaEy0rYhBrK0DXjb6xJbJVt45AyH6cu9Nz3fVdXuF3OLundtsOgHT4HUIdLuIIbCOEOHuAJOtAFBgm8w8JbeB/e56rWmrfu9wI24H39Asd+my0=</latexit><latexit sha1_base64="Y6zvR4zQcVlMRr3qXn2hhOnlnAs=">AAACv3ichVHfaxQxEM6uv+pa9fRVHwaLcEU4dvuiLwXRF6F9qOC1hdvrkc3N7oXLZpdkVruE/Fv+L/4xgrnrib224EDIx/dNZibfFK2SltL0VxTfu//g4aOdx8mT3afPng9e7J7apjMCx6JRjTkvuEUlNY5JksLz1iCvC4VnxfLzSj/7jsbKRn+jvsVpzSstSyk4BWo2+JkvOLnewyHkhJfkrKy0h1xhSUPIS8OFy7zLbVfP3PIw8xdHIC7ccLnv/2VdF/+q8GNz50ZWC9pf97n08A7+W/UOttguBrPBXjpK1wG3QbYBe2wTJ7PB73zeiK5GTUJxaydZ2tLUcUNSKPRJ3llsuVjyCicBal6jnbq1vx7eBmYOZWPC0QRr9voLx2tr+7oImTWnhb2prci7tElH5Yepk7rtCLW4alR2CqiB1bJgLg0KUn0AXBgZZgWx4ME9CitNttrM7Wq2rY+4tirD0D5JkuBXdtOd2+D0YJSlo+xrynbYK/aGDVnG3rOP7As7YWMmotfRp+goOo6PYxP3V87G0cbil2wrYvcH1/zbLg==</latexit><latexit sha1_base64="Y6zvR4zQcVlMRr3qXn2hhOnlnAs=">AAACv3ichVHfaxQxEM6uv+pa9fRVHwaLcEU4dvuiLwXRF6F9qOC1hdvrkc3N7oXLZpdkVruE/Fv+L/4xgrnrib224EDIx/dNZibfFK2SltL0VxTfu//g4aOdx8mT3afPng9e7J7apjMCx6JRjTkvuEUlNY5JksLz1iCvC4VnxfLzSj/7jsbKRn+jvsVpzSstSyk4BWo2+JkvOLnewyHkhJfkrKy0h1xhSUPIS8OFy7zLbVfP3PIw8xdHIC7ccLnv/2VdF/+q8GNz50ZWC9pf97n08A7+W/UOttguBrPBXjpK1wG3QbYBe2wTJ7PB73zeiK5GTUJxaydZ2tLUcUNSKPRJ3llsuVjyCicBal6jnbq1vx7eBmYOZWPC0QRr9voLx2tr+7oImTWnhb2prci7tElH5Yepk7rtCLW4alR2CqiB1bJgLg0KUn0AXBgZZgWx4ME9CitNttrM7Wq2rY+4tirD0D5JkuBXdtOd2+D0YJSlo+xrynbYK/aGDVnG3rOP7As7YWMmotfRp+goOo6PYxP3V87G0cbil2wrYvcH1/zbLg==</latexit><latexit sha1_base64="L2NpxAqVM4d36zzSfjLzNk62fXM=">AAACynichVFda9swFJW9deu8bsvax+3hsjBIGQR7L9tLobQvg/ahhaUtxGmQFdkRkWUjXW81Qm/7Tfsv+zGDKqnLmrawC0KHc+6XjrJaCoNx/CcInzzdePZ880X0cuvV6ze9t9tnpmo04yNWyUpfZNRwKRQfoUDJL2rNaZlJfp4tDpf6+Q+ujajUd2xrPilpoUQuGEVPTXu/0zlF2zrYgxT5FVojCuUglTzHAaS5pswmzqamKad2sZe4yyNgl3aw2HX/su6Ktyr87O5Ui2KOu6s5Vw4+wX+7PsJm681g2uvHw3gV8BAkHeiTLk6mvb/prGJNyRUySY0ZJ3GNE0s1Cia5i9LG8JqyBS342ENFS24mduWvg4+emUFeaX8Uwoq9W2FpaUxbZj6zpDg397Ul+Zg2bjD/OrFC1Q1yxW4G5Y0ErGD5WTATmjOUrQeUaeF3BTan3j30XxqtjZmZ5W5rD7F1kfulXRRF3q/kvjsPwdnnYRIPk9O4v3/QObdJ3pEPZEAS8oXsk2/khIwIC94HB8FRcBwehzpsQ3uTGgZdzQ5Zi/DXNWZT3GA=</latexit><latexit sha1_base64="7VTdIS0pzFXHTdgbGWNpaikKEtE=">AAACynichVFda9swFJW9buu8r2x73B4uC4OUQbDLYH0plO5l0D500LSFOA2yIjsismyk665G6G2/af9lP6ZQJfVY0xZ2Qehwzv3SUVZLYTCO/wTho43HT55uPouev3j56nXvzdsTUzWa8RGrZKXPMmq4FIqPUKDkZ7XmtMwkP80W35b66QXXRlTqGNuaT0paKJELRtFT097vdE7Rtg52IUV+idaIQjlIJc9xAGmuKbOJs6lpyqld7Cbu/ADYuR0stty/rNviXxV+dneqRTHHrdWcSwef4b9dH2Cz9WYw7fXjYbwKuA+SDvRJF0fT3lU6q1hTcoVMUmPGSVzjxFKNgknuorQxvKZsQQs+9lDRkpuJXfnr4JNnZpBX2h+FsGJvV1haGtOWmc8sKc7NXW1JPqSNG8x3JlaoukGu2M2gvJGAFSw/C2ZCc4ay9YAyLfyuwObUu4f+S6O1MTOz3G3tIbYucr+0i6LI+5Xcdec+ONkeJvEw+fGlv7ffObdJ3pOPZEAS8pXske/kiIwICz4E+8FBcBgehjpsQ3uTGgZdzTuyFuGva2eT3GQ=</latexit><latexit sha1_base64="7VTdIS0pzFXHTdgbGWNpaikKEtE=">AAACynichVFda9swFJW9buu8r2x73B4uC4OUQbDLYH0plO5l0D500LSFOA2yIjsismyk665G6G2/af9lP6ZQJfVY0xZ2Qehwzv3SUVZLYTCO/wTho43HT55uPouev3j56nXvzdsTUzWa8RGrZKXPMmq4FIqPUKDkZ7XmtMwkP80W35b66QXXRlTqGNuaT0paKJELRtFT097vdE7Rtg52IUV+idaIQjlIJc9xAGmuKbOJs6lpyqld7Cbu/ADYuR0stty/rNviXxV+dneqRTHHrdWcSwef4b9dH2Cz9WYw7fXjYbwKuA+SDvRJF0fT3lU6q1hTcoVMUmPGSVzjxFKNgknuorQxvKZsQQs+9lDRkpuJXfnr4JNnZpBX2h+FsGJvV1haGtOWmc8sKc7NXW1JPqSNG8x3JlaoukGu2M2gvJGAFSw/C2ZCc4ay9YAyLfyuwObUu4f+S6O1MTOz3G3tIbYucr+0i6LI+5Xcdec+ONkeJvEw+fGlv7ffObdJ3pOPZEAS8pXske/kiIwICz4E+8FBcBgehjpsQ3uTGgZdzTuyFuGva2eT3GQ=</latexit><latexit sha1_base64="7VTdIS0pzFXHTdgbGWNpaikKEtE=">AAACynichVFda9swFJW9buu8r2x73B4uC4OUQbDLYH0plO5l0D500LSFOA2yIjsismyk665G6G2/af9lP6ZQJfVY0xZ2Qehwzv3SUVZLYTCO/wTho43HT55uPouev3j56nXvzdsTUzWa8RGrZKXPMmq4FIqPUKDkZ7XmtMwkP80W35b66QXXRlTqGNuaT0paKJELRtFT097vdE7Rtg52IUV+idaIQjlIJc9xAGmuKbOJs6lpyqld7Cbu/ADYuR0stty/rNviXxV+dneqRTHHrdWcSwef4b9dH2Cz9WYw7fXjYbwKuA+SDvRJF0fT3lU6q1hTcoVMUmPGSVzjxFKNgknuorQxvKZsQQs+9lDRkpuJXfnr4JNnZpBX2h+FsGJvV1haGtOWmc8sKc7NXW1JPqSNG8x3JlaoukGu2M2gvJGAFSw/C2ZCc4ay9YAyLfyuwObUu4f+S6O1MTOz3G3tIbYucr+0i6LI+5Xcdec+ONkeJvEw+fGlv7ffObdJ3pOPZEAS8pXske/kiIwICz4E+8FBcBgehjpsQ3uTGgZdzTuyFuGva2eT3GQ=</latexit><latexit sha1_base64="7VTdIS0pzFXHTdgbGWNpaikKEtE=">AAACynichVFda9swFJW9buu8r2x73B4uC4OUQbDLYH0plO5l0D500LSFOA2yIjsismyk665G6G2/af9lP6ZQJfVY0xZ2Qehwzv3SUVZLYTCO/wTho43HT55uPouev3j56nXvzdsTUzWa8RGrZKXPMmq4FIqPUKDkZ7XmtMwkP80W35b66QXXRlTqGNuaT0paKJELRtFT097vdE7Rtg52IUV+idaIQjlIJc9xAGmuKbOJs6lpyqld7Cbu/ADYuR0stty/rNviXxV+dneqRTHHrdWcSwef4b9dH2Cz9WYw7fXjYbwKuA+SDvRJF0fT3lU6q1hTcoVMUmPGSVzjxFKNgknuorQxvKZsQQs+9lDRkpuJXfnr4JNnZpBX2h+FsGJvV1haGtOWmc8sKc7NXW1JPqSNG8x3JlaoukGu2M2gvJGAFSw/C2ZCc4ay9YAyLfyuwObUu4f+S6O1MTOz3G3tIbYucr+0i6LI+5Xcdec+ONkeJvEw+fGlv7ffObdJ3pOPZEAS8pXske/kiIwICz4E+8FBcBgehjpsQ3uTGgZdzTuyFuGva2eT3GQ=</latexit><latexit sha1_base64="7VTdIS0pzFXHTdgbGWNpaikKEtE=">AAACynichVFda9swFJW9buu8r2x73B4uC4OUQbDLYH0plO5l0D500LSFOA2yIjsismyk665G6G2/af9lP6ZQJfVY0xZ2Qehwzv3SUVZLYTCO/wTho43HT55uPouev3j56nXvzdsTUzWa8RGrZKXPMmq4FIqPUKDkZ7XmtMwkP80W35b66QXXRlTqGNuaT0paKJELRtFT097vdE7Rtg52IUV+idaIQjlIJc9xAGmuKbOJs6lpyqld7Cbu/ADYuR0stty/rNviXxV+dneqRTHHrdWcSwef4b9dH2Cz9WYw7fXjYbwKuA+SDvRJF0fT3lU6q1hTcoVMUmPGSVzjxFKNgknuorQxvKZsQQs+9lDRkpuJXfnr4JNnZpBX2h+FsGJvV1haGtOWmc8sKc7NXW1JPqSNG8x3JlaoukGu2M2gvJGAFSw/C2ZCc4ay9YAyLfyuwObUu4f+S6O1MTOz3G3tIbYucr+0i6LI+5Xcdec+ONkeJvEw+fGlv7ffObdJ3pOPZEAS8pXske/kiIwICz4E+8FBcBgehjpsQ3uTGgZdzTuyFuGva2eT3GQ=</latexit><latexit sha1_base64="7VTdIS0pzFXHTdgbGWNpaikKEtE=">AAACynichVFda9swFJW9buu8r2x73B4uC4OUQbDLYH0plO5l0D500LSFOA2yIjsismyk665G6G2/af9lP6ZQJfVY0xZ2Qehwzv3SUVZLYTCO/wTho43HT55uPouev3j56nXvzdsTUzWa8RGrZKXPMmq4FIqPUKDkZ7XmtMwkP80W35b66QXXRlTqGNuaT0paKJELRtFT097vdE7Rtg52IUV+idaIQjlIJc9xAGmuKbOJs6lpyqld7Cbu/ADYuR0stty/rNviXxV+dneqRTHHrdWcSwef4b9dH2Cz9WYw7fXjYbwKuA+SDvRJF0fT3lU6q1hTcoVMUmPGSVzjxFKNgknuorQxvKZsQQs+9lDRkpuJXfnr4JNnZpBX2h+FsGJvV1haGtOWmc8sKc7NXW1JPqSNG8x3JlaoukGu2M2gvJGAFSw/C2ZCc4ay9YAyLfyuwObUu4f+S6O1MTOz3G3tIbYucr+0i6LI+5Xcdec+ONkeJvEw+fGlv7ffObdJ3pOPZEAS8pXske/kiIwICz4E+8FBcBgehjpsQ3uTGgZdzTuyFuGva2eT3GQ=</latexit>

Voted Perceptron:

Averaged Perceptron:


Efficiency Trick

Efficient Parameter Averaging

» Naive solution: » Keep a running sum of the parameters

at each step » At the end: divide by the total number of

steps » Problem: can still be quite slow

» Parameters won’t change on most iterations

» Would like to only add vectors when we make an error


PKk=1 c

(k)w(k)

PKk=1 c

(k)<latexit sha1_base64="BuDcQDG/Fmn+wamhoKxVO28eYUI=">AAACTnicbVDLSsNAFJ3UV42vqks3g0Wom5KIoAhC0Y3gpoJ9QJOGyWRSh0wezEyUEvIvfo1bu/VH3BWdtlmY1gvDHM65l3vucRNGhTSML62ytr6xuVXd1nd29/YPaodHXRGnHJMOjlnM+y4ShNGIdCSVjPQTTlDoMtJzg/uZ3nslXNA4epbjhNghGkXUpxhJRTm1G8vnCGeWSEMnC27NfPgIIR5mjeA8h2+LPy/LhZo7tbrRNOYFV4FZgDooqu3UppYX4zQkkcQMCTEwjUTaGeKSYkZy3UoFSRAO0IgMFIxQSISdzW/M4ZliPOjHXL1Iwjn7dyJDoRDj0FWdIZIvYlmbkf9pg1T613ZGoySVJMKLRX7KoIzhLDDoUU6wZGMFEOZUeYX4BanQpIpVL63xxMxb6ZAsGfnKdK7rusrLXE5nFXQvmqbRNJ8u6627IrkqOAGnoAFMcAVa4AG0QQdg8A4+wCeYaBPtW5tqP4vWilbMHINSVaq/E/Kz/w==</latexit><latexit sha1_base64="BuDcQDG/Fmn+wamhoKxVO28eYUI=">AAACTnicbVDLSsNAFJ3UV42vqks3g0Wom5KIoAhC0Y3gpoJ9QJOGyWRSh0wezEyUEvIvfo1bu/VH3BWdtlmY1gvDHM65l3vucRNGhTSML62ytr6xuVXd1nd29/YPaodHXRGnHJMOjlnM+y4ShNGIdCSVjPQTTlDoMtJzg/uZ3nslXNA4epbjhNghGkXUpxhJRTm1G8vnCGeWSEMnC27NfPgIIR5mjeA8h2+LPy/LhZo7tbrRNOYFV4FZgDooqu3UppYX4zQkkcQMCTEwjUTaGeKSYkZy3UoFSRAO0IgMFIxQSISdzW/M4ZliPOjHXL1Iwjn7dyJDoRDj0FWdIZIvYlmbkf9pg1T613ZGoySVJMKLRX7KoIzhLDDoUU6wZGMFEOZUeYX4BanQpIpVL63xxMxb6ZAsGfnKdK7rusrLXE5nFXQvmqbRNJ8u6627IrkqOAGnoAFMcAVa4AG0QQdg8A4+wCeYaBPtW5tqP4vWilbMHINSVaq/E/Kz/w==</latexit><latexit sha1_base64="BuDcQDG/Fmn+wamhoKxVO28eYUI=">AAACTnicbVDLSsNAFJ3UV42vqks3g0Wom5KIoAhC0Y3gpoJ9QJOGyWRSh0wezEyUEvIvfo1bu/VH3BWdtlmY1gvDHM65l3vucRNGhTSML62ytr6xuVXd1nd29/YPaodHXRGnHJMOjlnM+y4ShNGIdCSVjPQTTlDoMtJzg/uZ3nslXNA4epbjhNghGkXUpxhJRTm1G8vnCGeWSEMnC27NfPgIIR5mjeA8h2+LPy/LhZo7tbrRNOYFV4FZgDooqu3UppYX4zQkkcQMCTEwjUTaGeKSYkZy3UoFSRAO0IgMFIxQSISdzW/M4ZliPOjHXL1Iwjn7dyJDoRDj0FWdIZIvYlmbkf9pg1T613ZGoySVJMKLRX7KoIzhLDDoUU6wZGMFEOZUeYX4BanQpIpVL63xxMxb6ZAsGfnKdK7rusrLXE5nFXQvmqbRNJ8u6627IrkqOAGnoAFMcAVa4AG0QQdg8A4+wCeYaBPtW5tqP4vWilbMHINSVaq/E/Kz/w==</latexit><latexit sha1_base64="rIc4elZHT3oXILhfcJEssMOX7as=">AAACBHicbVDLSgMxFL1TX3WsWtdugkVwVWbc6FJw47KCfUAdSiaTaUMzyZDcEcrQH3Dbr3En/oUfI5hpu7CtBy4czkm45544l8JiEHx7tb39g8Oj+rF/0vBPz86bjZ7VhWG8y7TUZhBTy6VQvIsCJR/khtMslrwfTx8rv//GjRVaveAs51FGx0qkglF0UmfUbAXtYAmyS8I1acEao+bPa6JZkXGFTFJrh2GQY1RSg4JJPvdfC8tzyqZ0zIeOKppxG5XLmHNy7ZSEpNq4UUiW6t8fJc2snWWxe5lRnNhtrxL/84YFpvdRKVReIFdstSgtJEFNqptJIgxnKGeOUGaEy0rYhBrK0DXjb6xJbJVt45AyH6cu9Nz3fVdXuF3OLundtsOgHT4HUIdLuIIbCOEOHuAJOtAFBgm8w8JbeB/e56rWmrfu9wI24H39Asd+my0=</latexit><latexit sha1_base64="n0mbPTYOpLcUvfE+a0Rncmi0fVM=">AAACQ3icbZDLSsNAFIZP6q3GqtGtm8Ei1E1J3CiCILgR3FSwF2jSMJlO2qGTCzMTpYS8i0/j1m59EXdFp5eFbT0wzM//z3DO+YKUM6ls+8sobW3v7O6V982DyuHRsXVSackkE4Q2ScIT0QmwpJzFtKmY4rSTCoqjgNN2MHqY5e1XKiRL4hc1TqkX4UHMQkaw0pZv3bqhwCR3ZRb5+ejOKXpPCJFeXhtdFuhtcRer8TItfKtq1+15oU3hLEUVltXwranbT0gW0VgRjqXsOnaqvBwLxQinhelmkqaYjPCAdrWMcUSll893LNCFdvooTIQ+sUJz9++PHEdSjqNAv4ywGsr1bGb+l3UzFd54OYvTTNGYLBqFGUcqQTNgqM8EJYqPtcBEMD0rIkOsoSmN1Vxp05ez2VYWydNBqIcuTNPUvJx1OpuidVV37LrzbEMZzuAcauDANdzDIzSgCQTe4QM+YWJMjG9juiBbMpaIT2GljJ9fVF6zRQ==</latexit><latexit sha1_base64="n0mbPTYOpLcUvfE+a0Rncmi0fVM=">AAACQ3icbZDLSsNAFIZP6q3GqtGtm8Ei1E1J3CiCILgR3FSwF2jSMJlO2qGTCzMTpYS8i0/j1m59EXdFp5eFbT0wzM//z3DO+YKUM6ls+8sobW3v7O6V982DyuHRsXVSackkE4Q2ScIT0QmwpJzFtKmY4rSTCoqjgNN2MHqY5e1XKiRL4hc1TqkX4UHMQkaw0pZv3bqhwCR3ZRb5+ejOKXpPCJFeXhtdFuhtcRer8TItfKtq1+15oU3hLEUVltXwranbT0gW0VgRjqXsOnaqvBwLxQinhelmkqaYjPCAdrWMcUSll893LNCFdvooTIQ+sUJz9++PHEdSjqNAv4ywGsr1bGb+l3UzFd54OYvTTNGYLBqFGUcqQTNgqM8EJYqPtcBEMD0rIkOsoSmN1Vxp05ez2VYWydNBqIcuTNPUvJx1OpuidVV37LrzbEMZzuAcauDANdzDIzSgCQTe4QM+YWJMjG9juiBbMpaIT2GljJ9fVF6zRQ==</latexit><latexit sha1_base64="9pK+9EKLhAcfRCjphZ924E/TIg0=">AAACTnicbVDLSsNAFJ3UV42vqEs3g0Wom5K4UQSh6EZwU8E+oEnLZDJph0wezEyUEvIvfo1bu/VH3BWdtFnY1gvDHM65l3vucRNGhTTNL62ysbm1vVPd1ff2Dw6PjOOTjohTjkkbxyzmPRcJwmhE2pJKRnoJJyh0Gem6wUOhd18JFzSOXuQkIU6IRhH1KUZSUUPj1vY5wpkt0nCYBXdWPniCEA+yenCZw7fFny/LpZoPjZrZMOcF14FVghooqzU0ZrYX4zQkkcQMCdG3zEQ6GeKSYkZy3U4FSRAO0Ij0FYxQSISTzW/M4YViPOjHXL1Iwjn7dyJDoRCT0FWdIZJjsaoV5H9aP5X+jZPRKEklifBikZ8yKGNYBAY9ygmWbKIAwpwqrxCPkQpNqlj1pTWeKLwtHZIlI1+ZznVdV3lZq+msg85VwzIb1rNZa96XyVXBGTgHdWCBa9AEj6AF2gCDd/ABPsFUm2rf2kz7WbRWtHLmFCxVpfoLErKz+w==</latexit><latexit sha1_base64="BuDcQDG/Fmn+wamhoKxVO28eYUI=">AAACTnicbVDLSsNAFJ3UV42vqks3g0Wom5KIoAhC0Y3gpoJ9QJOGyWRSh0wezEyUEvIvfo1bu/VH3BWdtlmY1gvDHM65l3vucRNGhTSML62ytr6xuVXd1nd29/YPaodHXRGnHJMOjlnM+y4ShNGIdCSVjPQTTlDoMtJzg/uZ3nslXNA4epbjhNghGkXUpxhJRTm1G8vnCGeWSEMnC27NfPgIIR5mjeA8h2+LPy/LhZo7tbrRNOYFV4FZgDooqu3UppYX4zQkkcQMCTEwjUTaGeKSYkZy3UoFSRAO0IgMFIxQSISdzW/M4ZliPOjHXL1Iwjn7dyJDoRDj0FWdIZIvYlmbkf9pg1T613ZGoySVJMKLRX7KoIzhLDDoUU6wZGMFEOZUeYX4BanQpIpVL63xxMxb6ZAsGfnKdK7rusrLXE5nFXQvmqbRNJ8u6627IrkqOAGnoAFMcAVa4AG0QQdg8A4+wCeYaBPtW5tqP4vWilbMHINSVaq/E/Kz/w==</latexit><latexit sha1_base64="BuDcQDG/Fmn+wamhoKxVO28eYUI=">AAACTnicbVDLSsNAFJ3UV42vqks3g0Wom5KIoAhC0Y3gpoJ9QJOGyWRSh0wezEyUEvIvfo1bu/VH3BWdtlmY1gvDHM65l3vucRNGhTSML62ytr6xuVXd1nd29/YPaodHXRGnHJMOjlnM+y4ShNGIdCSVjPQTTlDoMtJzg/uZ3nslXNA4epbjhNghGkXUpxhJRTm1G8vnCGeWSEMnC27NfPgIIR5mjeA8h2+LPy/LhZo7tbrRNOYFV4FZgDooqu3UppYX4zQkkcQMCTEwjUTaGeKSYkZy3UoFSRAO0IgMFIxQSISdzW/M4ZliPOjHXL1Iwjn7dyJDoRDj0FWdIZIvYlmbkf9pg1T613ZGoySVJMKLRX7KoIzhLDDoUU6wZGMFEOZUeYX4BanQpIpVL63xxMxb6ZAsGfnKdK7rusrLXE5nFXQvmqbRNJ8u6627IrkqOAGnoAFMcAVa4AG0QQdg8A4+wCeYaBPtW5tqP4vWilbMHINSVaq/E/Kz/w==</latexit><latexit sha1_base64="BuDcQDG/Fmn+wamhoKxVO28eYUI=">AAACTnicbVDLSsNAFJ3UV42vqks3g0Wom5KIoAhC0Y3gpoJ9QJOGyWRSh0wezEyUEvIvfo1bu/VH3BWdtlmY1gvDHM65l3vucRNGhTSML62ytr6xuVXd1nd29/YPaodHXRGnHJMOjlnM+y4ShNGIdCSVjPQTTlDoMtJzg/uZ3nslXNA4epbjhNghGkXUpxhJRTm1G8vnCGeWSEMnC27NfPgIIR5mjeA8h2+LPy/LhZo7tbrRNOYFV4FZgDooqu3UppYX4zQkkcQMCTEwjUTaGeKSYkZy3UoFSRAO0IgMFIxQSISdzW/M4ZliPOjHXL1Iwjn7dyJDoRDj0FWdIZIvYlmbkf9pg1T613ZGoySVJMKLRX7KoIzhLDDoUU6wZGMFEOZUeYX4BanQpIpVL63xxMxb6ZAsGfnKdK7rusrLXE5nFXQvmqbRNJ8u6627IrkqOAGnoAFMcAVa4AG0QQdg8A4+wCeYaBPtW5tqP4vWilbMHINSVaq/E/Kz/w==</latexit><latexit sha1_base64="BuDcQDG/Fmn+wamhoKxVO28eYUI=">AAACTnicbVDLSsNAFJ3UV42vqks3g0Wom5KIoAhC0Y3gpoJ9QJOGyWRSh0wezEyUEvIvfo1bu/VH3BWdtlmY1gvDHM65l3vucRNGhTSML62ytr6xuVXd1nd29/YPaodHXRGnHJMOjlnM+y4ShNGIdCSVjPQTTlDoMtJzg/uZ3nslXNA4epbjhNghGkXUpxhJRTm1G8vnCGeWSEMnC27NfPgIIR5mjeA8h2+LPy/LhZo7tbrRNOYFV4FZgDooqu3UppYX4zQkkcQMCTEwjUTaGeKSYkZy3UoFSRAO0IgMFIxQSISdzW/M4ZliPOjHXL1Iwjn7dyJDoRDj0FWdIZIvYlmbkf9pg1T613ZGoySVJMKLRX7KoIzhLDDoUU6wZGMFEOZUeYX4BanQpIpVL63xxMxb6ZAsGfnKdK7rusrLXE5nFXQvmqbRNJ8u6627IrkqOAGnoAFMcAVa4AG0QQdg8A4+wCeYaBPtW5tqP4vWilbMHINSVaq/E/Kz/w==</latexit><latexit sha1_base64="BuDcQDG/Fmn+wamhoKxVO28eYUI=">AAACTnicbVDLSsNAFJ3UV42vqks3g0Wom5KIoAhC0Y3gpoJ9QJOGyWRSh0wezEyUEvIvfo1bu/VH3BWdtlmY1gvDHM65l3vucRNGhTSML62ytr6xuVXd1nd29/YPaodHXRGnHJMOjlnM+y4ShNGIdCSVjPQTTlDoMtJzg/uZ3nslXNA4epbjhNghGkXUpxhJRTm1G8vnCGeWSEMnC27NfPgIIR5mjeA8h2+LPy/LhZo7tbrRNOYFV4FZgDooqu3UppYX4zQkkcQMCTEwjUTaGeKSYkZy3UoFSRAO0IgMFIxQSISdzW/M4ZliPOjHXL1Iwjn7dyJDoRDj0FWdIZIvYlmbkf9pg1T613ZGoySVJMKLRX7KoIzhLDDoUU6wZGMFEOZUeYX4BanQpIpVL63xxMxb6ZAsGfnKdK7rusrLXE5nFXQvmqbRNJ8u6627IrkqOAGnoAFMcAVa4AG0QQdg8A4+wCeYaBPtW5tqP4vWilbMHINSVaq/E/Kz/w==</latexit><latexit sha1_base64="BuDcQDG/Fmn+wamhoKxVO28eYUI=">AAACTnicbVDLSsNAFJ3UV42vqks3g0Wom5KIoAhC0Y3gpoJ9QJOGyWRSh0wezEyUEvIvfo1bu/VH3BWdtlmY1gvDHM65l3vucRNGhTSML62ytr6xuVXd1nd29/YPaodHXRGnHJMOjlnM+y4ShNGIdCSVjPQTTlDoMtJzg/uZ3nslXNA4epbjhNghGkXUpxhJRTm1G8vnCGeWSEMnC27NfPgIIR5mjeA8h2+LPy/LhZo7tbrRNOYFV4FZgDooqu3UppYX4zQkkcQMCTEwjUTaGeKSYkZy3UoFSRAO0IgMFIxQSISdzW/M4ZliPOjHXL1Iwjn7dyJDoRDj0FWdIZIvYlmbkf9pg1T613ZGoySVJMKLRX7KoIzhLDDoUU6wZGMFEOZUeYX4BanQpIpVL63xxMxb6ZAsGfnKdK7rusrLXE5nFXQvmqbRNJ8u6627IrkqOAGnoAFMcAVa4AG0QQdg8A4+wCeYaBPtW5tqP4vWilbMHINSVaq/E/Kz/w==</latexit>

Averaged Parameters: our goal is to compute This:


PKk=1 c

(k)w(k)

PKk=1 c



Parameters After kth update


PKk=1 c

(k)w(k)

PKk=1 c




How long did the kth parameters survive?


PKk=1 c

(k)w(k)

PKk=1 c





Total number of steps


PKk=1 c

(k)w(k)

PKk=1 c





Total number of stepsLet’s focus on the numerator (denominator is easy to compute)


KX

k=1

c(k)w(k)

<latexit sha1_base64="70ybiU0W8G4QcnSc471a8+vvGWc=">AAACMXicbVDLSsNAFJ3UV42vqEsRBotQNyURQTdC0Y3gpoJ9QJuGyWTSDp08mJkoJWTl17i1X9OduPUbBCdtFrb1wjCHc+7lnnvcmFEhTXOqldbWNza3ytv6zu7e/oFxeNQSUcIxaeKIRbzjIkEYDUlTUslIJ+YEBS4jbXd0n+vtF8IFjcJnOY6JHaBBSH2KkVSUY5z2RBI46ejWyvqPEOJ+Wh1dZPB1/jtGxayZs4KrwCpABRTVcIyfnhfhJCChxAwJ0bXMWNop4pJiRjK9lwgSIzxCA9JVMEQBEXY6OyOD54rxoB9x9UIJZ+zfiRQFQowDV3UGSA7FspaT/2ndRPo3dkrDOJEkxPNFfsKgjGCeCfQoJ1iysQIIc6q8QjxEHGGpktMX1ngi97ZwSBoPfGU603Vd5WUtp7MKWpc1y6xZT1eV+l2RXBmcgDNQBRa4BnXwABqgCTB4A+/gA0y0iTbVPrWveWtJK2aOwUJp3792zKjV</latexit><latexit sha1_base64="70ybiU0W8G4QcnSc471a8+vvGWc=">AAACMXicbVDLSsNAFJ3UV42vqEsRBotQNyURQTdC0Y3gpoJ9QJuGyWTSDp08mJkoJWTl17i1X9OduPUbBCdtFrb1wjCHc+7lnnvcmFEhTXOqldbWNza3ytv6zu7e/oFxeNQSUcIxaeKIRbzjIkEYDUlTUslIJ+YEBS4jbXd0n+vtF8IFjcJnOY6JHaBBSH2KkVSUY5z2RBI46ejWyvqPEOJ+Wh1dZPB1/jtGxayZs4KrwCpABRTVcIyfnhfhJCChxAwJ0bXMWNop4pJiRjK9lwgSIzxCA9JVMEQBEXY6OyOD54rxoB9x9UIJZ+zfiRQFQowDV3UGSA7FspaT/2ndRPo3dkrDOJEkxPNFfsKgjGCeCfQoJ1iysQIIc6q8QjxEHGGpktMX1ngi97ZwSBoPfGU603Vd5WUtp7MKWpc1y6xZT1eV+l2RXBmcgDNQBRa4BnXwABqgCTB4A+/gA0y0iTbVPrWveWtJK2aOwUJp3792zKjV</latexit><latexit sha1_base64="70ybiU0W8G4QcnSc471a8+vvGWc=">AAACMXicbVDLSsNAFJ3UV42vqEsRBotQNyURQTdC0Y3gpoJ9QJuGyWTSDp08mJkoJWTl17i1X9OduPUbBCdtFrb1wjCHc+7lnnvcmFEhTXOqldbWNza3ytv6zu7e/oFxeNQSUcIxaeKIRbzjIkEYDUlTUslIJ+YEBS4jbXd0n+vtF8IFjcJnOY6JHaBBSH2KkVSUY5z2RBI46ejWyvqPEOJ+Wh1dZPB1/jtGxayZs4KrwCpABRTVcIyfnhfhJCChxAwJ0bXMWNop4pJiRjK9lwgSIzxCA9JVMEQBEXY6OyOD54rxoB9x9UIJZ+zfiRQFQowDV3UGSA7FspaT/2ndRPo3dkrDOJEkxPNFfsKgjGCeCfQoJ1iysQIIc6q8QjxEHGGpktMX1ngi97ZwSBoPfGU603Vd5WUtp7MKWpc1y6xZT1eV+l2RXBmcgDNQBRa4BnXwABqgCTB4A+/gA0y0iTbVPrWveWtJK2aOwUJp3792zKjV</latexit><latexit sha1_base64="70ybiU0W8G4QcnSc471a8+vvGWc=">AAACMXicbVDLSsNAFJ3UV42vqEsRBotQNyURQTdC0Y3gpoJ9QJuGyWTSDp08mJkoJWTl17i1X9OduPUbBCdtFrb1wjCHc+7lnnvcmFEhTXOqldbWNza3ytv6zu7e/oFxeNQSUcIxaeKIRbzjIkEYDUlTUslIJ+YEBS4jbXd0n+vtF8IFjcJnOY6JHaBBSH2KkVSUY5z2RBI46ejWyvqPEOJ+Wh1dZPB1/jtGxayZs4KrwCpABRTVcIyfnhfhJCChxAwJ0bXMWNop4pJiRjK9lwgSIzxCA9JVMEQBEXY6OyOD54rxoB9x9UIJZ+zfiRQFQowDV3UGSA7FspaT/2ndRPo3dkrDOJEkxPNFfsKgjGCeCfQoJ1iysQIIc6q8QjxEHGGpktMX1ngi97ZwSBoPfGU603Vd5WUtp7MKWpc1y6xZT1eV+l2RXBmcgDNQBRa4BnXwABqgCTB4A+/gA0y0iTbVPrWveWtJK2aOwUJp3792zKjV</latexit>


KX

k=1

c(k)w(k)


= c(1)w(1) + c(2)w(2) + · · ·+ c(K)w(K)<latexit sha1_base64="0zCfeUUFkGv2vqrCGq3pvax43dg=">AAACUnicbVJNS8NAEN3ErxqrRj16WSxCRShJEfSiFL0IXirYD2hj2Ww2denmg92NUkL+jb/Gq734VwTBTZqDbR1Y5vHeDDPzWDdmVEjL+tL0tfWNza3KtrFT3d3bNw8OuyJKOCYdHLGI910kCKMh6UgqGenHnKDAZaTnTu5yvfdKuKBR+CSnMXECNA6pTzGSihqZN9cQP6d1+yyDb2U+L5hmyTQLZoi9SIpSeigllUdmzWpYRcBVYJegBspoj8yfoRfhJCChxAwJMbCtWDop4pJiRjJjmAgSIzxBYzJQMEQBEU5a3JnBU8V40I+4eqGEBfu3I0WBENPAVZUBki9iWcvJ/7RBIv0rJ6VhnEgS4vkgP2FQRjA3DXqUEyzZVAGEOVW7QvyCOMJSWWssjPFEvtvCIWk89tXSmWEYyi972Z1V0G02bKthP17UWrelcxVwDE5AHdjgErTAPWiDDsDgHXyATzDTZtq3rn7JvFTXyp4jsBB69Re5jK+F</latexit><latexit sha1_base64="0zCfeUUFkGv2vqrCGq3pvax43dg=">AAACUnicbVJNS8NAEN3ErxqrRj16WSxCRShJEfSiFL0IXirYD2hj2Ww2denmg92NUkL+jb/Gq734VwTBTZqDbR1Y5vHeDDPzWDdmVEjL+tL0tfWNza3KtrFT3d3bNw8OuyJKOCYdHLGI910kCKMh6UgqGenHnKDAZaTnTu5yvfdKuKBR+CSnMXECNA6pTzGSihqZN9cQP6d1+yyDb2U+L5hmyTQLZoi9SIpSeigllUdmzWpYRcBVYJegBspoj8yfoRfhJCChxAwJMbCtWDop4pJiRjJjmAgSIzxBYzJQMEQBEU5a3JnBU8V40I+4eqGEBfu3I0WBENPAVZUBki9iWcvJ/7RBIv0rJ6VhnEgS4vkgP2FQRjA3DXqUEyzZVAGEOVW7QvyCOMJSWWssjPFEvtvCIWk89tXSmWEYyi972Z1V0G02bKthP17UWrelcxVwDE5AHdjgErTAPWiDDsDgHXyATzDTZtq3rn7JvFTXyp4jsBB69Re5jK+F</latexit><latexit sha1_base64="0zCfeUUFkGv2vqrCGq3pvax43dg=">AAACUnicbVJNS8NAEN3ErxqrRj16WSxCRShJEfSiFL0IXirYD2hj2Ww2denmg92NUkL+jb/Gq734VwTBTZqDbR1Y5vHeDDPzWDdmVEjL+tL0tfWNza3KtrFT3d3bNw8OuyJKOCYdHLGI910kCKMh6UgqGenHnKDAZaTnTu5yvfdKuKBR+CSnMXECNA6pTzGSihqZN9cQP6d1+yyDb2U+L5hmyTQLZoi9SIpSeigllUdmzWpYRcBVYJegBspoj8yfoRfhJCChxAwJMbCtWDop4pJiRjJjmAgSIzxBYzJQMEQBEU5a3JnBU8V40I+4eqGEBfu3I0WBENPAVZUBki9iWcvJ/7RBIv0rJ6VhnEgS4vkgP2FQRjA3DXqUEyzZVAGEOVW7QvyCOMJSWWssjPFEvtvCIWk89tXSmWEYyi972Z1V0G02bKthP17UWrelcxVwDE5AHdjgErTAPWiDDsDgHXyATzDTZtq3rn7JvFTXyp4jsBB69Re5jK+F</latexit><latexit sha1_base64="0zCfeUUFkGv2vqrCGq3pvax43dg=">AAACUnicbVJNS8NAEN3ErxqrRj16WSxCRShJEfSiFL0IXirYD2hj2Ww2denmg92NUkL+jb/Gq734VwTBTZqDbR1Y5vHeDDPzWDdmVEjL+tL0tfWNza3KtrFT3d3bNw8OuyJKOCYdHLGI910kCKMh6UgqGenHnKDAZaTnTu5yvfdKuKBR+CSnMXECNA6pTzGSihqZN9cQP6d1+yyDb2U+L5hmyTQLZoi9SIpSeigllUdmzWpYRcBVYJegBspoj8yfoRfhJCChxAwJMbCtWDop4pJiRjJjmAgSIzxBYzJQMEQBEU5a3JnBU8V40I+4eqGEBfu3I0WBENPAVZUBki9iWcvJ/7RBIv0rJ6VhnEgS4vkgP2FQRjA3DXqUEyzZVAGEOVW7QvyCOMJSWWssjPFEvtvCIWk89tXSmWEYyi972Z1V0G02bKthP17UWrelcxVwDE5AHdjgErTAPWiDDsDgHXyATzDTZtq3rn7JvFTXyp4jsBB69Re5jK+F</latexit>


KX

k=1

c(k)w(k)



= c(1)y(1)x(1) + c(2)⇣y(1)x(1) + y(2)x(2)

⌘+ · · ·+ c(K)

⇣y(1)x(1) + y(2)x(2) + · · ·+ y(K)x(K)

⌘

<latexit sha1_base64="SpZIrweXSGQr0Jh2nRvHEaBZfBM=">AAACxXiclVHbSgMxEM2u9/VW9cEHX4JFqAhltwj6IoiCCr4oWBXaWrLZbBuavZDMSpel+E1+jh8jmLYptNYXB8Ic5syZmcz4qeAKXPfLshcWl5ZXVtec9Y3Nre3Szu6zSjJJWZ0mIpGvPlFM8JjVgYNgr6lkJPIFe/F710P+5Z1JxZP4CfKUtSLSiXnIKQEdapc+LzB9Kyre8QDnxveNPxkxNY2agoVQmedzw/cneZJ3unCsmSYNElCmxP0/SkxJcyPtT0qMq7dLZbfqjgzPA8+AMjL20C59N4OEZhGLgQqiVMNzU2gVRAKngg2cZqZYSmiPdFhDw5hETLWK0WoH+EhHAhwmUr8Y8Cg6rShIpFQe+TozItBVv7lh8C+ukUF43ip4nGbAYjpuFGYCQ4KHd8IBl4yCyDUgVHI9K6ZdIgkFfU1npk2ghrPNfKRIO6EeeuA4jt6X93s78+C5VvXcqvd4Wr68MptbRQfoEFWQh87QJbpDD6iOqLVvXVg31q19a0c22O/jVNsymj00Y/bHD0cc0cg=</latexit><latexit sha1_base64="SpZIrweXSGQr0Jh2nRvHEaBZfBM=">AAACxXiclVHbSgMxEM2u9/VW9cEHX4JFqAhltwj6IoiCCr4oWBXaWrLZbBuavZDMSpel+E1+jh8jmLYptNYXB8Ic5syZmcz4qeAKXPfLshcWl5ZXVtec9Y3Nre3Szu6zSjJJWZ0mIpGvPlFM8JjVgYNgr6lkJPIFe/F710P+5Z1JxZP4CfKUtSLSiXnIKQEdapc+LzB9Kyre8QDnxveNPxkxNY2agoVQmedzw/cneZJ3unCsmSYNElCmxP0/SkxJcyPtT0qMq7dLZbfqjgzPA8+AMjL20C59N4OEZhGLgQqiVMNzU2gVRAKngg2cZqZYSmiPdFhDw5hETLWK0WoH+EhHAhwmUr8Y8Cg6rShIpFQe+TozItBVv7lh8C+ukUF43ip4nGbAYjpuFGYCQ4KHd8IBl4yCyDUgVHI9K6ZdIgkFfU1npk2ghrPNfKRIO6EeeuA4jt6X93s78+C5VvXcqvd4Wr68MptbRQfoEFWQh87QJbpDD6iOqLVvXVg31q19a0c22O/jVNsymj00Y/bHD0cc0cg=</latexit><latexit sha1_base64="SpZIrweXSGQr0Jh2nRvHEaBZfBM=">AAACxXiclVHbSgMxEM2u9/VW9cEHX4JFqAhltwj6IoiCCr4oWBXaWrLZbBuavZDMSpel+E1+jh8jmLYptNYXB8Ic5syZmcz4qeAKXPfLshcWl5ZXVtec9Y3Nre3Szu6zSjJJWZ0mIpGvPlFM8JjVgYNgr6lkJPIFe/F710P+5Z1JxZP4CfKUtSLSiXnIKQEdapc+LzB9Kyre8QDnxveNPxkxNY2agoVQmedzw/cneZJ3unCsmSYNElCmxP0/SkxJcyPtT0qMq7dLZbfqjgzPA8+AMjL20C59N4OEZhGLgQqiVMNzU2gVRAKngg2cZqZYSmiPdFhDw5hETLWK0WoH+EhHAhwmUr8Y8Cg6rShIpFQe+TozItBVv7lh8C+ukUF43ip4nGbAYjpuFGYCQ4KHd8IBl4yCyDUgVHI9K6ZdIgkFfU1npk2ghrPNfKRIO6EeeuA4jt6X93s78+C5VvXcqvd4Wr68MptbRQfoEFWQh87QJbpDD6iOqLVvXVg31q19a0c22O/jVNsymj00Y/bHD0cc0cg=</latexit><latexit sha1_base64="rIc4elZHT3oXILhfcJEssMOX7as=">AAACBHicbVDLSgMxFL1TX3WsWtdugkVwVWbc6FJw47KCfUAdSiaTaUMzyZDcEcrQH3Dbr3En/oUfI5hpu7CtBy4czkm45544l8JiEHx7tb39g8Oj+rF/0vBPz86bjZ7VhWG8y7TUZhBTy6VQvIsCJR/khtMslrwfTx8rv//GjRVaveAs51FGx0qkglF0UmfUbAXtYAmyS8I1acEao+bPa6JZkXGFTFJrh2GQY1RSg4JJPvdfC8tzyqZ0zIeOKppxG5XLmHNy7ZSEpNq4UUiW6t8fJc2snWWxe5lRnNhtrxL/84YFpvdRKVReIFdstSgtJEFNqptJIgxnKGeOUGaEy0rYhBrK0DXjb6xJbJVt45AyH6cu9Nz3fVdXuF3OLundtsOgHT4HUIdLuIIbCOEOHuAJOtAFBgm8w8JbeB/e56rWmrfu9wI24H39Asd+my0=</latexit><latexit sha1_base64="cmFyKRk9YjQx0kmMYm/8vmNkfwI=">AAACuniclVHLSsNAFJ3EV41VoxsXbgZFaBFK0o1uBEGwgpsKVoWmlslk0g5OHszcSEPoR/k5fozgNE3BWjdeGO7hnvuac/1UcAWO82mYa+sbm1u1bWunvru3bx/Un1SSScp6NBGJfPGJYoLHrAccBHtJJSORL9iz/3Yz45/fmVQ8iR8hT9kgIqOYh5wS0KGh/XGF6WvRcJtTnFd+Uvnzkmlr5AkWQmOVzyt+ssiTfDSGpmY8GiSgqhb3/2jxozSvSieLFvPuQ/vUaTml4VXgVuAUVdYd2l9ekNAsYjFQQZTqu04Kg4JI4FSwqeVliqWEvpER62sYk4ipQVFKO8VnOhLgMJH6xYDL6M+KgkRK5ZGvMyMCY/WbmwX/4voZhJeDgsdpBiym80FhJjAkeHYnHHDJKIhcA0Il17tiOiaSUNDXtJbGBGq229JHinQU6qWnlmVpvdzf6qyCp3bLdVrug4Nq6BidoAZy0QW6Rneoi3qIGkfGlXFrdMyOGZkwV9Y0KokP0ZKZ798rYtCX</latexit><latexit sha1_base64="cmFyKRk9YjQx0kmMYm/8vmNkfwI=">AAACuniclVHLSsNAFJ3EV41VoxsXbgZFaBFK0o1uBEGwgpsKVoWmlslk0g5OHszcSEPoR/k5fozgNE3BWjdeGO7hnvuac/1UcAWO82mYa+sbm1u1bWunvru3bx/Un1SSScp6NBGJfPGJYoLHrAccBHtJJSORL9iz/3Yz45/fmVQ8iR8hT9kgIqOYh5wS0KGh/XGF6WvRcJtTnFd+Uvnzkmlr5AkWQmOVzyt+ssiTfDSGpmY8GiSgqhb3/2jxozSvSieLFvPuQ/vUaTml4VXgVuAUVdYd2l9ekNAsYjFQQZTqu04Kg4JI4FSwqeVliqWEvpER62sYk4ipQVFKO8VnOhLgMJH6xYDL6M+KgkRK5ZGvMyMCY/WbmwX/4voZhJeDgsdpBiym80FhJjAkeHYnHHDJKIhcA0Il17tiOiaSUNDXtJbGBGq229JHinQU6qWnlmVpvdzf6qyCp3bLdVrug4Nq6BidoAZy0QW6Rneoi3qIGkfGlXFrdMyOGZkwV9Y0KokP0ZKZ798rYtCX</latexit><latexit sha1_base64="/ZNcamsz6TihzeKvzZaqaO4IHLQ=">AAACxXiclVFLSwMxEM6ur7q+qh48eAkWoSKUXS96EUTBCl4UrAptLdlstg3NPkhmpctS/E3+HH+MYNpG6MOLA2E+5ptvZjLjp4IrcN0vy15aXlldK607G5tb2zvl3b1nlWSSsgZNRCJffaKY4DFrAAfBXlPJSOQL9uL3b0b8yzuTiifxE+Qpa0ekG/OQUwI61Cl/XmL6VlS9kyHOjR8YfzpmzjRqCRZCdZHPDT/4zZO824MTzbRokIAyJe7/UWJKmhvp4LfEpHqnXHFr7tjwIvAMqCBjD53ydytIaBaxGKggSjU9N4V2QSRwKtjQaWWKpYT2SZc1NYxJxFS7GK92iI91JMBhIvWLAY+j04qCRErlka8zIwI9Nc+Ngn9xzQzCi3bB4zQDFtNJozATGBI8uhMOuGQURK4BoZLrWTHtEUko6Gs6M20CNZpt5iNF2g310EPHcfS+vPntLILns5rn1rxHt3J1bTZXQofoCFWRh87RFbpDD6iBqHVgXVq3Vt2u25EN9vsk1baMZh/NmP3xA0Xc0cQ=</latexit><latexit sha1_base64="SpZIrweXSGQr0Jh2nRvHEaBZfBM=">AAACxXiclVHbSgMxEM2u9/VW9cEHX4JFqAhltwj6IoiCCr4oWBXaWrLZbBuavZDMSpel+E1+jh8jmLYptNYXB8Ic5syZmcz4qeAKXPfLshcWl5ZXVtec9Y3Nre3Szu6zSjJJWZ0mIpGvPlFM8JjVgYNgr6lkJPIFe/F710P+5Z1JxZP4CfKUtSLSiXnIKQEdapc+LzB9Kyre8QDnxveNPxkxNY2agoVQmedzw/cneZJ3unCsmSYNElCmxP0/SkxJcyPtT0qMq7dLZbfqjgzPA8+AMjL20C59N4OEZhGLgQqiVMNzU2gVRAKngg2cZqZYSmiPdFhDw5hETLWK0WoH+EhHAhwmUr8Y8Cg6rShIpFQe+TozItBVv7lh8C+ukUF43ip4nGbAYjpuFGYCQ4KHd8IBl4yCyDUgVHI9K6ZdIgkFfU1npk2ghrPNfKRIO6EeeuA4jt6X93s78+C5VvXcqvd4Wr68MptbRQfoEFWQh87QJbpDD6iOqLVvXVg31q19a0c22O/jVNsymj00Y/bHD0cc0cg=</latexit><latexit sha1_base64="SpZIrweXSGQr0Jh2nRvHEaBZfBM=">AAACxXiclVHbSgMxEM2u9/VW9cEHX4JFqAhltwj6IoiCCr4oWBXaWrLZbBuavZDMSpel+E1+jh8jmLYptNYXB8Ic5syZmcz4qeAKXPfLshcWl5ZXVtec9Y3Nre3Szu6zSjJJWZ0mIpGvPlFM8JjVgYNgr6lkJPIFe/F710P+5Z1JxZP4CfKUtSLSiXnIKQEdapc+LzB9Kyre8QDnxveNPxkxNY2agoVQmedzw/cneZJ3unCsmSYNElCmxP0/SkxJcyPtT0qMq7dLZbfqjgzPA8+AMjL20C59N4OEZhGLgQqiVMNzU2gVRAKngg2cZqZYSmiPdFhDw5hETLWK0WoH+EhHAhwmUr8Y8Cg6rShIpFQe+TozItBVv7lh8C+ukUF43ip4nGbAYjpuFGYCQ4KHd8IBl4yCyDUgVHI9K6ZdIgkFfU1npk2ghrPNfKRIO6EeeuA4jt6X93s78+C5VvXcqvd4Wr68MptbRQfoEFWQh87QJbpDD6iOqLVvXVg31q19a0c22O/jVNsymj00Y/bHD0cc0cg=</latexit><latexit sha1_base64="SpZIrweXSGQr0Jh2nRvHEaBZfBM=">AAACxXiclVHbSgMxEM2u9/VW9cEHX4JFqAhltwj6IoiCCr4oWBXaWrLZbBuavZDMSpel+E1+jh8jmLYptNYXB8Ic5syZmcz4qeAKXPfLshcWl5ZXVtec9Y3Nre3Szu6zSjJJWZ0mIpGvPlFM8JjVgYNgr6lkJPIFe/F710P+5Z1JxZP4CfKUtSLSiXnIKQEdapc+LzB9Kyre8QDnxveNPxkxNY2agoVQmedzw/cneZJ3unCsmSYNElCmxP0/SkxJcyPtT0qMq7dLZbfqjgzPA8+AMjL20C59N4OEZhGLgQqiVMNzU2gVRAKngg2cZqZYSmiPdFhDw5hETLWK0WoH+EhHAhwmUr8Y8Cg6rShIpFQe+TozItBVv7lh8C+ukUF43ip4nGbAYjpuFGYCQ4KHd8IBl4yCyDUgVHI9K6ZdIgkFfU1npk2ghrPNfKRIO6EeeuA4jt6X93s78+C5VvXcqvd4Wr68MptbRQfoEFWQh87QJbpDD6iOqLVvXVg31q19a0c22O/jVNsymj00Y/bHD0cc0cg=</latexit><latexit sha1_base64="SpZIrweXSGQr0Jh2nRvHEaBZfBM=">AAACxXiclVHbSgMxEM2u9/VW9cEHX4JFqAhltwj6IoiCCr4oWBXaWrLZbBuavZDMSpel+E1+jh8jmLYptNYXB8Ic5syZmcz4qeAKXPfLshcWl5ZXVtec9Y3Nre3Szu6zSjJJWZ0mIpGvPlFM8JjVgYNgr6lkJPIFe/F710P+5Z1JxZP4CfKUtSLSiXnIKQEdapc+LzB9Kyre8QDnxveNPxkxNY2agoVQmedzw/cneZJ3unCsmSYNElCmxP0/SkxJcyPtT0qMq7dLZbfqjgzPA8+AMjL20C59N4OEZhGLgQqiVMNzU2gVRAKngg2cZqZYSmiPdFhDw5hETLWK0WoH+EhHAhwmUr8Y8Cg6rShIpFQe+TozItBVv7lh8C+ukUF43ip4nGbAYjpuFGYCQ4KHd8IBl4yCyDUgVHI9K6ZdIgkFfU1npk2ghrPNfKRIO6EeeuA4jt6X93s78+C5VvXcqvd4Wr68MptbRQfoEFWQh87QJbpDD6iOqLVvXVg31q19a0c22O/jVNsymj00Y/bHD0cc0cg=</latexit><latexit sha1_base64="SpZIrweXSGQr0Jh2nRvHEaBZfBM=">AAACxXiclVHbSgMxEM2u9/VW9cEHX4JFqAhltwj6IoiCCr4oWBXaWrLZbBuavZDMSpel+E1+jh8jmLYptNYXB8Ic5syZmcz4qeAKXPfLshcWl5ZXVtec9Y3Nre3Szu6zSjJJWZ0mIpGvPlFM8JjVgYNgr6lkJPIFe/F710P+5Z1JxZP4CfKUtSLSiXnIKQEdapc+LzB9Kyre8QDnxveNPxkxNY2agoVQmedzw/cneZJ3unCsmSYNElCmxP0/SkxJcyPtT0qMq7dLZbfqjgzPA8+AMjL20C59N4OEZhGLgQqiVMNzU2gVRAKngg2cZqZYSmiPdFhDw5hETLWK0WoH+EhHAhwmUr8Y8Cg6rShIpFQe+TozItBVv7lh8C+ukUF43ip4nGbAYjpuFGYCQ4KHd8IBl4yCyDUgVHI9K6ZdIgkFfU1npk2ghrPNfKRIO6EeeuA4jt6X93s78+C5VvXcqvd4Wr68MptbRQfoEFWQh87QJbpDD6iOqLVvXVg31q19a0c22O/jVNsymj00Y/bHD0cc0cg=</latexit><latexit sha1_base64="SpZIrweXSGQr0Jh2nRvHEaBZfBM=">AAACxXiclVHbSgMxEM2u9/VW9cEHX4JFqAhltwj6IoiCCr4oWBXaWrLZbBuavZDMSpel+E1+jh8jmLYptNYXB8Ic5syZmcz4qeAKXPfLshcWl5ZXVtec9Y3Nre3Szu6zSjJJWZ0mIpGvPlFM8JjVgYNgr6lkJPIFe/F710P+5Z1JxZP4CfKUtSLSiXnIKQEdapc+LzB9Kyre8QDnxveNPxkxNY2agoVQmedzw/cneZJ3unCsmSYNElCmxP0/SkxJcyPtT0qMq7dLZbfqjgzPA8+AMjL20C59N4OEZhGLgQqiVMNzU2gVRAKngg2cZqZYSmiPdFhDw5hETLWK0WoH+EhHAhwmUr8Y8Cg6rShIpFQe+TozItBVv7lh8C+ukUF43ip4nGbAYjpuFGYCQ4KHd8IBl4yCyDUgVHI9K6ZdIgkFfU1npk2ghrPNfKRIO6EeeuA4jt6X93s78+C5VvXcqvd4Wr68MptbRQfoEFWQh87QJbpDD6iOqLVvXVg31q19a0c22O/jVNsymj00Y/bHD0cc0cg=</latexit>


KX

k=1

c(k)w(k)



= c(1)y(1)x(1) + c(2)⇣y(1)x(1) + y(2)x(2)

⌘+ · · ·+ c(K)

⇣y(1)x(1) + y(2)x(2) + · · ·+ y(K)x(K)

⌘


= y(1)x(1)⇣c(1) + c(2) + · · ·+ c(K)

⌘+ y(2)x(2) +

⇣c(2) + c(3) + · · ·+ c(K)

⌘+ · · ·+ y(K)x(K) +

⇣c(K)

⌘

<latexit sha1_base64="6tlbfLpzvCm8HV9fETa/nLhrYuM=">AAAC1XichVFda9swFJXdj3VeP9LucS+iYSylEOx00L4UwvpSGIMOlqSQZEGW5URU/kC6LjHGb2Wv/U37Gf0xhcqOaJu20AtCR+ece3V15aeCK3DdO8teWV1b/7Dx0fm0ubW909jd66skk5T1aCISeekTxQSPWQ84CHaZSkYiX7CBf3VW6YNrJhVP4j+Qp2wckWnMQ04JaGrS+H+K879Fyzso8dzsI8FCaGFqjoc16tRoRIMElKF+Vl7JpzM40ExuTPMn81OdzmOdo/fqPEq5keZmXyr4LGfSaLpttw78GngGNJGJi0njfhQkNItYDFQQpYaem8K4IBI4Fax0RpliKaFXZMqGGsYkYmpc1KMu8VfNBDhMpF4x4Jp9nlGQSKk88rUzIjBTL7WKfEsbZhCejAsepxmwmC4uCjOBIcHVv+GAS0ZB5BoQKrnuFdMZkYSC/l1n6ZpAVb0tPaRIp6FuunQcR8/Lezmd16DfaXtu2/v9vdn9YSa3gb6gfdRCHjpGXXSOLlAPUeub9cvqWwN7YJf2jf1vYbUtk/MZLYV9+wC88dZ9</latexit><latexit sha1_base64="6tlbfLpzvCm8HV9fETa/nLhrYuM=">AAAC1XichVFda9swFJXdj3VeP9LucS+iYSylEOx00L4UwvpSGIMOlqSQZEGW5URU/kC6LjHGb2Wv/U37Gf0xhcqOaJu20AtCR+ece3V15aeCK3DdO8teWV1b/7Dx0fm0ubW909jd66skk5T1aCISeekTxQSPWQ84CHaZSkYiX7CBf3VW6YNrJhVP4j+Qp2wckWnMQ04JaGrS+H+K879Fyzso8dzsI8FCaGFqjoc16tRoRIMElKF+Vl7JpzM40ExuTPMn81OdzmOdo/fqPEq5keZmXyr4LGfSaLpttw78GngGNJGJi0njfhQkNItYDFQQpYaem8K4IBI4Fax0RpliKaFXZMqGGsYkYmpc1KMu8VfNBDhMpF4x4Jp9nlGQSKk88rUzIjBTL7WKfEsbZhCejAsepxmwmC4uCjOBIcHVv+GAS0ZB5BoQKrnuFdMZkYSC/l1n6ZpAVb0tPaRIp6FuunQcR8/Lezmd16DfaXtu2/v9vdn9YSa3gb6gfdRCHjpGXXSOLlAPUeub9cvqWwN7YJf2jf1vYbUtk/MZLYV9+wC88dZ9</latexit><latexit sha1_base64="6tlbfLpzvCm8HV9fETa/nLhrYuM=">AAAC1XichVFda9swFJXdj3VeP9LucS+iYSylEOx00L4UwvpSGIMOlqSQZEGW5URU/kC6LjHGb2Wv/U37Gf0xhcqOaJu20AtCR+ece3V15aeCK3DdO8teWV1b/7Dx0fm0ubW909jd66skk5T1aCISeekTxQSPWQ84CHaZSkYiX7CBf3VW6YNrJhVP4j+Qp2wckWnMQ04JaGrS+H+K879Fyzso8dzsI8FCaGFqjoc16tRoRIMElKF+Vl7JpzM40ExuTPMn81OdzmOdo/fqPEq5keZmXyr4LGfSaLpttw78GngGNJGJi0njfhQkNItYDFQQpYaem8K4IBI4Fax0RpliKaFXZMqGGsYkYmpc1KMu8VfNBDhMpF4x4Jp9nlGQSKk88rUzIjBTL7WKfEsbZhCejAsepxmwmC4uCjOBIcHVv+GAS0ZB5BoQKrnuFdMZkYSC/l1n6ZpAVb0tPaRIp6FuunQcR8/Lezmd16DfaXtu2/v9vdn9YSa3gb6gfdRCHjpGXXSOLlAPUeub9cvqWwN7YJf2jf1vYbUtk/MZLYV9+wC88dZ9</latexit><latexit sha1_base64="6tlbfLpzvCm8HV9fETa/nLhrYuM=">AAAC1XichVFda9swFJXdj3VeP9LucS+iYSylEOx00L4UwvpSGIMOlqSQZEGW5URU/kC6LjHGb2Wv/U37Gf0xhcqOaJu20AtCR+ece3V15aeCK3DdO8teWV1b/7Dx0fm0ubW909jd66skk5T1aCISeekTxQSPWQ84CHaZSkYiX7CBf3VW6YNrJhVP4j+Qp2wckWnMQ04JaGrS+H+K879Fyzso8dzsI8FCaGFqjoc16tRoRIMElKF+Vl7JpzM40ExuTPMn81OdzmOdo/fqPEq5keZmXyr4LGfSaLpttw78GngGNJGJi0njfhQkNItYDFQQpYaem8K4IBI4Fax0RpliKaFXZMqGGsYkYmpc1KMu8VfNBDhMpF4x4Jp9nlGQSKk88rUzIjBTL7WKfEsbZhCejAsepxmwmC4uCjOBIcHVv+GAS0ZB5BoQKrnuFdMZkYSC/l1n6ZpAVb0tPaRIp6FuunQcR8/Lezmd16DfaXtu2/v9vdn9YSa3gb6gfdRCHjpGXXSOLlAPUeub9cvqWwN7YJf2jf1vYbUtk/MZLYV9+wC88dZ9</latexit>

Rearranging Terms:

= y(1)x(1)⇣c(1) + c(2) + · · ·+ c(K)

⌘+ y(2)x(2) +

⇣c(2) + c(3) + · · ·+ c(K)

⌘+ · · ·+ y(K)x(K) +

⇣c(K)

⌘


= y(1)x(1)⇣c(1) + c(2) + · · ·+ c(K)

⌘+ y(2)x(2) +

⇣c(2) + c(3) + · · ·+ c(K)

⌘+ · · ·+ y(K)x(K) +

⇣c(K)

⌘



KX

k=1

c(k)w(k)



= c(1)y(1)x(1) + c(2)⇣y(1)x(1) + y(2)x(2)

⌘+ · · ·+ c(K)

⇣y(1)x(1) + y(2)x(2) + · · ·+ y(K)x(K)

⌘


= y(1)x(1)⇣c(1) + c(2) + · · ·+ c(K)

⌘+ y(2)x(2) +

⇣c(2) + c(3) + · · ·+ c(K)

⌘+ · · ·+ y(K)x(K) +

⇣c(K)

⌘


Rearranging Terms:

= y(1)x(1)⇣c(1) + c(2) + · · ·+ c(K)

⌘+ y(2)x(2) +

⇣c(2) + c(3) + · · ·+ c(K)

⌘+ · · ·+ y(K)x(K) +

⇣c(K)

⌘


= y(1)x(1)⇣c(1) + c(2) + · · ·+ c(K)

⌘+ y(2)x(2) +

⇣c(2) + c(3) + · · ·+ c(K)

⌘+ · · ·+ y(K)x(K) +

⇣c(K)

⌘


=KX

i=1

x(i)y(i)

0

@iX

j=1

cj

1

A

<latexit sha1_base64="/Owmeuq0nRf8GFxRCS2YuVuvqzo=">AAACUnicbVJNSwMxEE3rV12rVj16CRahXsquCHqpFL0IXirYD+i2JZtm27TZD5JZsSz7b/w1Xu3FvyIIZts9WOvAkMebN8zMI04ouALT/MzlNza3tncKu8Zecf/gsHR03FJBJClr0kAEsuMQxQT3WRM4CNYJJSOeI1jbmd6n9fYLk4oH/jPMQtbzyMjnLqcENDUo3dawrSJvEPOalfQf8Ws/rvCLBM+y1xbMhUqmmaQajml/gm3JR2O4GJTKZtVcBF4HVgbKKIvGoPRtDwMaecwHKohSXcsMoRcTCZwKlhh2pFhI6JSMWFdDn3hM9eLFnQk+18wQu4HU6QNesL87YuIpNfMcrfQIjNXfWkr+V+tG4N70Yu6HETCfLge5kcAQ4NQ0POSSURAzDQiVXO+K6ZhIQkFba6yMGap0t5VD4nDk6qUTwzC0X9Zfd9ZB67JqmVXr6apcv8ucK6BTdIYqyELXqI4eUAM1EUVv6B19oHlunvvK61+ylOZzWc8JWol88QdFMbKU</latexit><latexit sha1_base64="/Owmeuq0nRf8GFxRCS2YuVuvqzo=">AAACUnicbVJNSwMxEE3rV12rVj16CRahXsquCHqpFL0IXirYD+i2JZtm27TZD5JZsSz7b/w1Xu3FvyIIZts9WOvAkMebN8zMI04ouALT/MzlNza3tncKu8Zecf/gsHR03FJBJClr0kAEsuMQxQT3WRM4CNYJJSOeI1jbmd6n9fYLk4oH/jPMQtbzyMjnLqcENDUo3dawrSJvEPOalfQf8Ws/rvCLBM+y1xbMhUqmmaQajml/gm3JR2O4GJTKZtVcBF4HVgbKKIvGoPRtDwMaecwHKohSXcsMoRcTCZwKlhh2pFhI6JSMWFdDn3hM9eLFnQk+18wQu4HU6QNesL87YuIpNfMcrfQIjNXfWkr+V+tG4N70Yu6HETCfLge5kcAQ4NQ0POSSURAzDQiVXO+K6ZhIQkFba6yMGap0t5VD4nDk6qUTwzC0X9Zfd9ZB67JqmVXr6apcv8ucK6BTdIYqyELXqI4eUAM1EUVv6B19oHlunvvK61+ylOZzWc8JWol88QdFMbKU</latexit><latexit sha1_base64="/Owmeuq0nRf8GFxRCS2YuVuvqzo=">AAACUnicbVJNSwMxEE3rV12rVj16CRahXsquCHqpFL0IXirYD+i2JZtm27TZD5JZsSz7b/w1Xu3FvyIIZts9WOvAkMebN8zMI04ouALT/MzlNza3tncKu8Zecf/gsHR03FJBJClr0kAEsuMQxQT3WRM4CNYJJSOeI1jbmd6n9fYLk4oH/jPMQtbzyMjnLqcENDUo3dawrSJvEPOalfQf8Ws/rvCLBM+y1xbMhUqmmaQajml/gm3JR2O4GJTKZtVcBF4HVgbKKIvGoPRtDwMaecwHKohSXcsMoRcTCZwKlhh2pFhI6JSMWFdDn3hM9eLFnQk+18wQu4HU6QNesL87YuIpNfMcrfQIjNXfWkr+V+tG4N70Yu6HETCfLge5kcAQ4NQ0POSSURAzDQiVXO+K6ZhIQkFba6yMGap0t5VD4nDk6qUTwzC0X9Zfd9ZB67JqmVXr6apcv8ucK6BTdIYqyELXqI4eUAM1EUVv6B19oHlunvvK61+ylOZzWc8JWol88QdFMbKU</latexit><latexit sha1_base64="rIc4elZHT3oXILhfcJEssMOX7as=">AAACBHicbVDLSgMxFL1TX3WsWtdugkVwVWbc6FJw47KCfUAdSiaTaUMzyZDcEcrQH3Dbr3En/oUfI5hpu7CtBy4czkm45544l8JiEHx7tb39g8Oj+rF/0vBPz86bjZ7VhWG8y7TUZhBTy6VQvIsCJR/khtMslrwfTx8rv//GjRVaveAs51FGx0qkglF0UmfUbAXtYAmyS8I1acEao+bPa6JZkXGFTFJrh2GQY1RSg4JJPvdfC8tzyqZ0zIeOKppxG5XLmHNy7ZSEpNq4UUiW6t8fJc2snWWxe5lRnNhtrxL/84YFpvdRKVReIFdstSgtJEFNqptJIgxnKGeOUGaEy0rYhBrK0DXjb6xJbJVt45AyH6cu9Nz3fVdXuF3OLundtsOgHT4HUIdLuIIbCOEOHuAJOtAFBgm8w8JbeB/e56rWmrfu9wI24H39Asd+my0=</latexit><latexit sha1_base64="9WOekzeJAiV4uvfkd327aM45ILA=">AAACR3icbVBNS8NAEJ22fsav6tXLYhHqpSRe9FIRvAheKlgtNG3ZbDft6uaD3YlYQv6Nv8arvfhXBMFNjWBbB5Z5vPeWmXleLIVG2/4olSsrq2vrG5vW1vbO7l51f/teR4livM0iGamORzWXIuRtFCh5J1acBp7kD97TVa4/PHOlRRTe4STmvYCOQuELRtFQg+pFk7g6CQapaDpZ/4a89NO6OMnIpOiu5D7WC89j7hGE9R+Jq8RojCeDas1u2LMiy8ApQA2Kag2qX+4wYknAQ2SSat117Bh7KVUomOSZ5Saax5Q90RHvGhjSgOteOrszI8eGGRI/UuaFSGbs3x8pDbSeBJ5xBhTHelHLyf+0boL+eS8VYZwgD9nPID+RBCOSh0aGQnGGcmIAZUqYXQkbU0UZmmituTFDne82d0gaj3yzdGZZlsnLWUxnGdyfNhy74dzasAGHcAR1cOAMLuEaWtAGBq/wBu8wLU1Ln+XST7K/HQ5grsqVb6HisdY=</latexit><latexit sha1_base64="9WOekzeJAiV4uvfkd327aM45ILA=">AAACR3icbVBNS8NAEJ22fsav6tXLYhHqpSRe9FIRvAheKlgtNG3ZbDft6uaD3YlYQv6Nv8arvfhXBMFNjWBbB5Z5vPeWmXleLIVG2/4olSsrq2vrG5vW1vbO7l51f/teR4livM0iGamORzWXIuRtFCh5J1acBp7kD97TVa4/PHOlRRTe4STmvYCOQuELRtFQg+pFk7g6CQapaDpZ/4a89NO6OMnIpOiu5D7WC89j7hGE9R+Jq8RojCeDas1u2LMiy8ApQA2Kag2qX+4wYknAQ2SSat117Bh7KVUomOSZ5Saax5Q90RHvGhjSgOteOrszI8eGGRI/UuaFSGbs3x8pDbSeBJ5xBhTHelHLyf+0boL+eS8VYZwgD9nPID+RBCOSh0aGQnGGcmIAZUqYXQkbU0UZmmituTFDne82d0gaj3yzdGZZlsnLWUxnGdyfNhy74dzasAGHcAR1cOAMLuEaWtAGBq/wBu8wLU1Ln+XST7K/HQ5grsqVb6HisdY=</latexit><latexit sha1_base64="uCVC5ZpJIvNBW4JYMHOqpc2J1kM=">AAACUnicbVJNSwMxEE3rV12rVj16CRahvZRdL3qpFL0IXirYD+i2JZtm29TsB8msWJb9N/4ar/biXxEEs+0ebOvAkMebN8zMI04ouALT/Mrlt7Z3dvcK+8ZB8fDouHRy2lZBJClr0UAEsusQxQT3WQs4CNYNJSOeI1jHeblP651XJhUP/GeYhazvkbHPXU4JaGpYuq1jW0XeMOZ1Kxk84rdBXOHVBM+y1xbMhUqmmaYajulgim3JxxOoDktls2YuAm8CKwNllEVzWPqxRwGNPOYDFUSpnmWG0I+JBE4FSww7Uiwk9IWMWU9Dn3hM9ePFnQm+1MwIu4HU6QNesH87YuIpNfMcrfQITNR6LSX/q/UicG/6MffDCJhPl4PcSGAIcGoaHnHJKIiZBoRKrnfFdEIkoaCtNVbGjFS628ohcTh29dKJYRjaL2vdnU3QvqpZZs16MsuNu8y5AjpHF6iCLHSNGugBNVELUfSOPtAnmufmue+8/iVLaT6X9ZyhlcgXfwFD8bKQ</latexit><latexit sha1_base64="/Owmeuq0nRf8GFxRCS2YuVuvqzo=">AAACUnicbVJNSwMxEE3rV12rVj16CRahXsquCHqpFL0IXirYD+i2JZtm27TZD5JZsSz7b/w1Xu3FvyIIZts9WOvAkMebN8zMI04ouALT/MzlNza3tncKu8Zecf/gsHR03FJBJClr0kAEsuMQxQT3WRM4CNYJJSOeI1jbmd6n9fYLk4oH/jPMQtbzyMjnLqcENDUo3dawrSJvEPOalfQf8Ws/rvCLBM+y1xbMhUqmmaQajml/gm3JR2O4GJTKZtVcBF4HVgbKKIvGoPRtDwMaecwHKohSXcsMoRcTCZwKlhh2pFhI6JSMWFdDn3hM9eLFnQk+18wQu4HU6QNesL87YuIpNfMcrfQIjNXfWkr+V+tG4N70Yu6HETCfLge5kcAQ4NQ0POSSURAzDQiVXO+K6ZhIQkFba6yMGap0t5VD4nDk6qUTwzC0X9Zfd9ZB67JqmVXr6apcv8ucK6BTdIYqyELXqI4eUAM1EUVv6B19oHlunvvK61+ylOZzWc8JWol88QdFMbKU</latexit><latexit sha1_base64="/Owmeuq0nRf8GFxRCS2YuVuvqzo=">AAACUnicbVJNSwMxEE3rV12rVj16CRahXsquCHqpFL0IXirYD+i2JZtm27TZD5JZsSz7b/w1Xu3FvyIIZts9WOvAkMebN8zMI04ouALT/MzlNza3tncKu8Zecf/gsHR03FJBJClr0kAEsuMQxQT3WRM4CNYJJSOeI1jbmd6n9fYLk4oH/jPMQtbzyMjnLqcENDUo3dawrSJvEPOalfQf8Ws/rvCLBM+y1xbMhUqmmaQajml/gm3JR2O4GJTKZtVcBF4HVgbKKIvGoPRtDwMaecwHKohSXcsMoRcTCZwKlhh2pFhI6JSMWFdDn3hM9eLFnQk+18wQu4HU6QNesL87YuIpNfMcrfQIjNXfWkr+V+tG4N70Yu6HETCfLge5kcAQ4NQ0POSSURAzDQiVXO+K6ZhIQkFba6yMGap0t5VD4nDk6qUTwzC0X9Zfd9ZB67JqmVXr6apcv8ucK6BTdIYqyELXqI4eUAM1EUVv6B19oHlunvvK61+ylOZzWc8JWol88QdFMbKU</latexit><latexit sha1_base64="/Owmeuq0nRf8GFxRCS2YuVuvqzo=">AAACUnicbVJNSwMxEE3rV12rVj16CRahXsquCHqpFL0IXirYD+i2JZtm27TZD5JZsSz7b/w1Xu3FvyIIZts9WOvAkMebN8zMI04ouALT/MzlNza3tncKu8Zecf/gsHR03FJBJClr0kAEsuMQxQT3WRM4CNYJJSOeI1jbmd6n9fYLk4oH/jPMQtbzyMjnLqcENDUo3dawrSJvEPOalfQf8Ws/rvCLBM+y1xbMhUqmmaQajml/gm3JR2O4GJTKZtVcBF4HVgbKKIvGoPRtDwMaecwHKohSXcsMoRcTCZwKlhh2pFhI6JSMWFdDn3hM9eLFnQk+18wQu4HU6QNesL87YuIpNfMcrfQIjNXfWkr+V+tG4N70Yu6HETCfLge5kcAQ4NQ0POSSURAzDQiVXO+K6ZhIQkFba6yMGap0t5VD4nDk6qUTwzC0X9Zfd9ZB67JqmVXr6apcv8ucK6BTdIYqyELXqI4eUAM1EUVv6B19oHlunvvK61+ylOZzWc8JWol88QdFMbKU</latexit><latexit sha1_base64="/Owmeuq0nRf8GFxRCS2YuVuvqzo=">AAACUnicbVJNSwMxEE3rV12rVj16CRahXsquCHqpFL0IXirYD+i2JZtm27TZD5JZsSz7b/w1Xu3FvyIIZts9WOvAkMebN8zMI04ouALT/MzlNza3tncKu8Zecf/gsHR03FJBJClr0kAEsuMQxQT3WRM4CNYJJSOeI1jbmd6n9fYLk4oH/jPMQtbzyMjnLqcENDUo3dawrSJvEPOalfQf8Ws/rvCLBM+y1xbMhUqmmaQajml/gm3JR2O4GJTKZtVcBF4HVgbKKIvGoPRtDwMaecwHKohSXcsMoRcTCZwKlhh2pFhI6JSMWFdDn3hM9eLFnQk+18wQu4HU6QNesL87YuIpNfMcrfQIjNXfWkr+V+tG4N70Yu6HETCfLge5kcAQ4NQ0POSSURAzDQiVXO+K6ZhIQkFba6yMGap0t5VD4nDk6qUTwzC0X9Zfd9ZB67JqmVXr6apcv8ucK6BTdIYqyELXqI4eUAM1EUVv6B19oHlunvvK61+ylOZzWc8JWol88QdFMbKU</latexit><latexit sha1_base64="/Owmeuq0nRf8GFxRCS2YuVuvqzo=">AAACUnicbVJNSwMxEE3rV12rVj16CRahXsquCHqpFL0IXirYD+i2JZtm27TZD5JZsSz7b/w1Xu3FvyIIZts9WOvAkMebN8zMI04ouALT/MzlNza3tncKu8Zecf/gsHR03FJBJClr0kAEsuMQxQT3WRM4CNYJJSOeI1jbmd6n9fYLk4oH/jPMQtbzyMjnLqcENDUo3dawrSJvEPOalfQf8Ws/rvCLBM+y1xbMhUqmmaQajml/gm3JR2O4GJTKZtVcBF4HVgbKKIvGoPRtDwMaecwHKohSXcsMoRcTCZwKlhh2pFhI6JSMWFdDn3hM9eLFnQk+18wQu4HU6QNesL87YuIpNfMcrfQIjNXfWkr+V+tG4N70Yu6HETCfLge5kcAQ4NQ0POSSURAzDQiVXO+K6ZhIQkFba6yMGap0t5VD4nDk6qUTwzC0X9Zfd9ZB67JqmVXr6apcv8ucK6BTdIYqyELXqI4eUAM1EUVv6B19oHlunvvK61+ylOZzWc8JWol88QdFMbKU</latexit><latexit sha1_base64="/Owmeuq0nRf8GFxRCS2YuVuvqzo=">AAACUnicbVJNSwMxEE3rV12rVj16CRahXsquCHqpFL0IXirYD+i2JZtm27TZD5JZsSz7b/w1Xu3FvyIIZts9WOvAkMebN8zMI04ouALT/MzlNza3tncKu8Zecf/gsHR03FJBJClr0kAEsuMQxQT3WRM4CNYJJSOeI1jbmd6n9fYLk4oH/jPMQtbzyMjnLqcENDUo3dawrSJvEPOalfQf8Ws/rvCLBM+y1xbMhUqmmaQajml/gm3JR2O4GJTKZtVcBF4HVgbKKIvGoPRtDwMaecwHKohSXcsMoRcTCZwKlhh2pFhI6JSMWFdDn3hM9eLFnQk+18wQu4HU6QNesL87YuIpNfMcrfQIjNXfWkr+V+tG4N70Yu6HETCfLge5kcAQ4NQ0POSSURAzDQiVXO+K6ZhIQkFba6yMGap0t5VD4nDk6qUTwzC0X9Zfd9ZB67JqmVXr6apcv8ucK6BTdIYqyELXqI4eUAM1EUVv6B19oHlunvvK61+ylOZzWc8JWol88QdFMbKU</latexit>

i<latexit sha1_base64="MWkOpNlr4SvTH4EYxiU0eV9qT+0=">AAACE3icbVBNS8NAEJ3Urxq/qh69BIvgqSQi6LHoxWML9gPaUDbbSbt0swm7G6GE/gKv9td4E6/+AH+M4KbNwbYODDzem2HevCDhTGnX/bZKW9s7u3vlffvg8Oj4pHJ61lZxKim2aMxj2Q2IQs4EtjTTHLuJRBIFHDvB5DHXOy8oFYvFs54m6EdkJFjIKNGGarJBperW3EU5m8ArQBWKagwqP/1hTNMIhaacKNXz3ET7GZGaUY4zu58qTAidkBH2DBQkQuVnC6Mz58owQyeMpWmhnQX7dyMjkVLTKDCTEdFjta7l5H9aL9XhvZ8xkaQaBV0eClPu6NjJv3aGTCLVfGoAoZIZrw4dE0moNtnYK2eGKve28kiWjEJjembbtsnLW09nE7Rvap5b85q31fpDkVwZLuASrsGDO6jDEzSgBRQQXuEN5tbcerc+rM/laMkqds5hpayvX1oEng0=</latexit><latexit sha1_base64="MWkOpNlr4SvTH4EYxiU0eV9qT+0=">AAACE3icbVBNS8NAEJ3Urxq/qh69BIvgqSQi6LHoxWML9gPaUDbbSbt0swm7G6GE/gKv9td4E6/+AH+M4KbNwbYODDzem2HevCDhTGnX/bZKW9s7u3vlffvg8Oj4pHJ61lZxKim2aMxj2Q2IQs4EtjTTHLuJRBIFHDvB5DHXOy8oFYvFs54m6EdkJFjIKNGGarJBperW3EU5m8ArQBWKagwqP/1hTNMIhaacKNXz3ET7GZGaUY4zu58qTAidkBH2DBQkQuVnC6Mz58owQyeMpWmhnQX7dyMjkVLTKDCTEdFjta7l5H9aL9XhvZ8xkaQaBV0eClPu6NjJv3aGTCLVfGoAoZIZrw4dE0moNtnYK2eGKve28kiWjEJjembbtsnLW09nE7Rvap5b85q31fpDkVwZLuASrsGDO6jDEzSgBRQQXuEN5tbcerc+rM/laMkqds5hpayvX1oEng0=</latexit><latexit sha1_base64="MWkOpNlr4SvTH4EYxiU0eV9qT+0=">AAACE3icbVBNS8NAEJ3Urxq/qh69BIvgqSQi6LHoxWML9gPaUDbbSbt0swm7G6GE/gKv9td4E6/+AH+M4KbNwbYODDzem2HevCDhTGnX/bZKW9s7u3vlffvg8Oj4pHJ61lZxKim2aMxj2Q2IQs4EtjTTHLuJRBIFHDvB5DHXOy8oFYvFs54m6EdkJFjIKNGGarJBperW3EU5m8ArQBWKagwqP/1hTNMIhaacKNXz3ET7GZGaUY4zu58qTAidkBH2DBQkQuVnC6Mz58owQyeMpWmhnQX7dyMjkVLTKDCTEdFjta7l5H9aL9XhvZ8xkaQaBV0eClPu6NjJv3aGTCLVfGoAoZIZrw4dE0moNtnYK2eGKve28kiWjEJjembbtsnLW09nE7Rvap5b85q31fpDkVwZLuASrsGDO6jDEzSgBRQQXuEN5tbcerc+rM/laMkqds5hpayvX1oEng0=</latexit><latexit sha1_base64="MWkOpNlr4SvTH4EYxiU0eV9qT+0=">AAACE3icbVBNS8NAEJ3Urxq/qh69BIvgqSQi6LHoxWML9gPaUDbbSbt0swm7G6GE/gKv9td4E6/+AH+M4KbNwbYODDzem2HevCDhTGnX/bZKW9s7u3vlffvg8Oj4pHJ61lZxKim2aMxj2Q2IQs4EtjTTHLuJRBIFHDvB5DHXOy8oFYvFs54m6EdkJFjIKNGGarJBperW3EU5m8ArQBWKagwqP/1hTNMIhaacKNXz3ET7GZGaUY4zu58qTAidkBH2DBQkQuVnC6Mz58owQyeMpWmhnQX7dyMjkVLTKDCTEdFjta7l5H9aL9XhvZ8xkaQaBV0eClPu6NjJv3aGTCLVfGoAoZIZrw4dE0moNtnYK2eGKve28kiWjEJjembbtsnLW09nE7Rvap5b85q31fpDkVwZLuASrsGDO6jDEzSgBRQQXuEN5tbcerc+rM/laMkqds5hpayvX1oEng0=</latexit>

K<latexit sha1_base64="dyrEuVOlqVeFDEFNV6R/fc5dSFg=">AAACE3icbVDLSgNBEOyNr7i+oh69DAbBU9gVQY9BL4KXBMwDkiXMTnqTIbMPZmaFsOQLvJqv8SZe/QA/RnA22YOJNjQUVd10dfmJ4Eo7zpdV2tjc2t4p79p7+weHR5Xjk7aKU8mwxWIRy65PFQoeYUtzLbCbSKShL7DjT+5zvfOMUvE4etLTBL2QjiIecEa1oZqPg0rVqTmLIn+BW4AqFNUYVL77w5ilIUaaCapUz3US7WVUas4Ezux+qjChbEJH2DMwoiEqL1sYnZELwwxJEEvTkSYL9vdGRkOlpqFvJkOqx2pdy8n/tF6qg1sv41GSaozY8lCQCqJjkn9Nhlwi02JqAGWSG6+EjamkTJts7JUzQ5V7W3kkS0aBMT2zbdvk5a6n8xe0r2quU3Ob19X6XZFcGc7gHC7BhRuowwM0oAUMEF7gFebW3Hqz3q2P5WjJKnZOYaWszx8ngp3v</latexit><latexit sha1_base64="dyrEuVOlqVeFDEFNV6R/fc5dSFg=">AAACE3icbVDLSgNBEOyNr7i+oh69DAbBU9gVQY9BL4KXBMwDkiXMTnqTIbMPZmaFsOQLvJqv8SZe/QA/RnA22YOJNjQUVd10dfmJ4Eo7zpdV2tjc2t4p79p7+weHR5Xjk7aKU8mwxWIRy65PFQoeYUtzLbCbSKShL7DjT+5zvfOMUvE4etLTBL2QjiIecEa1oZqPg0rVqTmLIn+BW4AqFNUYVL77w5ilIUaaCapUz3US7WVUas4Ezux+qjChbEJH2DMwoiEqL1sYnZELwwxJEEvTkSYL9vdGRkOlpqFvJkOqx2pdy8n/tF6qg1sv41GSaozY8lCQCqJjkn9Nhlwi02JqAGWSG6+EjamkTJts7JUzQ5V7W3kkS0aBMT2zbdvk5a6n8xe0r2quU3Ob19X6XZFcGc7gHC7BhRuowwM0oAUMEF7gFebW3Hqz3q2P5WjJKnZOYaWszx8ngp3v</latexit><latexit sha1_base64="dyrEuVOlqVeFDEFNV6R/fc5dSFg=">AAACE3icbVDLSgNBEOyNr7i+oh69DAbBU9gVQY9BL4KXBMwDkiXMTnqTIbMPZmaFsOQLvJqv8SZe/QA/RnA22YOJNjQUVd10dfmJ4Eo7zpdV2tjc2t4p79p7+weHR5Xjk7aKU8mwxWIRy65PFQoeYUtzLbCbSKShL7DjT+5zvfOMUvE4etLTBL2QjiIecEa1oZqPg0rVqTmLIn+BW4AqFNUYVL77w5ilIUaaCapUz3US7WVUas4Ezux+qjChbEJH2DMwoiEqL1sYnZELwwxJEEvTkSYL9vdGRkOlpqFvJkOqx2pdy8n/tF6qg1sv41GSaozY8lCQCqJjkn9Nhlwi02JqAGWSG6+EjamkTJts7JUzQ5V7W3kkS0aBMT2zbdvk5a6n8xe0r2quU3Ob19X6XZFcGc7gHC7BhRuowwM0oAUMEF7gFebW3Hqz3q2P5WjJKnZOYaWszx8ngp3v</latexit><latexit sha1_base64="dyrEuVOlqVeFDEFNV6R/fc5dSFg=">AAACE3icbVDLSgNBEOyNr7i+oh69DAbBU9gVQY9BL4KXBMwDkiXMTnqTIbMPZmaFsOQLvJqv8SZe/QA/RnA22YOJNjQUVd10dfmJ4Eo7zpdV2tjc2t4p79p7+weHR5Xjk7aKU8mwxWIRy65PFQoeYUtzLbCbSKShL7DjT+5zvfOMUvE4etLTBL2QjiIecEa1oZqPg0rVqTmLIn+BW4AqFNUYVL77w5ilIUaaCapUz3US7WVUas4Ezux+qjChbEJH2DMwoiEqL1sYnZELwwxJEEvTkSYL9vdGRkOlpqFvJkOqx2pdy8n/tF6qg1sv41GSaozY8lCQCqJjkn9Nhlwi02JqAGWSG6+EjamkTJts7JUzQ5V7W3kkS0aBMT2zbdvk5a6n8xe0r2quU3Ob19X6XZFcGc7gHC7BhRuowwM0oAUMEF7gFebW3Hqz3q2P5WjJKnZOYaWszx8ngp3v</latexit>

Efficient Parameter AveragingPK

k=1 c(k)w(k)

PKk=1 c


Average Parameters:

=KX

i=1

x(i)y(i)

0

@iX

j=1

cj

1

A



K<latexit sha1_base64="dyrEuVOlqVeFDEFNV6R/fc5dSFg=">AAACE3icbVDLSgNBEOyNr7i+oh69DAbBU9gVQY9BL4KXBMwDkiXMTnqTIbMPZmaFsOQLvJqv8SZe/QA/RnA22YOJNjQUVd10dfmJ4Eo7zpdV2tjc2t4p79p7+weHR5Xjk7aKU8mwxWIRy65PFQoeYUtzLbCbSKShL7DjT+5zvfOMUvE4etLTBL2QjiIecEa1oZqPg0rVqTmLIn+BW4AqFNUYVL77w5ilIUaaCapUz3US7WVUas4Ezux+qjChbEJH2DMwoiEqL1sYnZELwwxJEEvTkSYL9vdGRkOlpqFvJkOqx2pdy8n/tF6qg1sv41GSaozY8lCQCqJjkn9Nhlwi02JqAGWSG6+EjamkTJts7JUzQ5V7W3kkS0aBMT2zbdvk5a6n8xe0r2quU3Ob19X6XZFcGc7gHC7BhRuowwM0oAUMEF7gFebW3Hqz3q2P5WjJKnZOYaWszx8ngp3v</latexit><latexit sha1_base64="dyrEuVOlqVeFDEFNV6R/fc5dSFg=">AAACE3icbVDLSgNBEOyNr7i+oh69DAbBU9gVQY9BL4KXBMwDkiXMTnqTIbMPZmaFsOQLvJqv8SZe/QA/RnA22YOJNjQUVd10dfmJ4Eo7zpdV2tjc2t4p79p7+weHR5Xjk7aKU8mwxWIRy65PFQoeYUtzLbCbSKShL7DjT+5zvfOMUvE4etLTBL2QjiIecEa1oZqPg0rVqTmLIn+BW4AqFNUYVL77w5ilIUaaCapUz3US7WVUas4Ezux+qjChbEJH2DMwoiEqL1sYnZELwwxJEEvTkSYL9vdGRkOlpqFvJkOqx2pdy8n/tF6qg1sv41GSaozY8lCQCqJjkn9Nhlwi02JqAGWSG6+EjamkTJts7JUzQ5V7W3kkS0aBMT2zbdvk5a6n8xe0r2quU3Ob19X6XZFcGc7gHC7BhRuowwM0oAUMEF7gFebW3Hqz3q2P5WjJKnZOYaWszx8ngp3v</latexit><latexit sha1_base64="dyrEuVOlqVeFDEFNV6R/fc5dSFg=">AAACE3icbVDLSgNBEOyNr7i+oh69DAbBU9gVQY9BL4KXBMwDkiXMTnqTIbMPZmaFsOQLvJqv8SZe/QA/RnA22YOJNjQUVd10dfmJ4Eo7zpdV2tjc2t4p79p7+weHR5Xjk7aKU8mwxWIRy65PFQoeYUtzLbCbSKShL7DjT+5zvfOMUvE4etLTBL2QjiIecEa1oZqPg0rVqTmLIn+BW4AqFNUYVL77w5ilIUaaCapUz3US7WVUas4Ezux+qjChbEJH2DMwoiEqL1sYnZELwwxJEEvTkSYL9vdGRkOlpqFvJkOqx2pdy8n/tF6qg1sv41GSaozY8lCQCqJjkn9Nhlwi02JqAGWSG6+EjamkTJts7JUzQ5V7W3kkS0aBMT2zbdvk5a6n8xe0r2quU3Ob19X6XZFcGc7gHC7BhRuowwM0oAUMEF7gFebW3Hqz3q2P5WjJKnZOYaWszx8ngp3v</latexit><latexit sha1_base64="dyrEuVOlqVeFDEFNV6R/fc5dSFg=">AAACE3icbVDLSgNBEOyNr7i+oh69DAbBU9gVQY9BL4KXBMwDkiXMTnqTIbMPZmaFsOQLvJqv8SZe/QA/RnA22YOJNjQUVd10dfmJ4Eo7zpdV2tjc2t4p79p7+weHR5Xjk7aKU8mwxWIRy65PFQoeYUtzLbCbSKShL7DjT+5zvfOMUvE4etLTBL2QjiIecEa1oZqPg0rVqTmLIn+BW4AqFNUYVL77w5ilIUaaCapUz3US7WVUas4Ezux+qjChbEJH2DMwoiEqL1sYnZELwwxJEEvTkSYL9vdGRkOlpqFvJkOqx2pdy8n/tF6qg1sv41GSaozY8lCQCqJjkn9Nhlwi02JqAGWSG6+EjamkTJts7JUzQ5V7W3kkS0aBMT2zbdvk5a6n8xe0r2quU3Ob19X6XZFcGc7gHC7BhRuowwM0oAUMEF7gFebW3Hqz3q2P5WjJKnZOYaWszx8ngp3v</latexit>Numerator


k=1 c(k)w(k)

PKk=1 c


Average Parameters:

=KX

i=1

x(i)y(i)

0

@iX

j=1

cj

1

A




Add and subtract the same term to the numerator:

=

PKi=1 x

(i)y(i)⇣Pi

j=1 cj⌘+

PKi=1 x

(i)y(i)⇣PK

j=i cj⌘�

PKi=1 x

(i)y(i)⇣PK

j=i cj⌘

PKk=1 ck

<latexit sha1_base64="+6ZtQTwIuiZHtYioPrFWengbG4E=">AAAC6XicpVFdS+NAFJ3EVWt2das+7suwRaiIJZEFBSmI+7KwLy5sq9C0YTKdtNNMPpi5EUvIj/BN9tXftA/+GMFJm4d+7MvihWEO957DnDnXTwVXYNsvhrnxYXNru7Zjffy0u/e5vn/QVUkmKevQRCTyzieKCR6zDnAQ7C6VjES+YLd++L2c394zqXgS/4ZpyvoRGcU84JSAbnn1v23sBpLQ3FVZ5OW87RSDn/hhkDf5cYGn1e0KFkATzzmTksMxHUywK/loDMf4BP+PnJecRfnpu+RF5T2cq6kXFl69YbfsWeF14FSggaq68eqv7jChWcRioIIo1XPsFPo5kcCpYIXlZoqlhIZkxHoaxiRiqp/P4i/wke4McZBIfWLAs+6iIieRUtPI18yIwFitzsrmv2a9DIKLfs7jNAMW0/lDQSYwJLjcJR5yySiIqQaESq69YjomepugN24tPTNUpbelj+TpKNCmC8uydF7OajrroHvWcuyW8+tb4+q6Sq6GvqCvqIkcdI6u0A90gzqIGpcGMSZGaIbmo/lk/plTTaPSHKKlMp/fAKbx50U=</latexit><latexit sha1_base64="+6ZtQTwIuiZHtYioPrFWengbG4E=">AAAC6XicpVFdS+NAFJ3EVWt2das+7suwRaiIJZEFBSmI+7KwLy5sq9C0YTKdtNNMPpi5EUvIj/BN9tXftA/+GMFJm4d+7MvihWEO957DnDnXTwVXYNsvhrnxYXNru7Zjffy0u/e5vn/QVUkmKevQRCTyzieKCR6zDnAQ7C6VjES+YLd++L2c394zqXgS/4ZpyvoRGcU84JSAbnn1v23sBpLQ3FVZ5OW87RSDn/hhkDf5cYGn1e0KFkATzzmTksMxHUywK/loDMf4BP+PnJecRfnpu+RF5T2cq6kXFl69YbfsWeF14FSggaq68eqv7jChWcRioIIo1XPsFPo5kcCpYIXlZoqlhIZkxHoaxiRiqp/P4i/wke4McZBIfWLAs+6iIieRUtPI18yIwFitzsrmv2a9DIKLfs7jNAMW0/lDQSYwJLjcJR5yySiIqQaESq69YjomepugN24tPTNUpbelj+TpKNCmC8uydF7OajrroHvWcuyW8+tb4+q6Sq6GvqCvqIkcdI6u0A90gzqIGpcGMSZGaIbmo/lk/plTTaPSHKKlMp/fAKbx50U=</latexit><latexit sha1_base64="+6ZtQTwIuiZHtYioPrFWengbG4E=">AAAC6XicpVFdS+NAFJ3EVWt2das+7suwRaiIJZEFBSmI+7KwLy5sq9C0YTKdtNNMPpi5EUvIj/BN9tXftA/+GMFJm4d+7MvihWEO957DnDnXTwVXYNsvhrnxYXNru7Zjffy0u/e5vn/QVUkmKevQRCTyzieKCR6zDnAQ7C6VjES+YLd++L2c394zqXgS/4ZpyvoRGcU84JSAbnn1v23sBpLQ3FVZ5OW87RSDn/hhkDf5cYGn1e0KFkATzzmTksMxHUywK/loDMf4BP+PnJecRfnpu+RF5T2cq6kXFl69YbfsWeF14FSggaq68eqv7jChWcRioIIo1XPsFPo5kcCpYIXlZoqlhIZkxHoaxiRiqp/P4i/wke4McZBIfWLAs+6iIieRUtPI18yIwFitzsrmv2a9DIKLfs7jNAMW0/lDQSYwJLjcJR5yySiIqQaESq69YjomepugN24tPTNUpbelj+TpKNCmC8uydF7OajrroHvWcuyW8+tb4+q6Sq6GvqCvqIkcdI6u0A90gzqIGpcGMSZGaIbmo/lk/plTTaPSHKKlMp/fAKbx50U=</latexit><latexit sha1_base64="+6ZtQTwIuiZHtYioPrFWengbG4E=">AAAC6XicpVFdS+NAFJ3EVWt2das+7suwRaiIJZEFBSmI+7KwLy5sq9C0YTKdtNNMPpi5EUvIj/BN9tXftA/+GMFJm4d+7MvihWEO957DnDnXTwVXYNsvhrnxYXNru7Zjffy0u/e5vn/QVUkmKevQRCTyzieKCR6zDnAQ7C6VjES+YLd++L2c394zqXgS/4ZpyvoRGcU84JSAbnn1v23sBpLQ3FVZ5OW87RSDn/hhkDf5cYGn1e0KFkATzzmTksMxHUywK/loDMf4BP+PnJecRfnpu+RF5T2cq6kXFl69YbfsWeF14FSggaq68eqv7jChWcRioIIo1XPsFPo5kcCpYIXlZoqlhIZkxHoaxiRiqp/P4i/wke4McZBIfWLAs+6iIieRUtPI18yIwFitzsrmv2a9DIKLfs7jNAMW0/lDQSYwJLjcJR5yySiIqQaESq69YjomepugN24tPTNUpbelj+TpKNCmC8uydF7OajrroHvWcuyW8+tb4+q6Sq6GvqCvqIkcdI6u0A90gzqIGpcGMSZGaIbmo/lk/plTTaPSHKKlMp/fAKbx50U=</latexit>


k=1 c(k)w(k)

PKk=1 c


Average Parameters:

=KX

i=1

x(i)y(i)

0

@iX

j=1

cj

1

A





=

PKi=1 x

(i)y(i)⇣Pi

j=1 cj⌘+

PKi=1 x

(i)y(i)⇣PK

j=i cj⌘�

PKi=1 x

(i)y(i)⇣PK

j=i cj⌘

PKk=1 ck


=

PKi=1 x

(i)y(i)⇣PK

j=1 cj⌘�PK

i=1 x(i)y(i)

⇣PKj=i c

j⌘

PKk=1 ck

<latexit sha1_base64="npxWby413h2zI5jW7I30sLyaE2M=">AAACqniclVHLSsNAFJ3Ed3xVXboZWoSKWhIRdCOIbgQ3VawPmjZMJpN27OTBzI1YQn7Df/NfFJy2WdjqxgvDHO49Z+6dc/1UcAW2/WGYc/MLi0vLK9bq2vrGZmVr+0ElmaSsRRORyCefKCZ4zFrAQbCnVDIS+YI9+oOrUf3xlUnFk/gehinrRKQX85BTAjrlVd7PsRtKQnNXZZGX83On6N7gt25e5/sFHpa3K1gIdTzhvEw4tPuCXcl7fdjHR/g/cj4jL8rmg/Jhb1B4lZrdsMeBfwOnBDVURtOrfLlBQrOIxUAFUart2Cl0ciKBU8EKy80USwkdkB5raxiTiKlOPvavwHs6E+AwkfrEgMfZn4qcREoNI18zIwJ9NVsbJf+qtTMIzzo5j9MMWEwnjcJMYEjwaBk44JJREEMNCJVcz4ppn+h1gF6ZNdUmUKPZpj6Sp71QD11YlqX9cmbd+Q0ejhuO3XBuT2oXl6Vzy2gXVVEdOegUXaBr1EQtRNGnUTUOjEPz0Lwzn832hGoapWYHTYUZfAOViNAu</latexit><latexit sha1_base64="npxWby413h2zI5jW7I30sLyaE2M=">AAACqniclVHLSsNAFJ3Ed3xVXboZWoSKWhIRdCOIbgQ3VawPmjZMJpN27OTBzI1YQn7Df/NfFJy2WdjqxgvDHO49Z+6dc/1UcAW2/WGYc/MLi0vLK9bq2vrGZmVr+0ElmaSsRRORyCefKCZ4zFrAQbCnVDIS+YI9+oOrUf3xlUnFk/gehinrRKQX85BTAjrlVd7PsRtKQnNXZZGX83On6N7gt25e5/sFHpa3K1gIdTzhvEw4tPuCXcl7fdjHR/g/cj4jL8rmg/Jhb1B4lZrdsMeBfwOnBDVURtOrfLlBQrOIxUAFUart2Cl0ciKBU8EKy80USwkdkB5raxiTiKlOPvavwHs6E+AwkfrEgMfZn4qcREoNI18zIwJ9NVsbJf+qtTMIzzo5j9MMWEwnjcJMYEjwaBk44JJREEMNCJVcz4ppn+h1gF6ZNdUmUKPZpj6Sp71QD11YlqX9cmbd+Q0ejhuO3XBuT2oXl6Vzy2gXVVEdOegUXaBr1EQtRNGnUTUOjEPz0Lwzn832hGoapWYHTYUZfAOViNAu</latexit><latexit sha1_base64="npxWby413h2zI5jW7I30sLyaE2M=">AAACqniclVHLSsNAFJ3Ed3xVXboZWoSKWhIRdCOIbgQ3VawPmjZMJpN27OTBzI1YQn7Df/NfFJy2WdjqxgvDHO49Z+6dc/1UcAW2/WGYc/MLi0vLK9bq2vrGZmVr+0ElmaSsRRORyCefKCZ4zFrAQbCnVDIS+YI9+oOrUf3xlUnFk/gehinrRKQX85BTAjrlVd7PsRtKQnNXZZGX83On6N7gt25e5/sFHpa3K1gIdTzhvEw4tPuCXcl7fdjHR/g/cj4jL8rmg/Jhb1B4lZrdsMeBfwOnBDVURtOrfLlBQrOIxUAFUart2Cl0ciKBU8EKy80USwkdkB5raxiTiKlOPvavwHs6E+AwkfrEgMfZn4qcREoNI18zIwJ9NVsbJf+qtTMIzzo5j9MMWEwnjcJMYEjwaBk44JJREEMNCJVcz4ppn+h1gF6ZNdUmUKPZpj6Sp71QD11YlqX9cmbd+Q0ejhuO3XBuT2oXl6Vzy2gXVVEdOegUXaBr1EQtRNGnUTUOjEPz0Lwzn832hGoapWYHTYUZfAOViNAu</latexit><latexit sha1_base64="rIc4elZHT3oXILhfcJEssMOX7as=">AAACBHicbVDLSgMxFL1TX3WsWtdugkVwVWbc6FJw47KCfUAdSiaTaUMzyZDcEcrQH3Dbr3En/oUfI5hpu7CtBy4czkm45544l8JiEHx7tb39g8Oj+rF/0vBPz86bjZ7VhWG8y7TUZhBTy6VQvIsCJR/khtMslrwfTx8rv//GjRVaveAs51FGx0qkglF0UmfUbAXtYAmyS8I1acEao+bPa6JZkXGFTFJrh2GQY1RSg4JJPvdfC8tzyqZ0zIeOKppxG5XLmHNy7ZSEpNq4UUiW6t8fJc2snWWxe5lRnNhtrxL/84YFpvdRKVReIFdstSgtJEFNqptJIgxnKGeOUGaEy0rYhBrK0DXjb6xJbJVt45AyH6cu9Nz3fVdXuF3OLundtsOgHT4HUIdLuIIbCOEOHuAJOtAFBgm8w8JbeB/e56rWmrfu9wI24H39Asd+my0=</latexit><latexit sha1_base64="GfoHkVFGkm69VWJ98f1mxc5y6iA=">AAACn3iclVFNSwMxEM2u32vV6tVLsAgtatn1opeC4EXwomJtpdsu2TTbps1+kMyKZdm/4X/zvyiYtnuwrRcHQh5v5mUmb/xEcAW2/WmYa+sbm1vbO9ZuaW//oHxYelFxKilr0ljEsu0TxQSPWBM4CNZOJCOhL1jLH99O8603JhWPo2eYJKwbkkHEA04JaMorfzSwG0hCM1eloZfxhpP37vF7L6vyWo4nxe0KFkAVz2tG8xraG2FX8sEQavgC/0fOl+R50XxcPOyNc69csev2LPAqcApQQUU8eOVvtx/TNGQRUEGU6jh2At2MSOBUsNxyU8USQsdkwDoaRiRkqpvN/MvxqWb6OIilPhHgGftbkZFQqUno68qQwFAt56bkX7lOCsF1N+NRkgKL6LxRkAoMMZ4uA/e5ZBTERANCJdezYjokeh2gV2YttOmr6WwLH8mSQaCHzi3L0n45y+6sgpfLumPXnUcbbaNjdIKqyEFX6AbdoQfURBR9GSfGmXFunptP5uvcWdMoLD5CC2F2fgCPic8Y</latexit><latexit sha1_base64="GfoHkVFGkm69VWJ98f1mxc5y6iA=">AAACn3iclVFNSwMxEM2u32vV6tVLsAgtatn1opeC4EXwomJtpdsu2TTbps1+kMyKZdm/4X/zvyiYtnuwrRcHQh5v5mUmb/xEcAW2/WmYa+sbm1vbO9ZuaW//oHxYelFxKilr0ljEsu0TxQSPWBM4CNZOJCOhL1jLH99O8603JhWPo2eYJKwbkkHEA04JaMorfzSwG0hCM1eloZfxhpP37vF7L6vyWo4nxe0KFkAVz2tG8xraG2FX8sEQavgC/0fOl+R50XxcPOyNc69csev2LPAqcApQQUU8eOVvtx/TNGQRUEGU6jh2At2MSOBUsNxyU8USQsdkwDoaRiRkqpvN/MvxqWb6OIilPhHgGftbkZFQqUno68qQwFAt56bkX7lOCsF1N+NRkgKL6LxRkAoMMZ4uA/e5ZBTERANCJdezYjokeh2gV2YttOmr6WwLH8mSQaCHzi3L0n45y+6sgpfLumPXnUcbbaNjdIKqyEFX6AbdoQfURBR9GSfGmXFunptP5uvcWdMoLD5CC2F2fgCPic8Y</latexit><latexit sha1_base64="DdkH4C0YzqecJjH+Xbl3yBoZK+w=">AAACqniclVHJTsMwEHXCVsJW4MjFokIqAqqEC1yQEFyQuBREWdS0keM4rVtnkT1BVFF+g3/jX0DCbXOghQsjWX6aec8zfuOngiuw7Q/DXFhcWl6prFpr6xubW9XtnUeVZJKyFk1EIp99opjgMWsBB8GeU8lI5Av25A+vx/WnVyYVT+IHGKWsE5FezENOCeiUV32/wG4oCc1dlUVezi+conuL37p5nR8WeFTermAh1PGUM5hyaHeAXcl7fTjEJ/g/cj4nL8rmw/Jhb1h41ZrdsCeBfwOnBDVURtOrfrlBQrOIxUAFUart2Cl0ciKBU8EKy80USwkdkh5raxiTiKlOPvGvwAc6E+AwkfrEgCfZn4qcREqNIl8zIwJ9NV8bJ/+qtTMIzzs5j9MMWEynjcJMYEjweBk44JJRECMNCJVcz4ppn+h1gF6ZNdMmUOPZZj6Sp71QD11YlqX9cubd+Q0eTxuO3XDu7NrlVelcBe2hfVRHDjpDl+gGNVELUfRp7BtHxrF5bN6bL2Z7SjWNUrOLZsIMvgGUSNAq</latexit><latexit sha1_base64="npxWby413h2zI5jW7I30sLyaE2M=">AAACqniclVHLSsNAFJ3Ed3xVXboZWoSKWhIRdCOIbgQ3VawPmjZMJpN27OTBzI1YQn7Df/NfFJy2WdjqxgvDHO49Z+6dc/1UcAW2/WGYc/MLi0vLK9bq2vrGZmVr+0ElmaSsRRORyCefKCZ4zFrAQbCnVDIS+YI9+oOrUf3xlUnFk/gehinrRKQX85BTAjrlVd7PsRtKQnNXZZGX83On6N7gt25e5/sFHpa3K1gIdTzhvEw4tPuCXcl7fdjHR/g/cj4jL8rmg/Jhb1B4lZrdsMeBfwOnBDVURtOrfLlBQrOIxUAFUart2Cl0ciKBU8EKy80USwkdkB5raxiTiKlOPvavwHs6E+AwkfrEgMfZn4qcREoNI18zIwJ9NVsbJf+qtTMIzzo5j9MMWEwnjcJMYEjwaBk44JJREEMNCJVcz4ppn+h1gF6ZNdUmUKPZpj6Sp71QD11YlqX9cmbd+Q0ejhuO3XBuT2oXl6Vzy2gXVVEdOegUXaBr1EQtRNGnUTUOjEPz0Lwzn832hGoapWYHTYUZfAOViNAu</latexit><latexit sha1_base64="npxWby413h2zI5jW7I30sLyaE2M=">AAACqniclVHLSsNAFJ3Ed3xVXboZWoSKWhIRdCOIbgQ3VawPmjZMJpN27OTBzI1YQn7Df/NfFJy2WdjqxgvDHO49Z+6dc/1UcAW2/WGYc/MLi0vLK9bq2vrGZmVr+0ElmaSsRRORyCefKCZ4zFrAQbCnVDIS+YI9+oOrUf3xlUnFk/gehinrRKQX85BTAjrlVd7PsRtKQnNXZZGX83On6N7gt25e5/sFHpa3K1gIdTzhvEw4tPuCXcl7fdjHR/g/cj4jL8rmg/Jhb1B4lZrdsMeBfwOnBDVURtOrfLlBQrOIxUAFUart2Cl0ciKBU8EKy80USwkdkB5raxiTiKlOPvavwHs6E+AwkfrEgMfZn4qcREoNI18zIwJ9NVsbJf+qtTMIzzo5j9MMWEwnjcJMYEjwaBk44JJREEMNCJVcz4ppn+h1gF6ZNdUmUKPZpj6Sp71QD11YlqX9cmbd+Q0ejhuO3XBuT2oXl6Vzy2gXVVEdOegUXaBr1EQtRNGnUTUOjEPz0Lwzn832hGoapWYHTYUZfAOViNAu</latexit><latexit sha1_base64="npxWby413h2zI5jW7I30sLyaE2M=">AAACqniclVHLSsNAFJ3Ed3xVXboZWoSKWhIRdCOIbgQ3VawPmjZMJpN27OTBzI1YQn7Df/NfFJy2WdjqxgvDHO49Z+6dc/1UcAW2/WGYc/MLi0vLK9bq2vrGZmVr+0ElmaSsRRORyCefKCZ4zFrAQbCnVDIS+YI9+oOrUf3xlUnFk/gehinrRKQX85BTAjrlVd7PsRtKQnNXZZGX83On6N7gt25e5/sFHpa3K1gIdTzhvEw4tPuCXcl7fdjHR/g/cj4jL8rmg/Jhb1B4lZrdsMeBfwOnBDVURtOrfLlBQrOIxUAFUart2Cl0ciKBU8EKy80USwkdkB5raxiTiKlOPvavwHs6E+AwkfrEgMfZn4qcREoNI18zIwJ9NVsbJf+qtTMIzzo5j9MMWEwnjcJMYEjwaBk44JJREEMNCJVcz4ppn+h1gF6ZNdUmUKPZpj6Sp71QD11YlqX9cmbd+Q0ejhuO3XBuT2oXl6Vzy2gXVVEdOegUXaBr1EQtRNGnUTUOjEPz0Lwzn832hGoapWYHTYUZfAOViNAu</latexit><latexit sha1_base64="npxWby413h2zI5jW7I30sLyaE2M=">AAACqniclVHLSsNAFJ3Ed3xVXboZWoSKWhIRdCOIbgQ3VawPmjZMJpN27OTBzI1YQn7Df/NfFJy2WdjqxgvDHO49Z+6dc/1UcAW2/WGYc/MLi0vLK9bq2vrGZmVr+0ElmaSsRRORyCefKCZ4zFrAQbCnVDIS+YI9+oOrUf3xlUnFk/gehinrRKQX85BTAjrlVd7PsRtKQnNXZZGX83On6N7gt25e5/sFHpa3K1gIdTzhvEw4tPuCXcl7fdjHR/g/cj4jL8rmg/Jhb1B4lZrdsMeBfwOnBDVURtOrfLlBQrOIxUAFUart2Cl0ciKBU8EKy80USwkdkB5raxiTiKlOPvavwHs6E+AwkfrEgMfZn4qcREoNI18zIwJ9NVsbJf+qtTMIzzo5j9MMWEwnjcJMYEjwaBk44JJREEMNCJVcz4ppn+h1gF6ZNdUmUKPZpj6Sp71QD11YlqX9cmbd+Q0ejhuO3XBuT2oXl6Vzy2gXVVEdOegUXaBr1EQtRNGnUTUOjEPz0Lwzn832hGoapWYHTYUZfAOViNAu</latexit><latexit sha1_base64="npxWby413h2zI5jW7I30sLyaE2M=">AAACqniclVHLSsNAFJ3Ed3xVXboZWoSKWhIRdCOIbgQ3VawPmjZMJpN27OTBzI1YQn7Df/NfFJy2WdjqxgvDHO49Z+6dc/1UcAW2/WGYc/MLi0vLK9bq2vrGZmVr+0ElmaSsRRORyCefKCZ4zFrAQbCnVDIS+YI9+oOrUf3xlUnFk/gehinrRKQX85BTAjrlVd7PsRtKQnNXZZGX83On6N7gt25e5/sFHpa3K1gIdTzhvEw4tPuCXcl7fdjHR/g/cj4jL8rmg/Jhb1B4lZrdsMeBfwOnBDVURtOrfLlBQrOIxUAFUart2Cl0ciKBU8EKy80USwkdkB5raxiTiKlOPvavwHs6E+AwkfrEgMfZn4qcREoNI18zIwJ9NVsbJf+qtTMIzzo5j9MMWEwnjcJMYEjwaBk44JJREEMNCJVcz4ppn+h1gF6ZNdUmUKPZpj6Sp71QD11YlqX9cmbd+Q0ejhuO3XBuT2oXl6Vzy2gXVVEdOegUXaBr1EQtRNGnUTUOjEPz0Lwzn832hGoapWYHTYUZfAOViNAu</latexit><latexit sha1_base64="npxWby413h2zI5jW7I30sLyaE2M=">AAACqniclVHLSsNAFJ3Ed3xVXboZWoSKWhIRdCOIbgQ3VawPmjZMJpN27OTBzI1YQn7Df/NfFJy2WdjqxgvDHO49Z+6dc/1UcAW2/WGYc/MLi0vLK9bq2vrGZmVr+0ElmaSsRRORyCefKCZ4zFrAQbCnVDIS+YI9+oOrUf3xlUnFk/gehinrRKQX85BTAjrlVd7PsRtKQnNXZZGX83On6N7gt25e5/sFHpa3K1gIdTzhvEw4tPuCXcl7fdjHR/g/cj4jL8rmg/Jhb1B4lZrdsMeBfwOnBDVURtOrfLlBQrOIxUAFUart2Cl0ciKBU8EKy80USwkdkB5raxiTiKlOPvavwHs6E+AwkfrEgMfZn4qcREoNI18zIwJ9NVsbJf+qtTMIzzo5j9MMWEwnjcJMYEjwaBk44JJREEMNCJVcz4ppn+h1gF6ZNdUmUKPZpj6Sp71QD11YlqX9cmbd+Q0ejhuO3XBuT2oXl6Vzy2gXVVEdOegUXaBr1EQtRNGnUTUOjEPz0Lwzn832hGoapWYHTYUZfAOViNAu</latexit>

Combine the first two terms in the numerator:


k=1 c(k)w(k)

PKk=1 c


Average Parameters:

=KX

i=1

x(i)y(i)

0

@iX

j=1

cj

1

A





=

PKi=1 x

(i)y(i)⇣Pi

j=1 cj⌘+

PKi=1 x

(i)y(i)⇣PK

j=i cj⌘�

PKi=1 x

(i)y(i)⇣PK

j=i cj⌘

PKk=1 ck


=

PKi=1 x

(i)y(i)⇣PK

j=1 cj⌘�PK

i=1 x(i)y(i)

⇣PKj=i c

j⌘

PKk=1 ck



=

⇣PKj=1 c

j⌘⇣PK

i=1 x(i)y(i)

⌘�PK

i=1 x(i)y(i)

⇣PKj=i c

j⌘

PKk=1 ck

<latexit sha1_base64="yEMAtGSed2y7rQpfugjJ9n10VnE=">AAACuHicfVHbSisxFM2M1zPeqj76EiwH6oNlRgQvIIi+CL54wKrQacdMmpnGZi4ke8QS5m/8KT9GMO3MgdMqZ0PIYq0d9sraYS64Atf9sOyFxaXlldVfztr6xuZWY3vnQWWFpKxDM5HJp5AoJnjKOsBBsKdcMpKEgj2Go+uJ/vjKpOJZeg/jnPUSEqc84pSAoYLG+wX2I0mo9gWLoIV9VSSBfrnwyv4tpv0X7EseD+EAV3ol80p+6+sWPyjxuL7/th7i/7fNTuJzk0pdKaPaQzAqg0bTbbvTwt+BV4MmqusuaHz6g4wWCUuBCqJU13Nz6GkigVPBSscvFMsJHZGYdQ1MScJUT0/TLPFvwwxwlElzUsBT9t8XmiRKjZPQdCYEhmpem5A/ad0CotOe5mleAEtpNSgqBIYMT1aDB1wyCmJsAKGSG6+YDolZDpgFOjNjBmribeYjOo8jY7p0HMfk5c2n8x08HLU9t+39OW5eXtXJraI9tI9ayEMn6BLdoDvUQdRyLNc6s87tc/vZjm1etdpW/WYXzZQtvwDY49St</latexit><latexit sha1_base64="yEMAtGSed2y7rQpfugjJ9n10VnE=">AAACuHicfVHbSisxFM2M1zPeqj76EiwH6oNlRgQvIIi+CL54wKrQacdMmpnGZi4ke8QS5m/8KT9GMO3MgdMqZ0PIYq0d9sraYS64Atf9sOyFxaXlldVfztr6xuZWY3vnQWWFpKxDM5HJp5AoJnjKOsBBsKdcMpKEgj2Go+uJ/vjKpOJZeg/jnPUSEqc84pSAoYLG+wX2I0mo9gWLoIV9VSSBfrnwyv4tpv0X7EseD+EAV3ol80p+6+sWPyjxuL7/th7i/7fNTuJzk0pdKaPaQzAqg0bTbbvTwt+BV4MmqusuaHz6g4wWCUuBCqJU13Nz6GkigVPBSscvFMsJHZGYdQ1MScJUT0/TLPFvwwxwlElzUsBT9t8XmiRKjZPQdCYEhmpem5A/ad0CotOe5mleAEtpNSgqBIYMT1aDB1wyCmJsAKGSG6+YDolZDpgFOjNjBmribeYjOo8jY7p0HMfk5c2n8x08HLU9t+39OW5eXtXJraI9tI9ayEMn6BLdoDvUQdRyLNc6s87tc/vZjm1etdpW/WYXzZQtvwDY49St</latexit><latexit sha1_base64="yEMAtGSed2y7rQpfugjJ9n10VnE=">AAACuHicfVHbSisxFM2M1zPeqj76EiwH6oNlRgQvIIi+CL54wKrQacdMmpnGZi4ke8QS5m/8KT9GMO3MgdMqZ0PIYq0d9sraYS64Atf9sOyFxaXlldVfztr6xuZWY3vnQWWFpKxDM5HJp5AoJnjKOsBBsKdcMpKEgj2Go+uJ/vjKpOJZeg/jnPUSEqc84pSAoYLG+wX2I0mo9gWLoIV9VSSBfrnwyv4tpv0X7EseD+EAV3ol80p+6+sWPyjxuL7/th7i/7fNTuJzk0pdKaPaQzAqg0bTbbvTwt+BV4MmqusuaHz6g4wWCUuBCqJU13Nz6GkigVPBSscvFMsJHZGYdQ1MScJUT0/TLPFvwwxwlElzUsBT9t8XmiRKjZPQdCYEhmpem5A/ad0CotOe5mleAEtpNSgqBIYMT1aDB1wyCmJsAKGSG6+YDolZDpgFOjNjBmribeYjOo8jY7p0HMfk5c2n8x08HLU9t+39OW5eXtXJraI9tI9ayEMn6BLdoDvUQdRyLNc6s87tc/vZjm1etdpW/WYXzZQtvwDY49St</latexit><latexit sha1_base64="rIc4elZHT3oXILhfcJEssMOX7as=">AAACBHicbVDLSgMxFL1TX3WsWtdugkVwVWbc6FJw47KCfUAdSiaTaUMzyZDcEcrQH3Dbr3En/oUfI5hpu7CtBy4czkm45544l8JiEHx7tb39g8Oj+rF/0vBPz86bjZ7VhWG8y7TUZhBTy6VQvIsCJR/khtMslrwfTx8rv//GjRVaveAs51FGx0qkglF0UmfUbAXtYAmyS8I1acEao+bPa6JZkXGFTFJrh2GQY1RSg4JJPvdfC8tzyqZ0zIeOKppxG5XLmHNy7ZSEpNq4UUiW6t8fJc2snWWxe5lRnNhtrxL/84YFpvdRKVReIFdstSgtJEFNqptJIgxnKGeOUGaEy0rYhBrK0DXjb6xJbJVt45AyH6cu9Nz3fVdXuF3OLundtsOgHT4HUIdLuIIbCOEOHuAJOtAFBgm8w8JbeB/e56rWmrfu9wI24H39Asd+my0=</latexit><latexit sha1_base64="bIrM6GXkMre6bLzOgQoifgaSrTI=">AAACrXicfVFdS8MwFE3rd/2avvoSFGE+OFpf1IEg+CL4ouBUWLeaZmkXl36Q3Ioj9N/4p/wxgtlawU3xQsjhnhvOyblhLrgC1/2w7IXFpeWV1TVnfWNza7uxs/GgskJS1qGZyORTSBQTPGUd4CDYUy4ZSULBHsPR1YR/fGVS8Sy9h3HOegmJUx5xSsC0gsb7BfYjSaj2BYugiX1VJIF+ufDK/g2m/RfsSx4P4QhXfEXzin7r6yY/KvG4vr9Hj/H/Y7NKfE6p1BUzqj0EozJoHLgtd1r4N/BqcIDqug0an/4go0XCUqCCKNX13Bx6mkjgVLDS8QvFckJHJGZdA1OSMNXT0zRLfGg6Axxl0pwU8LT784UmiVLjJDSTCYGhmucmzb+4bgHRWU/zNC+ApbQSigqBIcOT1eABl4yCGBtAqOTGK6ZDYpYDZoHOjMxATbzNfETncWRMl47jmLy8+XR+g4eTlue2vDsXraI9tI+ayEOn6BJdo1vUQdRyLNc6t9p223624ypZ26oj3kUzZfMvmvzTiQ==</latexit><latexit sha1_base64="bIrM6GXkMre6bLzOgQoifgaSrTI=">AAACrXicfVFdS8MwFE3rd/2avvoSFGE+OFpf1IEg+CL4ouBUWLeaZmkXl36Q3Ioj9N/4p/wxgtlawU3xQsjhnhvOyblhLrgC1/2w7IXFpeWV1TVnfWNza7uxs/GgskJS1qGZyORTSBQTPGUd4CDYUy4ZSULBHsPR1YR/fGVS8Sy9h3HOegmJUx5xSsC0gsb7BfYjSaj2BYugiX1VJIF+ufDK/g2m/RfsSx4P4QhXfEXzin7r6yY/KvG4vr9Hj/H/Y7NKfE6p1BUzqj0EozJoHLgtd1r4N/BqcIDqug0an/4go0XCUqCCKNX13Bx6mkjgVLDS8QvFckJHJGZdA1OSMNXT0zRLfGg6Axxl0pwU8LT784UmiVLjJDSTCYGhmucmzb+4bgHRWU/zNC+ApbQSigqBIcOT1eABl4yCGBtAqOTGK6ZDYpYDZoHOjMxATbzNfETncWRMl47jmLy8+XR+g4eTlue2vDsXraI9tI+ayEOn6BJdo1vUQdRyLNc6t9p223624ypZ26oj3kUzZfMvmvzTiQ==</latexit><latexit sha1_base64="M3v/jmEarDVLbdI+53dzuwUgCVY=">AAACuHicfVFdS+NAFJ1k/dr4VXcffRksQn2wJL74AYKsL4IvLtgqNG2cTCfp2MkHMzdiGfJv9k/5Y4SdNhFsK14Y5nDOHe6Zc8NccAWu+2bZP1ZW19Y3fjqbW9s7u429X12VFZKyDs1EJh9DopjgKesAB8Eec8lIEgr2EI6vp/rDC5OKZ+k9THLWT0ic8ohTAoYKGv8usR9JQrUvWAQt7KsiCfTzpVcObjEdPGNf8ngER7jSK5lX8utAt/hRiSf1/dF6jL9vm5/EFyaVulLGtYdgXAaNptt2Z4WXgVeDJqrrLmi8+8OMFglLgQqiVM9zc+hrIoFTwUrHLxTLCR2TmPUMTEnCVF/P0izxoWGGOMqkOSngGfv5hSaJUpMkNJ0JgZFa1KbkV1qvgOisr3maF8BSWg2KCoEhw9PV4CGXjIKYGECo5MYrpiNilgNmgc7cmKGaepv7iM7jyJguHccxeXmL6SyD7knbc9veX7d59adObgPtowPUQh46RVfoBt2hDqKWY7nWuXVhX9hPdmzzqtW26je/0VzZ8j/Xo9Sp</latexit><latexit sha1_base64="yEMAtGSed2y7rQpfugjJ9n10VnE=">AAACuHicfVHbSisxFM2M1zPeqj76EiwH6oNlRgQvIIi+CL54wKrQacdMmpnGZi4ke8QS5m/8KT9GMO3MgdMqZ0PIYq0d9sraYS64Atf9sOyFxaXlldVfztr6xuZWY3vnQWWFpKxDM5HJp5AoJnjKOsBBsKdcMpKEgj2Go+uJ/vjKpOJZeg/jnPUSEqc84pSAoYLG+wX2I0mo9gWLoIV9VSSBfrnwyv4tpv0X7EseD+EAV3ol80p+6+sWPyjxuL7/th7i/7fNTuJzk0pdKaPaQzAqg0bTbbvTwt+BV4MmqusuaHz6g4wWCUuBCqJU13Nz6GkigVPBSscvFMsJHZGYdQ1MScJUT0/TLPFvwwxwlElzUsBT9t8XmiRKjZPQdCYEhmpem5A/ad0CotOe5mleAEtpNSgqBIYMT1aDB1wyCmJsAKGSG6+YDolZDpgFOjNjBmribeYjOo8jY7p0HMfk5c2n8x08HLU9t+39OW5eXtXJraI9tI9ayEMn6BLdoDvUQdRyLNc6s87tc/vZjm1etdpW/WYXzZQtvwDY49St</latexit><latexit sha1_base64="yEMAtGSed2y7rQpfugjJ9n10VnE=">AAACuHicfVHbSisxFM2M1zPeqj76EiwH6oNlRgQvIIi+CL54wKrQacdMmpnGZi4ke8QS5m/8KT9GMO3MgdMqZ0PIYq0d9sraYS64Atf9sOyFxaXlldVfztr6xuZWY3vnQWWFpKxDM5HJp5AoJnjKOsBBsKdcMpKEgj2Go+uJ/vjKpOJZeg/jnPUSEqc84pSAoYLG+wX2I0mo9gWLoIV9VSSBfrnwyv4tpv0X7EseD+EAV3ol80p+6+sWPyjxuL7/th7i/7fNTuJzk0pdKaPaQzAqg0bTbbvTwt+BV4MmqusuaHz6g4wWCUuBCqJU13Nz6GkigVPBSscvFMsJHZGYdQ1MScJUT0/TLPFvwwxwlElzUsBT9t8XmiRKjZPQdCYEhmpem5A/ad0CotOe5mleAEtpNSgqBIYMT1aDB1wyCmJsAKGSG6+YDolZDpgFOjNjBmribeYjOo8jY7p0HMfk5c2n8x08HLU9t+39OW5eXtXJraI9tI9ayEMn6BLdoDvUQdRyLNc6s87tc/vZjm1etdpW/WYXzZQtvwDY49St</latexit><latexit sha1_base64="yEMAtGSed2y7rQpfugjJ9n10VnE=">AAACuHicfVHbSisxFM2M1zPeqj76EiwH6oNlRgQvIIi+CL54wKrQacdMmpnGZi4ke8QS5m/8KT9GMO3MgdMqZ0PIYq0d9sraYS64Atf9sOyFxaXlldVfztr6xuZWY3vnQWWFpKxDM5HJp5AoJnjKOsBBsKdcMpKEgj2Go+uJ/vjKpOJZeg/jnPUSEqc84pSAoYLG+wX2I0mo9gWLoIV9VSSBfrnwyv4tpv0X7EseD+EAV3ol80p+6+sWPyjxuL7/th7i/7fNTuJzk0pdKaPaQzAqg0bTbbvTwt+BV4MmqusuaHz6g4wWCUuBCqJU13Nz6GkigVPBSscvFMsJHZGYdQ1MScJUT0/TLPFvwwxwlElzUsBT9t8XmiRKjZPQdCYEhmpem5A/ad0CotOe5mleAEtpNSgqBIYMT1aDB1wyCmJsAKGSG6+YDolZDpgFOjNjBmribeYjOo8jY7p0HMfk5c2n8x08HLU9t+39OW5eXtXJraI9tI9ayEMn6BLdoDvUQdRyLNc6s87tc/vZjm1etdpW/WYXzZQtvwDY49St</latexit><latexit sha1_base64="yEMAtGSed2y7rQpfugjJ9n10VnE=">AAACuHicfVHbSisxFM2M1zPeqj76EiwH6oNlRgQvIIi+CL54wKrQacdMmpnGZi4ke8QS5m/8KT9GMO3MgdMqZ0PIYq0d9sraYS64Atf9sOyFxaXlldVfztr6xuZWY3vnQWWFpKxDM5HJp5AoJnjKOsBBsKdcMpKEgj2Go+uJ/vjKpOJZeg/jnPUSEqc84pSAoYLG+wX2I0mo9gWLoIV9VSSBfrnwyv4tpv0X7EseD+EAV3ol80p+6+sWPyjxuL7/th7i/7fNTuJzk0pdKaPaQzAqg0bTbbvTwt+BV4MmqusuaHz6g4wWCUuBCqJU13Nz6GkigVPBSscvFMsJHZGYdQ1MScJUT0/TLPFvwwxwlElzUsBT9t8XmiRKjZPQdCYEhmpem5A/ad0CotOe5mleAEtpNSgqBIYMT1aDB1wyCmJsAKGSG6+YDolZDpgFOjNjBmribeYjOo8jY7p0HMfk5c2n8x08HLU9t+39OW5eXtXJraI9tI9ayEMn6BLdoDvUQdRyLNc6s87tc/vZjm1etdpW/WYXzZQtvwDY49St</latexit><latexit sha1_base64="yEMAtGSed2y7rQpfugjJ9n10VnE=">AAACuHicfVHbSisxFM2M1zPeqj76EiwH6oNlRgQvIIi+CL54wKrQacdMmpnGZi4ke8QS5m/8KT9GMO3MgdMqZ0PIYq0d9sraYS64Atf9sOyFxaXlldVfztr6xuZWY3vnQWWFpKxDM5HJp5AoJnjKOsBBsKdcMpKEgj2Go+uJ/vjKpOJZeg/jnPUSEqc84pSAoYLG+wX2I0mo9gWLoIV9VSSBfrnwyv4tpv0X7EseD+EAV3ol80p+6+sWPyjxuL7/th7i/7fNTuJzk0pdKaPaQzAqg0bTbbvTwt+BV4MmqusuaHz6g4wWCUuBCqJU13Nz6GkigVPBSscvFMsJHZGYdQ1MScJUT0/TLPFvwwxwlElzUsBT9t8XmiRKjZPQdCYEhmpem5A/ad0CotOe5mleAEtpNSgqBIYMT1aDB1wyCmJsAKGSG6+YDolZDpgFOjNjBmribeYjOo8jY7p0HMfk5c2n8x08HLU9t+39OW5eXtXJraI9tI9ayEMn6BLdoDvUQdRyLNc6s87tc/vZjm1etdpW/WYXzZQtvwDY49St</latexit><latexit sha1_base64="yEMAtGSed2y7rQpfugjJ9n10VnE=">AAACuHicfVHbSisxFM2M1zPeqj76EiwH6oNlRgQvIIi+CL54wKrQacdMmpnGZi4ke8QS5m/8KT9GMO3MgdMqZ0PIYq0d9sraYS64Atf9sOyFxaXlldVfztr6xuZWY3vnQWWFpKxDM5HJp5AoJnjKOsBBsKdcMpKEgj2Go+uJ/vjKpOJZeg/jnPUSEqc84pSAoYLG+wX2I0mo9gWLoIV9VSSBfrnwyv4tpv0X7EseD+EAV3ol80p+6+sWPyjxuL7/th7i/7fNTuJzk0pdKaPaQzAqg0bTbbvTwt+BV4MmqusuaHz6g4wWCUuBCqJU13Nz6GkigVPBSscvFMsJHZGYdQ1MScJUT0/TLPFvwwxwlElzUsBT9t8XmiRKjZPQdCYEhmpem5A/ad0CotOe5mleAEtpNSgqBIYMT1aDB1wyCmJsAKGSG6+YDolZDpgFOjNjBmribeYjOo8jY7p0HMfk5c2n8x08HLU9t+39OW5eXtXJraI9tI9ayEMn6BLdoDvUQdRyLNc6s87tc/vZjm1etdpW/WYXzZQtvwDY49St</latexit>

Pull counts out of the first sum (doesn’t depend on i):


k=1 c(k)w(k)

PKk=1 c


Average Parameters:

=KX

i=1

x(i)y(i)

0

@iX

j=1

cj

1

A





=

PKi=1 x

(i)y(i)⇣Pi

j=1 cj⌘+

PKi=1 x

(i)y(i)⇣PK

j=i cj⌘�

PKi=1 x

(i)y(i)⇣PK

j=i cj⌘

PKk=1 ck


=

PKi=1 x

(i)y(i)⇣PK

j=1 cj⌘�PK

i=1 x(i)y(i)

⇣PKj=i c

j⌘

PKk=1 ck



=

⇣PKj=1 c

j⌘⇣PK

i=1 x(i)y(i)

⌘�PK

i=1 x(i)y(i)

⇣PKj=i c

j⌘

PKk=1 ck


Pull counts out of the first sum (doesn’t depend on i):

Now these are all quantities we can easily compute in the perceptron algorithm!

Efficient Parameter Averaging=

⇣PKj=1 c

j⌘⇣PK

i=1 x(i)y(i)

⌘�PK

i=1 x(i)y(i)

⇣PKj=i c

j⌘

PKk=1 ck


Efficient Parameter Averaging=

⇣PKj=1 c

j⌘⇣PK

i=1 x(i)y(i)

⌘�PK

i=1 x(i)y(i)

⇣PKj=i c

j⌘

PKk=1 ck


=KX

i=1

x(i)y(i) �

PKi=1 x

(i)y(i)⇣PK

j=i cj⌘

PKk=1 ck

<latexit sha1_base64="+FWcBRqm7INaLNWX+Xn4OUpsV0Q=">AAACi3icfVFdS8MwFE3rd3U69dGX4BDmg6NVQREHogiCLwpOhXUraZZu2dIPklt1lP5QH/wpgunWB6fihZDDueckNyd+IrgC2343zLn5hcWl5RVrda2yvlHd3HpUcSopa9FYxPLZJ4oJHrEWcBDsOZGMhL5gT/7oqug/vTCpeBw9wDhhnZD0Ix5wSkBTXvW1iV2Vhl7Gm07evcVv3azO93M8LvcD7AaS0OxfkStYAPXyoGGTFxraHWJX8v4A9vPSPZq6qTfKLa9asxv2pPBv4JSghsq686qfbi+macgioIIo1XbsBDoZkcCpYLnlpoolhI5In7U1jEjIVCebBJTjPc30cBBLvSLAE/a7IyOhUuPQ18qQwED97BXkX712CsFpJ+NRkgKL6PSiIBUYYlykjXtcMgpirAGhkutZMR0QHSjoP7FmrumpYraZh2RJP9BD55ZV5OX8TOc3eDxsOHbDuT+uXVyWyS2jHbSL6shBJ+gC3aA71EIUfRhzxppRMSvmkXlmnk+lplF6ttFMmddfyYHD/A==</latexit><latexit sha1_base64="+FWcBRqm7INaLNWX+Xn4OUpsV0Q=">AAACi3icfVFdS8MwFE3rd3U69dGX4BDmg6NVQREHogiCLwpOhXUraZZu2dIPklt1lP5QH/wpgunWB6fihZDDueckNyd+IrgC2343zLn5hcWl5RVrda2yvlHd3HpUcSopa9FYxPLZJ4oJHrEWcBDsOZGMhL5gT/7oqug/vTCpeBw9wDhhnZD0Ix5wSkBTXvW1iV2Vhl7Gm07evcVv3azO93M8LvcD7AaS0OxfkStYAPXyoGGTFxraHWJX8v4A9vPSPZq6qTfKLa9asxv2pPBv4JSghsq686qfbi+macgioIIo1XbsBDoZkcCpYLnlpoolhI5In7U1jEjIVCebBJTjPc30cBBLvSLAE/a7IyOhUuPQ18qQwED97BXkX712CsFpJ+NRkgKL6PSiIBUYYlykjXtcMgpirAGhkutZMR0QHSjoP7FmrumpYraZh2RJP9BD55ZV5OX8TOc3eDxsOHbDuT+uXVyWyS2jHbSL6shBJ+gC3aA71EIUfRhzxppRMSvmkXlmnk+lplF6ttFMmddfyYHD/A==</latexit><latexit sha1_base64="+FWcBRqm7INaLNWX+Xn4OUpsV0Q=">AAACi3icfVFdS8MwFE3rd3U69dGX4BDmg6NVQREHogiCLwpOhXUraZZu2dIPklt1lP5QH/wpgunWB6fihZDDueckNyd+IrgC2343zLn5hcWl5RVrda2yvlHd3HpUcSopa9FYxPLZJ4oJHrEWcBDsOZGMhL5gT/7oqug/vTCpeBw9wDhhnZD0Ix5wSkBTXvW1iV2Vhl7Gm07evcVv3azO93M8LvcD7AaS0OxfkStYAPXyoGGTFxraHWJX8v4A9vPSPZq6qTfKLa9asxv2pPBv4JSghsq686qfbi+macgioIIo1XbsBDoZkcCpYLnlpoolhI5In7U1jEjIVCebBJTjPc30cBBLvSLAE/a7IyOhUuPQ18qQwED97BXkX712CsFpJ+NRkgKL6PSiIBUYYlykjXtcMgpirAGhkutZMR0QHSjoP7FmrumpYraZh2RJP9BD55ZV5OX8TOc3eDxsOHbDuT+uXVyWyS2jHbSL6shBJ+gC3aA71EIUfRhzxppRMSvmkXlmnk+lplF6ttFMmddfyYHD/A==</latexit><latexit sha1_base64="rIc4elZHT3oXILhfcJEssMOX7as=">AAACBHicbVDLSgMxFL1TX3WsWtdugkVwVWbc6FJw47KCfUAdSiaTaUMzyZDcEcrQH3Dbr3En/oUfI5hpu7CtBy4czkm45544l8JiEHx7tb39g8Oj+rF/0vBPz86bjZ7VhWG8y7TUZhBTy6VQvIsCJR/khtMslrwfTx8rv//GjRVaveAs51FGx0qkglF0UmfUbAXtYAmyS8I1acEao+bPa6JZkXGFTFJrh2GQY1RSg4JJPvdfC8tzyqZ0zIeOKppxG5XLmHNy7ZSEpNq4UUiW6t8fJc2snWWxe5lRnNhtrxL/84YFpvdRKVReIFdstSgtJEFNqptJIgxnKGeOUGaEy0rYhBrK0DXjb6xJbJVt45AyH6cu9Nz3fVdXuF3OLundtsOgHT4HUIdLuIIbCOEOHuAJOtAFBgm8w8JbeB/e56rWmrfu9wI24H39Asd+my0=</latexit><latexit sha1_base64="x/X77zpwS2b8V2ce6PDoCGDp1Mg=">AAACgHicfZHPS8MwFMfTzh+zbjq9egmKsB0crR4UcSB4EbxMcDpYt5Jm6RaX/iB5VUfpH+rBP0Uw3XpwKj4I+fJ97yUvn/iJ4Aps+90wK2vrG5vVLWu7Vt/ZbezVHlWcSsp6NBax7PtEMcEj1gMOgvUTyUjoC/bkz26K/NMLk4rH0QPMEzYMySTiAacEtOU1XjvYVWnoZbzj5KM7/DbKmryV43m5n2A3kIRm/xa5ggXQLA967vCiho6esSv5ZAqtvOyeLbupN8str3Fkt+1F4N/CKcURKqPrNT7dcUzTkEVABVFq4NgJDDMigVPBcstNFUsInZEJG2gZkZCpYbYAlONj7YxxEEu9IsAL93tHRkKl5qGvK0MCU/UzV5h/5QYpBBfDjEdJCiyiy4uCVGCIcUEbj7lkFMRcC0Il17NiOiUaKOg/sVauGatitpWHZMkk0EPnllXwcn7S+S0eT9uO3XbubVRFB+gQNZGDztE1ukVd1EMUfRgVo2bUzbp5Zl4uyZpGiXgfrYR59QVEJcMF</latexit><latexit sha1_base64="x/X77zpwS2b8V2ce6PDoCGDp1Mg=">AAACgHicfZHPS8MwFMfTzh+zbjq9egmKsB0crR4UcSB4EbxMcDpYt5Jm6RaX/iB5VUfpH+rBP0Uw3XpwKj4I+fJ97yUvn/iJ4Aps+90wK2vrG5vVLWu7Vt/ZbezVHlWcSsp6NBax7PtEMcEj1gMOgvUTyUjoC/bkz26K/NMLk4rH0QPMEzYMySTiAacEtOU1XjvYVWnoZbzj5KM7/DbKmryV43m5n2A3kIRm/xa5ggXQLA967vCiho6esSv5ZAqtvOyeLbupN8str3Fkt+1F4N/CKcURKqPrNT7dcUzTkEVABVFq4NgJDDMigVPBcstNFUsInZEJG2gZkZCpYbYAlONj7YxxEEu9IsAL93tHRkKl5qGvK0MCU/UzV5h/5QYpBBfDjEdJCiyiy4uCVGCIcUEbj7lkFMRcC0Il17NiOiUaKOg/sVauGatitpWHZMkk0EPnllXwcn7S+S0eT9uO3XbubVRFB+gQNZGDztE1ukVd1EMUfRgVo2bUzbp5Zl4uyZpGiXgfrYR59QVEJcMF</latexit><latexit sha1_base64="zT/Qja7Zdcp81VD01VBqgPPRQ/A=">AAACi3icfVFdS8MwFE3rd3U69dGX4BDmg6PVB0UciCIIvkxwKqxbSbN0i0s/SG7VUfpDffCnCKZbH5wTL4Qczj0nuTnxE8EV2PaHYS4sLi2vrK5Z6xuVza3q9s6jilNJWZvGIpbPPlFM8Ii1gYNgz4lkJPQFe/JH10X/6ZVJxePoAcYJ64ZkEPGAUwKa8qpvTeyqNPQy3nTy3h1+72V1fpjjcbkfYTeQhGb/ilzBAqiXB700eaGhvRfsSj4YwmFeukdTN/VGueVVa3bDnhSeB04Jaqisllf9cvsxTUMWARVEqY5jJ9DNiAROBcstN1UsIXREBqyjYURCprrZJKAcH2imj4NY6hUBnrA/HRkJlRqHvlaGBIbqd68g/+p1UgjOuhmPkhRYRKcXBanAEOMibdznklEQYw0IlVzPiumQ6EBB/4k1c01fFbPNPCRLBoEeOresIi/ndzrz4PG44dgN596uXV6Vya2iPbSP6shBp+gS3aIWaiOKPo0FY8OomBXzxDw3L6ZS0yg9u2imzJtvyEHD+A==</latexit><latexit sha1_base64="+FWcBRqm7INaLNWX+Xn4OUpsV0Q=">AAACi3icfVFdS8MwFE3rd3U69dGX4BDmg6NVQREHogiCLwpOhXUraZZu2dIPklt1lP5QH/wpgunWB6fihZDDueckNyd+IrgC2343zLn5hcWl5RVrda2yvlHd3HpUcSopa9FYxPLZJ4oJHrEWcBDsOZGMhL5gT/7oqug/vTCpeBw9wDhhnZD0Ix5wSkBTXvW1iV2Vhl7Gm07evcVv3azO93M8LvcD7AaS0OxfkStYAPXyoGGTFxraHWJX8v4A9vPSPZq6qTfKLa9asxv2pPBv4JSghsq686qfbi+macgioIIo1XbsBDoZkcCpYLnlpoolhI5In7U1jEjIVCebBJTjPc30cBBLvSLAE/a7IyOhUuPQ18qQwED97BXkX712CsFpJ+NRkgKL6PSiIBUYYlykjXtcMgpirAGhkutZMR0QHSjoP7FmrumpYraZh2RJP9BD55ZV5OX8TOc3eDxsOHbDuT+uXVyWyS2jHbSL6shBJ+gC3aA71EIUfRhzxppRMSvmkXlmnk+lplF6ttFMmddfyYHD/A==</latexit><latexit sha1_base64="+FWcBRqm7INaLNWX+Xn4OUpsV0Q=">AAACi3icfVFdS8MwFE3rd3U69dGX4BDmg6NVQREHogiCLwpOhXUraZZu2dIPklt1lP5QH/wpgunWB6fihZDDueckNyd+IrgC2343zLn5hcWl5RVrda2yvlHd3HpUcSopa9FYxPLZJ4oJHrEWcBDsOZGMhL5gT/7oqug/vTCpeBw9wDhhnZD0Ix5wSkBTXvW1iV2Vhl7Gm07evcVv3azO93M8LvcD7AaS0OxfkStYAPXyoGGTFxraHWJX8v4A9vPSPZq6qTfKLa9asxv2pPBv4JSghsq686qfbi+macgioIIo1XbsBDoZkcCpYLnlpoolhI5In7U1jEjIVCebBJTjPc30cBBLvSLAE/a7IyOhUuPQ18qQwED97BXkX712CsFpJ+NRkgKL6PSiIBUYYlykjXtcMgpirAGhkutZMR0QHSjoP7FmrumpYraZh2RJP9BD55ZV5OX8TOc3eDxsOHbDuT+uXVyWyS2jHbSL6shBJ+gC3aA71EIUfRhzxppRMSvmkXlmnk+lplF6ttFMmddfyYHD/A==</latexit><latexit sha1_base64="+FWcBRqm7INaLNWX+Xn4OUpsV0Q=">AAACi3icfVFdS8MwFE3rd3U69dGX4BDmg6NVQREHogiCLwpOhXUraZZu2dIPklt1lP5QH/wpgunWB6fihZDDueckNyd+IrgC2343zLn5hcWl5RVrda2yvlHd3HpUcSopa9FYxPLZJ4oJHrEWcBDsOZGMhL5gT/7oqug/vTCpeBw9wDhhnZD0Ix5wSkBTXvW1iV2Vhl7Gm07evcVv3azO93M8LvcD7AaS0OxfkStYAPXyoGGTFxraHWJX8v4A9vPSPZq6qTfKLa9asxv2pPBv4JSghsq686qfbi+macgioIIo1XbsBDoZkcCpYLnlpoolhI5In7U1jEjIVCebBJTjPc30cBBLvSLAE/a7IyOhUuPQ18qQwED97BXkX712CsFpJ+NRkgKL6PSiIBUYYlykjXtcMgpirAGhkutZMR0QHSjoP7FmrumpYraZh2RJP9BD55ZV5OX8TOc3eDxsOHbDuT+uXVyWyS2jHbSL6shBJ+gC3aA71EIUfRhzxppRMSvmkXlmnk+lplF6ttFMmddfyYHD/A==</latexit><latexit sha1_base64="+FWcBRqm7INaLNWX+Xn4OUpsV0Q=">AAACi3icfVFdS8MwFE3rd3U69dGX4BDmg6NVQREHogiCLwpOhXUraZZu2dIPklt1lP5QH/wpgunWB6fihZDDueckNyd+IrgC2343zLn5hcWl5RVrda2yvlHd3HpUcSopa9FYxPLZJ4oJHrEWcBDsOZGMhL5gT/7oqug/vTCpeBw9wDhhnZD0Ix5wSkBTXvW1iV2Vhl7Gm07evcVv3azO93M8LvcD7AaS0OxfkStYAPXyoGGTFxraHWJX8v4A9vPSPZq6qTfKLa9asxv2pPBv4JSghsq686qfbi+macgioIIo1XbsBDoZkcCpYLnlpoolhI5In7U1jEjIVCebBJTjPc30cBBLvSLAE/a7IyOhUuPQ18qQwED97BXkX712CsFpJ+NRkgKL6PSiIBUYYlykjXtcMgpirAGhkutZMR0QHSjoP7FmrumpYraZh2RJP9BD55ZV5OX8TOc3eDxsOHbDuT+uXVyWyS2jHbSL6shBJ+gC3aA71EIUfRhzxppRMSvmkXlmnk+lplF6ttFMmddfyYHD/A==</latexit><latexit sha1_base64="+FWcBRqm7INaLNWX+Xn4OUpsV0Q=">AAACi3icfVFdS8MwFE3rd3U69dGX4BDmg6NVQREHogiCLwpOhXUraZZu2dIPklt1lP5QH/wpgunWB6fihZDDueckNyd+IrgC2343zLn5hcWl5RVrda2yvlHd3HpUcSopa9FYxPLZJ4oJHrEWcBDsOZGMhL5gT/7oqug/vTCpeBw9wDhhnZD0Ix5wSkBTXvW1iV2Vhl7Gm07evcVv3azO93M8LvcD7AaS0OxfkStYAPXyoGGTFxraHWJX8v4A9vPSPZq6qTfKLa9asxv2pPBv4JSghsq686qfbi+macgioIIo1XbsBDoZkcCpYLnlpoolhI5In7U1jEjIVCebBJTjPc30cBBLvSLAE/a7IyOhUuPQ18qQwED97BXkX712CsFpJ+NRkgKL6PSiIBUYYlykjXtcMgpirAGhkutZMR0QHSjoP7FmrumpYraZh2RJP9BD55ZV5OX8TOc3eDxsOHbDuT+uXVyWyS2jHbSL6shBJ+gC3aA71EIUfRhzxppRMSvmkXlmnk+lplF6ttFMmddfyYHD/A==</latexit><latexit sha1_base64="+FWcBRqm7INaLNWX+Xn4OUpsV0Q=">AAACi3icfVFdS8MwFE3rd3U69dGX4BDmg6NVQREHogiCLwpOhXUraZZu2dIPklt1lP5QH/wpgunWB6fihZDDueckNyd+IrgC2343zLn5hcWl5RVrda2yvlHd3HpUcSopa9FYxPLZJ4oJHrEWcBDsOZGMhL5gT/7oqug/vTCpeBw9wDhhnZD0Ix5wSkBTXvW1iV2Vhl7Gm07evcVv3azO93M8LvcD7AaS0OxfkStYAPXyoGGTFxraHWJX8v4A9vPSPZq6qTfKLa9asxv2pPBv4JSghsq686qfbi+macgioIIo1XbsBDoZkcCpYLnlpoolhI5In7U1jEjIVCebBJTjPc30cBBLvSLAE/a7IyOhUuPQ18qQwED97BXkX712CsFpJ+NRkgKL6PSiIBUYYlykjXtcMgpirAGhkutZMR0QHSjoP7FmrumpYraZh2RJP9BD55ZV5OX8TOc3eDxsOHbDuT+uXVyWyS2jHbSL6shBJ+gC3aA71EIUfRhzxppRMSvmkXlmnk+lplF6ttFMmddfyYHD/A==</latexit>

Linear Separators

$  Which of these linear separators is optimal?

Support Vector Machines $  Maximizing the margin: good according to intuition,

theory, practice $  Support vector machines (SVMs) find the separator with

max margin

SVM

8i, y wy⇤ · xi � wy · xi + 1

Instructor: Alan Ritter

Documents

Transcript of Instructor: Alan Ritter