Post on 03-Apr-2018
7/28/2019 1 Linear Regression With One Variable
1/49
Linearregress
withonevaria
Model
representaMachineLearning
7/28/2019 1 Linear Regression With One Variable
2/49
0
100
200
300
400
500
0 500 1000 1500 2000
0
100
200
300
400
500
0 500 1000 1500 2000
HousingPrices
(Portland,OR)
Price(in1000s
ofdollars)
Size(feet2)
SupervisedLearning
Giventherightanswerfor
eachexampleinthedata.
RegressionProblem
Predictreal-valuedoutp
7/28/2019 1 Linear Regression With One Variable
3/49
Nota6on:
m=Numberoftrainingexamplesxs=inputvariable/features
ys=outputvariable/targetvariable
Sizeinfeet2(x) Price($)in1
2104 40
141 232
1534 315852 178
Trainingsetof
housingprices
(Portland,OR)
7/28/2019 1 Linear Regression With One Variable
4/49
TrainingSet
LearningAlgorithm
h
Sizeof
house
Es6mated
price
Howdowerep
Linearregressionwith
Univariatelinearregre
7/28/2019 1 Linear Regression With One Variable
5/49
Costfunc6
MachineLearning
Linearregress
withonevaria
7/28/2019 1 Linear Regression With One Variable
6/49
Howtochooses?
TrainingSet
Hypothesis:
s:Parameters
Sizeinfeet2(x) Price($)in1
2104 40
141 232
1534 315852 178
7/28/2019 1 Linear Regression With One Variable
7/49
0
1
2
3
0 1 2 3
0
1
2
3
0 1 2 3
0
1
2
3
0
7/28/2019 1 Linear Regression With One Variable
8/49
y
x
Idea:Choosesothat
isclosetoforour
trainingexamples
7/28/2019 1 Linear Regression With One Variable
9/49
Costfunc6
intui6onIMachineLearning
Linearregress
withonevaria
7/28/2019 1 Linear Regression With One Variable
10/49
Hypothesis:
Parameters:
CostFunc6on:
Goal:
Simplifie
7/28/2019 1 Linear Regression With One Variable
11/49
0
1
2
3
0 1 2 3
y
x
(forfixed,thisisafunc6onofx) (func6onoftheparam
0
1
2
3
-0.5 0 0.5 1
7/28/2019 1 Linear Regression With One Variable
12/49
0
1
2
3
0 1 2 3
y
x
(forfixed,thisisafunc6onofx) (func6onoftheparam
0
1
2
3
-0.5 0 0.5 1
7/28/2019 1 Linear Regression With One Variable
13/49
0
1
2
3
-0.5 0 0.5 1
y
x
(forfixed,thisisafunc6onofx) (func6onoftheparam
0
1
2
3
0 1 2 3
7/28/2019 1 Linear Regression With One Variable
14/49
Costfunc6
intui6onIMachineLearning
Linearregress
withonevaria
7/28/2019 1 Linear Regression With One Variable
15/49
Hypothesis:
Parameters:
CostFunc6on:
Goal:
7/28/2019 1 Linear Regression With One Variable
16/49
7/28/2019 1 Linear Regression With One Variable
17/49
7/28/2019 1 Linear Regression With One Variable
18/49
(forfixed,thisisafunc6onofx) (func6onofthepara
7/28/2019 1 Linear Regression With One Variable
19/49
(forfixed,thisisafunc6onofx) (func6onofthepara
7/28/2019 1 Linear Regression With One Variable
20/49
(forfixed,thisisafunc6onofx) (func6onofthepara
7/28/2019 1 Linear Regression With One Variable
21/49
(forfixed,thisisafunc6onofx) (func6onofthepara
7/28/2019 1 Linear Regression With One Variable
22/49
Gradient
descentMachineLearning
Linearregress
withonevaria
7/28/2019 1 Linear Regression With One Variable
23/49
Havesomefunc6on
Want
Outline:
Startwithsome Keepchangingtoreduce
un6lwehopefullyendupataminimum
7/28/2019 1 Linear Regression With One Variable
24/49
1
0
J(0,1)
7/28/2019 1 Linear Regression With One Variable
25/49
0
1
J(0,1)
7/28/2019 1 Linear Regression With One Variable
26/49
Gradientdescentalgorithm
Correct:Simultaneousupdate Incorrect:
7/28/2019 1 Linear Regression With One Variable
27/49
Gradientdes
intui6onMachineLearning
Linearregress
withonevaria
7/28/2019 1 Linear Regression With One Variable
28/49
Gradientdescentalgorithm
7/28/2019 1 Linear Regression With One Variable
29/49
7/28/2019 1 Linear Regression With One Variable
30/49
Ifistoosmall,gradientdescentcanbeslow.
Ifistoolarge,gradientdescent
canovershoottheminimum.Itmay
failtoconverge,orevendiverge.
7/28/2019 1 Linear Regression With One Variable
31/49
atlocalop6ma
Currentvalueof
Gradient descent can converge to a local
7/28/2019 1 Linear Regression With One Variable
32/49
Gradientdescentcanconvergetoalocal
minimum,evenwiththelearningratefixed.
Asweapproachalocal
minimum,gradient
descentwillautoma6callytakesmallersteps.So,no
needtodecreaseover
6me.
7/28/2019 1 Linear Regression With One Variable
33/49
Gradientdesc
linearregress
MachineLearning
Linearregress
withonevaria
7/28/2019 1 Linear Regression With One Variable
34/49
Gradientdescentalgorithm LinearRegressionM
7/28/2019 1 Linear Regression With One Variable
35/49
7/28/2019 1 Linear Regression With One Variable
36/49
Gradientdescentalgorithm
u
simu
7/28/2019 1 Linear Regression With One Variable
37/49
1
0
J(0,1)
7/28/2019 1 Linear Regression With One Variable
38/49
7/28/2019 1 Linear Regression With One Variable
39/49
7/28/2019 1 Linear Regression With One Variable
40/49
(forfixed,thisisafunc6onofx) (func6onofthepara
7/28/2019 1 Linear Regression With One Variable
41/49
(forfixed,thisisafunc6onofx) (func6onofthepara
7/28/2019 1 Linear Regression With One Variable
42/49
(forfixed,thisisafunc6onofx) (func6onofthepara
7/28/2019 1 Linear Regression With One Variable
43/49
(forfixed,thisisafunc6onofx) (func6onofthepara
7/28/2019 1 Linear Regression With One Variable
44/49
(forfixed,thisisafunc6onofx) (func6onofthepara
7/28/2019 1 Linear Regression With One Variable
45/49
(forfixed,thisisafunc6onofx) (func6onofthepara
7/28/2019 1 Linear Regression With One Variable
46/49
(forfixed,thisisafunc6onofx) (func6onofthepara
7/28/2019 1 Linear Regression With One Variable
47/49
(forfixed,thisisafunc6onofx) (func6onofthepara
7/28/2019 1 Linear Regression With One Variable
48/49
(forfixed,thisisafunc6onofx) (func6onofthepara
7/28/2019 1 Linear Regression With One Variable
49/49
BatchGradientDescent
Batch:Eachstepofgradientdescusesallthetrainingexamples.