Post on 01-Oct-2021
1 DYNAMICPROGAMMINGOPTIMIZINGOVERTIME
FINDTHESHORTESTPATH FROMTHE ROOTLOWESTcost
r r n r
I s l 0
7 T
E S 4 Lauer
Geer g 7 Craw
SOLVEDBACKTO FRONT GENERALSOLUTIONTO
SHORTESTPATHS IsPROBLEMREPEATS ITSELF ie SOLVE BELLMANFORD J
L MIN Cat La BELLMAN CQIae LEFTRIGHT
ABSTRACT DEFINITION
Tti
i
Xo
I _floe aPOLICY ITCH OR ItOBJECTIVET MAXIMIZE sum of REWARDS
type MAX rGet it r XT OVER Ito Ita c A
pVALUEFUNCTION 126,1T
T
TIEBELLMNEQUATOITHEOREM Vtc aMeAax rcx.asi V.fi VCx rk
PROOF
Veluach q
g.arC4aorG4iaDt trGee.iiati trCxta
an iMAX rbc da t MAX r x 9 t t rGen.ae trGeta CA 9 __ atyEA
Vt Cic
ma.A.yafrhia.ltVetch D
SOME CODE
THE PRINCIPLEOFOPTIMALITYaka WHENSHOULDDYNAMICPROGRAMMINGWork
EXAMPLEGHORTEST LONGESTPATHS PATHGoes
m are.EEuEIFaraj
77 D 7 7u
JrJr 77L 7C 7
THE CURSE OF DIMENSIONALITY
TRIEHAIRE AGAIN
T112 I
i STATESTooBIG
RUBIK'sCUBE
43 1019
A SHORTESTPATHPROBLEMBUTOPTIMALSolutionUNKNOWN
JieweknowHowtoFINDTHEoptimesource SolutionButComputersAren'tFastcwoundwww.oeoia
R
T
OTHEROBSERVATIONS
MINIMIZE MIN CGE.ae 404 over a at eA
BELLMAN CQQLecoq Maley CGctia 1 Luck
MEMOIZE SOMETIMES PROBLEM REPEATS ez I 0 20 20 1013 of 13so STORE VALUES ACTIONS 10 Oz
TELPSOVERCOME CURSEOFDIMENSIONALITY 10 3
GENERALIZE DP STILLWORKSFOR
MAX II EGG at Ceti Flat
5 t AoCHo AT1EATI
EXAMPLE AN INVESTOR HAS A FUNDIT HAS POUNDS ATTIMEZERO
MONEY CAN'T BE WITHDRAWNITPAYS INTEREST r x 100 PERYEARFORT YEARSTHE INVESTOR CONSUMES PROPORTION at OF INTEREST REINVESTSTHERES
WHAT SHOULD YOUDOTO MAXIMIZE CONSUMPTIONOVERTYEARS
ANGIER LET It TIME t FOND VALUE
WANT TO SOLVE BACKWARDS INTIMET l
x MAX E rxt.atT t o
S t 04 1 04 t r xt tatINTERESTNOTCONSUMED
OVER 0 E At E I
IF IT'STHE LASTYEAR CONSUME EVERYTHING
At 1 1 V se r x
IF 2 YEARS LEFT
VGd 1k roca Y aDrx tix ta
roc MAX aG r t Hr roc 2 Hr03 a El
p f e
or
CALLED BANGBANGCONTROL
LETS SOLVEFOR t.tlYEARS LEFTA GUESS
ex roc ft FOR SOME
ftPROOF BY INDUCTION
VtoW roca Vt xtructa
p x trace a I
rxffta.ve all re pathr x peer IvHp
70 ATS
fu rx f Hr V Hp
APICTURE plur Hen f I pCHr
x Pcooisonex
x
xx
x I a e
re'tii.e.li
eq1q 2fs s fsti Gtrfs fe psCt.yt.s
OPTIMAL TO CONSUMEFOR LAST Lrt YEARS
OTHERWISE REINVEST ALL INTEREST
SUMMARY
DP Vecoco MAX rheoao 1 rlxt i.at tr xtao at cA
BELLMAN EQI yea rarityr x a tf I V x RG
BELLMAN CQQ CAN BESOLVED INDUCTIVELY t o 1,2
BUTLOTS OFTRICKSEXISTCOMPLEXITY SCALESBADLYLeg
memoizing function approximation
PRINCIPLEOF OPTIMALITY ToSOLVEANOPTIMIZATION
ALWAYSTAKEOPTIMALDESCIONS FROMEACHSTATEONWARDS