Dynamic Programming

Key characteristic of a Dynamic Program: Breaking up a large,unwieldy problem into a series of smaller, more tractable problems.

Shortest Path Example:

Dynamic Programming

General Idea: work backward from day 4 to day 1. For day 4, what is the shortest path to reach LA given you began the day in either Denver or San Antonio?

f4(8) = 1030

f4(9) = 1390

Dynamic Programming

Now consider day 3.

Shortest path from Kansas City to LA is the minimum of the distance traveled in day 3 to a day 4 city, plus the shortest path from a day 4 city to LA :

f3(5) = min{ 610 + f4(8) , 790 + f4(9)} = min{1640*,2180} = 1640

Repeat for Omaha and Dallas:

f3(6) = min{ 540 + f4(8)* , 940 + f4(9)} = 1570f3(7) = min{ 790 + f4(8) , 270 + f4(9)*} = 2330

Dynamic Programming

Now consider day 2.

f2(2) = min{ 680 + f3(5) , 790 + f3(6), 1050 + f3(7) } = min{2320*,2360,2710} = 2320

f2(3) = min{ 580 + f3(5)* , 760 + f3(6), 660 + f3(7) } = 2220f2(4) = min{ 510 + f3(5)* , 700 + f3(6), 830 + f3(7) }= 2150

Now consider day 1.

f1(1) = min{ 550 + f2(2)* , 900 + f2(3), 770 + f2(4) } = 2870

Dynamic Programming

The shortest distance is 2870.

The route that obtained the shortest distance can be found by backtracking from the optimal(*) solution found for day 1 through day 4.

1 – 2 – 5 – 8 – 10

Dynamic Programming

Considering a single equation from the shortest route problem, a recursion formula can be developed:

f2(2) = min{ 680 + f3(5) , 790 + f3(6), 1050 + f3(7) }

fn-1(i) = min{ dij + fn(j)} where i is the city at the start of the day j and j is the set of possible cities at the end of the day.

Dynamic ProgrammingKey characteristics of a dynamic program:

1) The problem can be subdivided into stages, with a policy decision at each stage.

2) Each stage has a number of states associated with the beginning of that stage.

3) The effect of the policy decision at each stage is to transform the current state to a state associated with the beginning of the next stage.

4) The solution procedure is designed to find an optimal policy for the overall problem.

5) Given the current state, an optimal policy for the remaining stages is independent of the policy decisions adopted in previous stages.

6) The solution procedure begins by finding the optimal policy for the last stage.

7) A recursive relationship that identifies the optimal policy for stage n, given the optimal policy for stage n + 1, is available.