Dynamic Programming

Click this box to toggle showing the answers!

Dynamic Programming and Recursion

• DP used for recursively-defined problems


• Consider recurrences:
• Describe performance of recursive algorithm

       procedure merge_sort(A, l, r) is
          b: array(a'range)
          mid :Natural := ...
       begin
          merge_sort(A, l, mid)
          merge_sort(B, mid + 1, r)
          merge(A, mid, B);
       

• T(n) = $2T(n/2) + \Theta(n)$ = Θ(n lg n )


• Now, can we go the other way?

Recursion: Going the other way

• These are all related to each other:
• Recursive Functions and Procedures
• Recurrence Equations describing performance
• Recursively Definitions of functions


• Fibonacci Numbers - Recursive definition (think rabbits): F(n) = $F(n - 1) + F(n - 2)$
• Is this all that's needed for the definition? No, also need ...
•     Initial Conditions: $F(0)=F(1)=1$


• Recursive algorithm:

       function fibonacci(n: Natural) is
          if n = 0 then
             return 1
          elsif n = 1 then
             ans := 1
          else
             ans1 := fibonacci(n - 1)
             ans2 := fibonacci(n - 2)
             ans  := ans1 + ans2
          return ans
       

• Notice that we have taken a recursive definition and produced a recursive algorithm
• Where are initial conditions?


• Performance: Number of calls of Fibonacci: C(n) = $C(n - 1) + C(n - 2) + 1$
• Is this all that's needed for the definition? No, also need ...
•     Initial Conditions: $C(0)=C(1)=1$


• Performance: Number of Pluses: P(n) = $P(n - 1) + P(n - 2) + 1 $
• Initial Conditions: $P(0)=P(1)=0$


• Performance: Number of assignments: A(n) = $A(n - 1) + A(n - 2) + 3 $
• Initial Conditions: $A(0)=0, A(1)=1$

What's the Point? And Where Are We Going?

• The point: These are all related to each other:
• Recursive Functions and Procedures
• Recurrence Equations describing performance
• Recursively Definitions of functions


• Where are we going: DP involves taking a recursively defined function
• And implementing it ... Recursively? No, ... with an EFFICIENT Iterative Solution!

DP Intro

• Dynamic Programming:
• Efficient (ie non-recursive) solution to recursively defined problem
• Typically used for optimization problems
• Optimization means ... Find the BEST solution


• DP approach: Time/Space tradeoff
• Use a table to store solutions to subproblems
• Use solutions to subproblems to solve larger problems


• For simplicity: Start with non-optimization problem: stick with Fibonacci


• Look at three kinds of solutions:
1. Recursve
2. Memoized
3. Dynamic Programming


4. Important: Three general approaches to implementing recursive problems!


• Start by looking at improving Fibonacci

Improving Fibonacci

• Why is Fibonacci slow?
• Answer: Many subproblems are computed many times


• Let's look at the PPTS at the tree of recursive calls


• Now, three solutions to fib: Code and pretty
• Three levels of efficiency


• So, the iterative solution is a (very simple) dynamic programming solution


• That doesn't seem hard or earth shattering ...


• Yes, but, it's not an optimization problem ...

Now What?

• Now, back over to the PPTs


• Look at the Coin Selection Problem
• An Optimization Problem

Overview

• Dynamic programming (DP) can be used to solve certain optimization problems


• We will also look at DP solutions to several problems:
• Rod Cutting


• Chained Matrix Multiplication


• We will briefly consider these:
• Longest Common Subsequence
• Optimal Binary Search Tree
• Knapsack Problem


• We will skip these for now:
• Graphs: All Pairs Shortest Path (Floyd's Algorithm)
• TSP


• We will look at Memoized algorithms , a technique used in DP


• We also looked at a linear solution to maximal subarrays, but it's not really a DP solution


• We will characterize problems for which DP is appropriate

Overview

• Dynamic Programming
• General algorithmic design technique: Approach can be used to solve many kinds of problems
• Frequently used for optimization problems: finding best way to do something


• Like Divide and Conquer
• General design technique
• Uses solutions to subproblems to solve larger problems
• Difference: DP subproblems typically overlap


• Typically used when brute-force solution is to enumerate all possibilities
• May not know which subproblems to solve, so we solve many or all!
• Reduce number of possibilities by:
• Finding optimal solutions to subproblems
• Avoiding non-optimal subproblems (when possible)
• Frequently gives a polynomial algorithm for brute force exponential one

Student Outcomes

• Characterize problems that can be solved using dynamic programming
• Recognize problems that can be solved using dynamic programming
• Develop DP solutions to specified problems
• Distinguish between finding the value of a solution and constructing a solution to a problem
• Simulate execution of DP solutions to specified problems

Memoized Algorithms

• Memoized algorithms , are a technique used in DP


• Used to avoid computing recursive results more than once


• Solutions to recursive subproblems are stored in an array (ie in a memo)

Two DP Problems

• Rod Cutting


• Chained Matrix Multiplication

Principle of Optimalty

• To have a Dymamic Programming solution, a problem must have the Principle of Optimality


• This means that an optimal solution to a problem can be broken into one or more subproblems that are solved optimally
• Example - Rod Cutting Problem: an optimal solution to $n=7$, (ie 2, 2, 3) can be broken into a first cut and a subproblem of $n=5$, whose optimal solution is (2, 3).


• Key to finding a DP solution is to express the solution to a larger problem in terms of smaller problems.


• General Approach: first solve all of the subproblems, then choose the best subproblems to make up a solution to the origina, larger problem

Dynamic Programming - Steps

• Express a solution mathematically


• Express a solution recursively


• Either develop a bottom up algorithm
• Find a bottom up algorithm to find the optimal value
• Find a bottom up algorithm to construct the solution


• Or develop a memoized recursive algorithm

More Example DP Problems

• Longest Common Subsequence
• Optimal Binary Search Tree
• Knapsack Problem

Dynamic Programming - Summary

• Optimal substructure: optimal solution to a problem uses optimal solutions to related subproblems, which may be solved independently
• First find optimal solution to smallest subproblem, then use that in solution to next largest sbuproblem


• Use stored solutions of smaller problems in solutions to larger problems


• Cut and paste proof: optimal solution to problem must use optimal solution to subproblem: otherwise we could remove suboptimal solution to subproblem and replace it with a better solution, which is a contradiction


• Usually uses overlapping subproblems
• Example: Fibb(5) depends on Fibb(4) and Fibb(3) and Fibb(4) depends on Fibb(3) and Fibb(2).
• In this case, Fibb(3) overlaps as part of the solution of both Fibb(5) and Fibb(3)
• Divide and conquer: subproblems usually not overlapping



• Two approaches:
• Top down: memoize recursion
• Bottom up: find and store optimal solutions to subproblems, then use stored solutions to find optimal solution to problem


• Trades time for space
• Many times gives polynomial time rather than exponential for brute force algorithm


• Does not solve all optimization problems

Dynamic Programming: Name

• Origin: Richard Bellman, 1957
• Not related to computer programming
• Programming referred to a series of choices
• Dynamic: choices are made on the fly, not in beginning