# Dynamic Programming - Knapsack Problem

### Knapsack Problem

• Knapsack Problem:
• Knapsack of capacity $m$
• $n$ types of items of varying sizes and values
• Find combination of items that maximizes value in knapsack
• Knapsack not required to be full
• Can use any number of each item

### Knapsack Example

• $m$ = 17
• Sizes and Values
•  $i$ 1 2 3 4 5 Name A B C D E Size $s_i$ 3 4 7 8 9 Value $v_i$ 4 5 10 11 12
• Some solutions
• A, A, A, A, A: 20
• D, E: 23

### Knapsack Problem Formalized

• Given a knapsack with capacity $m$, and $n$ items with sizes $s_1 \dots s_n$ and values $v_1 .. v_n$

• Problem: Maximize $\displaystyle \sum_{i=1} ^k v_i$, subject to $\displaystyle m ≥ \sum_{i=1}^k s_i$, for some $k$ in $0 .. n$

• Solution: $B(i, c) =$ total value of best packing of items $1 .. i$ in a knapsack of size $c$

• $B(i, c) = \begin{cases} 0, & \text{if } i = 0 \\ B(i-1, c), & \text{if } i > 0 \text{ and }s_i > c \\ \max\{ B(i-1, c), v_i + B(i, c-s_i) \}, & \text {otherwise } \end{cases}$

• Intuition:
• If no items, best is 0
• If item $i$ won't fit, then use best for remaining items
• Otherwise, choose between
1. Best that can be packed with items $1 \dots i-1$
2. Sum of value of item i and best that can be done to pack rest of knapsack after item $i$ is added (to an empty knapsack)

### Knapsack Algorithm

• Since calculating a given value only needs a value to its left (and not above), we collapse B into a 1D array
• Effectively reusing the array for each item
Knapsack(m, n)
B: array(0 .. m) := (others => 0);  -- B(j) is best packing of size j knapsack
L: array(1 .. m) := (others => 0);  -- L(j) is last item added for B(j)
-- Initial Row of the table below is printed here

for i in 1 .. n loop      -- i is index for each item size and value
for c in 1 .. m loop  -- c is index for each knapsack Capacity
if c >= size(i) then
tempC := c - size(i)
tempB := value(i) + B(tempC)
if tempB > B(c) then
B(c) := tempB
L(c) := i
end if
end if
end loop
-- Row i of the table below is printed here - POINT AA
end loop

• Fill knapsacks of increasing sizes with one element, then with two, ...

### Trace of Knapsack Algorithm

• Look at the changes from one pair of rows to the next - Knapsack($M=>17, n=>5$):

•  $i \downarrow$ $c \rightarrow$ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 initial $B(c)$ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 $L(c)$ - - - - - - - - - - - - - - - - - 1 $B(c)$ 0 0 4 4 4 8 8 8 12 12 12 16 16 16 20 20 20 $L(c)$ - - A A A A A A A A A A A A A A A 2 $B(c)$ 0 0 4 5 5 8 9 10 12 13 14 16 17 18 20 21 22 $L(c)$ - - A B B A B B A B B A B B A B B 3 $B(c)$ 0 0 4 5 5 8 10 10 12 14 15 16 18 20 20 22 24 $L(c)$ - - A B B A C B A C C A C C A C C 4 $B(c)$ 0 0 4 5 5 8 10 11 12 14 15 16 18 20 21 22 24 $L(c)$ - - A B B A C D A C C A C C D C C 5 $B(c)$ 0 0 4 5 5 8 10 11 12 14 15 16 18 20 21 22 24 $L(c)$ - - A B B A C D A C C A C C D C C $i \uparrow$ $c \rightarrow$ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

• Size and Value table repeated
•  Name A B C D E Size 3 4 7 8 9 Value 4 5 10 11 12

• This table is produced by printing the B and L arrays five times, at POINT AA

• Example: What is the best way to fill up row 3, column 13?
• In other words, how can we best fill a size 13 knapsack with items A, B, C
• Choose which is better:
1. best way to fill up knapsack of size 13 with items A, B (but not C)
• Find this amount in the B row for item=2 and c=13, which is 17
2. 10 + best way to fill up knapsack of size 6 with items A, B, C
• 10 comes from B(C)
• 6 comes from 13 - size(C) = 13 - 7
• Imagine: put one C in an empty knapsack and then look up the best way to fill the remaining space
• Result is 10 + [B(6) when item=3] = 10 + 8 = 18
• 18 > 17, so we update B(13) in row item=2 from 17 to 18

• How to find items in best packing: look at L(17), then L(17-size(last))

### Knapsack Problem Variations

• Only use one of each type of type of item

• Take fractional parts of each item
• Consider gold dust and silver dust