Dynamic Programming - Knapsack Problem
Knapsack Problem
- Knapsack Problem:
- Knapsack of capacity $m$
- $n$ types of items of varying sizes and values
- Find combination of items that maximizes value in knapsack
- Knapsack not required to be full
- Can use any number of each item
Knapsack Example
- $m$ = 17
- Sizes and Values
| $i$ |
1 |
2 |
3 |
4 |
5 |
| Name |
A |
B |
C |
D |
E |
| Size $s_i$ |
3 |
4 |
7 |
8 |
9 |
| Value $v_i$ |
4 |
5 |
10 |
11 |
12 |
- Some solutions
- A, A, A, A, A: 20
- D, E: 23
Knapsack Problem Formalized
- Given a knapsack with capacity $m$, and $n$ items with sizes
$s_1 \dots s_n$ and values $v_1 .. v_n$
- Problem: Maximize $\displaystyle \sum_{i=1} ^k v_i$, subject to
$\displaystyle m ≥ \sum_{i=1}^k s_i$, for some $k$ in $0 .. n$
- Solution: $B(i, c) =$ total value of best packing of items $1 .. i$ in a
knapsack of size $c$
$
B(i, c) =
\begin{cases}
0, & \text{if } i = 0 \\
B(i-1, c), & \text{if } i > 0 \text{ and }s_i > c \\
\max\{ B(i-1, c), v_i + B(i, c-s_i) \}, & \text {otherwise }
\end{cases}
$
- Intuition:
- If no items, best is 0
- If item $i$ won't fit, then use best for remaining items
- Otherwise, choose between
- Best that can be packed with items $1 \dots i-1$
- Sum of value of item i and best that can be done to pack rest of knapsack
after item $i$ is added (to an empty knapsack)
Knapsack Algorithm
- Since calculating a given value only needs a value to its left (and
not above),
we collapse B into a 1D array
- Effectively reusing the array for each item
Knapsack(m, n)
B: array(0 .. m) := (others => 0); -- B(j) is best packing of size j knapsack
L: array(1 .. m) := (others => 0); -- L(j) is last item added for B(j)
-- Initial Row of the table below is printed here
for i in 1 .. n loop -- i is index for each item size and value
for c in 1 .. m loop -- c is index for each knapsack Capacity
if c >= size(i) then
tempC := c - size(i)
tempB := value(i) + B(tempC)
if tempB > B(c) then
B(c) := tempB
L(c) := i
end if
end if
end loop
-- Row i of the table below is printed here - POINT AA
end loop
Fill knapsacks of increasing sizes with one element, then with two, ...
Trace of Knapsack Algorithm
- Look at the changes from one pair of rows to the next - Knapsack($M=>17, n=>5$):
| $i \downarrow$ |
$c \rightarrow$ |
1 | 2 | 3 | 4 | 5 |
6 | 7 | 8 | 9 | 10 |
11 | 12 | 13 | 14 | 15 |
16 | 17 |
| initial |
$B(c)$ |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
| |
$L(c)$ |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 1 |
$B(c)$ |
0 |
0 |
4 |
4 |
4 |
8 |
8 |
8 |
12 |
12 |
12 |
16 |
16 |
16 |
20 |
20 |
20 |
| |
$L(c)$ |
- |
- |
A |
A |
A |
A |
A |
A |
A |
A |
A |
A |
A |
A |
A |
A |
A |
| 2 |
$B(c)$ |
0 |
0 |
4 |
5 |
5 |
8 |
9 |
10 |
12 |
13 |
14 |
16 |
17 |
18 |
20 |
21 |
22 |
| |
$L(c)$ |
- |
- |
A |
B |
B |
A |
B |
B |
A |
B |
B |
A |
B |
B |
A |
B |
B |
| 3 |
$B(c)$ |
0 |
0 |
4 |
5 |
5 |
8 |
10 |
10 |
12 |
14 |
15 |
16 |
18 |
20 |
20 |
22 |
24 |
| |
$L(c)$ |
- |
- |
A |
B |
B |
A |
C |
B |
A |
C |
C |
A |
C |
C |
A |
C |
C |
| 4 |
$B(c)$ |
0 |
0 |
4 |
5 |
5 |
8 |
10 |
11 |
12 |
14 |
15 |
16 |
18 |
20 |
21 |
22 |
24 |
| |
$L(c)$ |
- |
- |
A |
B |
B |
A |
C |
D |
A |
C |
C |
A |
C |
C |
D |
C |
C |
| 5 |
$B(c)$ |
0 |
0 |
4 |
5 |
5 |
8 |
10 |
11 |
12 |
14 |
15 |
16 |
18 |
20 |
21 |
22 |
24 |
| |
$L(c)$ |
- |
- |
A |
B |
B |
A |
C |
D |
A |
C |
C |
A |
C |
C |
D |
C |
C |
| $i \uparrow$ |
$c \rightarrow$ |
1 | 2 | 3 | 4 | 5 |
6 | 7 | 8 | 9 | 10 |
11 | 12 | 13 | 14 | 15 |
16 | 17 |
- Size and Value table repeated
| Name |
A |
B |
C |
D |
E |
| Size |
3 |
4 |
7 |
8 |
9 |
| Value |
4 |
5 |
10 |
11 |
12 |
- This table is produced by printing the B and L arrays five times,
at POINT AA
- Example: What is the best way to fill up row 3, column 13?
- In other words, how can we best fill a size 13 knapsack with items A, B, C
- Choose which is better:
- best way to fill up knapsack of size 13 with items A, B (but not C)
- Find this amount in the B row for item=2 and c=13, which is 17
- 10 + best way to fill up knapsack of size 6 with items A, B, C
- 10 comes from B(C)
- 6 comes from 13 - size(C) = 13 - 7
- Imagine: put one C in an empty knapsack and
then look up the best way to fill the remaining space
- Result is 10 + [B(6) when item=3] = 10 + 8 = 18
- 18 > 17, so we update B(13) in row item=2 from 17 to 18
- How to find items in best packing: look at L(17), then L(17-size(last))
Knapsack Problem Variations
- Only use one of each type of type of item
- Take fractional parts of each item
- Consider gold dust and silver dust