- Problem: Given a series of $n$ arrays (of appropriate sizes) to multiply: $\color{WildStrawberry}{A_1\times A_2\times\dots\times A_n}$
- Determine where to place parentheses to minimize the number of multiplications.
- Multiplying an $i\times j$ array with a $j\times k$ array takes $i\times j\times k$ array
- Matrix multiplication is associative, so all placements give same result

- Problem: Given a sequence of matrices $A_1, A_2, \dots, A_n$, insert parentheses so that the product of the matrices, in order, is unambiguous and needs the minimal number of multiplication
- Assume that the matrix dimensions allow multiplication, in order
- Matrix multiplication is associative: $A_1(A_2 A_3) = (A_1 A_2)A_3$
- A product is unambiguous if no factor is multiplied on both the left and the right and all factors are either a single matrix or an unambiguous product (in parentheses)

- Multiplying an $i\times j$ and a $j\times k$ matrix requires $ijk$ multiplications
- Each element of the product requires $j$ multiplications, and there are $ik$ elements

- Example:
- Given the matrices $A_1, A_2, A_3, A_4$
- Assume the dimensions of $A_1=d_0\times d_1$, etc
- Below are the five possible parenthesizations of these arrays, along with the number of multiplications:
- $(A_1A_2) (A_3 A_4): d_0d_1d_2 + d_2d_3d_4 + d_0d_2d_4$
- $((A_1A_2) A_3) A_4: d_0d_1d_2 + d_0d_2d_3 + d_0d_3d_4 $
- $(A_1(A_2 A_3)) A_4: d_1d_2d_3 + d_0d_1d_3 + d_0d_3d_4 $
- $A_1((A_2 A_3) A_4): d_1d_2d_3 + d_1d_3d_4 + d_0d_1d_4 $
- $A_1(A_2(A_3 A_4)): d_2d_3d_4 + d_1d_2d_4 + d_0d_1d_4 $
- The number of parenthesizations is at least $T(n) ≥ T(n-1) + T(n-1)$
- Since the number with the first element removed is $T(n-1)$, which is also the number with the last removed
- Thus the number of parenthesizations is $\Omega(2^n)$
- The number is actually $T(n) = \displaystyle \sum_{k=1}^{n-1} T(k)T(n-k)$ which is related to the
*Catalan numbers*. - This is because the original product can be split into 2 subproducts in $k$ places. Each split is to be parenthesized optimally.
- This recurrence is related to the
*Catalan numbers*.

- An optimal parenthesization of $A_1\dots A_n$ must break the product into two expressions, each of which is parenthesized or is a single array
- Assume the break occurs at position $k$
- In the optimal solution, the solution to the product $A_1\dots A_k$ must be optimal
- Otherwise, we could improve $A_1\dots A_n$ by improving $A_1\dots A_k$
- But the solution to $A_1\dots A_n$ is known to be optimal
- This is a contradiction
- Thus the solution to $A_1\dots An$ is known to be optimal

- This problem exhibits the Principle of Optimality:
- The optimal solution to product $A_1\dots A_n$ contains the optimal solution to two subproducts
- Thus we can use Dynamic Programming
- Consider a recursive solution
- Then improve it's performance with memoization or by rewriting bottom up

- Consider matrix prodcut $A_1\times \dots\times A_n$
- Let the dimensions of matrix $A_i$ be $d_{i-1}\times d_i$
- Thus the dimensions of matrix product $A_i \times \dots\times A_j$ is $d_{i-1}\times d_j$

- Let $M[i,j]$ represent the number of multiplications required for matrix product $A_i\times\dots \times A_j$
- For $1 ≤ i ≤ j < n$
- $M[i,i] = 0$ since no product is required
- The optimal solution of $A_i\times A_j$ must break at some point, $k$, with $i ≤ k < j$
- Thus, $M[i,j] = M[i,k] + M[k+1,j] + d_{i-1}d_k d_j$
- Thus, $ \displaystyle M[i,j] = \begin{cases} 0 & \text{if } i = j \\ \min_{i ≤ k < j} \{M[i,k] + M[k+1,j] + d_{i-1}d_k d_j\} & \text{if } i < j \end{cases} $
- This is easily expressed as a recursive function (with exponential complexity)
- What would this look like?
- Complexity:
- P(1) = 1
- P(n) = Sum (k=1, n-1) P(k)P(n-k)
- P(n) = $\Omega(f^n / n^{\frac{3}{2}})$

- We must find a way to calculate this bottom up
- Which values does $M[i,j]$ depend on?
- Consider a $n \times n$ matrix of values M[i,j]
- Diagonal is 0
- $1 \le i \le j \le n$ is the upper right triangle
- Consider some element $M[i,j]$
- Where are $M[i,k]$ (for $i ≤ k ≤ j$):
- Where are $M[k+1,j]$ (for $i ≤ k ≤ j$):
- This tells us the order in which to build the table: By diagonals
- By diagonal
- Moving up and right from the diagonal that goes top-left to bottom-right
- Diagonal indices:
- On diagonal $0, j=i$
- On diagonal $1, j = i+1$
- On diagonal $q, j = i+q$
- On diagonal $n-1, j = i+n-1$

- Bottom Up Algorithm to Calculate Minimum Number of Multiplications
- n -- Number of arrays
- d -- array of dimensions of arrays 1 .. n

minimum_multiplication(n: integer; d: array(0..n)) M: array(1..n, 1..n) := (others => 0); for diagonal in 1 .. n-1 loop for i in 1 .. n - diagonal loop j := i + diagonal; min := integer'last; for k in i .. j-1 loop current := M[i, k] + M[k+1, j] + d(i-1)*d(k)*d(j) if current < min then min := current end if end loop M[i, j] := min end loop end loop

- Bottom Up Algorithm to Calculate Minimum Number of Multiplications
- n -- Number of arrays
- d -- array of dimensions of arrays 1 .. n
- P -- 2D array of locations of M[i, j]

minimum_multiplication(n: integer; d: array(0..n)) M: array(1..n, 1..n) := (others => 0); for diagonal in 1 .. n-1 loop for i in 1 .. n-diagonal loop j := i + diagonal; min_value := integer'last; for k in i .. j-1 loop current := M[i, k] + M[k+1, j] + d(i-1)*d(k)*d(j) if current < min_value then min_value := current mink := k end if end loop M[i, j] := mi_nvalue P(i, j) := mink end loop end loop printbest(P, i, j): if i=j then print "A"_i else print '(' printbest(P, i, M[i, j]) printbest(P, M[i, j] + 1, j) print ')'

- Array dimensions:
- A
_{1}: 2 x 3 - A
_{2}: 3 x 5 - A
_{3}: 5 x 2 - A
_{4}: 2 x 4 - A
_{5}: 4 x 3 - Tables of $M_{ij}$ and $d_j$:
- Calculating $M_{25}$ = number of multiplies in the optimal way to parenthesize $A_2A_3A_4A_5$:
- Optimal locations for parentheses: $ \begin{array}{c|ccccc} i,j & 1 & 2 & 3 & 4 & 5 \\ \hline 1 & & 1 & 1 & 3 & 3 \\ 2 & & & 2 & 3 & \color{orange}{3} \\ 3 & & & & 3 & 3 \\ 4 & & & & & 4 \\ 5 & & & & & \end{array} $

Array sizes: A

M

$ \color{cyan}{M_{25}} = \min \begin{cases} \color{blue }{M_{22} + M_{35}} + \color{magenta}{d_1}\color{blue }{d_2}\color{magenta}{d_5}, \\ \color{orange}{M_{23} + M_{45}} +\color{magenta}{d_1}\color{orange}{d_3}\color{magenta}{d_5}, \\ \color{red}{M_{24} + M_{55}} + \color{magenta}{d_1}\color{red}{d_4}\color{magenta}{d_5} \end{cases} $ $ = \min \begin{cases} 0 + 54 + 45 = 99 \\ 30 + 24 + 18 = 72 \\ 54 + 0 + 36 = 90 \end{cases} $ $ = \min \begin{cases} (A_2)(A_3A_4A_5)\\ (A_2A_3)(A_4A_5)\\ (A_2A_3A_4)(A_5) \end{cases} $