Recursive Algorithms and Recurrence Equations
Introduction
Initial Conditions
Up and Down Substitution
Closed Form
Overview
- Performance of recursive algorithms typically specified with
recurrence equations
- Recurrence Equations aka Recurrence and Recurrence Relations
- Recurrence relations have specifically to do with sequences (eg Fibonacci Numbers)
- Recurrence equations require special techniques for solving
- We will focus on induction and the Master Method (and its variants)
- And touch on other methods
Analyzing Performance of Non-Recursive Routines is (relatively) Easy
for i in 1 .. n loop
Loop: T(n) = $\Theta(n^2)$
for i in 1 .. n loop
for j in 1 .. n loop
end loop;
end loop;
for i in 1 .. n loop
for j in 1 .. i loop
end loop;
end loop;
Loop: T(n) = $\Theta(n^3)$ ... obvious
Analyzing Recursive Routines
- Analysis of recursive routines is not as easy: consider factorial
fac(n) is
if n = 1 then return 1
else return fac(n-1) * 1
How many times is fac called for fac(n)?
To find an answer, Use a recurrence
Recurrences
- A recurrence defines T(n) in terms of T for smaller values
- Example: T(n) = T(n-1) + 1
- T(n) is defined in terms of T(n-1)
- Recurrences are used in analyzing recursive algorithms
- AKA: Recurrence Equation, Recurrence Relation
Evaluating a Recurrence
- How to think about T(n) = T(n-1) + 1
- How to find the value of a T(k) for a particular k:
- Substitute up from T(1) to T(k)
- Substitute down from T(k) to T(1)
- Solving the recurrence and evaluate the resulting expression
- All three methods require having the initial conditions for the recurrence
Initial Conditions
- The initial conditions are the values of the recurrence for small values of n
- For example, the values of T(0), T(1), T(2)
- We will see that the initial conditions are determined by
the specific problem being solved
- A Recurrence Equation has multiple solutions,
- The initial conditions determines which of those solutions applies.
Substituting Up and Down
- Problem: Find value of T(n) = T(n-1) + 1 for n=4, with initial condition T(1)=2
- Substituting up from T(1):
- T(1) = 2, Initial condition
- T(2) = T(1) + 1 = 2+1 = 3
- T(3) = T(2) + 1 = 3+1 = 4
- T(4) = T(3) + 1 = 4+1 = 5
- Subsituting down from T(4):
- Example: T(4) = T(3) + 1 = [T(2) + 1] + 1 = [[T(1) + 1] + 1] + 1 = 2+1+1+1 = 5
Solving the Recurrence: Closed Forms
- To solve a recurrence, we find a closed form for it
- Closed form for T(n): An equation that defines T(n) using an expression that does
not involve T
- Example: A closed form for T(n) = T(n-1)+1 is T(n) = n.
- Solution techniques - no single method works for all:
- Guess and Check
- Forward substitution
- Backward Substitution
- Characteristic equation
- Master method (2 versions)
- Recurrence trees help us think about recurrences and show intuition in Master Method
Solving RE
Forward and Backward Substitution, Initial Conditions
Guess and Check: Forward Substitution
- Recurrence: T(n) = T(n-1) + 1, with initial condition t(1) = 2
- Look for a pattern:
- T(1) = 2, Initial condition
- T(2) = T(1) + 1 = 2+1 = 3
- T(3) = T(2) + 1 = 3+1 = 4
- T(4) = T(3) + 1 = 4+1 = 5
- T(5) = T(4) + 1 = 5+1 = 6
- Guess:
- Informal Check:
- T(n) = T(n-1)+1 = [(n-1)+1] + 1 = n+1
- T(1) = 1+1 = 2
- Informally: a closed form is correct if it satisfies the open form and
the initial conditions
- Formally: Prove the closed form is correct using induction (example below)
Guess and Check: Backward Substitution
- Look for a pattern (how many 1's in each line?):
- T(n) = T(n-1) + 1
- T(n) = [T(n-2)+1] + 1
- T(n) = [[T(n-3)+1]+1] + 1
- T(n) = [[[T(n-4)+1]+1]+1] + 1
- ...
- T(n) = [...[[T(n-k)+1]+1] ... +1] + 1 [has k ones]
- ... [Let k = n-1]
- T(n) = [...[[T(n-(n-1))+1]+1] ... +1] + 1 [has k=n-1 ones]
- T(n) = T(n-(n-1)) + (n - 1)
- T(n) = T(1) + (n-1)
- T(n) = 2 + (n - 1)
- T(n) = n + 1
- This process leads to this Guess:
- Check the guess, by substituting, as in the previous example:
- T(n) = T(n-1)+1 = [(n-1)+1] + 1 = n+1
- T(1) = 1+1 = 2
Recurrences Without Initial Conditions
- A recurrence without initial conditions is satisfied by a family of closed forms
- Example: Recurrence T(n) = T(n-1) + 1 is satisfied by each member of this
family of closed forms:
- T(n) = n
- T(n) = n + 1
- T(n) = n + 2
- ...
- T(n) = n - 1
- T(n) = n - 2
- ...
- With specific initial conditions, only one of that family satisfies both the recurrence
and the initial conditions
- Example: Recurrence T(n) = T(n-1) + 1 with Initial Condition T(1) = 1 is satisfied by:
- Example: Recurrence T(n) = T(n-1) + 1 with Initial Condition T(1) = 0 is satisfied by:
Initial Conditions and Solutions - More Examples
- The solution to a recurrence is determined by its initial condition
- Example 1:
- T(n) = T(n-1) + 1
- T(1) = 2
- Guess: T(n) = n + 1
- Check: as above
- T(n) = T(n-1)+1 = [n-1+1] + 1 = n+1
- T(1) = 1+1 = 2
- Example 2:
- T(n) = T(n-1) + 1
- T(1) = 1
- Guess: T(n) = n
- Check, as above:
- T(n) = T(n-1)+1 = [n-1] + 1 = n
- T(1) = 1
- Initial conditions restrict number of solutions
- In this case, restricts to one condition
Recurrence Equations for Sample Algorithms
Performance of Recursive Routines
- Let's develop recurrences for some recursive algorithms:
- Factorial
- Fibonacci
- Binary Search
- Towers of Hanoi
- What shall we count: number of calls, number of multiplies, ...
Performance of Factorial
fac(n)
if n = 1 return 1
else return n * fac(n-1)
For fac(n), how many times is fac called?
- T(n) = T(n-1) + 1
- T(1) = 1
- $T(n) = n$
Whatever the number of calls is for n-1 is, the calls for n is one larger
Performance of Factorial, revisited
- For fac(n), how many multiplies?
- T(n) = T(n-1) + 1
- T(1) = 0
- Guess: T(n) = ???
- Check: ...
Performance of Fibonacci
fib(n)
if n in 1 .. 2 return 1
else return fib(n-1) + fib(n-2)
For fib(n), how many times is fib called?
- $T(1) = 1$
- $T(2) = 1$
- $T(3) = T(1) + T(2) + 1 = 1 + 1 + 1 = 3 $
- $T(4) = T(3) + T(2) + 1 = 3 + 1 + 1 = 5 $
- $T(5) = T(4) + T(3) + 1 = 5 + 3 + 1 = 9 $
- $T(n) = T(n-1) + T(n-2) + 1 $
- $T(n) = ?$ [This one's not so easy]
- The text shows that this has exponential behavior
- Book also uses T(0) = 0 as an initial condition.
Performance of Recursive Binary Search
index binsearch(index low, high, keytype S[], keytype x)
if low ≤ high then
mid = (low + high) / 2
if x = S[mid] then
return mid
elsif x < s[mid] then
return binsearch(n, low, mid-1, S, x)
else
return binsearch(n, mid+1, high, S, x)
else
return 0
end binsearch
For binsearch(n), how many times is binsearch called in the worst case?
- T(0) = 1
- T(1) = 2
- T(2) = T(1) + 1 = 3
- T(4) = T(2) + 1 = 4
- T(8) = T(4) + 1 = 4 + 1 = 5
- T(n) = T(n/2) + 1
- Guess: $T(n) = \lg n + 2$ for $n=2^k$
- Informal Check:
$\begin{align*}
T(n) & = T(n/2) + 1 \\
& = (\lg \frac{n}{2} + 2) + 1 \\
& = (\lg n - \lg 2 + 2) + 1 \\
& = (\lg n - 1 + 2) + 1 \\
& = \lg n + 2
\end{align*}
$
- We can prove $T(n) = \lg n + 2$ by induction ...
- $T(n) = \Theta(\lg n)$
- What do we assume about n?
Towers of Hanoi
- Recursive algorithm
- Recurrence Equation: T(n) = ...
- Closed form: ...
More Example Algorithms and their Recurrence Equations
- More Eamples - Let's review or figure out these:
- Factorial (Every Case):
- Fibonacci (Every Case):
- Binary Search (Worst Case):
- Quick Sort (Worst Case):
- Quick Sort(Best Case):
- Merge Sort(Every Case):
More Example Algorithms and their Recurrence Equations
- More Eamples - Let's review or figure out these:
- Factorial (Every Case): $T(n) = T(n-1) + 1$
- Fibonacci (Every Case): $T(n) = T(n-1) + T(n-2) + 1$
- Binary Search (Worst Case): $T(n) = T(n/2) + 1$
- Quick Sort (Worst Case): $T(n) = T(n-1) + \Theta(n)$
- Quick Sort(Best Case): $T(n) = 2T(\frac{n}{2}) + \Theta(n)$
- Merge Sort(Every Case): $T(n) = 2T(\frac{n}{2}) + \Theta(n)$
- General Forms:
- $T(n) = aT(\frac{n}{b}) + f(n)$
- $T(n) = aT(n - b) + f(n)$
Technical Issues and Θ on RHS
Technical Issues
- Floors and Ceilings
- Usually Ignore
- Or assume size is power of 2
- Initial (or Boundary) conditions
- Frequently we use asymptotic values
- Example: $T(1) = \Theta(1)$
- Exact vs asymptotic functions
- Usually asymptotic is good enough
- Finding exact solutions requires exact boundary conditions
$\Theta(n)$ on the RHS
- In $T(n) = T(n-1) + \Theta(n)$, the $\Theta(n)$ terms refers to some function
in $\Theta(n)$
- That is, $f(n) \in \Theta(n)$
- In most cases, the exact function does not affect the asymptotic behavior of $T(n)$
- This allows us to simplify by ignoring the exact function
Solution Techniques
Recurrence Equations - Solution Techniques
- No simple way to solve all recurrence equations
- Following techniques are used:
- Guess a solution and use induction to prove its correctness
- Use Forward and Backward Substitution to guess, if needed
- Use a general formula (ie the Master Method)
- For $T(n) = aT(\frac{n}{b}) + cn^k$ [Version in Text]
- For $T(n) = aT(\frac{n}{b}) + f(n)$ [More general version,
NIB, but in CLR
- Solve using Characteristic Equation
- Linear homogeneous equations with constant coefficients
- Non-linear homogeneous equations with constant coefficients
- Change of Variable
- We focus on the general formulae and touch on the others
- General formulae can be understood using recursion trees
- First we see an example of induction
Checking Recurrence Closed Forms Using Induction
Check Using Induction: Factorial [SKIP]
- For fac(n), how many times is fac called (every case)?
- Above we derived:
- T(1) = 1
- T(k) = T(k - 1) + 1
- T(n) = n
- Inductive proof that $T(n) = n \text{ is a solution of }T(n) = T(n-1) + 1$, with initial
condition T(1)=1:
- Base Case: Prove $T(n) = n \text{ is a solution }$ for n = 1. T(1) = 1 is an initial condition.
- Inductive Step: Prove: For all $n \gt 1, T(n) = n \text{ is a solution} \implies T(n+1) =
n+1$ is a solution
- Inductive Hypothesis: $T(n) = n$ is a solution
- Now consider T(n) + 1:
$\begin{align*}
T(n+1) & = T((n+1)-1) + 1 \\
& = T(n) + 1 \\
& = n + 1\textrm{, by IH} \\
\end{align*}
$
- This proves the Inductive Step
- Therefore from 1 and 2, by induction, we conclude that $T(n) = n \text{ for all } n ≥ 1$
Example Using Induction: Binary Search [SKIP]
- Worst case number of calls of binary search (assume n is a power of 2
- Recurrence: $T(n) = T(n/2) + 1 \text{ with } T(1)=2 $ [Derived above]
- Closed form: $T(n) = \lg n + 2$ [Guessed above.]
- Inductive proof that closed form is solution for recurrence (assume powers of 2):
- Base Case: Prove for 1: $T(1) = \lg 1 + 2 = 0 + 2 = 2$.
- Inductive Step: Prove: that $T(k) = \lg k + 2, \textrm{ for all } k < n$
- Inductive Step: Prove: For all $n \gt 1, T(n) = \lg n + 2, \text{ is a solution}
\implies T(2n) = \lg 2n +2 = \lg n +3$ is a solution
- IH: Assume that $T(k) = \lg k + 2, \textrm{ for all } k < n$
- Now consider $T(2n)$:
$\begin{align*}
T(2n) & = T(2n/2) + 1\text{, by recurrence} \\
& = T(n) + 1 \\
& = (\lg n + 2) + 1, \text{ by IH} \\
& = \lg n + 3
\end{align*}
$
- This proves the Inductive Step
- Therefore from 1 and 2 we conclude that $T(n) =\lg n +2 \text{ is a solution to the
recurrence for all } n ≥ 1$, where n is a power of 2
- Can also be proved for non-powers of 2
Master Method
Master Method from Text
- Cookbook way of solving recurrences of this form:
- $T(n) = aT(\frac{n}{b}) + \Theta(n^d)$
- $T(1) = c$
- $n = b^k$, for positive integer $k$
- $a \ge 1, b \ge 2, c \gt 0, d \ge 0$ are constants
- $
T(n) \in
\begin{cases}
\Theta(n^d), & a < b^d \\
\Theta(n^d\lg n), & a = b^d \\
\Theta(n^{\log_b a)}, & a > b^d
\end{cases}
$
- We call these Case 1, Case 2, and Case 3.
- Intuition: Which term of recurrence dominates (ie contributes more)
- In the recurrence we can replace
"T(n)=" with "T(n)≤" or "T(n)≥" and get $O$ or $\Omega$ performance
Examples using Master Method from Text
- $T(n) = 16T(n/4) + 5n^3$
- $a=16, b=4, d=3$
- 16 < 43 and so Case 1
- Thus, $T(n) = \Theta(n^d) = \Theta(n^3)$
- $T(n) = 2T(n/2) + n$
- $a=2, b=2, d=1$
- 2 = 21 and so Case 2
- Thus, $T(n) = \Theta(n^1\lg n)$
- $T(n) = 8T(n/2) + n$
- $a=8, b=2, d=1$,
- 8 > 21 and so Case 3
- Thus, $T(n) = \Theta(n^{\log_b a}) = \Theta(n^{\lg 8}) = \Theta(n^3)$
General Master Method [Not In Book]
- Solve $T(n) = aT(\frac{n}{b}) + f(n)$
- The MM in the text is a special case version of this one
- Text's MM's 3 cases are special cases of the GMM 3 cases
- $
T(n) \in
\begin{cases}
\Theta(f(n)), & f(n) = \Omega(n^{log_b a+\epsilon}) \text{ and } af(n/b) ≤
cf(n) \text{ for some constant } c ≤ 1 \text{ and large } n
\\
\Theta(n^{\log_b a}\lg n), & f(n) = \Theta(n^{log_b a})\\
\Theta(n^{\log_b a)}, & f(n) = O(n^{log_b a-\epsilon})
\end{cases}
$
- Assume $0 < \epsilon$
- Case 1 can also be written as $n^{log_b a+\epsilon} = O(f(n))$
- We ignore Case 1 because of its more complex condition
Examples
- $T(n) = 2T(n/2) + n$
- $a=2, b=2, f(n)=n, \log_b a = \lg 2 = 1$
- Case 2: $f(n) = n = n^1 = \Theta(n^{\log_b a})$
- Thus, $T(n) = \Theta(n^{\log_b a}\lg n) = \Theta(n \lg n)$
- $T(n) = 8T(n/2) + n$
- $a=8, b=2, f(n)=n, \log_b a = \log_2 8 = 3$
- Case 3: $f(n) = n = O(n^{3-1})$, with $\epsilon=1$
- Thus, $T(n) = \Theta(n^{\log_b a}) = \Theta(n^{\lg 8}) = \Theta(n^3)$
Text's Master Method vs General MM
- The text's Master Method is a special case of the general Master Method:
- Assume $T(n) = aT(n/k) + \Theta(n^d)$ meets the conditions for the Text's Case 2, ie $a = b^k$.
-
Taking log of both sides of $a = b^d$ gives $log_b a = log_b b^d = d$.
-
Thus, $f(n) \in \Theta( n^d)$ means $cn^{log_b a} \in \Theta(n^{log_b a})$
which is the condition for Case 2 in the General Master Method.
-
Therefore, if a recurrence meets Case 2 of the Text's MM, then it also meets Case 2 of
the General MM.
-
Similar arguments can be made for the other cases.
Other Solution Techniques
Other Methods for Solving Recurrence Equations
- Characteristic Equation
- Linear homogeneous equations with constant coefficients
- Non-linear homogeneous equations with constant coefficients
- Change of Variable
- Substitution
- We look at briefly at the first and ignore the second two
Characteristic Equations
Homogenous Linear Recurrence Equations
- General form: $a_0t_n + a_1t_{n-1} + \dots + a_kt_{n-k} = 0$
- Slightly different from previous recurrence equations:
- Notation: $t_n$ vs $T(n)$
- all $t$ terms on LHS of equation
- 0 on RHS of equation
- Overview of how to solve:
- Assume $t_n = r^n$ is a solution for all $n$ and for some
real number r
- Generate Characteristic Equation:
- Recurrence: $a_0t_n + a_1t_{n-1} + \dots + a_kt_{n-k} = 0$
- Substitute $r^n$ for $t_n$: $a_0r^n + a_1r^{n-1} + \dots + a_kr^{n-k} = 0$
- Factor out $r^{n-k}$: $(r^{n-k})(a_0r^k + a_1r^{k-1} + \dots + a_kr^{k-k}) = 0$
- Divide by $r^{n-k}$ (since it gives a root of r=0, which is not useful)
- Resulting Characteristic Equation: $a_0r^k + a_1r^{k-1} + \dots + a_kr^{k-k} = 0$
- Find roots of Characteristic Equation
- Use roots of Characteristic Equation to form recurrence solution
Homogenous Linear Recurrence Equation: Example
- Notation: $t_n$ vs $T(n)$
- Recurrence:
- $t_n - 5t_{n-1} + 6t_{n-2} = 0$
- $t_0 = 0$
- $t_1 = 1$
- Assume $t_n = r^n$ is a solution, for all $n$ and for some
real number r
- Substitute: $r^n - 5r^{n-1} + 6r^{n-2} = 0$
- Factor: $ r^{n-2}(r^2 - 5r^1 + 6r^0) = 0$
- Divide by $ r^{n-2}$ (since it gives a root of r=0, which is not useful)
- Resulting Characteristic Equation (CE): $r^2 - 5r^1 + 6r^0 = 0$
- Find roots of CE: $r^2 - 5r^1 + 6r^0 = (r-3)(r-2) = 0$
- Roots are 3 and 2
- That is, $r^2 - 5r^1 + 6r^0 = 0$ when $r_1=3$ or $r_2=2$
- By assumption, $t_n=3^n$ and $t_n=2^n$, are both solutions to the RE (and $t_n=0$, which we ignore).
- (Can verify that T(n) = 5T(-1) -6T(n-2) holds for each solution.)
- Since $t_n=3^n$ and $t_n=2^n$, are both solutions, so is the linear combination: $t_n = \alpha r_1^n + \beta r_2^n= \alpha 3^n + \beta 2^n$
- Plugging general solution into recurrence and distributing will give 0
- General solution defines an infinite number of solutions
- Use initial conditions to find a single solution
- If roots are identical, use this equation: $t_n = \alpha r^n + \beta nr^n$
- Choose $\alpha$ and $\beta$ to meet the initial conditions:
$t_0=0,t_1=1$
- $t_0 = \alpha 3^0 + \beta 2^0 = 0$
- $t_1 = \alpha 3^1 + \beta 2^1 = 1$
- Solving gives: $\alpha=1$ and $\beta = -1$
- Thus: general solution is $t_n = \alpha 3^n + \beta 2^n== 3^n - 2^n$
Homogenous Linear Recurrence Equation: Fibonacci Numbers
- Recurrence:
- $T(n) = T(n-1) + T(n-2)
- $t_n - t_{n-1} - t_{n-2} = 0$
- $t_0 = 0$
- $t_1 = 1$
- Assume solution is of form $t_n = r^n$, for all $n$
- Substitute: $r^n - r^{n-1} - r^{n-2} = 0$
- Factor: $ r^{n-2}(r^2 - r^1 - r^0) = 0$
- Divide by $ r^{n-2}$ give CE: $r^2 - r^1 - r^0 = 0$
- Roots of CE: $(1+\sqrt{5})/2=\phi$ and $(1-\sqrt{5})/2=1-\phi = 1/\phi =
\hat{\phi}$
- Choose $\alpha$ and $\beta$ to meet the initial conditions:
$t_0=0,t_1=1$
- $t_0 = \alpha \phi^0 + \beta \hat{\phi}^0 = 0$
- $t_1 = \alpha \phi^1 + \beta \hat{\phi}^1 = 1$
- Solving gives: $\alpha=1/\sqrt{5}$ and $\beta = -1/\sqrt{5}$
- Thus: general solution is $t_n = \alpha \phi^n + \beta \hat{\phi}^n
= \frac{1}{\sqrt{5}}(\phi^n - \hat{\phi}^n)$
Non-homogenous Linear Recurrence Equations
- General form: $a_0t_n + a_1t_{n-1} + \dots + a_kt_{n-k} = c \ne 0$
- RHS ≠ 0
- General plan:
- Find roots of related homogeneous equation and find alpha and beta.
- Form equation (eg $t_n = \alpha r_1^n + \beta r_2^n$ if distinct roots) and solve for alpha and beta
- Solution to non-homogeneous is based on solution to homogeneous
- We mostly ignore these
Change of Variable
- Substitute, for example, $2^k$ for $n$
- Example: For Fibonacci, number of additions
- $t_n = t_{n-1} + t_{n-2} + 1 = 0$
- $t_n - t_{n-1} - t_{n-2} = 1 $
- $t_n - t_{n-1} - t_{n-2} -1 = 0 $
- Substitute: $u_n = t_n + 1$
- We mostly ignore these
Recursion Tree - Intuition for Master Method
Recursion Tree
- A Recursion Tree is a technique for calculating the amount of work expressed by a recurrence equation
- Nice illustration of a recurrence
- Gives intuition behind Master Methods
- Each level of the tree shows the non-recursive work for a given parameter
value
- Write each node with two parts:
- Upper part: $T(s)$ for some $s$
- Lower part: non-recursive part of $T(s)$
- Relation between parts: Upper part = lower part + upper parts of children
Example Recursion Tree
- Consider $T(n) = 2T(\frac{n}{3}) + 5n$
- $T(0) = 7$
- These values don't represent a specific algorithm.
They are just chosen for illustration.
- First rewrite as $T(s) = 5s + 2T(\frac{s}{3})$
- Putting non-recursive term first is (slightly) easier
- Using a different variable allows us to focus on calculating $T(n)$
- Now calculate $T(27) = T(3^3)$:
- Root is $T(27)$:
- Upper part: $T(27)$
- Lower part: $5s=5\times 27 = 135$
- Children of root:
- How many children?
- What's in each child?
- Calculating $T(27)$:
$\begin{align*}
T(27) & = 5\times 27 + 2 \times T(27/3) \\
& = 135 + 2 \times T(27/3) \\
& = 135 + 2 \times [45 + 2 \times T(9/3)] \\
& = 135 + 2 \times [45 + 2 \times [15 + 2 \times T(3/3)]] \\
& = 135 + 2 \times [45 + 2 \times [15 + 2 \times 7]] \\
& = 135 + 2 \times [45 + 2 \times 29] \\
& = 135 + 2 \times 103 \\
& = 135 + 206 \\
& = 341
\end{align*}
$
Example Recursion Tree
- Consider $T(s) = 2T(\frac{s}{3}) + 5s = 5s + 2T(\frac{s}{3}) $
- Calculate $T(27)$
- Where do 2, 3, log3, log3 27 come from?
Tree Properties
- Consider $T(s) = 5s + 2T(\frac{s}{3}) $
- Bottom level is level $\log_3 27$ = level 3
- Number of levels: $ 1 + \log_3 27 = 1 + 3 = 4$
- Sum of level 0: $2^0 \times (5\times 27) = 135$
- Sum of level 1: $2^1 \times (5\times 9) = 90$
- Sum of level 2: $2^2 \times (5\times 3) = 60$
- Sum of bottom level: $2^{\log_3 27} * 7 = 2^3 \times 7 = 56$
- Sum of all levels: $135 + 90 + 60 + 56 = 341$
Recursion Tree for $T(n) = aT(\frac{n}{b}) + f(n)$
- Recurrence Equation: $T(s) = f(s) + a T(s/b)$
- Find $T(n)$
Summing the Values in the Tree
- Recurrence Equation: $T(s) = f(s) + a T(s/b)$
- Bottom level is level $\log_b n$
- Number of levels: $ 1 + \log_b n$
- Sum of all levels except leaves: cost of dividing and joining
- Sum of level 0: $a^0 \times f(\frac{n}{b^0}) $
- Sum of level 1: $a^1 \times f(\frac{n}{b^1}) $
- Sum of level 2: $a^2 \times f(\frac{n}{b^2}) $
- Sum of level i: $a^i \times f(\frac{n}{b^i}) $
- Sum of all levels except leaves: $\sum_{i=0}^{\log_b n - 1} a^i f(\frac{n}{b^i})$
- Sum of bottom level: cost of solving all problems of size 1
- $d a^{log_b n} = d n^{log_b a} $
- Sum of all levels:
$$
\sum_{i=0}^{\log_b n - 1} a^i f(\frac{n}{b^i}) + d n^{log_b a}
$$
Fine Points
- $a ≥ 1, b > 1, \text{ are constants}$
- If $n$ is not a power of $b$, then replace $n/b$ by $\lfloor n/b \rfloor$ (or
$\lceil n/b \rceil$)
- It can be proved that this does not affect the complexity
Evaluating the Complexity of the Sum of the Tree Levels
- Sum of all levels:
$\displaystyle \sum_{i=0}^{\log_b n - 1} a^i f(\frac{n}{b^i}) + \Theta(n^{log_b a})
$
- Simplify base case of recursion using $\Theta(1)$ instead of $d$
- When this formula is a polynomial in $n$, we can simplify it by
considering the power of $n$ in each term:
- Either first or second term can dominate or they can be the same order
- Intuitively:
- $\Theta(f(n))$, if the first term (ie the sum of non-leaf levels) dominates (Case 1)
- $\Theta(n^{log_b a}\lg n )$, if neither term dominates
- $\Theta(n^{log_b a})$, if the second term (ie sum of leaves) dominates (Case 3)
- Some restrictions occur
- These results can be formalized to obtain the General Master Method
Formalizing Gives the General Master Method
- To formalize, we consider the relation
between the terms of the sum of the levels.
- Sum of all levels:
$\displaystyle
\sum_{i=0}^{\log_b n - 1} a^i f(\frac{n}{b^i})
+
\Theta(n^{log_b a})
$
- Formal definition of one term dominating is below (assume $0 < \epsilon$):
- $\Theta(f(n))$, if $f(n) = \Omega(n^{log_b a+\epsilon})$ and $af(n/b) ≤ cf(n)$ for some constant $c ≤ 1$ and large $n$ [ie Case 1:
second term (sum of non-leaves) dominates]
- $\Theta(n^{log_b a}\lg n )$, if $f(n) = \Theta(n^{log_b a})$ [ie neither term dominates]
- $\Theta(n^{log_b a})$, if $f(n) = O(n^{log_b a-\epsilon})$ [ie Case 3: second term (sum of leaves) dominates]
- This can also be written as $n^{log_b a+\epsilon} = O(f(n))$
- We ignore this case
- These results give the general Master Method