# Recursive Algorithms and Recurrence Equations

## Closed Form

### Overview

• Performance of recursive algorithms typically specified with recurrence equations

• Recurrence Equations aka Recurrence and Recurrence Relations
• Recurrence relations have specifically to do with sequences (eg Fibonacci Numbers)

• Recurrence equations require special techniques for solving

• We will focus on induction and the Master Method (and its variants)
• And touch on other methods

### Analyzing Performance of Non-Recursive Routines is (relatively) Easy

• Loop: T(n) = $\Theta(n)$
•     for i in 1 .. n loop

• Loop: T(n) = $\Theta(n^2)$
•     for i in 1 .. n loop
for j in 1 .. n loop
end loop;
end loop;

for i in 1 .. n loop
for j in 1 .. i loop
end loop;
end loop;

• Loop: T(n) = $\Theta(n^3)$ ... obvious

### Analyzing Recursive Routines

• Analysis of recursive routines is not as easy: consider factorial
•     fac(n) is
if n = 1 then return 1
else return fac(n-1) * 1

• How many times is fac called for fac(n)?

• To find an answer, Use a recurrence

### Recurrences

• A recurrence defines T(n) in terms of T for smaller values

• Example: T(n) = T(n-1) + 1
• T(n) is defined in terms of T(n-1)

• Recurrences are used in analyzing recursive algorithms

• AKA: Recurrence Equation, Recurrence Relation

### Evaluating a Recurrence

• How to think about T(n) = T(n-1) + 1

• How to find the value of a T(k) for a particular k:
• Substitute up from T(1) to T(k)
• Substitute down from T(k) to T(1)

• Solving the recurrence and evaluate the resulting expression

• All three methods require having the initial conditions for the recurrence

### Initial Conditions

• The initial conditions are the values of the recurrence for small values of n

• For example, the values of T(0), T(1), T(2)

• We will see that the initial conditions are determined by the specific problem being solved
• A Recurrence Equation has multiple solutions,
• The initial conditions determines which of those solutions applies.

### Substituting Up and Down

• Problem: Find value of T(n) = T(n-1) + 1 for n=4, with initial condition T(1)=2

• Substituting up from T(1):
• T(1) = 2, Initial condition
• T(2) = T(1) + 1 = 2+1 = 3
• T(3) = T(2) + 1 = 3+1 = 4
• T(4) = T(3) + 1 = 4+1 = 5

• Subsituting down from T(4):
• Example: T(4) = T(3) + 1 = [T(2) + 1] + 1 = [[T(1) + 1] + 1] + 1 = 2+1+1+1 = 5

### Solving the Recurrence: Closed Forms

• To solve a recurrence, we find a closed form for it
• Closed form for T(n): An equation that defines T(n) using an expression that does not involve T
• Example: A closed form for T(n) = T(n-1)+1 is T(n) = n.

• Solution techniques - no single method works for all:
• Guess and Check
• Forward substitution
• Backward Substitution
• Characteristic equation

• Master method (2 versions)
• Recurrence trees help us think about recurrences and show intuition in Master Method

## Forward and Backward Substitution, Initial Conditions

### Guess and Check: Forward Substitution

• Recurrence: T(n) = T(n-1) + 1, with initial condition t(1) = 2
• Look for a pattern:
• T(1) = 2, Initial condition
• T(2) = T(1) + 1 = 2+1 = 3
• T(3) = T(2) + 1 = 3+1 = 4
• T(4) = T(3) + 1 = 4+1 = 5
• T(5) = T(4) + 1 = 5+1 = 6
• Guess:
• T(n) = n + 1

• Informal Check:
• T(n) = T(n-1)+1 = [(n-1)+1] + 1 = n+1
• T(1) = 1+1 = 2

• Informally: a closed form is correct if it satisfies the open form and the initial conditions

• Formally: Prove the closed form is correct using induction (example below)

### Guess and Check: Backward Substitution

• Look for a pattern (how many 1's in each line?):
• T(n) = T(n-1) + 1
• T(n) = [T(n-2)+1] + 1
• T(n) = [[T(n-3)+1]+1] + 1
• T(n) = [[[T(n-4)+1]+1]+1] + 1
• ...
• T(n) = [...[[T(n-k)+1]+1] ... +1] + 1 [has k ones]
• ... [Let k = n-1]
• T(n) = [...[[T(n-(n-1))+1]+1] ... +1] + 1 [has k=n-1 ones]
• T(n) = T(n-(n-1)) + (n - 1)
• T(n) = T(1) + (n-1)
• T(n) = 2 + (n - 1)
• T(n) = n + 1
• This process leads to this Guess:
• T(n) = n + 1

• Check the guess, by substituting, as in the previous example:
• T(n) = T(n-1)+1 = [(n-1)+1] + 1 = n+1
• T(1) = 1+1 = 2

### Recurrences Without Initial Conditions

• A recurrence without initial conditions is satisfied by a family of closed forms

• Example: Recurrence T(n) = T(n-1) + 1 is satisfied by each member of this family of closed forms:
• T(n) = n
• T(n) = n + 1
• T(n) = n + 2
• ...
• T(n) = n - 1
• T(n) = n - 2
• ...

• With specific initial conditions, only one of that family satisfies both the recurrence and the initial conditions

• Example: Recurrence T(n) = T(n-1) + 1 with Initial Condition T(1) = 1 is satisfied by:
• T(n) = n
• Example: Recurrence T(n) = T(n-1) + 1 with Initial Condition T(1) = 0 is satisfied by:
• T(n) = n - 1

### Initial Conditions and Solutions - More Examples

• The solution to a recurrence is determined by its initial condition
• Example 1:
• T(n) = T(n-1) + 1
• T(1) = 2
• Guess: T(n) = n + 1
• Check: as above
• T(n) = T(n-1)+1 = [n-1+1] + 1 = n+1
• T(1) = 1+1 = 2

• Example 2:
• T(n) = T(n-1) + 1
• T(1) = 1
• Guess: T(n) = n
• Check, as above:
• T(n) = T(n-1)+1 = [n-1] + 1 = n
• T(1) = 1

• Initial conditions restrict number of solutions
• In this case, restricts to one condition

## Recurrence Equations for Sample Algorithms

### Performance of Recursive Routines

• Let's develop recurrences for some recursive algorithms:
• Factorial
• Fibonacci
• Binary Search
• Towers of Hanoi

• What shall we count: number of calls, number of multiplies, ...

### Performance of Factorial

• Algorithm:
•     fac(n)
if n = 1 return 1
else return n * fac(n-1)

• For fac(n), how many times is fac called?
• T(n) = T(n-1) + 1
• T(1) = 1

• $T(n) = n$

• Whatever the number of calls is for n-1 is, the calls for n is one larger

### Performance of Factorial, revisited

• For fac(n), how many multiplies?
• T(n) = T(n-1) + 1
• T(1) = 0

• Guess: T(n) = ???
• Check: ...

### Performance of Fibonacci

• Algorithm:
•     fib(n)
if n in 1 .. 2 return 1
else return fib(n-1) + fib(n-2)

• For fib(n), how many times is fib called?
• $T(1) = 1$
• $T(2) = 1$
• $T(3) = T(1) + T(2) + 1 = 1 + 1 + 1 = 3$
• $T(4) = T(3) + T(2) + 1 = 3 + 1 + 1 = 5$
• $T(5) = T(4) + T(3) + 1 = 5 + 3 + 1 = 9$

• $T(n) = T(n-1) + T(n-2) + 1$

• $T(n) = ?$ [This one's not so easy]
• The text shows that this has exponential behavior
• Book also uses T(0) = 0 as an initial condition.

### Performance of Recursive Binary Search

• Algorithm (n?, count?):
•     index binsearch(index low, high, keytype S[], keytype x)
if low ≤ high then
mid = (low + high) / 2
if x = S[mid] then
return mid
elsif x < s[mid] then
return binsearch(n, low, mid-1, S, x)
else
return binsearch(n, mid+1, high, S, x)
else
return 0
end binsearch

• For binsearch(n), how many times is binsearch called in the worst case?

• T(0) = 1
• T(1) = 2
• T(2) = T(1) + 1 = 3
• T(4) = T(2) + 1 = 4
• T(8) = T(4) + 1 = 4 + 1 = 5

• T(n) = T(n/2) + 1

• Guess: $T(n) = \lg n + 2$ for $n=2^k$
• Informal Check:
• \begin{align*} T(n) & = T(n/2) + 1 \\ & = (\lg \frac{n}{2} + 2) + 1 \\ & = (\lg n - \lg 2 + 2) + 1 \\ & = (\lg n - 1 + 2) + 1 \\ & = \lg n + 2 \end{align*}
• We can prove $T(n) = \lg n + 2$ by induction ...
• $T(n) = \Theta(\lg n)$
• What do we assume about n?

### Towers of Hanoi

• Recursive algorithm
• Recurrence Equation: T(n) = ...
• Closed form: ...

### More Example Algorithms and their Recurrence Equations

• More Eamples - Let's review or figure out these:
• Factorial (Every Case):

• Fibonacci (Every Case):

• Binary Search (Worst Case):

• Quick Sort (Worst Case):

• Quick Sort(Best Case):

• Merge Sort(Every Case):

### More Example Algorithms and their Recurrence Equations

• More Eamples - Let's review or figure out these:
• Factorial (Every Case): $T(n) = T(n-1) + 1$

• Fibonacci (Every Case): $T(n) = T(n-1) + T(n-2) + 1$

• Binary Search (Worst Case): $T(n) = T(n/2) + 1$

• Quick Sort (Worst Case): $T(n) = T(n-1) + \Theta(n)$

• Quick Sort(Best Case): $T(n) = 2T(\frac{n}{2}) + \Theta(n)$

• Merge Sort(Every Case): $T(n) = 2T(\frac{n}{2}) + \Theta(n)$

• General Forms:
• $T(n) = aT(\frac{n}{b}) + f(n)$
• $T(n) = aT(n - b) + f(n)$

## Technical Issues and Θ on RHS

### Technical Issues

• Floors and Ceilings
• Usually Ignore
• Or assume size is power of 2

• Initial (or Boundary) conditions
• Frequently we use asymptotic values
• Example: $T(1) = \Theta(1)$

• Exact vs asymptotic functions
• Usually asymptotic is good enough
• Finding exact solutions requires exact boundary conditions

### $\Theta(n)$ on the RHS

• In $T(n) = T(n-1) + \Theta(n)$, the $\Theta(n)$ terms refers to some function in $\Theta(n)$
• That is, $f(n) \in \Theta(n)$

• In most cases, the exact function does not affect the asymptotic behavior of $T(n)$

• This allows us to simplify by ignoring the exact function

## Solution Techniques

### Recurrence Equations - Solution Techniques

• No simple way to solve all recurrence equations

• Following techniques are used:
• Guess a solution and use induction to prove its correctness
• Use Forward and Backward Substitution to guess, if needed

• Use a general formula (ie the Master Method)
• For $T(n) = aT(\frac{n}{b}) + cn^k$ [Version in Text]

• For $T(n) = aT(\frac{n}{b}) + f(n)$ [More general version, NIB, but in CLR

• Solve using Characteristic Equation
• Linear homogeneous equations with constant coefficients
• Non-linear homogeneous equations with constant coefficients

• Change of Variable

• We focus on the general formulae and touch on the others
• General formulae can be understood using recursion trees
• First we see an example of induction

## Checking Recurrence Closed Forms Using Induction

### Check Using Induction: Factorial [SKIP]

• For fac(n), how many times is fac called (every case)?
• Above we derived:
• T(1) = 1
• T(k) = T(k - 1) + 1

• T(n) = n

• Inductive proof that $T(n) = n \text{ is a solution of }T(n) = T(n-1) + 1$, with initial condition T(1)=1:
1. Base Case: Prove $T(n) = n \text{ is a solution }$ for n = 1. T(1) = 1 is an initial condition.
2. Inductive Step: Prove: For all $n \gt 1, T(n) = n \text{ is a solution} \implies T(n+1) = n+1$ is a solution
• Inductive Hypothesis: $T(n) = n$ is a solution
• Now consider T(n) + 1:
• \begin{align*} T(n+1) & = T((n+1)-1) + 1 \\ & = T(n) + 1 \\ & = n + 1\textrm{, by IH} \\ \end{align*}
• This proves the Inductive Step
3. Therefore from 1 and 2, by induction, we conclude that $T(n) = n \text{ for all } n ≥ 1$

### Example Using Induction: Binary Search [SKIP]

• Worst case number of calls of binary search (assume n is a power of 2
• Recurrence: $T(n) = T(n/2) + 1 \text{ with } T(1)=2$ [Derived above]

• Closed form: $T(n) = \lg n + 2$ [Guessed above.]

• Inductive proof that closed form is solution for recurrence (assume powers of 2):
1. Base Case: Prove for 1: $T(1) = \lg 1 + 2 = 0 + 2 = 2$.
2. Inductive Step: Prove: that $T(k) = \lg k + 2, \textrm{ for all } k < n$
3. Inductive Step: Prove: For all $n \gt 1, T(n) = \lg n + 2, \text{ is a solution} \implies T(2n) = \lg 2n +2 = \lg n +3$ is a solution
• IH: Assume that $T(k) = \lg k + 2, \textrm{ for all } k < n$
• Now consider $T(2n)$:
• \begin{align*} T(2n) & = T(2n/2) + 1\text{, by recurrence} \\ & = T(n) + 1 \\ & = (\lg n + 2) + 1, \text{ by IH} \\ & = \lg n + 3 \end{align*}
• This proves the Inductive Step

4. Therefore from 1 and 2 we conclude that $T(n) =\lg n +2 \text{ is a solution to the recurrence for all } n ≥ 1$, where n is a power of 2
• Can also be proved for non-powers of 2

## Master Method

### Master Method from Text

• Cookbook way of solving recurrences of this form:
• $T(n) = aT(\frac{n}{b}) + \Theta(n^d)$
• $T(1) = c$

• $n = b^k$, for positive integer $k$
• $a \ge 1, b \ge 2, c \gt 0, d \ge 0$ are constants

• $T(n) \in \begin{cases} \Theta(n^d), & a < b^d \\ \Theta(n^d\lg n), & a = b^d \\ \Theta(n^{\log_b a)}, & a > b^d \end{cases}$

• We call these Case 1, Case 2, and Case 3.

• Intuition: Which term of recurrence dominates (ie contributes more)

• In the recurrence we can replace "T(n)=" with "T(n)≤" or "T(n)≥" and get $O$ or $\Omega$ performance

### Examples using Master Method from Text

• $T(n) = 16T(n/4) + 5n^3$
• $a=16, b=4, d=3$
• 16 < 43 and so Case 1
• Thus, $T(n) = \Theta(n^d) = \Theta(n^3)$

• $T(n) = 2T(n/2) + n$
• $a=2, b=2, d=1$
• 2 = 21 and so Case 2
• Thus, $T(n) = \Theta(n^1\lg n)$

• $T(n) = 8T(n/2) + n$
• $a=8, b=2, d=1$,
• 8 > 21 and so Case 3
• Thus, $T(n) = \Theta(n^{\log_b a}) = \Theta(n^{\lg 8}) = \Theta(n^3)$

### General Master Method [Not In Book]

• Solve $T(n) = aT(\frac{n}{b}) + f(n)$
• The MM in the text is a special case version of this one
• Text's MM's 3 cases are special cases of the GMM 3 cases

• $T(n) \in \begin{cases} \Theta(f(n)), & f(n) = \Omega(n^{log_b a+\epsilon}) \text{ and } af(n/b) ≤ cf(n) \text{ for some constant } c ≤ 1 \text{ and large } n \\ \Theta(n^{\log_b a}\lg n), & f(n) = \Theta(n^{log_b a})\\ \Theta(n^{\log_b a)}, & f(n) = O(n^{log_b a-\epsilon}) \end{cases}$

• Assume $0 < \epsilon$

• Case 1 can also be written as $n^{log_b a+\epsilon} = O(f(n))$
• We ignore Case 1 because of its more complex condition

### Examples

• $T(n) = 2T(n/2) + n$
• $a=2, b=2, f(n)=n, \log_b a = \lg 2 = 1$
• Case 2: $f(n) = n = n^1 = \Theta(n^{\log_b a})$
• Thus, $T(n) = \Theta(n^{\log_b a}\lg n) = \Theta(n \lg n)$

• $T(n) = 8T(n/2) + n$
• $a=8, b=2, f(n)=n, \log_b a = \log_2 8 = 3$
• Case 3: $f(n) = n = O(n^{3-1})$, with $\epsilon=1$
• Thus, $T(n) = \Theta(n^{\log_b a}) = \Theta(n^{\lg 8}) = \Theta(n^3)$

### Text's Master Method vs General MM

• The text's Master Method is a special case of the general Master Method:

• Assume $T(n) = aT(n/k) + \Theta(n^d)$ meets the conditions for the Text's Case 2, ie $a = b^k$.
• Taking log of both sides of $a = b^d$ gives $log_b a = log_b b^d = d$.
• Thus, $f(n) \in \Theta( n^d)$ means $cn^{log_b a} \in \Theta(n^{log_b a})$ which is the condition for Case 2 in the General Master Method.
• Therefore, if a recurrence meets Case 2 of the Text's MM, then it also meets Case 2 of the General MM.
• Similar arguments can be made for the other cases.

## Other Solution Techniques

### Other Methods for Solving Recurrence Equations

• Characteristic Equation
• Linear homogeneous equations with constant coefficients
• Non-linear homogeneous equations with constant coefficients

• Change of Variable

• Substitution

• We look at briefly at the first and ignore the second two

## Characteristic Equations

### Homogenous Linear Recurrence Equations

• General form: $a_0t_n + a_1t_{n-1} + \dots + a_kt_{n-k} = 0$
• Slightly different from previous recurrence equations:
• Notation: $t_n$ vs $T(n)$
• all $t$ terms on LHS of equation
• 0 on RHS of equation

• Overview of how to solve:
• Assume $t_n = r^n$ is a solution for all $n$ and for some real number r

• Generate Characteristic Equation:
• Recurrence: $a_0t_n + a_1t_{n-1} + \dots + a_kt_{n-k} = 0$
• Substitute $r^n$ for $t_n$: $a_0r^n + a_1r^{n-1} + \dots + a_kr^{n-k} = 0$
• Factor out $r^{n-k}$: $(r^{n-k})(a_0r^k + a_1r^{k-1} + \dots + a_kr^{k-k}) = 0$
• Divide by $r^{n-k}$ (since it gives a root of r=0, which is not useful)
• Resulting Characteristic Equation: $a_0r^k + a_1r^{k-1} + \dots + a_kr^{k-k} = 0$

• Find roots of Characteristic Equation

• Use roots of Characteristic Equation to form recurrence solution

### Homogenous Linear Recurrence Equation: Example

• Notation: $t_n$ vs $T(n)$

• Recurrence:
• $t_n - 5t_{n-1} + 6t_{n-2} = 0$
• $t_0 = 0$
• $t_1 = 1$

• Assume $t_n = r^n$ is a solution, for all $n$ and for some real number r

• Substitute: $r^n - 5r^{n-1} + 6r^{n-2} = 0$

• Factor: $r^{n-2}(r^2 - 5r^1 + 6r^0) = 0$

• Divide by $r^{n-2}$ (since it gives a root of r=0, which is not useful)
• Resulting Characteristic Equation (CE): $r^2 - 5r^1 + 6r^0 = 0$

• Find roots of CE: $r^2 - 5r^1 + 6r^0 = (r-3)(r-2) = 0$
• Roots are 3 and 2
• That is, $r^2 - 5r^1 + 6r^0 = 0$ when $r_1=3$ or $r_2=2$

• By assumption, $t_n=3^n$ and $t_n=2^n$, are both solutions to the RE (and $t_n=0$, which we ignore).

• (Can verify that T(n) = 5T(-1) -6T(n-2) holds for each solution.)
• Since $t_n=3^n$ and $t_n=2^n$, are both solutions, so is the linear combination: $t_n = \alpha r_1^n + \beta r_2^n= \alpha 3^n + \beta 2^n$
• Plugging general solution into recurrence and distributing will give 0
• General solution defines an infinite number of solutions
• Use initial conditions to find a single solution
• If roots are identical, use this equation: $t_n = \alpha r^n + \beta nr^n$

• Choose $\alpha$ and $\beta$ to meet the initial conditions: $t_0=0,t_1=1$
• $t_0 = \alpha 3^0 + \beta 2^0 = 0$
• $t_1 = \alpha 3^1 + \beta 2^1 = 1$
• Solving gives: $\alpha=1$ and $\beta = -1$

• Thus: general solution is $t_n = \alpha 3^n + \beta 2^n== 3^n - 2^n$

• Recurrence:
• $T(n) = T(n-1) + T(n-2) •$t_n - t_{n-1} - t_{n-2} = 0$•$t_0 = 0$•$t_1 = 1$• Assume solution is of form$t_n = r^n$, for all$n$• Substitute:$r^n - r^{n-1} - r^{n-2} = 0$• Factor:$ r^{n-2}(r^2 - r^1 - r^0) = 0$• Divide by$ r^{n-2}$give CE:$r^2 - r^1 - r^0 = 0$• Roots of CE:$(1+\sqrt{5})/2=\phi$and$(1-\sqrt{5})/2=1-\phi = 1/\phi = \hat{\phi}$• Choose$\alpha$and$\beta$to meet the initial conditions:$t_0=0,t_1=1$•$t_0 = \alpha \phi^0 + \beta \hat{\phi}^0 = 0$•$t_1 = \alpha \phi^1 + \beta \hat{\phi}^1 = 1$• Solving gives:$\alpha=1/\sqrt{5}$and$\beta = -1/\sqrt{5}$• Thus: general solution is$t_n = \alpha \phi^n + \beta \hat{\phi}^n = \frac{1}{\sqrt{5}}(\phi^n - \hat{\phi}^n)$### Non-homogenous Linear Recurrence Equations • General form:$a_0t_n + a_1t_{n-1} + \dots + a_kt_{n-k} = c \ne 0$• RHS ≠ 0 • General plan: • Find roots of related homogeneous equation and find alpha and beta. • Form equation (eg$t_n = \alpha r_1^n + \beta r_2^n$if distinct roots) and solve for alpha and beta • Solution to non-homogeneous is based on solution to homogeneous • We mostly ignore these ### Change of Variable • Substitute, for example,$2^k$for$n$• Example: For Fibonacci, number of additions •$t_n = t_{n-1} + t_{n-2} + 1 = 0$•$t_n - t_{n-1} - t_{n-2} = 1 $•$t_n - t_{n-1} - t_{n-2} -1 = 0 $• Substitute:$u_n = t_n + 1$• We mostly ignore these ## Recursion Tree - Intuition for Master Method ### Recursion Tree • A Recursion Tree is a technique for calculating the amount of work expressed by a recurrence equation • Nice illustration of a recurrence • Gives intuition behind Master Methods • Each level of the tree shows the non-recursive work for a given parameter value • See diagram • Write each node with two parts: • Upper part:$T(s)$for some$s$• Lower part: non-recursive part of$T(s)$• Relation between parts: Upper part = lower part + upper parts of children ### Example Recursion Tree • Consider$T(n) = 2T(\frac{n}{3}) + 5n$•$T(0) = 7$• These values don't represent a specific algorithm. They are just chosen for illustration. • First rewrite as$T(s) = 5s + 2T(\frac{s}{3})$• Putting non-recursive term first is (slightly) easier • Using a different variable allows us to focus on calculating$T(n)$• Now calculate$T(27) = T(3^3)$: • Root is$T(27)$: • Upper part:$T(27)$• Lower part:$5s=5\times 27 = 135$• Children of root: • How many children? • What's in each child? • Calculating$T(27)$: •$\begin{align*} T(27) & = 5\times 27 + 2 \times T(27/3) \\ & = 135 + 2 \times T(27/3) \\ & = 135 + 2 \times [45 + 2 \times T(9/3)] \\ & = 135 + 2 \times [45 + 2 \times [15 + 2 \times T(3/3)]] \\ & = 135 + 2 \times [45 + 2 \times [15 + 2 \times 7]] \\ & = 135 + 2 \times [45 + 2 \times 29] \\ & = 135 + 2 \times 103 \\ & = 135 + 206 \\ & = 341 \end{align*} $### Example Recursion Tree • Consider$T(s) = 2T(\frac{s}{3}) + 5s = 5s + 2T(\frac{s}{3}) $• Calculate$T(27)$• Where do 2, 3, log3, log3 27 come from? ### Tree Properties • Consider$T(s) = 5s + 2T(\frac{s}{3}) $• Bottom level is level$\log_3 27$= level 3 • Number of levels:$ 1 + \log_3 27 = 1 + 3 = 4$• Sum of level 0:$2^0 \times (5\times 27) = 135$• Sum of level 1:$2^1 \times (5\times 9) = 90$• Sum of level 2:$2^2 \times (5\times 3) = 60$• Sum of bottom level:$2^{\log_3 27} * 7 = 2^3 \times 7 = 56$• Sum of all levels:$135 + 90 + 60 + 56 = 341$### Recursion Tree for$T(n) = aT(\frac{n}{b}) + f(n)$• Recurrence Equation:$T(s) = f(s) + a T(s/b)$•$T(1) = d$• Find$T(n)$### Summing the Values in the Tree • Recurrence Equation:$T(s) = f(s) + a T(s/b)$•$f(0) = d$• Bottom level is level$\log_b n$• Number of levels:$ 1 + \log_b n$• Sum of all levels except leaves: cost of dividing and joining • Sum of level 0:$a^0 \times f(\frac{n}{b^0}) $• Sum of level 1:$a^1 \times f(\frac{n}{b^1}) $• Sum of level 2:$a^2 \times f(\frac{n}{b^2}) $• Sum of level i:$a^i \times f(\frac{n}{b^i}) $• Sum of all levels except leaves:$\sum_{i=0}^{\log_b n - 1} a^i f(\frac{n}{b^i})$• Sum of bottom level: cost of solving all problems of size 1 •$d a^{log_b n} = d n^{log_b a} $• Sum of all levels: • $$\sum_{i=0}^{\log_b n - 1} a^i f(\frac{n}{b^i}) + d n^{log_b a}$$ ### Fine Points •$a ≥ 1, b > 1, \text{ are constants}$• If$n$is not a power of$b$, then replace$n/b$by$\lfloor n/b \rfloor$(or$\lceil n/b \rceil$) • It can be proved that this does not affect the complexity ### Evaluating the Complexity of the Sum of the Tree Levels • Sum of all levels:$\displaystyle \sum_{i=0}^{\log_b n - 1} a^i f(\frac{n}{b^i}) + \Theta(n^{log_b a}) $• Simplify base case of recursion using$\Theta(1)$instead of$d$• When this formula is a polynomial in$n$, we can simplify it by considering the power of$n$in each term: • Either first or second term can dominate or they can be the same order • Intuitively: •$\Theta(f(n))$, if the first term (ie the sum of non-leaf levels) dominates (Case 1) •$\Theta(n^{log_b a}\lg n )$, if neither term dominates •$\Theta(n^{log_b a})$, if the second term (ie sum of leaves) dominates (Case 3) • Some restrictions occur • These results can be formalized to obtain the General Master Method ### Formalizing Gives the General Master Method • To formalize, we consider the relation between the terms of the sum of the levels. • Sum of all levels:$\displaystyle \sum_{i=0}^{\log_b n - 1} a^i f(\frac{n}{b^i}) + \Theta(n^{log_b a}) $• Formal definition of one term dominating is below (assume$0 < \epsilon$): •$\Theta(f(n))$, if$f(n) = \Omega(n^{log_b a+\epsilon})$and$af(n/b) ≤ cf(n)$for some constant$c ≤ 1$and large$n$[ie Case 1: second term (sum of non-leaves) dominates] •$\Theta(n^{log_b a}\lg n )$, if$f(n) = \Theta(n^{log_b a})$[ie neither term dominates] •$\Theta(n^{log_b a})$, if$f(n) = O(n^{log_b a-\epsilon})$[ie Case 3: second term (sum of leaves) dominates] • This can also be written as$n^{log_b a+\epsilon} = O(f(n))\$
• We ignore this case

• These results give the general Master Method