Recursive Algorithms and Recurrence Equations
Introduction
Initial Conditions
Up and Down Substitution
Closed Form
Overview
 Performance of recursive algorithms typically specified with
recurrence equations
 Recurrence Equations aka Recurrence and Recurrence Relations
 Recurrence relations have specifically to do with sequences (eg Fibonacci Numbers)
 Recurrence equations require special techniques for solving
 We will focus on induction and the Master Method (and its variants)
 And touch on other methods
Analyzing Performance of NonRecursive Routines is (relatively) Easy
for i in 1 .. n loop
Loop: T(n) = $\Theta(n^2)$
for i in 1 .. n loop
for j in 1 .. n loop
end loop;
end loop;
for i in 1 .. n loop
for j in 1 .. i loop
end loop;
end loop;
Loop: T(n) = $\Theta(n^3)$ ... obvious
Analyzing Recursive Routines
 Analysis of recursive routines is not as easy: consider factorial
fac(n) is
if n = 1 then return 1
else return fac(n1) * 1
How many times is fac called for fac(n)?
To find an answer, Use a recurrence
Recurrences
 A recurrence defines T(n) in terms of T for smaller values
 Example: T(n) = T(n1) + 1
 T(n) is defined in terms of T(n1)
 Recurrences are used in analyzing recursive algorithms
 AKA: Recurrence Equation, Recurrence Relation
Evaluating a Recurrence
 How to think about T(n) = T(n1) + 1
 How to find the value of a T(k) for a particular k:
 Substitute up from T(1) to T(k)
 Substitute down from T(k) to T(1)
 Solving the recurrence and evaluate the resulting expression
 All three methods require having the initial conditions for the recurrence
Initial Conditions
 The initial conditions are the values of the recurrence for small values of n
 For example, the values of T(0), T(1), T(2)
 We will see that the initial conditions are determined by
the specific problem being solved
 A Recurrence Equation has multiple solutions,
 The initial conditions determines which of those solutions applies.
Substituting Up and Down
 Problem: Find value of T(n) = T(n1) + 1 for n=4, with initial condition T(1)=2
 Substituting up from T(1):
 T(1) = 2, Initial condition
 T(2) = T(1) + 1 = 2+1 = 3
 T(3) = T(2) + 1 = 3+1 = 4
 T(4) = T(3) + 1 = 4+1 = 5
 Subsituting down from T(4):
 Example: T(4) = T(3) + 1 = [T(2) + 1] + 1 = [[T(1) + 1] + 1] + 1 = 2+1+1+1 = 5
Solving the Recurrence: Closed Forms
 To solve a recurrence, we find a closed form for it
 Closed form for T(n): An equation that defines T(n) using an expression that does
not involve T
 Example: A closed form for T(n) = T(n1)+1 is T(n) = n.
 Solution techniques  no single method works for all:
 Guess and Check
 Forward substitution
 Backward Substitution
 Characteristic equation
 Master method (2 versions)
 Recurrence trees help us think about recurrences and show intuition in Master Method
Solving RE
Forward and Backward Substitution, Initial Conditions
Guess and Check: Forward Substitution
 Recurrence: T(n) = T(n1) + 1, with initial condition t(1) = 2
 Look for a pattern:
 T(1) = 2, Initial condition
 T(2) = T(1) + 1 = 2+1 = 3
 T(3) = T(2) + 1 = 3+1 = 4
 T(4) = T(3) + 1 = 4+1 = 5
 T(5) = T(4) + 1 = 5+1 = 6
 Guess:
 Informal Check:
 T(n) = T(n1)+1 = [(n1)+1] + 1 = n+1
 T(1) = 1+1 = 2
 Informally: a closed form is correct if it satisfies the open form and
the initial conditions
 Formally: Prove the closed form is correct using induction (example below)
Guess and Check: Backward Substitution
 Look for a pattern (how many 1's in each line?):
 T(n) = T(n1) + 1
 T(n) = [T(n2)+1] + 1
 T(n) = [[T(n3)+1]+1] + 1
 T(n) = [[[T(n4)+1]+1]+1] + 1
 ...
 T(n) = [...[[T(nk)+1]+1] ... +1] + 1 [has k ones]
 ... [Let k = n1]
 T(n) = [...[[T(n(n1))+1]+1] ... +1] + 1 [has k=n1 ones]
 T(n) = T(n(n1)) + (n  1)
 T(n) = T(1) + (n1)
 T(n) = 2 + (n  1)
 T(n) = n + 1
 This process leads to this Guess:
 Check the guess, by substituting, as in the previous example:
 T(n) = T(n1)+1 = [(n1)+1] + 1 = n+1
 T(1) = 1+1 = 2
Recurrences Without Initial Conditions
 A recurrence without initial conditions is satisfied by a family of closed forms
 Example: Recurrence T(n) = T(n1) + 1 is satisfied by each member of this
family of closed forms:
 T(n) = n
 T(n) = n + 1
 T(n) = n + 2
 ...
 T(n) = n  1
 T(n) = n  2
 ...
 With specific initial conditions, only one of that family satisfies both the recurrence
and the initial conditions
 Example: Recurrence T(n) = T(n1) + 1 with Initial Condition T(1) = 1 is satisfied by:
 Example: Recurrence T(n) = T(n1) + 1 with Initial Condition T(1) = 0 is satisfied by:
Initial Conditions and Solutions  More Examples
 The solution to a recurrence is determined by its initial condition
 Example 1:
 T(n) = T(n1) + 1
 T(1) = 2
 Guess: T(n) = n + 1
 Check: as above
 T(n) = T(n1)+1 = [n1+1] + 1 = n+1
 T(1) = 1+1 = 2
 Example 2:
 T(n) = T(n1) + 1
 T(1) = 1
 Guess: T(n) = n
 Check, as above:
 T(n) = T(n1)+1 = [n1] + 1 = n
 T(1) = 1
 Initial conditions restrict number of solutions
 In this case, restricts to one condition
Recurrence Equations for Sample Algorithms
Performance of Recursive Routines
 Let's develop recurrences for some recursive algorithms:
 Factorial
 Fibonacci
 Binary Search
 Towers of Hanoi
 What shall we count: number of calls, number of multiplies, ...
Performance of Factorial
fac(n)
if n = 1 return 1
else return n * fac(n1)
For fac(n), how many times is fac called?
 T(n) = T(n1) + 1
 T(1) = 1
 $T(n) = n$
Whatever the number of calls is for n1 is, the calls for n is one larger
Performance of Factorial, revisited
 For fac(n), how many multiplies?
 T(n) = T(n1) + 1
 T(1) = 0
 Guess: T(n) = ???
 Check: ...
Performance of Fibonacci
fib(n)
if n in 1 .. 2 return 1
else return fib(n1) + fib(n2)
For fib(n), how many times is fib called?
 $T(1) = 1$
 $T(2) = 1$
 $T(3) = T(1) + T(2) + 1 = 1 + 1 + 1 = 3 $
 $T(4) = T(3) + T(2) + 1 = 3 + 1 + 1 = 5 $
 $T(5) = T(4) + T(3) + 1 = 5 + 3 + 1 = 9 $
 $T(n) = T(n1) + T(n2) + 1 $
 $T(n) = ?$ [This one's not so easy]
 The text shows that this has exponential behavior
 Book also uses T(0) = 0 as an initial condition.
Performance of Recursive Binary Search
index binsearch(index low, high, keytype S[], keytype x)
if low ≤ high then
mid = (low + high) / 2
if x = S[mid] then
return mid
elsif x < s[mid] then
return binsearch(n, low, mid1, S, x)
else
return binsearch(n, mid+1, high, S, x)
else
return 0
end binsearch
For binsearch(n), how many times is binsearch called in the worst case?
 T(0) = 1
 T(1) = 2
 T(2) = T(1) + 1 = 3
 T(4) = T(2) + 1 = 4
 T(8) = T(4) + 1 = 4 + 1 = 5
 T(n) = T(n/2) + 1
 Guess: $T(n) = \lg n + 2$ for $n=2^k$
 Informal Check:
$\begin{align*}
T(n) & = T(n/2) + 1 \\
& = (\lg \frac{n}{2} + 2) + 1 \\
& = (\lg n  \lg 2 + 2) + 1 \\
& = (\lg n  1 + 2) + 1 \\
& = \lg n + 2
\end{align*}
$
 We can prove $T(n) = \lg n + 2$ by induction ...
 $T(n) = \Theta(\lg n)$
 What do we assume about n?
Towers of Hanoi
 Recursive algorithm
 Recurrence Equation: T(n) = ...
 Closed form: ...
More Example Algorithms and their Recurrence Equations
 More Eamples  Let's review or figure out these:
 Factorial (Every Case):
 Fibonacci (Every Case):
 Binary Search (Worst Case):
 Quick Sort (Worst Case):
 Quick Sort(Best Case):
 Merge Sort(Every Case):
More Example Algorithms and their Recurrence Equations
 More Eamples  Let's review or figure out these:
 Factorial (Every Case): $T(n) = T(n1) + 1$
 Fibonacci (Every Case): $T(n) = T(n1) + T(n2) + 1$
 Binary Search (Worst Case): $T(n) = T(n/2) + 1$
 Quick Sort (Worst Case): $T(n) = T(n1) + \Theta(n)$
 Quick Sort(Best Case): $T(n) = 2T(\frac{n}{2}) + \Theta(n)$
 Merge Sort(Every Case): $T(n) = 2T(\frac{n}{2}) + \Theta(n)$
 General Forms:
 $T(n) = aT(\frac{n}{b}) + f(n)$
 $T(n) = aT(n  b) + f(n)$
Technical Issues and Θ on RHS
Technical Issues
 Floors and Ceilings
 Usually Ignore
 Or assume size is power of 2
 Initial (or Boundary) conditions
 Frequently we use asymptotic values
 Example: $T(1) = \Theta(1)$
 Exact vs asymptotic functions
 Usually asymptotic is good enough
 Finding exact solutions requires exact boundary conditions
$\Theta(n)$ on the RHS
 In $T(n) = T(n1) + \Theta(n)$, the $\Theta(n)$ terms refers to some function
in $\Theta(n)$
 That is, $f(n) \in \Theta(n)$
 In most cases, the exact function does not affect the asymptotic behavior of $T(n)$
 This allows us to simplify by ignoring the exact function
Solution Techniques
Recurrence Equations  Solution Techniques
 No simple way to solve all recurrence equations
 Following techniques are used:
 Guess a solution and use induction to prove its correctness
 Use Forward and Backward Substitution to guess, if needed
 Use a general formula (ie the Master Method)
 For $T(n) = aT(\frac{n}{b}) + cn^k$ [Version in Text]
 For $T(n) = aT(\frac{n}{b}) + f(n)$ [More general version,
NIB, but in CLR
 Solve using Characteristic Equation
 Linear homogeneous equations with constant coefficients
 Nonlinear homogeneous equations with constant coefficients
 Change of Variable
 We focus on the general formulae and touch on the others
 General formulae can be understood using recursion trees
 First we see an example of induction
Checking Recurrence Closed Forms Using Induction
Check Using Induction: Factorial [SKIP]
 For fac(n), how many times is fac called (every case)?
 Above we derived:
 T(1) = 1
 T(k) = T(k  1) + 1
 T(n) = n
 Inductive proof that $T(n) = n \text{ is a solution of }T(n) = T(n1) + 1$, with initial
condition T(1)=1:
 Base Case: Prove $T(n) = n \text{ is a solution }$ for n = 1. T(1) = 1 is an initial condition.
 Inductive Step: Prove: For all $n \gt 1, T(n) = n \text{ is a solution} \implies T(n+1) =
n+1$ is a solution
 Inductive Hypothesis: $T(n) = n$ is a solution
 Now consider T(n) + 1:
$\begin{align*}
T(n+1) & = T((n+1)1) + 1 \\
& = T(n) + 1 \\
& = n + 1\textrm{, by IH} \\
\end{align*}
$
 This proves the Inductive Step
 Therefore from 1 and 2, by induction, we conclude that $T(n) = n \text{ for all } n ≥ 1$
Example Using Induction: Binary Search [SKIP]
 Worst case number of calls of binary search (assume n is a power of 2
 Recurrence: $T(n) = T(n/2) + 1 \text{ with } T(1)=2 $ [Derived above]
 Closed form: $T(n) = \lg n + 2$ [Guessed above.]
 Inductive proof that closed form is solution for recurrence (assume powers of 2):
 Base Case: Prove for 1: $T(1) = \lg 1 + 2 = 0 + 2 = 2$.
 Inductive Step: Prove: that $T(k) = \lg k + 2, \textrm{ for all } k < n$
 Inductive Step: Prove: For all $n \gt 1, T(n) = \lg n + 2, \text{ is a solution}
\implies T(2n) = \lg 2n +2 = \lg n +3$ is a solution
 IH: Assume that $T(k) = \lg k + 2, \textrm{ for all } k < n$
 Now consider $T(2n)$:
$\begin{align*}
T(2n) & = T(2n/2) + 1\text{, by recurrence} \\
& = T(n) + 1 \\
& = (\lg n + 2) + 1, \text{ by IH} \\
& = \lg n + 3
\end{align*}
$
 This proves the Inductive Step
 Therefore from 1 and 2 we conclude that $T(n) =\lg n +2 \text{ is a solution to the
recurrence for all } n ≥ 1$, where n is a power of 2
 Can also be proved for nonpowers of 2
Master Method
Master Method from Text
 Cookbook way of solving recurrences of this form:
 $T(n) = aT(\frac{n}{b}) + \Theta(n^d)$
 $T(1) = c$
 $n = b^k$, for positive integer $k$
 $a \ge 1, b \ge 2, c \gt 0, d \ge 0$ are constants
 $
T(n) \in
\begin{cases}
\Theta(n^d), & a < b^d \\
\Theta(n^d\lg n), & a = b^d \\
\Theta(n^{\log_b a)}, & a > b^d
\end{cases}
$
 We call these Case 1, Case 2, and Case 3.
 Intuition: Which term of recurrence dominates (ie contributes more)
 In the recurrence we can replace
"T(n)=" with "T(n)≤" or "T(n)≥" and get $O$ or $\Omega$ performance
Examples using Master Method from Text
 $T(n) = 16T(n/4) + 5n^3$
 $a=16, b=4, d=3$
 16 < 4^{3} and so Case 1
 Thus, $T(n) = \Theta(n^d) = \Theta(n^3)$
 $T(n) = 2T(n/2) + n$
 $a=2, b=2, d=1$
 2 = 2^{1} and so Case 2
 Thus, $T(n) = \Theta(n^1\lg n)$
 $T(n) = 8T(n/2) + n$
 $a=8, b=2, d=1$,
 8 > 2^{1} and so Case 3
 Thus, $T(n) = \Theta(n^{\log_b a}) = \Theta(n^{\lg 8}) = \Theta(n^3)$
General Master Method [Not In Book]
 Solve $T(n) = aT(\frac{n}{b}) + f(n)$
 The MM in the text is a special case version of this one
 Text's MM's 3 cases are special cases of the GMM 3 cases
 $
T(n) \in
\begin{cases}
\Theta(f(n)), & f(n) = \Omega(n^{log_b a+\epsilon}) \text{ and } af(n/b) ≤
cf(n) \text{ for some constant } c ≤ 1 \text{ and large } n
\\
\Theta(n^{\log_b a}\lg n), & f(n) = \Theta(n^{log_b a})\\
\Theta(n^{\log_b a)}, & f(n) = O(n^{log_b a\epsilon})
\end{cases}
$
 Assume $0 < \epsilon$
 Case 1 can also be written as $n^{log_b a+\epsilon} = O(f(n))$
 We ignore Case 1 because of its more complex condition
Examples
 $T(n) = 2T(n/2) + n$
 $a=2, b=2, f(n)=n, \log_b a = \lg 2 = 1$
 Case 2: $f(n) = n = n^1 = \Theta(n^{\log_b a})$
 Thus, $T(n) = \Theta(n^{\log_b a}\lg n) = \Theta(n \lg n)$
 $T(n) = 8T(n/2) + n$
 $a=8, b=2, f(n)=n, \log_b a = \log_2 8 = 3$
 Case 3: $f(n) = n = O(n^{31})$, with $\epsilon=1$
 Thus, $T(n) = \Theta(n^{\log_b a}) = \Theta(n^{\lg 8}) = \Theta(n^3)$
Text's Master Method vs General MM
 The text's Master Method is a special case of the general Master Method:
 Assume $T(n) = aT(n/k) + \Theta(n^d)$ meets the conditions for the Text's Case 2, ie $a = b^k$.

Taking log of both sides of $a = b^d$ gives $log_b a = log_b b^d = d$.

Thus, $f(n) \in \Theta( n^d)$ means $cn^{log_b a} \in \Theta(n^{log_b a})$
which is the condition for Case 2 in the General Master Method.

Therefore, if a recurrence meets Case 2 of the Text's MM, then it also meets Case 2 of
the General MM.

Similar arguments can be made for the other cases.
Other Solution Techniques
Other Methods for Solving Recurrence Equations
 Characteristic Equation
 Linear homogeneous equations with constant coefficients
 Nonlinear homogeneous equations with constant coefficients
 Change of Variable
 Substitution
 We look at briefly at the first and ignore the second two
Characteristic Equations
Homogenous Linear Recurrence Equations
 General form: $a_0t_n + a_1t_{n1} + \dots + a_kt_{nk} = 0$
 Slightly different from previous recurrence equations:
 Notation: $t_n$ vs $T(n)$
 all $t$ terms on LHS of equation
 0 on RHS of equation
 Overview of how to solve:
 Assume $t_n = r^n$ is a solution for all $n$ and for some
real number r
 Generate Characteristic Equation:
 Recurrence: $a_0t_n + a_1t_{n1} + \dots + a_kt_{nk} = 0$
 Substitute $r^n$ for $t_n$: $a_0r^n + a_1r^{n1} + \dots + a_kr^{nk} = 0$
 Factor out $r^{nk}$: $(r^{nk})(a_0r^k + a_1r^{k1} + \dots + a_kr^{kk}) = 0$
 Divide by $r^{nk}$ (since it gives a root of r=0, which is not useful)
 Resulting Characteristic Equation: $a_0r^k + a_1r^{k1} + \dots + a_kr^{kk} = 0$
 Find roots of Characteristic Equation
 Use roots of Characteristic Equation to form recurrence solution
Homogenous Linear Recurrence Equation: Example
 Notation: $t_n$ vs $T(n)$
 Recurrence:
 $t_n  5t_{n1} + 6t_{n2} = 0$
 $t_0 = 0$
 $t_1 = 1$
 Assume $t_n = r^n$ is a solution, for all $n$ and for some
real number r
 Substitute: $r^n  5r^{n1} + 6r^{n2} = 0$
 Factor: $ r^{n2}(r^2  5r^1 + 6r^0) = 0$
 Divide by $ r^{n2}$ (since it gives a root of r=0, which is not useful)
 Resulting Characteristic Equation (CE): $r^2  5r^1 + 6r^0 = 0$
 Find roots of CE: $r^2  5r^1 + 6r^0 = (r3)(r2) = 0$
 Roots are 3 and 2
 That is, $r^2  5r^1 + 6r^0 = 0$ when $r_1=3$ or $r_2=2$
 By assumption, $t_n=3^n$ and $t_n=2^n$, are both solutions to the RE (and $t_n=0$, which we ignore).
 (Can verify that T(n) = 5T(1) 6T(n2) holds for each solution.)
 Since $t_n=3^n$ and $t_n=2^n$, are both solutions, so is the linear combination: $t_n = \alpha r_1^n + \beta r_2^n= \alpha 3^n + \beta 2^n$
 Plugging general solution into recurrence and distributing will give 0
 General solution defines an infinite number of solutions
 Use initial conditions to find a single solution
 If roots are identical, use this equation: $t_n = \alpha r^n + \beta nr^n$
 Choose $\alpha$ and $\beta$ to meet the initial conditions:
$t_0=0,t_1=1$
 $t_0 = \alpha 3^0 + \beta 2^0 = 0$
 $t_1 = \alpha 3^1 + \beta 2^1 = 1$
 Solving gives: $\alpha=1$ and $\beta = 1$
 Thus: general solution is $t_n = \alpha 3^n + \beta 2^n== 3^n  2^n$
Homogenous Linear Recurrence Equation: Fibonacci Numbers
 Recurrence:
 $T(n) = T(n1) + T(n2)
 $t_n  t_{n1}  t_{n2} = 0$
 $t_0 = 0$
 $t_1 = 1$
 Assume solution is of form $t_n = r^n$, for all $n$
 Substitute: $r^n  r^{n1}  r^{n2} = 0$
 Factor: $ r^{n2}(r^2  r^1  r^0) = 0$
 Divide by $ r^{n2}$ give CE: $r^2  r^1  r^0 = 0$
 Roots of CE: $(1+\sqrt{5})/2=\phi$ and $(1\sqrt{5})/2=1\phi = 1/\phi =
\hat{\phi}$
 Choose $\alpha$ and $\beta$ to meet the initial conditions:
$t_0=0,t_1=1$
 $t_0 = \alpha \phi^0 + \beta \hat{\phi}^0 = 0$
 $t_1 = \alpha \phi^1 + \beta \hat{\phi}^1 = 1$
 Solving gives: $\alpha=1/\sqrt{5}$ and $\beta = 1/\sqrt{5}$
 Thus: general solution is $t_n = \alpha \phi^n + \beta \hat{\phi}^n
= \frac{1}{\sqrt{5}}(\phi^n  \hat{\phi}^n)$
Nonhomogenous Linear Recurrence Equations
 General form: $a_0t_n + a_1t_{n1} + \dots + a_kt_{nk} = c \ne 0$
 RHS ≠ 0
 General plan:
 Find roots of related homogeneous equation and find alpha and beta.
 Form equation (eg $t_n = \alpha r_1^n + \beta r_2^n$ if distinct roots) and solve for alpha and beta
 Solution to nonhomogeneous is based on solution to homogeneous
 We mostly ignore these
Change of Variable
 Substitute, for example, $2^k$ for $n$
 Example: For Fibonacci, number of additions
 $t_n = t_{n1} + t_{n2} + 1 = 0$
 $t_n  t_{n1}  t_{n2} = 1 $
 $t_n  t_{n1}  t_{n2} 1 = 0 $
 Substitute: $u_n = t_n + 1$
 We mostly ignore these
Recursion Tree  Intuition for Master Method
Recursion Tree
 A Recursion Tree is a technique for calculating the amount of work expressed by a recurrence equation
 Nice illustration of a recurrence
 Gives intuition behind Master Methods
 Each level of the tree shows the nonrecursive work for a given parameter
value
 Write each node with two parts:
 Upper part: $T(s)$ for some $s$
 Lower part: nonrecursive part of $T(s)$
 Relation between parts: Upper part = lower part + upper parts of children
Example Recursion Tree
 Consider $T(n) = 2T(\frac{n}{3}) + 5n$
 $T(0) = 7$
 These values don't represent a specific algorithm.
They are just chosen for illustration.
 First rewrite as $T(s) = 5s + 2T(\frac{s}{3})$
 Putting nonrecursive term first is (slightly) easier
 Using a different variable allows us to focus on calculating $T(n)$
 Now calculate $T(27) = T(3^3)$:
 Root is $T(27)$:
 Upper part: $T(27)$
 Lower part: $5s=5\times 27 = 135$
 Children of root:
 How many children?
 What's in each child?
 Calculating $T(27)$:
$\begin{align*}
T(27) & = 5\times 27 + 2 \times T(27/3) \\
& = 135 + 2 \times T(27/3) \\
& = 135 + 2 \times [45 + 2 \times T(9/3)] \\
& = 135 + 2 \times [45 + 2 \times [15 + 2 \times T(3/3)]] \\
& = 135 + 2 \times [45 + 2 \times [15 + 2 \times 7]] \\
& = 135 + 2 \times [45 + 2 \times 29] \\
& = 135 + 2 \times 103 \\
& = 135 + 206 \\
& = 341
\end{align*}
$
Example Recursion Tree
 Consider $T(s) = 2T(\frac{s}{3}) + 5s = 5s + 2T(\frac{s}{3}) $
 Calculate $T(27)$
 Where do 2, 3, log_{3}, log_{3} 27 come from?
Tree Properties
 Consider $T(s) = 5s + 2T(\frac{s}{3}) $
 Bottom level is level $\log_3 27$ = level 3
 Number of levels: $ 1 + \log_3 27 = 1 + 3 = 4$
 Sum of level 0: $2^0 \times (5\times 27) = 135$
 Sum of level 1: $2^1 \times (5\times 9) = 90$
 Sum of level 2: $2^2 \times (5\times 3) = 60$
 Sum of bottom level: $2^{\log_3 27} * 7 = 2^3 \times 7 = 56$
 Sum of all levels: $135 + 90 + 60 + 56 = 341$
Recursion Tree for $T(n) = aT(\frac{n}{b}) + f(n)$
 Recurrence Equation: $T(s) = f(s) + a T(s/b)$
 Find $T(n)$
Summing the Values in the Tree
 Recurrence Equation: $T(s) = f(s) + a T(s/b)$
 Bottom level is level $\log_b n$
 Number of levels: $ 1 + \log_b n$
 Sum of all levels except leaves: cost of dividing and joining
 Sum of level 0: $a^0 \times f(\frac{n}{b^0}) $
 Sum of level 1: $a^1 \times f(\frac{n}{b^1}) $
 Sum of level 2: $a^2 \times f(\frac{n}{b^2}) $
 Sum of level i: $a^i \times f(\frac{n}{b^i}) $
 Sum of all levels except leaves: $\sum_{i=0}^{\log_b n  1} a^i f(\frac{n}{b^i})$
 Sum of bottom level: cost of solving all problems of size 1
 $d a^{log_b n} = d n^{log_b a} $
 Sum of all levels:
$$
\sum_{i=0}^{\log_b n  1} a^i f(\frac{n}{b^i}) + d n^{log_b a}
$$
Fine Points
 $a ≥ 1, b > 1, \text{ are constants}$
 If $n$ is not a power of $b$, then replace $n/b$ by $\lfloor n/b \rfloor$ (or
$\lceil n/b \rceil$)
 It can be proved that this does not affect the complexity
Evaluating the Complexity of the Sum of the Tree Levels
 Sum of all levels:
$\displaystyle \sum_{i=0}^{\log_b n  1} a^i f(\frac{n}{b^i}) + \Theta(n^{log_b a})
$
 Simplify base case of recursion using $\Theta(1)$ instead of $d$
 When this formula is a polynomial in $n$, we can simplify it by
considering the power of $n$ in each term:
 Either first or second term can dominate or they can be the same order
 Intuitively:
 $\Theta(f(n))$, if the first term (ie the sum of nonleaf levels) dominates (Case 1)
 $\Theta(n^{log_b a}\lg n )$, if neither term dominates
 $\Theta(n^{log_b a})$, if the second term (ie sum of leaves) dominates (Case 3)
 Some restrictions occur
 These results can be formalized to obtain the General Master Method
Formalizing Gives the General Master Method
 To formalize, we consider the relation
between the terms of the sum of the levels.
 Sum of all levels:
$\displaystyle
\sum_{i=0}^{\log_b n  1} a^i f(\frac{n}{b^i})
+
\Theta(n^{log_b a})
$
 Formal definition of one term dominating is below (assume $0 < \epsilon$):
 $\Theta(f(n))$, if $f(n) = \Omega(n^{log_b a+\epsilon})$ and $af(n/b) ≤ cf(n)$ for some constant $c ≤ 1$ and large $n$ [ie Case 1:
second term (sum of nonleaves) dominates]
 $\Theta(n^{log_b a}\lg n )$, if $f(n) = \Theta(n^{log_b a})$ [ie neither term dominates]
 $\Theta(n^{log_b a})$, if $f(n) = O(n^{log_b a\epsilon})$ [ie Case 3: second term (sum of leaves) dominates]
 This can also be written as $n^{log_b a+\epsilon} = O(f(n))$
 We ignore this case
 These results give the general Master Method