Heaps
Overview
- Heapsort
- Improved Selection Sort - fast way to find largest
- is O(n lg n) worst case - like merge sort
- sorts in place, like insertion sort
- Best of both worlds
- First study heaps and then heapsort
What is a Heap?
- Almost complete binary tree
- Complete: all levels are full
- Almost complete: all levels are full, except bottom which is
filled to left
- Heap property - at each level
- Stored in an array
Terminology
- Height of node: number of edges on longest path from root to a leaf
- Height of heap: height of root (Θ(lg n)
Array Storage of Trees
- Root: A(1)
- Parent of A(i): A(i/2) [integer division]
- Left Child of A(i): A(2i)
- Right Child of A(i): A(2i + 1)
- Example: What tree is this?
1 2 3 4 5 6 7 8 9 10
16 14 10 8 7 9 3 2 4 1
What is the height and number of nodes at each level?
Heap Property
- For all nodes i, except the root, parent is larger than node
- Questions:
- Is the example a heap?
- Where is largest element in the heap?
- A leaf is trivially a heap
- We ignore min heaps and focus on max heaps
- Min heap: A(parent(i)) < A(i)
Maintaining a Heap
- During the sorting process, we will undo the heap property
- Algorithm Heapify will restore the heap property
- Assume:
- A(i) may be smaller than its children (ie violate heap property)
- Subtrees rooted at A(left(i)) and A(right(i)) are themselves heaps
- Algorithm Heapify(A, i, n): restores a heap
l = left(i)
r = right(i)
-- Find largest of l and i
if l ≤ n and A(l) > A(i) then
largest := l
else
largest := i
-- Find largest of r and largest
if r ≤ n and A(r) > largest then
largest := r
-- If heap property violated, fix node i and heapify rest of tree
if largest ≠ i
exchange A(i) with A(largest)
heapify(A, largest, n)
More on Heapfiy
- Example: 16; 4, 10; 14, 7, 9, 3; 2, 8, 1
- Complexity: O(lg n)
- Why: How many levels? How much work at each level?
How to Create a Heap from an Array?
- Algorithm Build-Heap(A, n)
for in reverse 1 .. n/2 loop -- integer division
heapify(A, i, n)
Example: 4; 1, 3; 2, 16, 9, 10; 14, 8, 7
Some questions:
- What happens if we start at 10?
- What's important about the node n/2? Why does n/2 work?
How does it work: build the heap from the bottom up. As the
algorithm goes up the tree it heapifies each parent and its two child heaps
Two Approaches to Building a Heap
- In both cases, start with non-heap array
- Bottom Up:
- Algoithm above
- for i in reverse 1 .. n / 2: heapify (A, i, n)
- Top Down:
- Algoithm above
- for i in 1 .. n / 2: Heapify (A, i, n)
Correctness of Build-Heap
- Loop Invariant: At the start of each iteration of the loop, each of
the nodes i+1, i+2, ..., n is the root of a heap
- Initialization: Before the loop, the leaves are heaps, in the first
pass i is n/2, and the leaves are in positions n/2+1, n/2+2, ..., n
- Maintenance: For a given i, the children of i are heaps by the loop
invariant. Heapify makes node i a heap. Decrementing i reestablishes
the loop invariant
- Termination: When i = 0 the loop terminates, and by the loop
invariant, each node is the root of a heap. In particular, node 1 is the
root of a heap.
Performance
- Simple: O(n) calls to heapify, each of which is lg n = n lg n
- Better:
- Use the fact that the time of heapify is linear in the height of
the node it's called on and most nodes have small height.
- How many nodes at height h? Ceiling(n/(2^h+1)
- What is height of tree? floor(lg n)
- Time for heapify for node of height h is O(h)
- Total time for build-heap (ignoring floors and ceilings) is
- Σ h=0..lg n O(h) n/2^(h+1)
- = O(n Σh=0..lg n h/2^h
- But Σh=0..∞ h / 2^h = (1/2) / (1 - 1/2)^2
- = 2
- Thus, Build-Heap is O(n)
HeapSort Algorithm
- Algorithm (for an ascending sort):
- Build a heap
- Start with the root (the largest element), swap the root with
its correct location
- Discard last element (which is in its correct location)
- Heapify the new root, on the smaller array
- Repeat until only one node remains
Code for HeapSort
Build-Heap(A, n)
for i in reverse 2 .. n loop
exchange A(1) with A(i)
heapify(A, 1, i-1)
Example: 7; 4, 3; 1, 2
Analysis of Heap Sort
- Build-Heap: O(n)
- for loop: n-1 times
- exchange: O(1)
- Heapify: O(lg n)
- Total: O(n lg n)
- Final note: In practice, quicksort usually beats heapsort
Heap Implementation of Priority Ques
- Priority Queue: What is it???
- What is at the front of the queue?
- Example application: schedule jobs on shared computer
- Heaps give an efficient implementation of a priority queue
- Insertion: O(lg n)
- Extraction: O(lg n)
- Versus: one fast and the other slow
- Both max and min priority queues are useful. We consider only
max heap priority queues.
- Example: We will use our usual example:
16 14 10 8 7 9 3 2 4 1
Priority Queue
- Dynamic set S of elements
- Each element has a key - an associated value
- Operations
- Maximum(S): return element with largest key
- Extract-Max(S): remove and return element with largest key
- Increase-Key(S, x, k): increases value of element x's key to k
- Assume k ≥ x's current key value
- Insert(S, x): insert x into S
Operation: Maximum
- Easy: Algorithm??
- Time: Θ(1)
Operation: Extract Maximum Element
- Process: Remember first, move last to first, reheapify, return old first
- Algorithm:
Heap-Extract-Max(A, n)
if n < 1 then
error
max = A[1]
A[1] = A[n]
heapify(A, 1, n-1)
return max
Performance: Constant time assignments + heapify = ??
Example: 16 out, 1 up from node 10, remove 10, ...
Increase Key Value
- Process: Verify k ≥ x's current key, update x's key value to k, traverse key
upward, swapping keys as needed, until x's key smaller than its parent
- Algorithm:
Heap-Increase-Key(A, i, key)
if key < A[i] then
error new key smaller than current key
A[i] = key
while i > 1 and A[parent(i) < A(i)
swap A(i) and A(parent(i))
i = parent(i)
Performance: Upward path from node i has length O(lg n), and so time = ...
Example: increase key of 9 to 15, ...
Insert into the heap
- Given a key k to insert, insert new node in last position w/ key = -∞,
increase key to k
- Algorithm:
Heap-insert(A, key, n)
A[n+1] = -∞
heap-increase-key(A, n+1, key)
Performance: Constant time asst, and so time = ...