Algorithm Analysis -- Week 8
Introduction
This week we will look at the greedy method of solving problems.
Note that the links from this page are to handouts that will be distributed the night of class. Only print out those handouts if you could not attend class.
Main topics this week:
Web Research Assignment 2 Due
Greedy Method
Coin Changing Problem
Greedy Method, Part 2
Coin Changing, Part 2
Minimum Spanning Trees
Kruskal's Algorithm
Prim's Algorithm
Sollin's Algorithm
Minimum Spanning Tree Exercise
Dijkstra's Algorithm
Next Week
Your web research assignment 2 is due today. You should turn in a hard copy of your paper.
The greedy method is similar to dynamic programming, in that it is used to solve optimization problems. Superficially, it seems similar to dynamic programming, in that we expect an optimal solution to consist of optimal solutions to subproblems. In dynamic programming, though, at each step we make a choice among the various solutions to a subproblem to pick the optimal one. This might require changing to a more optimal solution. In the greedy method, at each step we simply make the most optimal choice we can based on where we are at. We never go back and change decisions in the greedy method.
For example, a greedy method to find the shortest path from one vertex to another vertex would simply pick the shortest edge at each step. This will probably not find the shortest path. So the greedy method does not always work.
Let's look at a problem that may or may not be solvable via the greedy method. The problem is that of giving a customer change using the fewest number of coins. For example, if the change were 26 cents, 1 quarter and 1 penny is better than 26 pennies.
We can picture an algorithm that works like this:
while (change is not equal to desired change)
{
grab the largest coinif (change + coin > desired change)
reject the coin
do not consider that size of coin again
else
accept the coin
}
If the desired change is 26 cents, we first pick up a quarter. The next time we try to pick up a quarter, we reject it. We also reject dimes and nickles. When we pick up a penny, that gives us the right amount of change and we're done.
Any algorithm that uses the greedy method goes through three stages:
Selection Procedure
This is where we make our choice based on the most optimal choice available to us right now. We do not consider any possible future choices.
Feasibility Check
This is where we check to see if the solution represented by our last selection is feasible. A solution is not feasible if it cannot be part of a solution to the larger problem.
Solution Check
This is where we check to see if we have solved the larger problem yet.
In the coin changing algorithm, which pieces of the algorithm correspond to which steps?
Does the greedy method always produce an optimal solution for the coin changing problem? What if we add in a 12 cent coin? Does it still produce optimal solutions? Use it to make 16 cents worth of change.
A common problem in graphing is to determine the subset of a graph that contains all the vertices, but only enough edges to result in a connected graph with no cycles (e.g. a tree). This is called a spanning tree. You can use a depth first search or a breadth first search to construct a spanning tree, by simply adding vertices and edges to the spanning tree as they are crossed, stopping when you have added all the vertices.
(show examples of depth first and breadth first spanning trees)
Quite often we care more about the spanning tree with the minimum total edge cost, or the minimum spanning tree. A minimum spanning tree can be used to help people who are building a network of some kind (a telecommunications network, a system of plumbing, etc) to use the least amount of physical cable or pipe.
To calculate all the possible spanning trees and compare them to find the minimum spanning tree would be a factorial, O(n!), algorithm in the worst case. The greedy method is used in different ways to more efficiently construct minimum spanning trees out of graphs. We will concern ourselves with weighted, undirected graphs.
We'll look at three different greedy algorithms for constructing a minimum spanning tree:
We'll use the following graph with each algorithm, so we can see the differences (if any) in the minimum spanning trees created by each algorithm.
V(G) = {0, 1, 2, 3, 4, 5, 6}
E(G) = {(0, 1, 28), (0,5,10), (1, 2, 16), (1, 6, 14), (2, 3, 12), (3, 6, 18), (3, 4, 22), (4, 6, 24), (4, 5, 25)}0
5 6 1
4 3 2
Kruskal's algorithm concerns itself with choosing which edges will be in the minimum spanning tree. It does this by making a greedy choice and selecting the minimum edge in the graph. That edge and the vertices connected to it are added to the minimum spanning tree. It then picks the next minimum edge, and so on. That's the selection procedure.
The feasibility check comes in because we cannot have cycles in the minimum spanning tree. So if selecting an edge would result in a cycle, then that edge is not part of a valid solution, so we reject it.
The solution check is simply to recognize when we have connected all the vertices. At that point we have a minimum spanning tree.
(show example)
Here's the algorithm.
(show example using the algorithm data structures)
The worst case order of complexity for this algorithm is O(n2 lg n). For graphs with a relatively few number of edges, the order of complexity is O(n lg n).
Prim's algorithm also constructs a minimum cost spanning tree, but it takes a slightly different approach. Rather than considering any edges, prim's algorithm makes certain that at every stage the set of edges picked constructs a tree. We start with any vertex as the root of the tree.
The selection procedure is then to select the minimum edge that is connected to a vertex in the tree, giving preference to edges connected to lower numbers vertices in the tree if there is a tie.
The feasibility check is to make certain that the edge does not lead to a vertex already in the tree (this prevents cycles).
The solution check is to see whether we have all the vertices in the tree. If so, we have constructed a minimum spanning tree.
(show example)
Here's the algorithm.
(show example on algorithm)
The worst case order of complexity is O(n2). This would seem to indicate that Prim's algorithm is better than Kruskal's algorithm. However, that is only true for the worst case. For graphs with fewer edges, Kruskal's algorithm performs better.
Sollin's algorithm takes a third approach to constructing a minimum spanning tree. In Sollin's algorithm, we consider each vertex in the graph to be the root of an otherwise empty tree.
The selection procedure is to select, for each tree, the minimum cost edge that connects to a vertex in that tree.
The feasibility check is to make certain that the edge chosen does not lead back into the same tree (this prevents cycles).
The solution check is to see if all the vertices are part of the same tree. If so, we have constructed a minimum spanning tree.
(show example)
I leave constructing a high level algorithm for Sollin's algorithm up to you. The order of complexity is O(n2 lg n) for a straight forward algorithm. However, Sollin's algorithm is extremely well suited to parallel programming, a technique by which we perform multiple steps of the algorithm at the same time, making the entire algorithm run faster. We'll talk more about parallel programming later in the course.
Minimum Spanning Tree Exercise
I will draw a graph on the board, and for the graph you should show me the order in which edges are added to the minimum spanning tree for Kruskal's algorithm, Prim's algorithm, and Sollin's algorithm. Identify edges based on the vertices they connect. This exercise is worth 10 points.
Recall that we talked about Floyd's all-pairs shortest path algorithm, a dynamic programming approach to finding the shortest path from every vertex to every other vertex in a graph. Floyd's algorithm was O(n3). Now we'll talk about Dijkstra's algorithm for finding the shortest path from one particular vertex to every other vertex. This is called a single-source shortest path algorithm, and is a greedy algorithm.
Here are the basic steps:
We use three sets, Y = {v1}, F = {}, V = {all vertices}
Choose the shortest path from a vertex in Y to a vertex in V - Y. The path can only use vertices in Y as intermediate nodes (i.e. every vertex in the path must be in Y except for the ending vertex, which must be in V). Add the ending vertex to Y, and add the edge chosen to F.
Note that we are looking for shortest paths, not shortest edges. It may well be that the new edge we add is not the shortest edge remaining, because the shortest edge would result in a longer path.
Repeat, stopping only when Y == V.
Let's see this in action on an example graph:
V(G) = {v1, v2, v3, v4, v5}
E(G) = {<v1, v2, 7>, <v1, v3, 4>, <v1, v4, 6>, <v1, v5, 1>, <v3, v2, 2>, <v3, v4, 5>,
<v4, v2, 3>, <v5, v4, 1>}
That's the algorithm conceptually, in terms that we can understand. To get a computer to execute this algorithm, though, we need to put it in terms of processing an adjacency matrix. Page 155 in your book has the full algorithm. From inspecting the algorithm, what's the big Oh of Dijkstra's algorithm?
Next week we will continue covering various greedy algorithms.