## Greedy Algorithms, Minimum Spanning Trees, and Dynamic Programming

Start Date: 07/05/2020

 Course Type: Common Course

The primary topics in this part of the specialization are: greedy algorithms (scheduling, minimum spanning trees, clustering, Huffman codes) and dynamic programming (knapsack, sequence alignment, optimal search trees).

#### Course Introduction

Greedy Algorithms, Minimum Spanning Trees, and Dynamic Programming Introduction to greedy algorithms (more advanced than the previous course) and dynamic programming. We will learn a lot of practical algorithms that work with arbitrary data structures, and a lot of very basic algorithms that work with fixed-length data structures. We will use dynamic programming to implement algorithms that require great flexibility: how we want the data structure to be extended, how quickly we want it extended, how efficiently we want a data structure to be used. We will start by introducing the concepts of dynamic programming and maximum spanning trees. We will then transition into algorithms that extend a data structure by adding new data layers. We will then introduce the technique for finding optimal solutions to problems in both typical (multithreading) and extreme (random) programs. We will see how efficient we can be using those algorithms. We will then see how to implement efficient algorithms using the NLT and AST. When we get to the idea of maximum spanning trees, we will see that they can solve some very efficient problems. We will discuss the basic idea behind weakly coupled optimization, and how to use it for dealing with elements of a data structure that are too large to be handled by conventional algorithms. We will then get into the algorithms that implement partial applications of greedy algorithms. We will see how the optimized code is compiled down to a form that can run on small machines. But the most interesting part of the course is the part that deals with real problems compiled down to a form that can run on large computers.

#### Course Tag

Spanning Tree Algorithms Dynamic Programming Greedy Algorithm

#### Related Wiki Topic

Article Example
Greedy algorithm If a greedy algorithm can be proven to yield the global optimum for a given problem class, it typically becomes the method of choice because it is faster than other optimization methods like dynamic programming. Examples of such greedy algorithms are Kruskal's algorithm and Prim's algorithm for finding minimum spanning trees, and the algorithm for finding optimum Huffman trees.
Dynamic programming Dynamic programming algorithms are often used for optimization. A dynamic programming algorithm will examine the previously solved subproblems and will combine their solutions to give the best solution for the given problem. In comparison, a greedy algorithm treats the solution as some sequence of steps and picks the locally optimal choice at each step. Using a greedy algorithm does not guarantee an optimal solution, because picking locally optimal choices may result in a bad global solution, but it is often faster to calculate. Fortunately, some greedy algorithms (such as Kruskal's or Prim's for minimum spanning trees) are proven to lead to the optimal solution.
Minimum spanning tree Research has also considered parallel algorithms for the minimum spanning tree problem.
Minimum spanning tree Maximum spanning trees find applications in parsing algorithms for natural languages
Minimum spanning tree More generally, if the edge weights are not all distinct then only the (multi-)set of weights in minimum spanning trees is certain to be unique; it is the same for all minimum spanning trees.
Minimum spanning tree Other specialized algorithms have been designed for computing minimum spanning trees of a graph so large that most of it must be stored on disk at all times. These "external storage" algorithms, for example as described in "Engineering an External Memory Minimum Spanning Tree Algorithm" by Roman, Dementiev et al., can operate, by authors' claims, as little as 2 to 5 times slower than a traditional in-memory algorithm. They rely on efficient external storage sorting algorithms and on graph contraction techniques for reducing the graph's size efficiently.
Minimum spanning tree A minimum spanning tree (MST) or minimum weight spanning tree is a subset of the edges of a connected, edge-weighted undirected graph that connects all the vertices together, without any cycles and with the minimum possible total edge weight. That is, it is a spanning tree whose sum of edge weights is as small as possible. More generally, any undirected graph (not necessarily connected) has a minimum spanning forest, which is a union of the minimum spanning trees for its connected components.
Distributed minimum spanning tree Two commonly used algorithms for the classical minimum spanning tree problem are Prim’s algorithm and Kruskal’s algorithm. However, it is difficult to apply these two algorithms in the distributed message-passing model. The main challenges are:
Dynamic programming Dynamic programming is widely used in bioinformatics for the tasks such as sequence alignment, protein folding, RNA structure prediction and protein-DNA binding. The first dynamic programming algorithms for protein-DNA binding were developed in the 1970s independently by Charles DeLisi in USA and Georgii Gurskii and Alexander Zasedatelev in USSR. Recently these algorithms have become very popular in bioinformatics and computational biology, particularly in the studies of nucleosome positioning and transcription factor binding.
Minimum spanning tree A set of "k-smallest spanning trees" is a subset of "k" spanning trees (out of all possible spanning trees) such that no spanning tree outside the subset has smaller weight. (Note that this problem is unrelated to the "k"-minimum spanning tree.)
Minimum spanning tree All these four are greedy algorithms. Since they run in polynomial time, the problem of finding such trees is in FP, and related decision problems such as determining whether a particular edge is in the MST or determining if the minimum total weight exceeds a certain value are in P.
Greedy embedding For more general graphs, some greedy embedding algorithms such as the one by Kleinberg start by finding a spanning tree of the given graph, and then construct a greedy embedding of the spanning tree. The result is necessarily also a greedy embedding of the whole graph. However, there exist graphs that have a greedy embedding in the Euclidean plane but for which no spanning tree has a greedy embedding.
Minimum spanning tree There may be several minimum spanning trees of the same weight; in particular, if all the edge weights of a given graph are the same, then every spanning tree of that graph is minimum.
Random minimum spanning tree Random minimum spanning trees of grid graphs may be used for invasion percolation models of liquid flow through a porous medium, and for maze generation.
Minimum degree spanning tree A minimum degree spanning tree formula_10 is a spanning tree which has the least maximum degree. The vertex of maximum degree in formula_10 is the least among all possible spanning trees of formula_1.
Minimum spanning tree Minimum spanning trees have direct applications in the design of networks, including computer networks, telecommunications networks, transportation networks, water supply networks, and electrical grids (which they were first invented for, as mentioned above). They are invoked as subroutines in algorithms for other problems, including the Christofides algorithm for approximating the traveling salesman problem, approximating the multi-terminal minimum cut problem (which is equivalent in the single-terminal case to the maximum flow problem),
Minimum bottleneck spanning tree In an undirected graph "G"("V", "E") and a function "w" : "E" → "R", let "S" be the set of all spanning trees "T". Let "B"("T") be the maximum weight edge for any spanning tree "T". We define subset of minimum bottleneck spanning trees "S"' such that for every "T" ∈ "S"' and "T" ∈ "S" we have "B"("T") ≤ "B"("T") for all "i" and "k".
Rectilinear minimum spanning tree It is possible to apply usual algorithms for the minimum spanning tree problem to the complete graph with "n" edges, which yields a complexity of O("n") with appropriate data structures such as the Fibonacci heap.
Euclidean minimum spanning tree The "realization problem" for Euclidean minimum spanning trees is stated as follows: Given a tree "T" = ("V", "E"), find a location "D"("u") for each vertex "u" ∈ "V" so that "T" is a minimum spanning tree of "D"("u"): u ∈ V, or determine that no such locations exist.
Spanning tree Because a graph may have exponentially many spanning trees, it is not possible to list them all in polynomial time. However, algorithms are known for listing all spanning trees in polynomial time per tree.