## Probabilistic Graphical Models 1: Representation

Start Date: 07/05/2020

 Course Type: Common Course

Probabilistic graphical models (PGMs) are a rich framework for encoding probability distributions over complex domains: joint (multivariate) distributions over large numbers of random variables that interact with each other. These representations sit at the intersection of statistics and computer science, relying on concepts from probability theory, graph algorithms, machine learning, and more. They are the basis for the state-of-the-art methods in a wide variety of applications, such as medical diagnosis, image understanding, speech recognition, natural language processing, and many, many more. They are also a foundational tool in formulating many machine learning problems. This course is the first in a sequence of three. It describes the two basic PGM representations: Bayesian Networks, which rely on a directed graph; and Markov networks, which use an undirected graph. The course discusses both the theoretical properties of these representations as well as their use in practice. The (highly recommended) honors track contains several hands-on assignments on how to represent some real-world problems. The course also presents some important extensions beyond the basic PGM representation, which allow more complex models to be encoded compactly.

#### Course Syllabus

In this module, we define the Bayesian network representation and its semantics. We also analyze the relationship between the graph structure and the independence properties of a distribution represented over that graph. Finally, we give some practical tips on how to model a real-world situation as a Bayesian network.

#### Course Introduction

Probabilistic Graphical Models 1: Representation In this course you will continue your understanding of probabilistic graphical models, and learn more about computing the weights and biases of such models. You will also learn more about the representation scheme of such models. We will apply various techniques to represent the information in a graph, including embedding of nodes in their respective color schemes, and also how to segment the nodes to improve the parsing efficiency of a program. These techniques are implemented in a fairly simple yet powerful computational framework, and the results are presented in a very elegant and streamlined way. You will also learn about the underlying concepts of the machine learning algorithms, and how they are applied to solve problems in a practical way. After finishing this course, you will have gained a solid foundation to proceed with most future training examples in probabilistic graphical models and supervised machine learning problems. You will also have a solid base to go into any other course on algorithms in data science, machine learning, or any other domain in which you would like to learn how to use machine learning methods. Note that the machine learning techniques discussed in this course are not available for use in Deep Learning. Recommended Background: You should have at least one year programming experience under the "Know How" or equivalent experience (C, C++, Java, Python, etc.). You should have experience in programming algorithms, especially C++. You should be proficient at manipulating files, including a good understanding of algorithms in the C++ standard library.You should have

#### Course Tag

Bayesian Network Graphical Model Markov Random Field

#### Related Wiki Topic

Article Example
Probabilistic programming language A probabilistic programming language (PPL) is a programming language designed to describe probabilistic models and then perform inference in those models. PPLs are closely related to graphical models and Bayesian networks, but are more expressive and flexible. Probabilistic programming represents an attempt to "[unify] general purpose programming with probabilistic modeling."
Probabilistic soft logic Probabilistic soft logic (PSL) is a framework for collective, probabilistic reasoning in relational domains. PSL uses first order logic rules as a template language for graphical models over random variables with soft truth values from the interval [0,1].
Graphical model A graphical model or probabilistic graphical model (PGM) is a probabilistic model for which a graph expresses the conditional dependence structure between random variables. They are commonly used in probability theory, statistics—particularly Bayesian statistics—and machine learning.
Graphical model Generally, probabilistic graphical models use a graph-based representation as the foundation for encoding a complete distribution over a multi-dimensional space and a graph that is a compact or factorized representation of a set of independences that hold in the specific distribution. Two branches of graphical representations of distributions are commonly used, namely, Bayesian networks and Markov random fields. Both families encompass the properties of factorization and independences, but they differ in the set of independences they can encode and the factorization of the distribution that they induce.
Probabilistic classification Some classification models, such as naive Bayes, logistic regression and multilayer perceptrons (when trained under an appropriate loss function) are naturally probabilistic. Other models such as support vector machines are not, but methods exist to turn them into probabilistic classifiers.
Probabilistic classification Binary probabilistic classifiers are also called binomial regression models in statistics. In econometrics, probabilistic classification in general is called discrete choice.
Probabilistic programming language A probabilistic relational programming language (PRPL) is a PPL specially designed to describe and infer with probabilistic relational models (PRMs).
Russ Salakhutdinov He specializes in deep learning, probabilistic graphical models, and large-scale optimization.
Graphical model This type of graphical model is known as a directed graphical model, Bayesian network, or belief network. Classic machine learning models like hidden Markov models, neural networks and newer models such as variable-order Markov models can be considered special cases of Bayesian networks.
Probabilistic database Probabilistic databases distinguish between the logical data model and the physical representation of the data much like relational databases do in the ANSI-SPARC Architecture.
Models of representation Models of representation refer to ways in which elected officials behave in representative democracies. There are three types: delegate, trustee, and politico.
Graphical models for protein structure Graphical models can still be used when the variables of choice are continuous. In these cases, the probability distribution is represented as a multivariate probability distribution over continuous variables. Each family of distribution will then impose certain properties on the graphical model. Multivariate Gaussian distribution is one of the most convenient distributions in this problem. The simple form of the probability, and the direct relation with the corresponding graphical model makes it a popular choice among researchers.
Graphical models for protein structure Graphical models have become powerful frameworks for protein structure prediction, protein–protein interaction and free energy calculations for protein structures. Using a graphical model to represent the protein structure allows the solution of many problems including secondary structure prediction, protein protein interactions, protein-drug interaction, and free energy calculations.
Graphical models for protein structure Gaussian graphical models are multivariate probability distributions encoding a network of dependencies among variables. Let formula_15 be a set of formula_16 variables, such as formula_16 dihedral angles, and let formula_18 be the value of the probability density function at a particular value "D". A multivariate Gaussian graphical model defines this probability as follows:
Systems Biology Graphical Notation The Systems Biology Graphical Notation (SBGN) is a standard graphical representation crafted over several years by a community of biochemists, modelers and computer scientists.
Graphical game theory For a general formula_1 players game, in which each player has formula_2 possible strategies, the size of a normal form representation would be formula_15. The size of the graphical representation for this game is formula_16 where formula_17 is the maximal node degree in the graph. If formula_18, then the graphical game representation is much smaller.
Graphical model The framework of the models, which provides algorithms for discovering and analyzing structure in complex distributions to describe them succinctly and extract the unstructured information, allows them to be constructed and utilized effectively. Applications of graphical models include causal inference, information extraction, speech recognition, computer vision, decoding of low-density parity-check codes, modeling of gene regulatory networks, gene finding and diagnosis of diseases, and graphical models for protein structure.
Daphne Koller In 2009, she published a textbook on probabilistic graphical models together with Nir Friedman. She offered a free online course on the subject starting in February 2012.
Probabilistic soft logic In recent years there has been a rise in the approaches that combine graphical models and first-order logic to allow the development of complex probabilistic models with relational structures. A notable example of such approaches is Markov logic networks (MLNs). Like MLNs PSL is a modelling language (with an accompanying implementation) for learning and predicting in relational domains. Unlike MLNs, PSL uses soft truth values for predicates in an interval between [0,1]. This allows for the integration of similarity functions in the into models. This is useful in problems such as Ontology Mapping and Entity Resolution. Also, in PSL the formula syntax is restricted to rules with conjunctive bodies.
Eclipse (software) The Concrete Syntax Development project contains the Graphical Modeling Framework, an Eclipse-based framework dedicated to the graphical representation of EMF based models.