## Mathematics for Machine Learning: PCA

Start Date: 07/05/2020

 Course Type: Common Course

This intermediate-level course introduces the mathematical foundations to derive Principal Component Analysis (PCA), a fundamental dimensionality reduction technique. We'll cover some basic statistics of data sets, such as mean values and variances, we'll compute distances and angles between vectors using inner products and derive orthogonal projections of data onto lower-dimensional subspaces. Using all these tools, we'll then derive PCA as a method that minimizes the average squared reconstruction error between data points and their reconstruction. At the end of this course, you'll be familiar with important mathematical concepts and you can implement PCA all by yourself. If you’re struggling, you'll find a set of jupyter notebooks that will allow you to explore properties of the techniques and walk you through what you need to do to get on track. If you are already an expert, this course may refresh some of your knowledge. The lectures, examples and exercises require: 1. Some ability of abstract thinking 2. Good background in linear algebra (e.g., matrix and vector algebra, linear independence, basis) 3. Basic background in multivariate calculus (e.g., partial derivatives, basic optimization) 4. Basic knowledge in python programming and numpy Disclaimer: This course is substantially more abstract and requires more programming than the other two courses of the specialization. However, this type of abstract thinking, algebraic manipulation and programming is necessary if you want to understand and develop machine learning algorithms.

#### Course Syllabus

Principal Component Analysis (PCA) is one of the most important dimensionality reduction algorithms in machine learning. In this course, we lay the mathematical foundations to derive and understand PCA from a geometric point of view. In this module, we learn how to summarize datasets (e.g., images) using basic statistics, such as the mean and the variance. We also look at properties of the mean and the variance when we shift or scale the original data set. We will provide mathematical intuition as well as the skills to derive the results. We will also implement our results in code (jupyter notebooks), which will allow us to practice our mathematical understand to compute averages of image data sets. #### Course Introduction

Mathematics for Machine Learning: PCA and Linear Algebra This course gives you an introduction to the PCA and linear algebra that you need in order to understand the workings of various machine learning algorithms. We will cover both the linear algebra and the mathematical foundations of PCA. We will start by introducing the math behind the most common machine learning algorithms: linear gradient descent, L2-weighted smoothing, and Keras. We will also cover the calculus required to fit these algorithms to datasets. We will also cover the general concepts required to understand the code generated by the algorithms. Finally, we will cover the differences between linear algebra and modern optimization.Mathematics for Machine Learning: PCA L2-Weighted Smoothing Linear Gradient Descent Binomial Coefficients Mathematics for Machine Learning: Differential Equations This course gives you an overview of mathematical topics in machine learning, including differential equations and approximate methods of computing values for variables in more advanced ways. We will start by introducing the mathematics behind the most common machine learning algorithms: linear gradient descent, L2-weighted smoothing, and Keras. We will also cover the calculus required to fit these algorithms to datasets. We will also cover the differences between linear algebra and modern optimization.Mathematics for Machine Learning: Differential Equations L2-Weighted Smoothing Linear Gradient Descent

#### Course Tag

Machine Learning Mathematics for Machine Learning Math Mathematics PCA Principal Component Analysis Machine Learning Mathematics for Machine Learning Math Mathematics PCA Principal Component Analysis Python Programming Principal Component Analysis (PCA) Projection Matrix Mathematical Optimization

#### Related Wiki Topic

Article Example
Machine learning Rule-based machine learning is a general term for any machine learning method that identifies, learns, or evolves `rules’ to store, manipulate or apply, knowledge. The defining characteristic of a rule-based machine learner is the identification and utilization of a set of relational rules that collectively represent the knowledge captured by the system. This is in contrast to other machine learners that commonly identify a singular model that can be universally applied to any instance in order to make a prediction. Rule-based machine learning approaches include learning classifier systems, association rule learning, and artificial immune systems.
Investigations in Mathematics Learning Investigations in Mathematics Learning is the official research journal of the Research Council for Mathematics Learning. Information about submission can be found here. RCML seeks to stimulate, generate, coordinate, and disseminate research efforts designed to understand and/or influence factors that affect mathematics learning.
British Society for Research into Learning Mathematics The British Society for Research into Learning Mathematics is a United Kingdom association for people interested in research in mathematics education.
Active learning (machine learning) Recent developments are dedicated to hybrid active learning and active learning in a single-pass (on-line) context, combining concepts from the field of Machine Learning (e.g., conflict and ignorance) with adaptive, incremental learning policies in the field of Online machine learning.
Machine learning Some statisticians have adopted methods from machine learning, leading to a combined field that they call "statistical learning".
Machine learning Machine learning tasks are typically classified into three broad categories, depending on the nature of the learning "signal" or "feedback" available to a learning system. These are
Machine learning Another categorization of machine learning tasks arises when one considers the desired "output" of a machine-learned system:
Relevance vector machine In mathematics, a Relevance Vector Machine (RVM) is a machine learning technique that uses Bayesian inference to obtain parsimonious solutions for regression and probabilistic classification.
Machine learning Machine Learning poses a host of ethical questions. Systems which are trained on datasets collected with biases may exhibit these biases upon use, thus digitizing cultural prejudices. Responsible collection of data thus is a critical part of machine learning.
Machine learning Machine learning is closely related to (and often overlaps with) computational statistics, which also focuses on prediction-making through the use of computers. It has strong ties to mathematical optimization, which delivers methods, theory and application domains to the field. Machine learning is sometimes conflated with data mining, where the latter subfield focuses more on exploratory data analysis and is known as unsupervised learning. Machine learning can also be unsupervised and be used to learn and establish baseline behavioral profiles for various entities and then used to find meaningful anomalies.
Quantum machine learning The term quantum machine learning is also used for approaches that apply classical methods of machine learning to the study of quantum systems, for instance in the context of quantum information theory or for the development of quantum technologies. For example, when experimentalists have to deal with incomplete information on a quantum system or source, Bayesian methods and concepts of algorithmic learning can be fruitfully applied. This includes the application of machine learning to tackle quantum state classification, Hamiltonian learning, or learning an unknown unitary transformation.
Quantum machine learning Quantum machine learning is an emerging interdisciplinary research area at the intersection of quantum physics and machine learning. One can distinguish four different ways of merging the two parent disciplines. Quantum machine learning algorithms can use the advantages of quantum computation in order to improve classical methods of machine learning, for example by developing efficient implementations of expensive classical algorithms on a quantum computer. On the other hand, one can apply classical methods of machine learning to analyse quantum systems. Most generally, one can consider situations wherein both the learning device and the system under study are fully quantum.
Tanagra (machine learning) Tanagra is a free suite of machine learning software for research and academic purposes
Adversarial machine learning Adversarial machine learning is a research field that lies at the intersection of machine learning and computer security. It aims to enable the safe adoption of machine learning techniques in adversarial settings like spam filtering, malware detection and biometric recognition.
Machine learning Among other categories of machine learning problems, learning to learn learns its own inductive bias based on previous experience. Developmental learning, elaborated for robot learning, generates its own sequences (also called curriculum) of learning situations to cumulatively acquire repertoires of novel skills through autonomous self-exploration and social interaction with human teachers and using guidance mechanisms such as active learning, maturation, motor synergies, and imitation.
Machine learning Software suites containing a variety of machine learning algorithms include the following :
Outline of machine learning [[Category:Artificial intelligence|Machine learning]]
Machine learning Learning classifier systems (LCS) are a family of rule-based machine learning algorithms that combine a discovery component (e.g. typically a genetic algorithm) with a learning component (performing either supervised learning, reinforcement learning, or unsupervised learning). They seek to identify a set of context-dependent rules that collectively store and apply knowledge in a piecewise manner in order to make predictions.
List of datasets for machine learning research These datasets are used for machine learning research and have been cited in peer-reviewed academic journals and other publications. Datasets are an integral part of the field of machine learning. Major advances in this field can result from advances in learning algorithms (such as deep learning), computer hardware, and, less-intuitively, the availability of high-quality training datasets. High-quality labeled training datasets for supervised and semi-supervised machine learning algorithms are usually difficult and expensive to produce because of the large amount of time needed to label the data. Although they do not need to be labeled, high-quality datasets for unsupervised learning can also be difficult and costly to produce. This list aggregates high-quality datasets that have been shown to be of value to the machine learning research community from multiple different data repositories to provide greater coverage of the topic than is otherwise available.
Machine learning Machine learning and statistics are closely related fields. According to Michael I. Jordan, the ideas of machine learning, from methodological principles to theoretical tools, have had a long pre-history in statistics. He also suggested the term data science as a placeholder to call the overall field.