Machine Learning.pdf

Categories of Learning

Unsupervised Learning: no supervision, but a reward function
Supervised Learning: learning from a training data set, inference on the test data
Reinforcement Learning: no supervision, but a reward function
Clustering, density estimation, discriminant function, discriminative model, generative model

Mathematical Formulation

Given a set of objects and a set of object categories (classes)
Search for a mapping such that similar elements in the input space are mapped to similar elements in the output space
Difference between regression and classification: regression is continuous, classification is discrete

Basis Functions

Map from the input space to the output space
Can be interpreted as functions that extract features from the input data
Features reflect the properties of the objects (width, height, etc.)

Linear Regression

Assume: identity mapping
Given: data points
Goal: predict the value of a new example
Parametric formulation: y = w0 + w1x
Error function: sum of squared errors

Polynomial Regression

Model complexity and data set size
Define: basis functions, outer product
Obtain: polynomial regression equation
Error function: sum of squared errors

Key Concepts

Supervised and unsupervised learning
Regression and classification
Basis functions and feature extraction
Linear and polynomial regression
Error functions and optimization