1. What is a representation?

A representation is a way of describing an idea. For example, with generative adversarial networks, a discriminant is a manifold in the space of all images defines images with certain qualities.

Types of Represetaions

• Mental models
• Data
• Display
• Programming abstraction (code)
• Human senses
• Interaction model

3. Interpolation

Refresher on constrained points

Taken from Prof. Hariharan's Linear Algebra Tutorial.

Over-constrained systems

Suppose $A$ is a $m\times n$ matrix where $m > n$ and is full rank.

$$ A = \begin{bmatrix} a_{1,1} & a_{1,2} & \dots & a_{1,n}\\ \vdots & \vdots & \vdots & \vdots\\ \vdots & \vdots & \vdots & \vdots\\ \vdots & \vdots & \vdots & \vdots\\ a_{m,1} & a_{m, 2} & \dots & a_{m ,n} \end{bmatrix} $$

Since there are more constraints then variables, there may not be a solution. Instead, we try to optimize for a solution via least squares.

$$ \min_x \|Ax-b\|^2 $$

We have that

$$ f(x) = \|Ax-b\|^2 = (Ax-b)^T(Ax-b)\\ = (x^TA^T-b^T)(Ax-b)\\ = x^TA^TAx - x^TA^Tb -b^TAx+b^Tb\\ = x^TA^TAx-2 b^TAx + b^Tb. $$