The mathematical basis of machine learning: Vector Chapter

Preface

In the above, I briefly summarize the basic operation of the Matrix, and give two examples of application. In this article we continue to talk about vectors.

Vectors are the basic concepts in linear algebra and are the fundamental data representations of machine learning. For example, the process of reading text on a computer begins by segmenting the text and then using a vector representation. This is because vectors are well suited for expression and processing in high-dimensional spaces. The concepts of projection and dimensionality that come into contact with machine learning are made on the basis of vectors.

The vector \ (\vec{\mathbf{v}}\) defined in \ (\mathbb{r}^{n}\) space can be represented by an ordered set of n real numbers, i.e. \ (\vec{\mathbf{v}} = \begin{bmatrix}v_1 \ v_2 \ \ \ldots \ v_n\end{bmatrix}\), each element in this ordered set is called the component of the vector. For example, in a \ (\mathbb{r}^{2}\) space in the vector \ (\begin{bmatrix}2 \ 1\end{bmatrix}\), some places will also use \ ((2, 1) \) or \ (<2, 1>\) such as the form of expression.

The drawing represents this variable:

The length of the vector is defined as \[\left\| \vec{\mathbf{v}} \right\| = \sqrt{v_{1}^{2} + v_{2}^{2} + \ldots + v_{n}^{2}}\], which is exactly the same as the distance formula we used to touch. A vector of length 1 is called a unit vector. Basic Operations Plus

The addition of vector \ (\mathbf{a}\) and Vector \ (\mathbf{b}\) is defined as:

\[\mathbf{a} + \mathbf{b} = \begin{bmatrix} a_1 + b_1 \ a_2 + b_2 \ \ldots \ A_n + b_n \end{bmatrix} \]

The plot schematic vector \ (\mathbf{a} = \begin{bmatrix}-1 \ 2\end{bmatrix}\) and \ (\mathbf{b} = \begin{bmatrix}3 \ 1\end{bmatrix}\) are added with a value of \ (\begin{bmatrix}2 \ 3\end{bmatrix}\):

In Python, vectors can be represented directly using Numpy's Ndarray.

Import NumPy as np
a = Np.array ([-1, 2])
B = Np.array ([3, 1])
print a + B # [2 3]

minus

\[\mathbf{a}-\mathbf{b} = \begin{bmatrix} a_1-b_1 \ a_2-b_2 \ \ldots \ A_n-b_n \end{bmatrix} \]

From a geometric point of view, the vector subtraction is equivalent to adding a reverse vector.

Import NumPy as np
a = Np.array ([-1, 2])
B = Np.array ([3, 1])
print  a B # [-4, 1]

Multiply scalar multiplication Vectors

Scalar \ (c\) times vector \ (\mathbf{a}\) is defined as:

\[c \cdot \mathbf{a} = \begin{bmatrix} c \cdot a_1 \ c \cdot a_2 \ \ldots \ c \cdot a_n \end{bmatrix} = \begin{bmatrix } a_1 \cdot c \ \ a_2 \cdot c \ \ \ldots \ a_n \cdot c \end{bmatrix} \]

Plot schematic vector \ (\mathbf{a} = \begin{bmatrix}-1 \ 2 \end{bmatrix}\) times a scalar 3 get \ (\begin{bmatrix}-3 \ 6 \end{bmatrix}\):

Python implementations:

Import NumPy as np
a = Np.array ([-1, 2])
print a * 3 #[-3, 6]

Vector dot Product

The dot product (also called point multiplication) of a vector is defined as follows:

\[\vec{\mathbf{a}}\cdot \vec{\mathbf{b}} = \begin{bmatrix} a_1 \ a_2 \ \ldots \ A_n\end{bmatrix} \cdot \begin{bmatrix} b_1 \ b_2 \ \ldots \ b_n \end{bmatrix} = a_{1}b_{1} + a_{2}b_{2} + \ldots + a_{n}b_{n}\]

The Visible dot product gets a scalar.

For example:

\[\begin{bmatrix} 3 \ 5 \ 2 \end{bmatrix} \cdot \begin{bmatrix} 1 \ 4 \ 7 \end{bmatrix} = 3 \cdot 1 + 5 \cdot 4 + 2 \c Dot 7 = 37\]

Python Example:

Import NumPy as np
a = Np.array ([3, 5, 2])
B = Np.array ([1, 4, 7])
print A.dot (b)  # PNS
print Np.dot (A, b)  # 37 (another equivalent notation)

It is easy to prove that dot product satisfies multiplication exchange law, distributive law and binding law.

We know that the length of the vector is defined as \ (\left\| \vec{\mathbf{v}} \right\| = \sqrt{v_{1}^{2} + v_{2}^{2} + \ldots + v_{n}^{2}}\), and the definition of the joint dot product can be drawn:

Eq:1»

\[\left\| \vec{\mathbf{v}} \right\| = \sqrt{v_{1}^{2} + v_{2}^{2} + \ldots + v_{n}^{2}} = \sqrt{\vec{\mathbf{v}} \cdot \ve C{\mathbf{v}}}\]

There is also a very important property about dot product, called Cauchy inequality: to two non-0 vectors \ (\vec{\mathbf{x}}, \vec{\mathbf{y}} \in \mathbb{r}^{n}\), \ (|\vec{\mathbf{x}} \cdot \vec{\mathbf{y}}| \le \left\|\vec{\mathbf{x}}\right\|\left\|\vec{\mathbf{y}}\right\|\). When and only if \ (\vec{\mathbf{x} = c\vec{\mathbf{y}}\), the equation is set.

Although confined to the space does not prove it, but this nature is very important, there will be a lot of vectors behind the theory is based on it. For example, for a vector \ ((\vec{\mathbf{x} + \vec{\mathbf{y}) \), using this property, combined with equation 1, we can get

\[\begin{align} \left\|\vec{\mathbf{x}} + \vec{\mathbf{y}}\right\|^2 & = (\vec{\mathbf{x} + \vec{\mathbf{y}}) \ CDOT (\vec{\mathbf{x} + \vec{\mathbf{y}}) \\\ & = \left\|\vec{\mathbf{x}}\right\|^2 + 2\vec{\mathbf{x}}\vec{\ Mathbf{y}} + \left\|\vec{\mathbf{y}}\right\|^2 \\\ & \le \left\|\vec{\mathbf{x}}\right\|^2 + 2\left\|\vec{\mathbf{ x}}\right\|\left\|\vec{\mathbf{y}}\right\| + \left\|\vec{\mathbf{y}}\right\|^2 \end{align} \]

So:

\[\left\|\vec{\mathbf{x}} + \vec{\mathbf{y}}\right\|^2 \le (\left\|\vec{\mathbf{x}}\right\| + \left\|\vec{\mathbf{y} }\right\|) ^2 \]

Open squares on both sides get:

\[\left\|\vec{\mathbf{x}} + \vec{\mathbf{y}}\right\| \le \left\|\vec{\mathbf{x}}\right\| + \left\|\vec{\mathbf{y}}\ right\| \]

The triangle inequality is obtained.

From a geometrical point of view, the dot product of a vector is related to the cosine of an angle between vectors (\theta\): \[\vec{\mathbf{a}}\cdot\vec{\mathbf{b}} = \left\|\vec{\mathbf{a}}\right\|\ Left\|\vec{\mathbf{b}}\right\|cos\theta\], which means that the dot product of a vector actually reflects the projection of the vector \ (\vec{\mathbf{a}}\) on the vector \ (\vec{\mathbf{b}}\), That is, two vectors are the same degree in the same direction. When two vectors are orthogonal, the value of \ (Cos\theta\) is 0, the value of the dot product is 0, and the projection is minimal. When two vectors are parallel, the value of \ (cos\theta\) is 1, the dot product value is the largest, and the projection is the largest.

In the view above, \ (l\) is a line extending from the \vec{\mathbf{v}}\ vector, i.e. \ (L={c\vec{\mathbf{v}}|c\in \mathbb{r}}\). \vec{\mathbf{x}}\ \ (l\) is projected on \ (proj_l (\vec{\mathbf{x}}) \). Depending on the nature of the dot product, it is possible to:

\[\begin{align} (\vec{\mathbf{x}}-\underbrace {c\vec{\mathbf{v}}}_{proj_l ({\vec{\mathbf{x}})}) \cdot \VEC{\MATHBF {v}} &= 0 \\\ \vec{\mathbf{x}}\cdot \vec{\mathbf{v}}-c\vec{\mathbf{v}}\cdot \vec{\mathbf{v}} &= 0\\\ C\cdot \vec {\mathbf{v}} \cdot \vec{\mathbf{v}} &= \vec{\mathbf{x}}\cdot \vec{\mathbf{v}}\\\ c &= \frac{\vec{\mathbf{x}}\ CDOT \vec{\mathbf{v}}}{\vec{\mathbf{v}}\cdot \vec{\mathbf{v}}} \end{align} \]

With \ (c\), we can find out that the projection \ (proj_l ({\vec{\mathbf{x}}) \) is:

\[proj_l ({\vec{\mathbf{x}}) = C\vec{\mathbf{v}} = (\frac{\vec{\mathbf{x}}\cdot \vec{\mathbf{v}}}{\vec{\mathbf{v}}\ CDOT \vec{\mathbf{v}}) \vec{\mathbf{v}}\]

For example, Vector \ (\vec{\mathbf{a}} = \begin{bmatrix}1 \ 2\end{bmatrix}\), Vector \ (\vec{\mathbf{b}} = \begin{bmatrix}1 \ 1\end{ bmatrix}\), then \ (\vec{\mathbf{a}}\) in \ (\vec{\mathbf{b}}\) direction \ (l\) The projection is:

\[proj_l ({\vec{\mathbf{a}}) = C\vec{\mathbf{b}} = (\frac{\vec{\mathbf{a}}\cdot \vec{\mathbf{b}}}{\vec{\mathbf{b}}\ CDOT \vec{\mathbf{b}}) \vec{\mathbf{b}} = \frac{3}{2}\vec{\mathbf{b}}\]

Python Example:

Defget_projection (A, B):
    return A.dot (b) *1.0*b/b.dot (b)

a = Np.array ([1, 2])
B = Np.array ([2, 2])
Print Get_projection (A, b)  # [1.5 1.5]

vector Outer product

Vectors (also called cross-multiplication, vector product, cross product) are defined only in \ (\mathbb{r}^{2}\) and \ (\mathbb{r}^{3}\):

The vector outer product of \ (\mathbb{r}^{2}\):

\[\begin{bmatrix} a_1 \ A_2\end{bmatrix} \times \begin{bmatrix} b_1 \ b_2 \end{bmatrix} = \begin{bmatrix} a_1 \cdot b_2 -A_2 \cdot b_1\end{bmatrix}\]

For example:

\[\begin{bmatrix} 1 \ 2 \end{bmatrix} \times \begin{bmatrix} 3 \ 4 \end{bmatrix} = \begin{bmatrix} 1 \cdot 4-3 \cdot 2 \end{bmatrix} = \begin{bmatrix}-2\end{bmatrix} \]

The vector outer product of \ (\mathbb{r}^{3}\):

\[\begin{bmatrix} a_1 \ a_2 \ A_3\end{bmatrix} \times \begin{bmatrix} b_1 \ b_2 \ b_3 \end{bmatrix} = \begin{bmatrix} a_2 \cdot b_3 - a_3 \cdot b_2 \\ a_3 \cdot b_1 - a_1 \cdot b_3 \\ a_1 \cdot b_2 - a_2 \cdot b_1\end{bmatrix}\]

For example:

\[\begin{bmatrix} 3 \ 5 \ 2 \end{bmatrix} \times \begin{bmatrix} 1 \ \ 4 \ 7 \end{bmatrix} = \begin{bmatrix} 5 \cdot 7 -2 \cdot 4 \ 2 \cdot 1-3 \cdot 7 \ 3 \cdot 4-5 \cdot 1\end{bmatrix} = \begin{bmatrix} + \ -19 \ 7\end{bmatrix} \ ]

A new vector is obtained by the result of the outer product of the visible vector.

Python Example:

Import NumPy as np
a = Np.array ([3, 5, 2])
B = Np.array ([1, 4, 7])
print Np.cross (A, b)  # [27,-19, 7]

An important function of the outer product is to get a new vector \ (\vec{\mathbf{c}}\) orthogonal to the two original vectors of \ (\vec{\mathbf{a}}\), \ (\vec{\mathbf{b}}\), and the direction of the new vector can be determined by the law of the right hand (a simple way to determine the direction of the result vector satisfying the "right-hand rule" is this: if the coordinate system is right-handed, when the right hand four refers to from \ (\vec{\mathbf{a}}\) to not more than 180 degrees from the corner of the turning \ (\vec{ \mathbf{b}}\), the thumbs up pointing to the direction of \ (\vec{\mathbf{c}}\).

From a geometrical point of view, the outer product of the vector is related to the sine of the angle between the vectors \ (\theta\): \[\left\|\vec{\mathbf{a}}\times\vec{\mathbf{b}}\right\| = \left\|\vec{\ Mathbf{a}}\right\|\left\|\vec{\mathbf{b}}\right