Linear Algebra is an important branch of mathematics. It is often applied to engineering problems. To understand deep learning and operate deep learning, a deep understanding of linear algebra is very important, the following are some interesting ideas that I have extracted from the second chapter of DL book linear algebra to understand the basic line generation problem.
2.3 identity and inverse matrices play an important role in solving linear equations. For details, see:
In actual application scenarios, inverse matrice
It may not be able to be found, and it may not be able to be preserved in digital computers without loss of precision.
2.4 linear dependence, span, and rank
To solve the equation, we can also look at this problem from another perspective. We regard each column in matrix A as a direction, the above equation can be seen as how to reach B from the origin (all 0), and each walk can only follow the direction represented by each column in. From this perspective, the above equations can be written as follows:
So
If you want to find the corresponding solution x (M dimension) for any B (n-dimensional), any vector in the n-d space can be linearly combined with a (nxm dimension) the m column in The Matrix, so the number of columns in matrix A must be greater than or equal to the row number, the equations can be resolved only when the number of equations is less than or equal to the number of unknowns. . In solving this equation, if
To find the inverse of matrix A as a tool for solving the equation, matrix A must be a square matrix and each column is column independent. .
Some very important concepts will be used in the line generation, but it is enough in the CNN network proposed by alexnet;
In addition, how can we design the loss function and calculate its gradient for the update of backward propagation using other line generation knowledge, I will discuss how to use the gradient descent method to optimize loss functions in my post. .
[Deep Learning an MIT Press book in preparation] Linear Algebra
