Mathematical knowledge of pattern recognition and mathematical derivation in machine learning

Each hand push formula will meet a variety of not, in the online search a summary of the commonly used derivation formula ... Continue to update in ...

The following original address: http://blog.163.com/live_freely/blog/static/151142060201023154057339/

On the internet to see someone posted the following derivation formula:

Y = A * X--dy/dx = a '

Y = X * A--DY/DX = a

Y = A ' * X * b--DY/DX = A * b '

Y = a ' * X ' * B--DY/DX = b * A '

So the previously learned matrix derivative part of the collation:

1. Matrix y derivative of scalar x:

The equivalent of each element after the derivation of the transpose, notice that the MXN matrix after the derivation into NXM

y = [y (IJ)]--DY/DX = [DY (JI)/dx]

2. Scalar y derivative of the column vector x:

Note that unlike above, this time the brackets are biased, not transpose, the Nx1 vector after derivation or Nx1 vector

y = f (x1,x2,.., xn)--dy/dx = (dy/dx1,dy/dx2,.., dy/dxn) '

3. Line vector y ' derivative of column vector x:

Note that the 1xM vector is the NXM matrix after the derivation of the Nx1 vector.

Each column of y is biased to X, and the columns form a matrix.

Important Conclusions:

DX '/DX = I

D (AX) '/dx = A '

4. The column vector y is the derivative of the row vector x ':

Converts the derivative of the row vector y ' to the column vector x, and then transpose.

Note that the Mx1 vector has a derivative of the 1xN vector to the MXN matrix.

Dy/dx ' = (DY '/dx) '

5. Vector product-to-column vector x derivation algorithm:

Note the derivation of the scalar is a little different.

D (UV ')/dx = (DU/DX) V ' + U (DV '/dx)

D (U ' V)/dx = (DU '/dx) V + (DV '/dx) U '

Important Conclusions:

D (x ' a)/dx = (DX '/dx) A + (DA/DX) X ' = IA + 0X ' = a

D (AX)/dx ' = (d (X ' a ')/dx) ' = (a ') ' = a

D (X ' ax)/dx = (DX '/dx) AX + (d (AX) '/dx) x = ax + A ' x

6. Matrix y derivative of the column vector x:

The y is biased to each component of X to form a hyper-vector.

Note that each element of the vector is a matrix.

7. Matrix product-to-column vector derivation rule:

D (uV)/dx = (DU/DX) V + u (DV/DX)

D (UV)/dx = (DU/DX) V + U (DV/DX)

Important Conclusions:

D (x ' a)/dx = (DX '/dx) A + x ' (DA/DX) = IA + X ' 0 = a

8. The derivative of the scalar y to the matrix x:

Similar to the derivative of the scalar y-to-column vector x,

The y is biased for the elements of each x without transpose.

DY/DX = [Dy/dx (IJ)]

Important Conclusions:

y = U ' XV =σσu (i) x (IJ) v (j) then Dy/dx = = UV '

y = U ' X ' XU then dy/dx = 2XUU '

y = (xu-v) ' (xu-v) then dy/dx = d (U ' X ' xu-2v ' XU + V ' V)/dx = 2XUU '-2VU ' + 0 = 2 (xu-v) U '

9. Derivative of matrix Y to matrix x:

Each element of Y is derivative of x, and then it is lined together to form a super matrix.

Mathematical knowledge of pattern recognition and mathematical derivation in machine learning