[Google Deep Learning notes] Logistic classification

Logistic classification

GitHub Project Address: https://github.com/ahangchen/GDLnotes
Welcome to star, you can discuss it in issue area.
Official Tutorial Address
Video/subtitle Download

About

Simple but important classifier

• Train your first simple model entirely end to end
• Download, preprocess some pictures to classify
• Run an actual logistic classifier on images data
• Connect bit of math and code

Detaillinear Classifier

This is modeled because linear formulas are the simplest mathematical models, and that's all.

• Input:x (e.g. the pixels in an image)
• Apply a linear function to X
  
  • Giant Matrix Multiply
  • Take inputs as a big vector
  • Multiply input vector with a matrix, W means weights
  • B means biased term
  • Machine learning adjust weights and bias for the best prediction

• Output:y, predictions for per output class

  • Y is a vector, represents the probability of each label
  • In a good prediction, the probability of the correct label should be closer to 1
  • Often get the start of Y is not a probability, but some specific value (scores/logits), so need to convert, by:

  Softmax regression model: Wikipedia

Softmax

• Code SOFT_MAX.PY:SOFTMAX Implementation and application
• The greater the difference in the score of input (which can be multiplied by 10), the greater the difference in the probability of each label of the output, the less the difference
• Softmax only cares about the probability of a few labels and doesn't care about the specific values
• Machine learning is a matter of increasing the success rate of prediction, so it is a process that makes the difference between score

One hot encoding

The correct prediction result should be that only one label is set, and the other label is not set. In this case, the most probable result is the most predictable probability.

Example:take this test

• One hot encoding in the case of a label in a lot of cases not work well, because the output vector is everywhere 0, very sparse, so inefficient
  
  • Solved by embeddings

• Benefits: Can measure the distance between us and the ideal situation (compare-vectors)

  Classifier output: [0.7 0.2 0.1] \<=> the actual condition corresponding to the label: [1 0 0]

• Compare vectors:cross-entropy
  

• D (S, l)! = d (l, s)

  Remember:label don ' t log, for Label Zero

Summary

Find the appropriate W and B, which makes the average of the distance D of S and L, the smallest in the entire data set N.

Minimized cross-entropy

The average of D is training loss, sum and matrix multiplication is a big data activity.

The error of two parameters results in a rounded loss, so all we have to do is find the weight that is as close to the center as possible.

Machine learning problem becomes a numerical optimization
-One solution: Gradient descent, derivative

Modify the parameters to check if the error is larger and change to a smaller direction until you reach bottom.

The weight is two-dimensional, but in fact there may be a great many weight

Next section Practice

If you feel that my article is helpful to you, please feel free to enjoy it ~

[Google Deep Learning notes] Logistic classification