Softmax Logistic Loss Detailed

Softmax function –softmax Layer

Softmax is used in the multi-classification process, it maps the output of multiple neurons into the (0,1) interval, which can be interpreted as probabilities to understand and thus be multi-categorized.

Assuming we have an array z= (Z1,Z2,... ZM) z= (z_1,z_2,... z_m), its softmax function is defined as follows:
Ōi (z) =exp (zi) ∑mj=1exp (ZJ), i=1,2..,m \sigma_i (z) =\frac{exp (z_i)}{\sum_{j=1}^mexp (Z_j)}, I=1,2..,m
That is, Softmax is a function map that maps z= (z1,z2,... ZM) z= (z_1,z_2,... z_m) to (σ1,σ2,... Σ m) (\sigma_1,\sigma_2,... \sigma_m).
Among them, ∑σi=1. \sum\sigma_i=1.

As shown in the following diagram, it can be more clearly indicated.

In logistic regression, it is assumed that the Zi=wtix+bi z_i=w_i^tx+b_i is the linear prediction result of category I, and the Oi=σi (z) o_i=\sigma_i (z) can be obtained in Softmax. can be interpreted as the probability that the observed data x belongs to Class I, or is called likelihood (likehood). Logistic regression

The objective function of the Logistic Regression is based on the maximum likelihood principle , assuming that the data x corresponds to the category Y, then according to X we just calculate the maximum likelihood is to maximize the value of Oy o_y

Typically use negative log-likelihood instead of likelihood, which means minimizing −log (OY) <