UFLDL Learning notes and programming Jobs: Softmax Regression (Softmax return)

UFLDL Learning notes and programming Jobs: Softmax Regression (Softmax regression)

UFLDL out a new tutorial. Feel better than before, starting from the basics. The system is clear and has programming practice.

In deep learning high-quality group inside listen to some predecessors said, do not delve into other machine learning algorithms, can directly to learn DL.

So I started doing this recently. The tutorial plus MATLAB programming, is perfect AH.

The address of the new tutorial is: http://ufldl.stanford.edu/tutorial/

Learn links in this section: http://ufldl.stanford.edu/tutorial/supervised/SoftmaxRegression/

Softmax regression is actually an extended form of logistic regression,

Logistic regression is often used as a classifier of Class 2,

The Softmax is used as a classifier for multiple classes.

In mathematical form, the fact that logistic regression is the k=2 of Softmax regression is the case. In this tutorial, too.

The derivation of the objective function and the derivative of the Softmax is also very clear.

For programming jobs. Because the MATLAB implementation is not ripe, jumped a lot of pits.

For a very long time, and it is only done with a for loop.

This time I finally realized that the performance of the For loop was poor. Iterated 200 times. 1 hours.

It is also related to the complexity of this model than the first two models.

The first version number of the code to paste first. Later came up with the quantification of programming to fill up.

Here is the code for SOFTMAX_REGRESSION.M

function [F,g] = Softmax_regression_vec (theta, x, y)% Arguments:% theta-a vector containing the parameter values  To optimize.  % in Minfunc, Theta are reshaped to a long vector.  So we need to% resize it to an n-by-(num_classes-1) matrix.  % Recall that we assume theta (:, num_classes) = 0.    % x-the examples stored in a matrix.  % X (i,j) is the i ' th coordinate of the J ' th example.  % y-the label for each example.  Y (j) is the J ' th example ' s label.  % m=size (x,2);  N=size (x,1); %theta was originally a matrix, and when it was passed, Theta (:) came in.  It's a vector, just one column, and now we have to turn her into a matrix. Theta is a vector;  Need to reshape to n x num_classes.  Theta=reshape (Theta, N, []);    Num_classes=size (theta,2) +1;  % initialize objective value and gradient.  f = 0;  g = zeros (Size (theta)); h = Theta ' *x;%h (k,i) K Theta.  The first sample paralysis still has to be circulated to seek AH A = exp (h);  A = [A;ones (1,size (a,2))];% plus line B = SUM (a,1);      For i=1:m for j=1:num_classes if Y (i)!=j continue;    End F+=log2 (A (j,i)/b (i));End end f=-f;% symbol flag=0;      For j=1:num_classes-1 for I=1:m if (Y (i) ==j) flag = 1;      else flag=0;    End G (:, J) +=x (:, i) * (A (j,i)/b (i)-flag);  End end% Todo:compute the softmax objective function and gradient using vectorized code.  % Store The objective function value in ' f ', and the gradient in ' G '.  % before returning g, make sure your form it back to a vector with g=g (:); %%%% YOUR CODE here%%% g=g (:);% make gradient a vector for Minfunc

Here is the result of the execution:

Old Tutorial Http://deeplearning.stanford.edu/wiki/index.php/Exercise:Softmax_Regression

There are also softmax programming jobs. It is also the recognition of handwritten numerals.

The question of accuracy is mentioned.

Our implementation achieved an accuracy of 92.6% . If your model ' s accuracy is significantly less than 91%, check your code, ensure this is using the trained Wei Ghts, and that is training your model on the full 60000 training images. Conversely, if your accuracy is too high (99-100%), ensure so you had not accidentally trained your model on the test s ET as well.

In other words, my code is still able to do so from a precise rate.

The next step is to achieve quantitative programming, heightening the speed.

Let's say you have a good idea. Remember to share a little bit!

This article linger

This article link: http://blog.csdn.net/lingerlanlan/article/details/38410123

Copyright notice: This article blog original articles, blogs, without consent, may not be reproduced.

UFLDL Learning notes and programming Jobs: Softmax Regression (Softmax return)