Ufldl Study Notes and programming assignments: softmax regression (vectorization acceleration)

Ufldl Study Notes and programming assignments: softmax regression (vectorization acceleration)

Ufldl provides a new tutorial, which is better than the previous one. Starting from the basics, the system is clear and has programming practices.

In the high-quality deep learning group, you can learn DL directly without having to delve into other machine learning algorithms.

So I started to do this recently. The tutorial, coupled with Matlab programming, is perfect.

The address of the new tutorial is: http://ufldl.stanford.edu/tutorial/

This section is an improvement on the ufldl learning notes and programming job: softmax regression (softmax regression) version.

Haha, I wrote the vectoring method. Nima is so fast. It takes only two minutes to complete the 200 iteration. Last night, I ran for an hour and a half.

In fact, various matrices should be written on paper to implement the vectorized writing.

1. Thanks to tornadomeet, although he is doing an experiment in the old tutorial, he has learned a few MATLAB functions from him. Http://www.cnblogs.com/tornadomeet/archive/2013/03/23/2977621.html

For example, sparse and full. '

2 There is also http://deeplearning.stanford.edu/wiki/index.php/Exercise:Softmax_Regression from the old tutorial

Learned

% M is the matrix as described in the textM = bsxfun(@rdivide, M, sum(M))

3. I learned the new tutorial.

I=sub2ind(size(A), 1:size(A,1), y);values = A(I);

The following code is softmax_regression_vec.m:

Function [f, g] = softmax_regression_vec (Theta, x, y) % arguments: % theta-a vector containing the parameter values to optimize. % in minfunc, Theta is reshaped to a long vector. so we need to % resize it to an n-by-(num_classes-1) matrix. % recall that we assume theta (:, num_classes) = 0.% x-the examples stored in a matrix. % x (I, j) is the I 'th coordinate of the J 'th example. % Y-the label for each example. Y (j) is the J 'th example's label. % m = size (x, 2); n = size (x, 1); % Theta is a matrix. When passing parameters, theta (:) comes in like this, it is a vector with only one column. Now we have to change her to the matrix % Theta is a vector; need to reshape to N x num_classes. theta = reshape (Theta, N, []); num_classes = size (Theta, 2) + 1; % initialize objective value and gradient. f = 0; G = zeros (SIZE (theta); H = Theta '* X; % H (K, I) The K Theta, sample I A = exp (h); A = [A; ones (1, size (A, 2)]; % Add 1 row p = bsxfun (@ rdivide, a, sum (a); C = log2 (p); I = sub2ind (SIZE (C), y, [1: size (C, 2)]); values = C (I); F =-sum (values); D = Full (sparse (1: M, Y, 1); D = D (:, 1 :( size (D, 2)-1); P = P (1 :( size (P, 1)-1 ),:); % minus 1 line G = x * (P '. -d); % todo: Compute the softmax objective function and gradient using vectorized code. % store the objective function value in 'F', and the gradient in 'G '. % before returning g, make sure you form it back into a vector with G = g (:); % Your code here % G = g (:); % make gradient a vector for minfunc

Linger

Link: http://blog.csdn.net/lingerlanlan/article/details/38425929