Deep Learning # lab1 #

Deep Learning lab1

I plan to paste the six related experiments of Professor Andrew Ng # deep learning # one by one ~

It is estimated that the length of time will be relatively long (after all, Jos's 7-level float has not been completed yet .)

Certificate -----------------------------------------------------------------------------------------------------------------------------------

Experiment content: Linear Fitting

Lab material: http://download.csdn.net/detail/u011368821/8114541

Certificate -----------------------------------------------------------------------------------------------------------------------------------

Linear fitting is simple, practical, and cool ~

Identity matrix -- Unit Matrix

The style of the experiment is the same as that of the foreign experiment ~ Is the function that completes the functions to be implemented one by one.

Linear regression with one variable

Problem Background:

In this part of this exercise, you will implement linear regression with one variable to predict profits for a food truck. suppose you are the CEO of a restaurant franchise and are considering different cities for opening a new outlet. the chain already has trucks in varous cities and you have data for profits and populations from the cities. you wowould like to use this data to help you select which city to expand to next.

Assume that you are the CEO of a catering company and have data in hand about the cities with different populations and the profitability of your shop in the city. based on the data, a solution is provided for a store.

The purpose of this experiment is to determine how a given data (essentially a single-Variable Function) can be used to fit the scattered points.

At the beginning of the experiment, a 5*5 unit matrix is required to be returned. Two methods are available:

1. directly call the eye () function

2. Write a two-layer for loop to initialize a 5*5 matrix.

The following two methods are provided:

function A = warmUpExercise()%WARMUPEXERCISE Example function in octave%   A = WARMUPEXERCISE() is an example function that returns the 5x5 identity matrixA = eye(5);% ============= YOUR CODE HERE ==============% Instructions: Return the 5x5 identity matrix %               In octave, we return values by defining which variables%               represent the return values (at the top of the file)%               and then set them accordingly. % % for row = 1:5%     for col = 1:5%         if row == col%             A(row,col) = 1;%         else%             A(row,col) = 0;%         end%     end% end% % ===========================================end

Plotdata implementation: follow the requirements of instructions.

function plotData(x, y)%PLOTDATA Plots the data points x and y into a new figure %   PLOTDATA(x,y) plots the data points and gives the figure axes labels of%   population and profit.% ====================== YOUR CODE HERE ======================% Instructions: Plot the training data into a figure using the %               "figure" and "plot" commands. Set the axes labels using%               the "xlabel" and "ylabel" commands. Assume the %               population and revenue data have been passed in%               as the x and y arguments of this function.%% Hint: You can use the 'rx' option with plot to have the markers%       appear as red crosses. Furthermore, you can make the%       markers larger by using plot(..., 'rx', 'MarkerSize', 10);figure; % open a new figure windowplot(x,y,'rx','MarkerSize',10);ylabel('Profit in $10,000s');xlabel('Population of City in 10,000s');% ============================================================end

The above are simple practical methods for Matlab & Ave ave.

This section describes how to calculate the cost function.

How can we evaluate the degree of fit?

When the square of the difference between the fitting value H and the actual value y tends to be 0, we think the fitting is good (because the closer Y and H are)

Note that H is a matrix of the same size as Y, and theta is the initial Theta = [0; 0] of a two-dimensional column vector. (emphasize the column vector again)

The problem is that the initial Theta is [0; 0]. How can we adjust the slope and intercept (theta) here?

If Theta is large, it is represented as H-y greater than 0, so alpha (a constant, used as the characterization precision) multiplied by this (H-Y ). it will get a positive number, so theta will decrease the value by subtracting this number.

Similarly, if Theta is smaller, it will subtract a negative number (because h-y is smaller than 0), then theta will become larger.

(It is emphasized that the theta used here becomes larger and smaller, which is not the whole matrix for a certain element in the theta matrix)

So long as the number of iterations is too large, we will get a very appropriate Theta so that the value of the cost function is small enough (in fact, we have iterated 1500 times in our experiment, so that the minimum cost function reaches 0 )!

Then we need to implement the computecost function. Here we provide the "two methods", which are essentially the same. method 1 uses the traditional calculation method, the second method is matrix-based computing ("more elegant)

function J = computeCost(X, y, theta)%COMPUTECOST Compute cost for linear regression%   J = COMPUTECOST(X, y, theta) computes the cost of using theta as the%   parameter for linear regression to fit the data points in X and y% Initialize some useful valuesm = length(y); % number of training examples% You need to return the following variables correctly J = 0;% ====================== YOUR CODE HERE ======================% Instructions: Compute the cost of a particular choice of theta%               You should set J to the cost.% % Implementation method one:% temp = (X*theta-y).^2;% J = sum(temp(:))./(2*m);% Implementation method two:temp = (X*theta - y);J = (temp'*temp)./(2*m);% =========================================================================end

Then complete the gradientdescent function. here you should be familiar with matrix operations...

function [theta, J_history] = gradientDescent(X, y, theta, alpha, num_iters)%GRADIENTDESCENT Performs gradient descent to learn theta%   theta = GRADIENTDESENT(X, y, theta, alpha, num_iters) updates theta by %   taking num_iters gradient steps with learning rate alpha% Initialize some useful valuesm = length(y); % number of training examplesJ_history = zeros(num_iters, 1);    for iter = 1:num_iters        % ====================== YOUR CODE HERE ======================        % Instructions: Perform a single gradient step on the parameter vector        %               theta.         %        % Hint: While debugging, it can be useful to print out the values        %       of the cost function (computeCost) and gradient here.        %       temp = (X'*(X*theta -y))./m;       theta = theta - (alpha*(temp));               % ============================================================        % Save the cost J in every iteration            J_history(iter) = computeCost(X, y, theta);    endend

OK! Finally, we will get the best Theta. The blue line is our result. Similarly, we can think that the larger the population, the more profitable the city.

Here we only analyze the relationship between profitability and population. If you are interested, we can also study the stability of profitability and use variance to portray it ~ I will not discuss it more here.

Then, when we know the optimal Theta, we analyze the changes of the cost function value in a certain range. The more we stay away from the best, the larger the cost function value, the worse the fitting effect!

Linear regression with multiple variables

Linear Fitting of multiple variables.

I am familiar with matrix operations and don't need to change the code. (the charm of the matrix ~)

Background:

The file ex1data2.txt contains a training set of housing prices in Portland, Oregon. the first column is the size of the house (in square feet), the second column is the number of bedrooms, and the third column is the price
Of the house.

Data about the relationship between room size, house size, and house price comes from Portland and Oregon.

X = [2104 3], y = 399900 the house size is 2104, there are 3 bedrooms, the house price is 399900, the other is the same
X = [1600 3], y = 329900
X = [2400 3], y = 369000
X = [1416 2], y = 232000
X = [3000 4], y = 539900

......

Why normalization?

By looking at the values, note that house sizes are about 1000 times the number of bedrooms. When features differ by orders of magncategory, first known Ming feature scaling can make gradient descent converge
Much more quickly.

This is not just to converge as soon as possible, but not to normalize, it will be a bug...

function [X_norm, mu, sigma] = featureNormalize(X)%FEATURENORMALIZE Normalizes the features in X %   FEATURENORMALIZE(X) returns a normalized version of X where%   the mean value of each feature is 0 and the standard deviation%   is 1. This is often a good preprocessing step to do when%   working with learning algorithms.% You need to set these values correctlyX_norm = X;mu = zeros(1, size(X, 2));sigma = zeros(1, size(X, 2));% ====================== YOUR CODE HERE ======================% Instructions: First, for each feature dimension, compute the mean%               of the feature and subtract it from the dataset,%               storing the mean value in mu. Next, compute the %               standard deviation of each feature and divide%               each feature by it's standard deviation, storing%               the standard deviation in sigma. %%               Note that X is a matrix where each column is a %               feature and each row is an example. You need %               to perform the normalization separately for %               each feature. %% Hint: You might find the 'mean' and 'std' functions useful.%           for temp = 1:size(X,2)        mu(temp) = mean(X(:,temp));        sigma(temp)    = std(X(:,temp));        X_norm(:,temp) = (X_norm(:,temp) - mu(temp))./sigma(temp);    end% ============================================================end

Computecostmulti function implementation

function J = computeCostMulti(X, y, theta)%COMPUTECOSTMULTI Compute cost for linear regression with multiple variables%   J = COMPUTECOSTMULTI(X, y, theta) computes the cost of using theta as the%   parameter for linear regression to fit the data points in X and y% Initialize some useful valuesm = length(y); % number of training examples% You need to return the following variables correctly J = 0;% ====================== YOUR CODE HERE ======================% Instructions: Compute the cost of a particular choice of theta%               You should set J to the cost.temp = (X*theta - y);J = (temp'*temp)./(2*m);% =========================================================================end

Implementation of gradientdescentmulti Functions

function [theta, J_history] = gradientDescentMulti(X, y, theta, alpha, num_iters)%GRADIENTDESCENTMULTI Performs gradient descent to learn theta%   theta = GRADIENTDESCENTMULTI(x, y, theta, alpha, num_iters) updates theta by%   taking num_iters gradient steps with learning rate alpha% Initialize some useful valuesm = length(y); % number of training examplesJ_history = zeros(num_iters, 1);    for iter = 1:num_iters        % ====================== YOUR CODE HERE ======================        % Instructions: Perform a single gradient step on the parameter vector        %               theta.         %        % Hint: While debugging, it can be useful to print out the values        %       of the cost function (computeCostMulti) and gradient here.        %        temp = X'*(X*theta - y)./m;        theta = theta - alpha*temp;        % ============================================================        % Save the cost J in every iteration            J_history(iter) = computeCostMulti(X, y, theta);    endend

The output value of the error function decreases rapidly as the number of iterations increases.

Finally, based on the formula to solve Theta, this method does not need to be iterated, but can be used to obtain an approximate solution (in essence, the transformation of the matrix, theta * x = y deformation is obtained)

function [theta] = normalEqn(X, y)%NORMALEQN Computes the closed-form solution to linear regression %   NORMALEQN(X,y) computes the closed-form solution to linear %   regression using the normal equations.theta = zeros(size(X, 2), 1);% ====================== YOUR CODE HERE ======================% Instructions: Complete the code to compute the closed form solution%               to linear regression and put the result in theta.%% ---------------------- Sample Solution ----------------------theta = (inv(X'*X))*X'*y;% -------------------------------------------------------------% ============================================================end

Solving with normal equations...
Theta computed from the normal equations:
89597.909543
139.210674
-8738.019112


Predicted price of a 1650 SQ-ft, 3 Br House (using normal equations ):
$91490.957664

Estimated 1650sq-ft size, 3 bedroom, the house needs about 91490 price

Inn brohe Danish canvas oil painting 110x70 cm private collection
This painting depicts the life scenes of Danes. Inn, small restaurant, dinner. The clothes in the painting are quite distinctive, and the characters are vividly painted by painters. The viewer can feel the male's poor eyes in the painting. The eyes of the two women on the opposite side also look into this direction, which makes people feel uneasy. What did they say? It must not be a good word. This is a manifestation of the techniques used by painters to send gods. They are amazed at the dilapidated walls behind the characters, which are a realistic photo. The author Carl Heinrich Bloch is a Danish painter in the 19th century.

Deep Learning # lab1 #