Robot Learning Cornerstone (Machine learning foundations) Learn the cornerstone of the work after three lessons to solve the problem

Today we share the coursera-ntu-machine learning Cornerstone (Machines learning foundations)-exercise solution for job three. I encountered a lot of difficulties in doing these topics, when I find the answer on the Internet but can not find, and Lin teacher does not provide answers, so I would like to do their own questions on how to think about the writing down, for everyone to provide some ideas. Of course, my understanding of the topic is not necessarily correct, if you bo friends found errors please contact, thank you! Again: Please do not use this blog as a way to pass the exam, or better to learn and understand the course! Hope my blog is helpful to your study!

The source of this article: http://blog.csdn.net/a1015553840/article/details/51103628

Other job analysis See summary stickers: http://blog.csdn.net/a1015553840/article/details/51085129

The third homework Lin teacher mainly talked about four pieces of content:

1. Linear regression (Linear Regression): Linear regression equation, error calculation method is square error, Parameter W calculation method, Pseudo-inverse, linear regression implementation step, etc.

2. Logistic Regression: Logistic regression theory, error calculation method cross entropy errors, gradient descent method to find the optimal solution and push to the gradient descent formula of logistic regression

3. Linear models (Linear model for classification): It is proved that linear regression and logistic regression can also be used for linear classification (VC bound), random gradient descent method (stochstic), Multi-class classification Ova,ovo

4. Non-linear conversion (Noliear transformation): This paper introduces how to transform the high-level curve in the low-dimensional space into the linear classification in the high-latitude space, and shows his limitation (the conversion cost is large).

1. First question

(1) Test instructions: Add noise epsilon to the objective function f, and the calculation formula for the noise intensity of Sigma^2,ein is also given. The minimum number of samples that can make Ein greater than 0.008 when a given sigma=0.1,d=8 n

(2) Analysis: This is a very common calculation problem, the value into the calculation can be

(3) Answer: 100

2. The second question

(1) Test instructions: The definition of Hat Marix H has been given to ask five statements which are correct

(2) Analysis: The teacher said in class that the role of H is to project Y from n-dimensional space to the X-d+1 dimension space. (i-h) refers to the vertical distance between Y and the expansion plane. Diagonal Trace (I-h) = N (d+1)

A.H is semi-definite, right, reason Bo master does not know

B.H is reversible, wrong, blogger personally think is wrong

Some eigenvalues of C.H may be greater than 1, error. The eigenvalues of semi-positive matrices are less than or equal to 1

D. The individual feels that it is correct. Trace (i-h) =n-(d+1)

E. Correct. Because the role of H is to project Y to hat y, the projection is constant no matter how many times it is

(3) Answer: none of above

3. Question Three

(1) Test instructions: Which of the following formulas is the upper limit of Chinese children?

(2) Analysis: The figure for making these six error is shown below

Can see that only the fifth one is his upper limit

(3) Answer: Max (0,1-YWX) ^ 2

4. Question Fourth

(1) Test instructions: Which of the following are not for W everywhere

(2) Analysis: It is clear that Max (0,-YWX) in 0 is not differentiable, the left derivative is-1, the right derivative is 0

(3) Answer: Last item

(1) Test instructions: Using the random gradient descent method to calculate the following error functions, ignoring individual non-micro points, which result is PLA

(2) Analysis:

(3) Answer: Max (0,-YWX)

6. Question Sixth

(1) Test instructions: Ask for the gradient of E near (0,0)

(2) Analysis: Very simple, respectively, the u,v of E to the reciprocal, and then will (0,0) into the can

(3) Answer: ( -2,0)

7. Question Seventh

(1) Test instructions: Using the gradient descent formula above, starting from (0,0) iteration, the step is 0.01, the iterative five times after the error E

(2) Analysis: This is too simple, just use C or matlab to write an algorithm can be, Bo master here no longer tired

(3) Answer: 2.825

8. Title

(1) Test instructions: This is about $ two Taylor unfold

(2) Analysis: Two Yuan Taylor expansion formula for Http://wenku.baidu.com/link?url=TFtNo_ C9ov4mqsshuxz6yfmnjf-2a4-94zmsss5qyepqhnir2al6ubn3yjscrmdfpy4_zt2andpoxv0gii3b3qjbom1xdmb-8czyugv-a3s

Bring in the calculation. Pay attention to the correspondence with each item.

(3) Answer: (1.5,4,-1,-2,0,3)

9. The ninth question

(1) Test instructions: Using Hessian matrix to calculate Newton's direction

(2) Analysis: Hessian Matrix See Http://baike.baidu.com/link?url=zCgekuYg4ViCDXyjWlpQZPEfGXZoUGl7bP8lpe_ N6ww7bselqyyikduortvabdjw9kbhixjmcml2s5zdeib2y_

Newton iteration See http://blog.csdn.net/luoleicn/article/details/6527049

(3) Answer: the first

10. Question Tenth

(1) Test instructions: Using Newton iterative iteration to find the e-value of iteration five times

(2) Analysis: We have learned from the 9th question Newton direction, their own use of MATLAB to write a simple implementation on it, here is no longer tired of

(3) Answer: 2.361 (as you can see, Newton iterates five times for 2.361, the gradient drops five times to 2.825, which means Newton's iteration is faster!) )

11. Question 11th

(1) Test instructions: The set of minimum x that can be shatter by all two times + one curve

(2) Analysis:

Use six lines on the way to easily shatter

(3) Answer: x1,x2,x3,x4,x5,x6

12. Question 12th

(1) Test instructions: Use the above equation to convert the D dimension of the X space to Z space for the converted D_vc

(2) Analysis: Adding x has n points, then each point using [x = xn] can be converted to a dimension of z, then the resulting z space after the conversion of the dimension is N. What we are going to do is to do a two-tuple classification in z space, because we have previously deduced the course T, and D is a space-aware machine (perceptrons) D_VC = d+1. So Z-space gets the D_VC = N +1. Then for any n points can be shatter. So n take any large, d_ve = n+1, any large n can be shatter, so D_VC = Infinity

(3) Answer: Infinity

The C + + implementation of question 13th to 15th see: http://blog.csdn.net/a1015553840/article/details/51085094

16. Question 16th

(1) Test instructions: This question is about the cost function derivation of multivariate logistic regression.

(2) Analysis: The author also does not understand the problem, and then search for information on Baidu, the domestic website does not seem to be too many to write multiple logistic regression article. So I found the relevant information on a foreign site, but did not see the derivation process. The URL is: http://blog.datumbox.com/machine-learning-tutorial-the-multinomial-logistic-regression-softmax-regression/. He did it according to Wunda's theory, where J (Theta) is what we call the Ein.

(3) Answer: the second

17. Question 17th

、

(1) Test instructions: The meaning of this problem is that we know from the 16 of the Ein method of calculation, his partial derivative of WI

(2) Analysis: derivative can be

(3) Answer: item Fourth

18.18–20 the C + + implementation of the question, see: http://blog.csdn.net/a1015553840/article/details/51085835

The source of this article: http://blog.csdn.net/a1015553840/article/details/51103628

For other questions please see summary stickers: http://blog.csdn.net/a1015553840/article/details/51085129

Robot Learning Cornerstone (Machine learning foundations) Learn the cornerstone of the work after three lessons to solve the problem