A little understanding of gradient descent algorithm

Yesterday, I started reading Professor Ng's machine learning class and found that it was a good course, and I saw the second lesson in one breath.

First Lesson

There is no new knowledge, it is the general situation of machine learning.

Lesson two

There are some concepts that you may not understand very well. In fact, this class is mainly about an algorithm, gradient descent algorithm . By the time the Professor deduced the formula, he felt a little bit blindfolded, but after thinking about it, he probably understood that the algorithm was not as obscure as it imagined. In this lesson, the gradient descent algorithm is used to solve the linear regression problem. The example in the video is to give you a bunch of training data (house size and corresponding room rate), if you give you a no-show house area data, can you give the correct price? The solution to the idea is to see that the relationship between the House area and the housing price is linear, is a linear regression problem, we need to fit an approximate curve to describe the relationship between the two. Then we want to know that the linear regression problem can be solved by gradient descent algorithm, then it is natural to use gradient drop to program. What is the principle of gradient descent? To simplify the problem to two-dimensional space, the only thing you have to do is to find a proper line segment to string all the training data in the space, like a string of candied fruit, so that as much data as possible passes through this line segment. How to describe a line in a mathematical way? Of course it is to find its slope, and the gradient in two-dimensional space is the slope, so the gradient descent algorithm is to calculate the slope of this line of all training data lines. Then you can get the relationship between the house price and the housing area. There is a clear linear relationship between the data can be done in this way. Prof Ng also mentions that for small data to be used in the most primitive gradient descent algorithm, the improved stochastic gradient descent algorithm can be used to improve the efficiency of big data.

A little understanding of gradient descent algorithm