Machine Learning notes: Logistic Regression

The logistic regression algorithm is well-known and is said to be widely used in engineering practice. As a newbie, I first heard about dragonstar. I didn't understand it at the time because Yu Kai spoke fast. I attended the cs229 lesson today and found the notes and procedures of the cool man.

Logistic regression is a regression algorithm, which is different from linear regression. Its Y is a discrete random variable with the Bernoulli distribution and values of 0 or 1, while the y of linear regression is a continuous random variable with Gaussian distribution. For linear regression, its H function is a linear combination of feature X, while the logistic regression H function is a sigmoid function of feature X-ray combination. The two regression (training) methods are different: linear regression obtains the weight parameter theta by using the quadratic loss function. For logistic regression, you can also use the loss function of linear, but in logistic regression, the function is "non-convex", so the maximum likelihood function p (Y | X) is used. (I think this is not the proper method. p (x | Y) is the likelihood function, P (Y | X) is the posterior probability. In fact, the maximum likelihood function is a common method. If we maximize the likelihood function of Y in the linear regression, we will also obtain the quadratic loss function ).

Since the output variable Y of Logistic regression is only 0 or 1, it is widely used in two categories of classification problems. If y = 1 ~ K, you can solve the k class classification problem, that is, the softmax algorithm to be studied later.

Reference link:

1. http://v.163.com/movie/2008/1/E/D/M6SGF6VB4_M6SGHKAED.html

2. http://52opencourse.com/125/coursera%E5%85%AC%E5%BC%80%E8%AF%BE%E7%AC%94%E8% AE %B0-%E6%96%AF%E5%9D%A6%E7%A6%8F%E5%A4%A7%E5%AD%A6%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0%E7%AC%AC%E5%85%AD%E8%AF%BE-%E9%80%BB%E8%BE%91%E5%9B%9E%E5%BD%92-logistic-regression

3. http://blog.csdn.net/pennyliang/article/details/7045372