[Cs229-Lecture4] GLMS: select the exponential distribution family, how to use it to derive GLM?

There are three parts in lecture4:

• Newton's method Newton Method
• Exceponential family exponential distribution family
• Generalized Linear models generalized linear model (GLMS)

The Newton method, as mentioned in the previous article, is another optimization algorithm parallel to the gradient descent algorithm.

Then, the video proves that the bernuoli distribution and Gaussian distribution belong to the special cases in the exponential distribution family. In fact, they are converted into the exponential distribution family, then compare them one by one to determine whether they are consistent.

Next, let's talk about how to use it to derive a generalized linear model (GLM) After selecting an exponential distribution family.

First, we give three assumptions. At the same time, we take the bernuoli distribution as an example (01 problem modeling) to illustrate:

Given an exponential distribution family, we have proved that the bernuoli distribution is a special case of the exponential distribution family. Therefore, it satisfies the first hypothesis.

Given X, the goal is to find the expected E [T (y) | x] of T (y) with X as the condition, that is, let the learning algorithm output h (x) = E [T (y) | x]. Here, the expected value is equal to the probability of the dependent variable taking 1, which is generally written as p. This P covers the information of the 0-1 dependent variable and can be used to intuitively understand our problem: how does the probability of 1 change for the target variable when the independent variable changes.

Assume that the probability of taking 1 as the dependent variable is linearly related to the independent variable.

 

An example will be added later.

[Cs229-Lecture4] GLMS: select the exponential distribution family, how to use it to derive GLM?