GLM (Generalized linear model) and LR (logistic regression) detailed

Source: Internet
Author: User
Tags constant
GLM Generalized linear model

George Box said: "All models is wrong, some is useful" 1. Starting with the Linear Model

As a foundation of GLM, this section review the classic Linear Regression, and expounds some basic terms.
The basic formula for our linear regression is essentially to look at x x and then predict Y y with a simple linear function h (x) H (x):
Y=h (x) =wtx y=h (x) =w^tx

1.1 dependent variable y y

This is our forecast target, also called response variable. There is an easily confusing point, in fact Y y can express three meanings (modeled by distribution, observed by sampling, predicted by expectation): distribution; When response variable is discussed abstractly, We are actually concerned about the given data and parameters, y| X,w y|\ x,w the distribution of obedience. Linear Regression y y obeys the Gaussian distribution, the specific value is the real number, but here we are concerned about the distribution. observed outcome; Our label, sometimes denoted by T-T, is the true observed result, just a value. Expected outcome; Y=e[y|x]=h (x) y=\mathbf{e}[y|x]=h (x) represents the prediction of the model; Note y y actually obeys a distribution, but the predicted result is the mean value of the entire distribution μ\mu, just a value. 1.2 Independent variable x x

This is our feature, which can contain many dimensions, a feature also known as a predictor. 1.3 hypothesis H (x) H (x)

The assumption of the linear model is very simple, i.e. h (x) =wtx h (x) = W^tx inner product of the weight vector and feature vector, known as linear predictor. This is the linear model, and GLM is also a generalization of the base.
In depth, the various dimension features (Predictor) XJ x_j through the coefficients WJ w_j linear Plus, this process integrates the information, while the different weight (coefficient) reflect the different contribution of the relevant characteristics. 2. Extended to generalized Linear Model 2.1 Motive & Definition

The linear model has a very strong limitation, that is, response variable y y must obey the Gaussian distribution; The main limitation is that the scale of the fitted target y y is a real number (−∞,+∞) (-\infty,+\infty). In particular, there are two problems: the range of values for Y Y and some common problems do not match. For example, count (number of visitors is constant) and binary (a two classification problem) the variance of Y Y is constant constant. Some problems may depend on the mean value of Y Y, for example, I predict the more generous the target value (the less accurate predictions)

So then we use the generalized Linear Model to overcome these two problems.
Define GLM as a sentence (from wiki):
In statistics, the generalized linear Model (GLM) was a flexible generalization of ordinary linear regression that allows F Or response variables that has the error distribution models other than a normal distribution.
in detail, we can decompose GLM into Random Component, System Component, and Link Function three parts . 2.2 Random Component

An exponential family model for the response

Here refers to the response variable must obey a exponential family distribution exponential family distribution, namely Y|x,w

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.