Introduction to Probability Density Estimation

1, Probability density function

In classifier design (especially Bayesian classifier), when the prior probability of a class and the probability density of a class are both known, determine the discriminant function and decision plane according to certain decision rules. However, in practice, the probability density of class conditions is generally unknown. Then, how can we determine the class when the prior probability and the probability density of the class condition are unknown or one of them is unknown? In fact, as long as we can collect a certain number of samples, we can infer the overall probability distribution from the sample set based on statistical knowledge. This estimation method is usually called probability density estimation. It is one of the basic problems of machine learning, and its purpose is to determine based on training samples.X(Random population) probability distribution. Density Estimation is divided into parameter estimation and non-parameter estimation.

 

2Parameter Estimation

Parameter Estimation: according to the general understanding of the problem, if the random variables are subject to a certain distribution (for example, normal distribution), the parameters of the distribution function can be estimated through training data. Parameter estimation can be divided into supervised parameter estimation and unsupervised parameter estimation. The two most common methods for parameter estimation are the maximum likelihood estimation method and Bayesian estimation method.

 

Supervised parameter estimation: the type of the sample and the general probability density of the condition are known. Some parameters that characterize the probability density are unknown.

Unsupervised parameter estimation: the type of a known sample, but the form of an unknown General probability density function, requires the probability density itself to be inferred.

 

3, Non-parameter estimation

Non-parameter estimation: the class of the known sample, but the form of the unknown overall probability density function requires us to directly infer the probability density function itself. That is, you do not need to use models to estimate the probability density only by using the training data.

The histogram method and the kernel method are commonly used for non-parameter estimation.PazenWindow Method andKNMethod.

Introduction to Probability Density Estimation