Parametric/non-parametric learning algorithms

First, parametric Learning Algorithm (parametric learning algorithm)

Definition:　　 assuming that the learning process can be minimized, and at the same time limiting what can be learned, the algorithm simplifies to a known function form, an algorithm that fits data by a fixed number of parameters . 　

parameter Learning The algorithm consists of two steps:

• Choose the form of a target function
• Learning the coefficients of objective functions from the training data

Some common examples of parametric learning algorithms include:

• Logistic Regression
• LDA (linear discriminant analysis)
• Perception Machine
• Naive Bayesian
• A simple neural network

? Advantages of the Parametric machine learning algorithm:

• Simple : These algorithms are easy to understand and interpret results
• fast : Parametric models can quickly learn from data
• small amount of data : they do not require much training data and can even fit a defective number well

? Limitations of parametric machine learning algorithms:

• constraints : These algorithms select a function form height low limit model itself
• Limited Complexity : This algorithm may be more suitable for simple problems
• not appropriate : In practice, these methods are unlikely to match potential target (mapping) functions

Second, nonparametric Learning Algorithm (Non-parametric learning algorithm)

　　definition: algorithms that do not make strong assumptions about the form of objective functions are called nonparametric machine learning Algorithms, and by not making assumptions, they can freely learn any function form from the training data, that is, the number of parameters will increase with the number of training samples.

Some common examples of non-parametric learning algorithms include:

• Knn
• Decision trees, such as cart and C4.5
• Svm

? Advantages of non-parametric machine learning algorithms:

• Flexibility : Fits a large number of different function forms
• Ability : There is no need for assumptions about potential functions (or, if assumed)
• Performance : You can get a high-performance model for forecasting

? Limitations of non-parametric machine learning algorithms:

• more Data : More training data needed to estimate the target function
• Slow : training is slow because they often need to train more parameters
• overfitting: More over-fitting training data risk, and it's more difficult to explain why specific predictions are made

Note:

Local weighted linear regression is actually a non-parametric learning algorithm (Non-parametric learning algorithm);

Linear regression is a parametric learning algorithm (parametric learning algorithm) because its parameters are fixed and the parameters of local weighted linear regression are different with the prediction points.

Since each prediction is based on an instance point near the forecast point, each prediction is rerun the algorithm, resulting in a set of parameter values, so it is computationally expensive.

Parametric/non-parametric learning algorithms