[Machine learning algorithm-python implementation] matrix denoising and normalization, python Machine Learning
1. The background project is required. We plan to use python to implement matrix denoising and normalization. The numpy mathematical library does not find ideal functions. Therefore, I wrote a de-noise and normalization algorithm in the standard library, which is a little inefficient, but still usable, you can use it if needed. (1) denoising algorithm: based on the knowledge of probability theory, if a group of data follows a normal distribution, we set the mean value to n and the variance to v, so for each discrete value more than 92% probability will be in (n-3 * v, n + 3 * v) range. Therefore, the noise removal function is mainly used to mark this value as the maximum value that can be tolerated if the range is exceeded. (2) Normalization: Find the maximum and minimum values of the input queue in min. For any independent variable x, the normalized value is (x-min/max-min ).
2. Implementation Code
from __future__ import divisiondef GetAverage(mat): n=len(mat) m= width(mat) num = [0]*m for j in range(0,m): for i in mat: num[j]=num[j]+i[j] num[j]=num[j]/n return numdef width(lst): i=0 for j in lst[0]: i=i+1 return idef GetVar(average,mat): ListMat=[] for i in mat: ListMat.append(list(map(lambda x: x[0]-x[1], zip(average, i)))) n=len(ListMat) m= width(ListMat) num = [0]*m for j in range(0,m): for i in ListMat: num[j]=num[j]+(i[j]*i[j]) num[j]=num[j]/n return num def DenoisMat(mat): average=GetAverage(mat) variance=GetVar(average,mat) section=list(map(lambda x: x[0]+x[1], zip(average, variance))) n=len(mat) m= width(mat) num = [0]*m denoisMat=[] for i in mat: for j in range(0,m): if i[j]>section[j]: i[j]=section[j] denoisMat.append(i) return denoisMat def AutoNorm(mat): n=len(mat) m= width(mat) MinNum=[9999999999]*m MaxNum = [0]*m for i in mat: for j in range(0,m): if i[j]>MaxNum[j]: MaxNum[j]=i[j] for p in mat: for q in range(0,m): if p[q]<=MinNum[q]: MinNum[q]=p[q] section=list(map(lambda x: x[0]-x[1], zip(MaxNum, MinNum))) print section NormMat=[] for k in mat: distance=list(map(lambda x: x[0]-x[1], zip(k, MinNum))) value=list(map(lambda x: x[0]/x[1], zip(distance,section))) NormMat.append(value) return NormMat
Library implementation: Input matrix mat,
GetAverage (mat): returns the mean value.
GetVar (average, mat): returns the variance
DenoisMat (mat): de-noise
AutoNorm (mat): normalization Matrix
: Https://github.com/jimenbian/AutoNorm-mat-
/********************************
* This article is from the blog "Li bogarvin"
* Reprinted please indicate the source: http://blog.csdn.net/buptgshengod
**************************************** **/
What languages are commonly used to develop machine learning algorithms? Generally, you can thoroughly study machine learning algorithms and customize the developer programming language based on your project needs.
Search for the following keywords on google by yourself: Multi-Machine Learning Algorithm Implementation
Machine learning in Java
Machine learning in C ++
Machine learning in Python
Machine learning in Matlab
Machine learning in R
What are common normalization methods in machine learning? Unitization, except for the token, I used two more books to search for information;
In addition, it is better to take the line of