Data preprocessing in MATLAB-normalization (Mapminmax) and normalization (MAPSTD)

First, Mapminmax

Process matrices by mapping row minimum and maximum values to [-1 1]

This means that each row of the matrix is processed into [ -1,1] intervals, for pattern recognition or other statistics, the data should be each column is a sample, each row is the same dimension of multiple samples, that is, for a m*n matrix, the sample dimension is M, the sample number is N, a total of n column n samples.

The main invocation forms are:

1. [Y,ps] = Mapminmax (X,ymin,ymax)

2. [Y,ps] = Mapminmax (X,FP)

3. Y = Mapminmax (' Apply ', X,ps)

4. X = Mapminmax (' reverse ', y,ps)

5. Dx_dy = Mapminmax (' Dx_dy ', x,y,ps)

For the 1 and 2 invocation forms, X is the preprocessed data, ymin and Ymax are expected to be the minimum and maximum value of each line, and the FP is a struct member primarily Fp.ymin, Fp.ymax. This structure can replace the effects of ymin and ymax,1 and 2. Only the parameters are brought in different forms.

Code:

X=[2,3,4,5,6;7,8,9,10,11];mapminmax (x,0,1) Fp.ymin=0;fp.ymax=1;mapminmax (X,FP)

For 3, in pattern recognition or statistics, PS is a training sample of data mapping, that is, PS contains the training data of the maximum and minimum value, here x is a test sample, for the test sample, the preprocessing should be consistent with the training sample, the maximum and minimum value should be the maximum and minimum value of the training set. Assuming Y is a test sample, with a total of two test samples, the code is as follows:

x=[2,3,4,5,6;7,8,9,10,11];y=[2,3;4,5]; [Xx,ps]=mapminmax (x,0,1); Mapminmax (' Apply ', Y,ps)

For 4-type, the data after preprocessing is reversed to get the original data.

x=[2,3,4,5,6;7,8,9,10,11];y=[2,3;4,5]; [Xx,ps]=mapminmax (x,0,1); Yy=mapminmax (' Apply ', y,ps); Mapminmax (' reverse ', yy,ps)

For 5, theinverse number of wizards is obtained based on the given matrix X, the normalized matrix Y , and the mapping PS (reverse derivative). If the given X and Y are the matrices of the m row n columns, the result Dx_dy is a 1xn struct array with each element being a m x N the diagonal matrix. This usage is not commonly used and is no longer an example here.

Second, Mapminmax principle and its realization

The mathematical formula for Mapminmax is y = (ymax-ymin) * (x-xmin)/(xmax-xmin) + ymin. If the data for a row is all the same, xmax=xminAt this point, thedivisor is 0, the data is unchanged at this time.

The MATLAB implementation is:

function [Out]=mymapminmax (X,ymin,ymax) out= (ymax-ymin). * (X-repmat (min (x,[],2), 1,size (x,2)))./repmat ((Max (x,[],2 )-min (x,[],2)), 1,size (x,2)) +ymin;index=isnan (out); Out (index) =x (index); end

Note that the above code assumes that the sample in data x is a column vector.

Iii. Standardization of MAPSTD

Process matrices by mapping each row's means to 0 and deviations to 1: maps each line of the matrix to 0 mean 1 variance data.

The main invocation forms are:

1. [Y,ps] = MAPSTD (X,YMEAN,YSTD)

2. [Y,ps] = MAPSTD (X,FP)

3. Y = MAPSTD (' Apply ', X,ps)

4. X = MAPSTD (' reverse ', y,ps)

5. Dx_dy = MAPSTD (' Dx_dy ', x,y,ps)

Similar to Mapminmax, the 1 and 2 are standardized for data x, where Ymean and YSTD are expected to get the mean and variance of each row of data, and similarly, we can also use a struct containing ymean and ystd to carry in.

x=[2,3,4,5,6;7,8,9,10,11];y=[2,3;4,5]; [Xx,ps]=mapstd (x,0,1) fp.ymean=0;fp.ystd=1; [Xx,ps]=mapstd (X,FP)

3 is the pretreatment of the test data, using the mean and variance of the training data to deal with, 4 is the data reversal after preprocessing.

x=[2,3,4,5,6;7,8,9,10,11];y=[2,3;4,5]; [Xx,ps]=mapstd (x,0,1); YY=MAPSTD (' Apply ', y,ps); MAPSTD (' reverse ', yy,ps)

IV. Realization of MAPSTD standardization

the formula is y = (x-xmean) * (YSTD/XSTD) + Ymean . If the ystd=0is set, or if thedata for a row is all the same ( xstd =0 at this point )

function [out] = MYMAPSTD (X,YMEAN,YSTD) out= (X-repmat (Mean (x,2), 1,size (x,2)))./repmat (STD (x,0,2), 1,size (x,2)). * Ystd+ymean;end

V. Description of functions such as mean, STD, etc.

Mean default is to sum each column, mean (x,2) is the sum of each row, the STD function by default is the unbiased estimate of the standard deviation, there are three usages, s = STD (x), s = STD (x,flag), s = STD (X,flag,dim)

Where flag is an unbiased estimate of the parameters, flag=0 is unbiased estimation, that is, the default is unbiased estimation, flag=1 is biased estimation, Dim indicates the variance of the first dimension, STD (x,0,2) for each row of X to do unbiased standard deviation estimation.

Data preprocessing in MATLAB-normalization (Mapminmax) and normalization (MAPSTD)