Machine learning Note (--sklearn) Examples of dimensionality reduction methods (Randomprojection,tsvd,t-sne)

Examples of Sklearn dimensionality reduction methods importing related packages with datasets.digits data as an example

Import NumPy as NP
import pandas as PD
import matplotlib.pyplot as Plt
import time from
sklearn.datasets I Mport Load_digits

Visualization of large sample data is a relatively troublesome thing, In general, we will use the dimensionality reduction method to deal with the characteristics first. Let's look at an example to see what we can do, such as our data set to take the classic "handwritten number set" Each datapoint is a 8x8 image of a digit. classes->10 Samples per class->180 Samples total->1797 dimensionality->64 Features integers->0-16 Import data, 0,1,2,3,4,5,6 These 7 numbers of handwritten images

digits = Load_digits (n_class=7) #integer
X = digits.data
y = digits.target
n_samples,n_features = X.shape

Output Image

From matplotlib import Offsetbox def plot_embedding (x, Title=none): x_min, X_max = np.min (x, 0), Np.max (x, 0) X = (x-x_min)/(x_max-x_min) #正则化 print X plt.figure (figsize= (ten)) ax = Plt.subplot (111) for I in rang
                 E (x.shape[0]): Plt.text (x[i, 0], x[i, 1], str (digits.target[i]), Color=plt.cm.set1 (Y[i]/10.),
        fontdict={' weight ': ' bold ', ' size ': ') #打印彩色字体 if hasattr (Offsetbox, ' Annotationbbox '):  # only print thumbnails with matplotlib > 1.0 shown_images = Np.array ([[1., 1.]]) # just something big for I in range (Digits.data.shape[0]): Dist = Np.sum ((x[i]-shown_images) * * 2, 1
            ) if Np.min (dist) < 4e-3: # don ' t show points that is too close continue Shown_images = Np.r_[shown_images, [x[i]]] Imagebox = Offsetbox. Annotationbbox (Offsetbox. Offsetimage (Digits.images[i], Cmap=plt.cm.graY_r), X[i]) ax.add_artist (imagebox) #输出图上输出图片 plt.xticks ([]), Plt.yticks ([]) if Title I
 s not None:plt.title (title)

1. Random projection (random-projection) First we look at a problem, if you have a set of data on hand x∈rn x∈r n X \in r^n, its dimensionality is too high, and thus have to be reduced to Rk R K r^k, what will you do. It is believed that many people x′x the x′x x ' x in a reflexive way, and then convert it to the extended maximum variance expansion problem. Of course, this method needs to solve intrinsic values and intrinsic vectors. It all seems good for small data for testing, but if I give you a 50G size CSV format for Twitter speaking. If you need to dimension all the words, I'm afraid half a month. So we need a more rapid approach. so now think again, since to be fast, then we will use the simplest method: randomly select several unit vectors in the high-dimensional space ei e i e_i, but note, here we only ask is the unit vector, and do not require that they must be orthogonal <ei,ej>=0 < e i, E J >= 0 = 0, so you can choose freely. Finally, we project the high-dimensional data onto the selected set of bases. Also completed the dimensionality reduction. How to choose the axis can guarantee the dimensionality reduction effect. But it is mathematically based, called Johnson-lindenstrauss Lemma. This theorem guarantees the upper and lower limits of the accuracy of any dimensionality reduction method. Johnson-lindenstrauss Lemma Suppose we have the data x∈rn x∈r n x \in r^n, and we reduce it by a method F (x) f (x) f (x) Wi Cheng y∈rk y∈r k y \in r^k, then, will be any two points before and after, A, B, a, The distance has an inequality guarantee:

(1−ϵ) ∥a−b∥2≤∥f (a) −f (b) ∥2≤ (1+ϵ) ∥a−b