Deeplenrnig study notes--what are features

The characteristic is the raw material of machine learning system, the influence to the final model is undoubted. If the data is well expressed as a feature, the linear model is usually able to achieve satisfactory accuracy.

First, the characteristics of the expression granularity:

　　What is the characteristic expression of the learning algorithm in a particle size that can play a role? In the case of a picture, pixel-level features have no value at all. For example the following motorcycles, from the pixel level, do not get any information at all, and they cannot be differentiated by motorcycles and non-motorcycles. And if the feature is a structural (or meaning) time, such as whether it has a handlebar (handle), whether it has a wheel (wheel), it is easy to distinguish between motorcycles and non-motorcycles, learning algorithm to play a role.

Second, the primary (shallow) characteristics of the expression:

Since the pixel-level feature representation does not work, what is the use of the representation?

Around 1995, Bruno Olshausen and David Field Two scholars served as Cornell University, who tried to use both physiology and computer techniques to study visual problems.

They collected a lot of black and white scenery photos, from these photos, extract 400 small fragments, each photo fragment size is 16x16 pixels, you may want to mark these 400 fragments as s[i], i = 0,.. 399. Next, from these black and white landscape photos, randomly extract another fragment, the size is 16x16 pixels, you might want to mark this fragment as T.

The question they raised was how to pick a set of fragments from these 400 fragments, s[k], and, by superposition, synthesize a new fragment, and this new fragment should be as similar as possible to the randomly chosen target fragment, while the number of s[k] is as small as possible. To describe in a mathematical language is:

Sum_k (a[k] * s[k])--T, where A[k] is the weighting factor when stacking fragments s[k].

To address this problem, Bruno Olshausen and David Field invented an algorithm for sparse coding (Sparse Coding).

Sparse encoding is the process of repeating iterations, with each iteration divided into two steps:

1) Select a group of s[k] and adjust the a[k] so that sum_k (a[k] * s[k]) is closest to T.

2) Fix a[k], in 400 fragments, select other more appropriate fragments S ' [K], replace the original s[k], so that Sum_k (a[k] * S ' [K]) closest to T.

After several iterations, the best s[k] combination was selected. Surprisingly, the selected S[k] are basically the edge lines of different objects on the photo, these segments are similar in shape and differ in direction.

The algorithmic results of Bruno Olshausen and David Field coincide with the physiological discoveries of David Hubel and Torsten Wiesel!

In other words, complex graphs are often made up of some basic structures. For example: A graph can be expressed linearly by using 64 orthogonal edges (which can be understood as an orthogonal basic structure). For example, the X can be used in 1-64 edges three in accordance with the weight of 0.8,0.3,0.5. The other basic edge has no contribution, so they are all 0.

Third, the structural characteristics of the expression:

Small pieces of graphics can be composed of basic edge, more structured, more complex, conceptual graphics how to express it? This requires a higher level of feature representation, such as v2,v4. So V1 see pixel level is pixel level. V2 See V1 is the pixel level, this is the level of progressive, high-level expression by the combination of the underlying expression. The professional point is that it is the base basis. V1 basis is the edge, and then V2 layer is V1 layer of these basis combination, this time V2 area is also a layer of basis. That is, the result of the basis combination on the upper layer is the upper layer of the combination of basis ...

Intuitively speaking, is to find makes sense of the small patch and then combine it, get the upper layer of feature, recursively learning feature upward.

Doing training on different objects is, the resulting edge basis is very similar, but the object parts and models will completely different (then we can distinguish car or face is not much easier)

Deeplenrnig study notes--what are features