Automatic sparse encoding-Visual Encoder

For a sparse automatic encoder trained, I want to see what the learned function looks like. For an image trained, calculate the output value of each hidden layer node:

The Visualized function is a function calculated by the hidden layer node, which takes a 2D image as the input parameter (ignoring the offset. In particular, we regard it as a non-linear feature of input. We would like to know: what kind of image can make it the biggest incentive? Another problem is that constraints must be added. Assuming that the input norm constraint is yes, it can be proved that hidden layer neurons can obtain the pixel input with maximum activation (all 100 pixels ,):

The image composed of these pixel gray values is displayed, and we can see what features are learned by hidden layer nodes.

If we train an automatic encoder containing 100 hidden layer nodes, We can visualize 100 images (one for each hidden layer node ). By testing the 100 images, we try to understand the overall effect of the hidden layer learning.

The following shows the learning result of a sparse encoder (100 hidden layer nodes with input images:

Each small square in provides an input image, which enables one of the 100 hidden units (hidden layer nodes) to obtain the maximum excitation. We can see that different hidden units have learned to perform Edge Detection in different positions and directions of the image. These features are useful for Object Recognition and other visual learning tasks. When applied to other fields (such as audio), this algorithm can also learn useful representations or features.

 

Learning Source: http://deeplearning.stanford.edu/wiki/index.php/Visualizing_a_Trained_Autoencoder

Automatic sparse encoding-Visual Encoder