The past and present of Sparse Coding (II)

To better understand the working mechanism (Strategy) of cerebral cortex for signal encoding, we need to convert them into mathematical languages, because mathematical languages are a rigorous language, you can use it to export the desired and desired program. This section uses Bayesian views to extend sparse encoding to images that change over time, because the signals acquired through the eyes during daily activities of humans or mammals change over time, there are still sparse coefficients and basis for such signals to describe them, processing Methods of the same type also involve slow Feature Analysis (slow features analysis ). If you don't talk much about it, go to the topic:

We regard the image stream (image sequence) as a linear combination of time-space basis functions plus noise. Of course, the time-space basis function can be imagined to be time-space unchanged, similar to the 3d-sift in behavior recognition, this seems to be related to the analysis of slow features. Similarly, the space-time basis function still has some coefficients. in representation, the image stream can be seen as the convolution of the space-time basis and coefficient plus some noise. The model is shown in (Formula 1:

(Formula 1)

The entire model can be presented in image (figure 1). The attention coefficient is a kind of things with a single peak similar to a burst. (Figure 1:

(Figure 1)

Of course, the space-time basis functions in (figure 1) should be as sparse as possible to reduce the amount of computing, otherwise the amount of computing on the image sequence will be too large. For solving model parameters, we assume that the coefficients are independent and sparse. Based on these assumptions, Bruno gives a prior formula for the coefficients, as shown in (formula 2:

(Formula 2)

Because the coefficients are independent, their joint distribution is decomposed into the product form of a single distribution, and each coefficient satisfies the sparse hypothesis. S is a non-convex function that controls the sparseness of the alpha coefficient. With these prior knowledge, the posterior probability of the alpha coefficient after the given image sequence is shown in (Formula 3:

(Formula 3)

To maximize the posterior probability, and then use the Gradient Descent Method to Solve the obtained alpha coefficient, the steps for solving all the solutions are shown in (formula 4:

(Formula 4)

Although there are so many formulas, they are not enough to explain the detailed steps for solving the coefficients. Because the last two items in Formula 3 are still unclear, we will make another assumption for these two items, as shown in formula 5:

(Formula 5)

Despite this assumption, P (I | alpha, theta) still cannot be directly computed and needs to be sampled to complete. This is where improvements are needed. However, we still stick to the learning base function step together to lay the foundation for subsequent improvement. The learning process is shown in Figure 2:

(Figure 2)

The alpha coefficient is completed by gradient descent, while the base function is updated by Hebbian learning. Hebbian learning is to strengthen the connection between activated cells ("Cells that firetogether, wire together. "), This can slightly explain the plastic working mechanism of cerebral cortex behind" hundreds of times. The learned basic functions are shown in Figure 3:

(Figure 3)

Well, the interpretation of the Sparse Coding life science is almost the same. We can see that the idea is good, but there are too many manual assumptions and the learning methods are not friendly. With the introduction of modern mathematics and lasso, sparse code gradually matures and begins to embark on the Application Path. In the deeplearning era, there are fewer and fewer manual components, and the power seems to be growing. (Well, I admit it is disgusting, but the biggest highlight of this section is coding in the empty time domain, which is helpful for behavior recognition and language recognition)

 

References:

Probabilistic models of the Brain: perception and neural function. MIT Press

Reprinted please indicate the link: http://blog.csdn.net/cuoqu/article/details/8989233