Fishe Vector Fisher VECOTR (i) _fisher vector

In the Gaussian mixture model, I mentioned the general process of feature processing:

In fact, the Gaussian mixture model completes the task of K-means, then through the Gaussian mixture model clustering, also uses the general distance based method to carry On feature encoding. No, the Gaussian mixture model is usually used in conjunction with Fisher Vector. Now let's make a brief introduction to Fisher Vector.

First of all, Fisher vector is a feature coding method. There are few Chinese materials available on the Internet, and only one of the more valuable is Fisher vector. But to me this kind of mathematics not very good person looks still somewhat difficult, therefore has deduced all formulas, and has written again the rationale. So the outline of this blog is roughly the same as Fisher Vector, with just a little bit of understanding added to the brief. Kernel trick

To understand Fisher Vector, you must first understand the kernel. I agree with the third answer about the question of kernel. Kernel is not a mysterious thing, but a trick of calculation. In the permanent topic classification in the CV World, it is assumed that to train a (−1,1) (−1,1) Two classifier, when a new sample is available, the probability of a class Y Y is: P (y|x) p (y|x)

It can be seen that this is a discriminant model, modeled using logistic function (why use logistic function to refer to Andrew Ng's machine learning Handout): P (y|x;θ) =11+e−θtx P (y|x;θ) =11+ E−θtx

Our goal is to find the best θθ (θθ is a vector), if the sample is a lot, we can find a group of best θ^θ^ through the maximum likelihood estimate to achieve the best classification effect. But in the case of fewer samples, it can be assumed that the prior probability distribution of θθ obeys the Gaussian distribution of the mean 0 0, that is, Θ∼n (0,ξ), Ξθ∼n (0,ξ), ξ can be obtained by statistical sample, then there is P (θ) =σ (θ) =12πξ−−−√e−θtξ−1θ2 p (θ) =σ (θ) = 12πξe−θtξ−1θ2

Using the maximum posteriori probability, it is assumed that a total of n N Samples XI (i=1,2,3,..., N) XI (i=1,2,3,..., N) and independent of each other, then Θ^MAP=ARGMAXΘ∏I=1NP (yi|xi;θ) P (θ) θ^map=arg maxθ∏i= 1NP (yi|xi;θ) p (θ) LΘ=∏I=1NP (yi|xi;θ) p (θ) LΘ=∏I=1NP (yi|xi;θ) p (θ)

To lθlθ on both sides of the ln logarithm, then: l (θ) =ln (lθ) =∑i=1n (LNP (yi|xi;θ) +LNP (θ)) L (θ) =ln (lθ) =∑i=1n (LNP (yi|xi;θ) +LNP (θ))

θθ derivation: ∂l (θ) ∂θ=∂∂θlnp (θ) +∑I=1N∂∂ΘLNP (yi|xi;θ) ∂l (θ) ∂θ=∂∂θlnp (θ) +∑I=1N∂∂ΘLNP (yi|xi;θ) ∂∂ΘLNP (theta) =−∂∂θθtξ−1θ2=−ξ−1θ ∂∂ΘLNP (θ) =−∂∂ΘΘTΞ−1Θ2=−Ξ−1Θ∂∂ΘLNP (yi|xi;θ) =∂∂θ (Ln1−ln (1+e−θtxi)) =XIE−ΘTXI1+E−ΘTXI∂∂ΘLNP (yi|xi;θ) =∂∂θ (LN1−LN (1 + E−ΘTXI)) =xie−θtxi1+e−θtxi

Make ∂l (theta) ∂θ=0∂l (theta) ∂θ=0, with ∑i=1nxie−θtxi1+e−θtxi−ξ−1θ=0∑i=1nxie−θtxi1+e−θtxi−ξ−1θ=0

The solution process is as follows: ∑i=1nxie−θtxi1+e−θtxi=∑i=1nxi1+eθtxi=ξ−1θ∑i=1nxie−θtxi1+e−θtxi=∑i=1nxi1+eθtxi=ξ−1θ

11+eθtxi=σ (−θ) =λi 11+eθtxi=σ (−θ) =λi, then there are: Θ^=∑i=1nxiλiξθ^=∑i=1nxiλiξλi=−1eθ^txiλi=−1eθ^txi

The result is brought into P (y|x;θ) p (y|x;θ), and eventually: P (y|x;θ) =11+e∑ni=1λi (xtiξx) p (y|x;θ) =11+e∑i=1nλi (xitξx)

The K (xi,x) =xtiξx K (xi,x) is =xitξx as a kernel function, which is a linear kernel.

from:http://bucktoothsir.github.io/blog/2014/11/24/9-th/