Locality-Constrained Linear coding for Image Classification

Introduction

This paper presents a simple but valid tive coding scheme called locality-constrained
Linear coding (LLC) in place of the VQ coding in traditional SPM.

Feature quantization Mechanism

LLC utilizes the locality constraints to project each descriptor into itslocal-coordinate system, and the projected coordinates are integrated bymax poolingto generate the final representation. Added local constraints

Compared with the Sparse Coding Strategy, the objective function used by LLC has an analytical solution. in addition, the paper proposes a fast approximated LLC method by first known Ming a K-nearest-neighbor search and then solving a constrained Least Square Fitting Problem

Compared with Sparse Coding, LLC provides resolution solutions with low computing cost and high computing speed. It can be used for real-time tasks.

Process

A typical flowchart of the SPM approach based on BOF is wrongly strated on the left of Figure 1. first, feature points are detected or densely located on the input image, and descriptors such as "Sift" or "color moment" are extracted from each feature point (highlighted in Blue Circle in Figure 1 ). this obtains the "descriptor" layer. then, a codebook with mentries is applied to quantize each Descriptor and generate the "code" layer, where each descriptor is converted into anrmcode (highlighted in green circle ). if hard Vector Quantization (VQ) is used, each code has only one non-zero element, while for soft-VQ, a small group of elements can be non-zero. next in the "SPM" layer, multiple codes from inside each sub-region are pooled together by averaging and
Normalizing into a histogram. Finally, the histograms from all sub-regions are concatenated together to generate the final representation of the image for classification.

1. Discover interest points from input images

2. Apply the feature descriptor to the point of interest to obtain the feature vector.

3. quantize features to get the codebook

4. Feature encoding-if it is hard voting, each feature corresponds to a code; if it is soft voting, each feature corresponds to a set of code; if it is SPM, the feature corresponds to the Code in sub-region, use averaging pooling and normalization to form a histogram, and connect the histograms of these subareas to form the final image expression.

However, to achieve better performance, we need to use nonlinear kernel SVM when using SPM, which results in a high computing cost and average performance.

 

Scspm uses sparse codeing for non-linear coding (replacing the original kmeans)

During the scspm experiment, we found that the feature is always related to the code that is close to it. We recommend that you add a local constraint to encourage local code -- locality is more essential than sparsity

In addition, the objective functions of LLC are optimized.Resolution,High computing cost ~~ Can be used for real-time tasks

Locality-Constrained Linear Coding

Sparse encoding is so awesome ~~

 

Locality is more essential than sparsity, as locality must lead to sparsity but not necessary vice versa

LLC uses local constraints to replace sparse constraints and achieves some good properties.

In the above formula, B is basis, C is coefficient, and D is a local constraint of the same dimension as the base.

Between D and C is an element-level multiplication.

(4) formula is the structure of D. theata is used to adjust the weight, and D must be normalized.

Properties of LLC

To achieve good classification performance, the coding scheme shoshould generate similar codes for similar descriptors.

1. Better reconstruction-or smaller Quantization Error

2. Local smoothing and sparsity-compared with Sparse Coding, the rule items of SC containing L1 paradigm are not smooth, and because the foundation of SC is too complete, similar features may be expressed by different bases, which may lead to certain errors.

3. Analytical Solution-the solution to SC is cumbersome

Approximated LLC for fast Encoding

LLC has a resolution solution, but we can select K codes near the feature to construct its local coordinate system to accelerate computing.

In addition, LLC can have a huge codebook, but each feature-related code is just a few

Codebook Optimization

According to our experimental results in subsection 5.4, the codebook generated by K-means can produce satisfactory accuracy. in this work, we use the LLC coding criteria to train the codebook, which further improves the performance.

Analysis

1. The learning method of codebook does not differ much between kmean and the author, but the recognition rate will increase with the increase of codebook.

2. The effect of the number of local codes, that is, the effect of K. The smaller the value, the higher the recognition rate ~~ (But cannot be less than 5)

3, we tested the performance under different constraints other than the shift-invariant constraint -- the shift-invariant constraint leads to the best performance.

Summary

LLC uses local constraints to replace sparse constraints of SC, and the performance is improved to the next level.

In particular, the proposed nearest neighbor coding method can complete real-time tasks.

I think the feature encoding is about to take a while ~~

Locality-Constrained Linear coding for Image Classification