The principle of similar image search

Two years ago, I wrote the principle of similar image search, and introduced the simplest method of implementation.

Yesterday, I saw on the Isnowfy website, there are two other methods also very simple, here do some notes.

First, the color distribution method

Each picture can generate a histogram of color distributions (histogram). If the histogram of the two pictures is close enough, you can think of them as similar.

Any one color is made up of red, green, and Blue (RGB), so there are 4 histogram (the histogram of the primary color + the last synthesized straight-side graph).

If you can take 256 values for each primary color, there are 16 million colors (256 of three) for the entire colour space. For these 16 million kinds of color comparison histogram, the calculation is too large, so the need to adopt a simplified method. 0~255 can be divided into four districts: 0~63 is the No. 0 district, 64~127 is 1th, 128~191 is 2nd, and 192~255 is the 3rd area. This means that red and green blue have 4 zones, which can form a total of 64 combinations (4 of 3).

Any one color is bound to belong to one of these 64 combinations, so you can count the number of pixels that each combination contains.

is a picture of the color distribution table, the last column in the table is extracted, composed of a 64-dimensional vector (7414, 230, 0, 0, 8, ..., 109, 0, 0, 3415, 53929). This vector is the characteristic value of this image or "fingerprint".

So, looking for a similar image becomes the vector that finds its closest resemblance. This can be calculated using Pearson correlation coefficients or cosine similarity.

Second, the content characteristic method

In addition to the color composition, you can also start by comparing the similarity of the picture content.

First, turn the original into a smaller grayscale image, assuming 50x50 pixels. Then, determine a threshold value to turn the grayscale image into black and white.

If the two images are similar, their black and white contours should be similar. So the question becomes, how does the first step determine a reasonable threshold and correctly present the outline in the photo?

Obviously, the larger the contrast between foreground and background color, the more obvious the contour. This means that if we find a value, we can make the difference between the foreground and background colors "Within the class" (Minimizing the Intra-class variance), or the "Maximum difference between classes" (Maximizing the Inter-class Variance), then this value is the ideal threshold value.

In 1979, the Japanese scholar Dajing show proved that "the smallest difference between classes" and "the most difference between classes" is the same thing, that is, the same threshold value. He proposes a simple algorithm that can be used to find this threshold, known as the "Dajing" (Otsu's method). Here's how he calculates it.

Suppose a picture has n pixels, where the grayscale value is less than the threshold pixel is N1, and pixels greater than or equal to the threshold are N2 (n1 + n2 = N). W1 and W2 represent the respective weights of the two pixels.

W1 = n1/n

W2 = n2/n

It is assumed that the mean and variance of all pixels with grayscale values less than the threshold are μ1 and σ1, respectively, and the average and variance of all pixels with a grayscale value greater than or equal to the threshold is μ2 and σ2. So, you can get

Intra-Class differences = W1 (σ1 squared) + W2 (σ2 squared)

inter-class differences = W1W2 (μ1-μ2) ^2

It can be proved that the two formulas are equivalent: The minimum value of the "in-class difference" is obtained, which equates to the maximum value of the "inter-class difference". However, from the computational difficulty, the latter is easier to calculate.

The next step is to use the "exhaustive method", the threshold from the lowest value of the gray level to the highest value, then take again, respectively, into the above formula. The value that makes the "smallest difference within the class" or "Maximum difference between classes" is the final threshold value. For specific examples and Java algorithms, see here.

With the black and white thumbnails of 50x50 pixels, it is equivalent to having a 50x50 0-1 matrix. Each value of the matrix corresponds to one pixel of the original, 0 for Black, and 1 for white. This matrix is the characteristic matrix of a picture.

The fewer of the two feature matrices are, the more similar the two images represent. This can be achieved with "XOR" (that is, only one of the two values is 1, then the result of the operation is 1, otherwise the result of the operation is 0). For different images of the characteristics of the matrix to "XOR", the results of less than 1, is the more similar pictures.

The principle of similar image search