High-end image theme Color extraction algorithm

The visual and Visual Analysis Team at the CAD&CG State Key laboratory of Zhejiang University has compiled the following articles, which are summarized below.

It is important to introduce the Stanford Visualization Group, one of the two leaders is Professor Pat Hanrahan, who spans two areas of information visualization and scientific visualization, Even people who have never heard of the name should also know that the 2013 red-hot data visualization listed company tableau, and he is the co-founder of the company, Tableau based on his Polaris system, another leader named Jeffrey Heer, In recent years He is very famous in the fields of information visualization and human-computer interaction, and the two characteristics of innovation and practicality are both in his paper and a talented person.

In this article, they solve the problem of color extraction of image theme, which belongs to the theme of color modeling. same year A paper on Eurovis, Chi, and Siggraph has published articles on Color modeling (SIGGRAPH's thesis is now in conditionally accepted state) and is one of the best papers in Eurovis and Chi. It can be said that they are thick and thin hair.

The general theme color extraction methods include K and fuzzy C-means clustering method of pixel color value and the method of peak color histogram. In fact, the idea of the paper is not complicated, the image defines a series of characteristics, apply multivariate linear regression model lasso, set up the task collection training set on the platform Amazon Turkish robot, lasso the weight of the key feature to reduce the influence of the redundancy characteristic through the learning of training set. So as to generate a better theme color extraction model. The following descriptions describe the feature definition, regression model, and user study three parts respectively.

In fact, the concept of the theme of color really is the public right, the woman said that the woman is reasonable, judge an image of the theme color, 1000 reader can not get the same answer, but their answers are mostly approximate. Therefore, it is reasonable to use the user-defined theme color as the standard answer. For each image, the article uses the K method to calculate the 40 colors of the image as the base color of the k=40. The user can only select 5 colors from these 40 colors as the theme color of the image.

This article defines the following 6 aspects of the characteristics of the extraction of the calculation of 79 feature variables, here for a simple description:

Visual significance saliency: This paper defines the visual significance of each pixel in the image by the user's eye movement tracking data. To define the visual significance of a set of theme colors in the image is the superposition of the visual significance of the pixels of all theme colors, while defining the ratio of the pixel to the number of pixels for a color visual significant density.

Coverage error coverage error: Coverage error is defined as the theme color to cover the image and the original image of the color error, hard error and soft error, the difference is whether a pixel is covered by a single theme color or by a number of theme color overlay. Similarly, the coverage error of pixels in color channels such as brightness, saturation, red-green, blue-yellow and so on is defined, and the coverage error is computed by region after the image is segmented.

Colour Diversity Color Diversity: Color diversity takes into account the average, maximum, and minimum distance between colors.

Color impurity: The color concentration considers the distance between the first 5% pixels that are similar to the theme color.

Color nameability and color statistics colour statistics: These two sounds intuitive, actually very vague, the text is not described in detail.

After defining these 79 features, it will be lasso to play. LASSO (least differs shrinkage selection operator) is a multivariate linear regression method, which achieves the feature selection through a constraint condition in the traditional multivariate linear regression equation (the following figure is excerpted from the LASSO text). where x is the feature, β is the weight of the feature, if the constraint T is an infinite value, then with the general multiple linear regression is not different, but t gradually reduce the time the feature weight received extrusion (shrinkage), so as to remove the redundancy feature selection (selection) role. The 79 features defined by the lasso approach to the training set are reduced to very limited.

User study is the author set up 40 images on this crowdsourcing platform, each user receives 10 images of the task, in the base color to find the image of the 5 theme colors. In contrast, the author also found 11 students of the art department to perform the same task.

In this task , the user selected theme colors and art students of the same, but with the automatic selection of the color of the larger difference.

For the theme color of modeling, the author also use user study to identify with the real theme of the image, judged by the user by the way, to give the topic to give 1 to 5 points, 5 points is very close and 1 points is very close, the user group scoring is widely praised to modeling the theme color, The other two methods are slightly inferior.

Finally, the weight of these 79 features is given, which seems to be used in a kind of image retrieval based on theme color. But in fact, because the visual significance is obtained by the user's eye movement tracking data, it is impossible to model the image without visual significance, which greatly reduces the usability. If this feature is improved, the method can be used more widely.

A few additions:

1. This post is an academic review of the article, which has been reported before.

2.Eurovis of the article for the data entity itself color semantics and design of the corresponding color consistency problem, such as the fruit of the data, the blueberry blue, banana with yellow and so on, interested in the reader can see the original paper.