Read paper, multi-block processing: Deep multi-patch Aggregation network forimage Style, aesthetics, and Quality estimation

Deep multi-patch Aggregation Network forimage Style, aesthetics, and Quality estimation

purpose and General methods

If you want to study the style quality of the picture, you need fine-grained fine-grained detail information. For the network presented in this paper, use a number of patches generated by a picture to train.

Existing methods, such as {24,17}, use a randomly generated patch chunk (e.g, cropping 224x224x3 patchesfrom 256x256x3 images), which is not representative, and poorly expressive to the picture. The approach of this paper is to generate a set of patch for the original image, and then package it as the representation of the original artwork. A small set or bag ofpatches cropped from it. The resulting picture is unordered in bag, and then gets the aggregation result of the picture set bag.

What is the aggregation of patches and its network structure

It takes a set of patches as input and completes patches aggregation in the middle tier through patchaggregation structures; This structure is related to max pooling, and it starts with hand-designed, but the parameters are available to learn.

The ordinary CNN network structure, when the high resolution image sampling can not recognize the original resolution of the fine-grained characteristics; randomly cropped patch the advantage is that it can maintain the primary resolution, but a single patch less information, not informative enough, even ambiguous. This network structure is to extract the characteristics of each patch (bag), aggregate features, and then predict this group of bag tags. The following figure is the network structure.

What is the key aggregation, exactly?

top two requirements for aggregation:

1 The features that CNN gets are comparable so that they can be aggregated. The response is CNN sharing parameters.

2) patches is not required orderless. For the disordered constraint authors put forward two methods, one is the use of ordinary statistical methods, such as Min,max,median,mean, is not sequential, but also a sort, add a sort of structure, and then aggregation of two methods have done the corresponding strategy: propose two Different structures for Multi-patch Aggregation:thestatistics aggregation Structure and the fully-connected sorting Egation Structure. The latter is to order, is by a sorting layer, the corresponding characteristics of patch each dimension of the value registration.

Methods of Statistics

s = {min; max; median mean}, which are grouped together by the structure of the S-calculated feature, concatenate. FC aggregation is used again.

FC Sorting aggregation structure

method is sorted by value, order by values

So far, the article is basically over, and don't forget that the purpose of this paper is to evaluate the image style and clarity of the network.