Application of Overview:end-to-end deep Learning Network in the field of hyper-resolution (to be continued)

Directory

• 1. srcnn
  • Contribution
  • Inspiration
  • Network
    • O. pre-processing
    • I. Patch extraction and representation
    • II. Non-linear Mapping
    • III. Reconstruction
  • Story
  • Further learning

1. srcnn

Home page
Http://mmlab.ie.cuhk.edu.hk/projects/SRCNN.html

ECCV , Tpami .

Contribution

1. End-to-end Deep Learning application in the field of super-resolution, 2018 was cited over 1000 times. (Non-end-to-end see story.3)
2. The relationship between traditional method (Sparse-coding-based SR methods) and deep learning method is pointed out, which is instructive.
3. SRCNN Network is very simple, PSNR, Ssim and so on have a small increase (< 1dB). In specific terms:
   • The method (SRCNN) directly learns an end-to-end mapping between the low/high-resolution images.
   • Because it is an end-to-end network, training is a comprehensive optimization of the overall framework (see INSPIRATION.2).
4. The application (test) is completely feed-forward, so the network speed is fast.

Inspiration

1. This problem (SR) are inherently ill-posed since a multiplicity of solutions exist for any given low-resolution pixel.
   Such a problem is typically mitigated by constraining the solution space by strong prior information.
   Note: Training CNN is learning a priori knowledge.

2. Traditional methods focus on learning and optimizing dictionaries, but few are optimized for other parts.
   But for CNN, its convolution layer is responsible for patch extraction and aggregation, and the hidden layer acts as dictionaries, so it will be uniformly optimized.
   In other words, we only need very little pre/post-processing.

3. In the past, we used a set of pre-trained bases such as PCA, DCT, Haar to represent patches.
   Now, with different convolution cores, we've achieved a variety of representations.

4. Because of overlapping, the convolution uses more pixel information than a simple dictionary map.

Network

O. pre-processing

A low-resolution picture is obtained by bicubic interpolation (\MATHBF y\) . Note We still call low-resolution image.

I. Patch extraction and representation

Extract overlapping patches from \ (\MATHBF y\) , each patch represents a high-dimensional vector.
These vectors together form a set of feature maps.
The dimensions of each vector are both the total number of features and the total number of feature maps.

\[f_1 (\mathbf y) = max (0, W_1 * \mathbf y + b_1) \]

II. Non-linear Mapping

Through a nonlinear transformation, the original high-dimensional vector is transformed to another high-dimensional vector.
The high-dimensional vector also consists of a set of feature maps, which conceptually represents the high-resolution patch.

\[f_2 (\mathbf y) = max (0, W_2 * f_1 (\MATHBF Y) + b_2) \]

III. Reconstruction

Generates output close to ground truth: \ (\MATHBF x\) .

The average method used in the past. In fact, the average is also a special convolution.
So we might as well just use a convolution.
At this point, the output patch is no longer a simple average, it can also be an average on the frequency domain (depending on the nature of the high-dimensional vector).

\[f_3 (\mathbf y) = w_2 * F_2 (\MATHBF y) + b_3\]

Be careful not to deal with nonlinearity.

Story

1. The 3 most popular causes of deep CNN's growing popularity:
   • more powerful GPU;
   • More data (e.g. imagenet);
   • Relu the proposed, accelerate the convergence while maintaining good quality.
2. CNN was previously used for natural image denoising and removing noisy patterns (dirt/rain), which was used for the first time in SR.
   This is the importance of telling good stories, nothing more than mapping pairs different.
3. Auto-encoder has also been used in the hyper-resolution network, but has yet to get rid of the drawbacks of the separated framework.

Further learning

1. Traditional sparse-coding-based SR methods

2. From low-resolution images to \ (\MATHBF y\) is bicubic interpolation, which is actually convolution. But why not as a convolutional layer?
   The paper explains that because the output is larger than the input, in order to effectively use well-optimized implementations Sucha as cuda-convnet, it is not regarded as a "layer" for the moment.

Application of Overview:end-to-end deep Learning Network in the field of hyper-resolution (to be continued)