Summary of the thesis (III.)--The basic and summary of super-resolution algorithm

the concept of image resolution

Image resolution refers to the imaging or display system to the details of the resolution, representing the amount of information stored in the image.
The amount of information stored in an image is the number of pixels per inch of the image, and the resolution is measured in PPI (Pixels per inch), usually called pixels per inch.
The size of the image is determined by the number of pixels, the resolution is the unit density, the higher the resolution of the same pixel picture, the smaller the area.
In general, the higher the image resolution, the more detail the image contains and the greater the amount of information. The image resolution is divided into spatial resolution and time resolution. Typically, the resolution is represented as a number of pixels in each direction, such as a 64*64 two-dimensional image. However, the level of resolution is not equal to the number of pixels, such as an image magnified 5 times times by interpolation does not indicate how much detail it contains. Image super-resolution reconstruction is concerned with recovering missing details in the image, i.e. high-frequency information. classification of super-resolution problems

Super resolution image reconstruction, Srir or SR) refers to the method of signal processing and image processing, in which the existing low-resolution (low-resolution, LR) image is transferred by software algorithm. Technology for high-resolution (high-resolution, HR) images

Generally speaking, the super-resolution problem can be divided into several sub-problems according to the different types of input and output of the algorithm, as shown in Figure 1. The input is a low-resolution image sequence (video), The output is a super resolution problem with a single frame high resolution image, called a reconstruction-based super-resolution problem (reconstruction-based super-resolution), and the input and output are image sequences (video) of the super-resolution problem called video hyper-resolution super-resolution), the input and output is a single-frame image of the super-resolution problem, called the single-frame image super-resolution problem ("one" Super Resolution,sisr). Depending on whether the training sample is dependent, Super resolution problem can be divided into edge-enhanced super-resolution problem (edge-focused Super-resolution) (no training sample) and learning-based hyper-resolution problem (learning-based Superresolution) (with training samples) Two kinds of. For the input is a single frame low-resolution image, the output is the image sequence (video) problem, because of its missing information too much, the actual significance of the study is little, almost no relevant research. Super-resolution of a single image

The study-based single-frame super-resolution problem is a hotspot in recent years, also known as Image hallucination or example-based-based hyper-resolution, which extracts the high frequency information model from the training sample set by machine learning method, Thus, the required information of the unknown test sample is predicted to improve the image resolution, see Figure 5. Most of the learning-based super-resolution methods are based on the Block (patch-based), the target image plane is divided into small image blocks, and the high-resolution image block corresponding to the low-resolution image block is obtained by calculation.

The core problems related to the learning-based super-resolution algorithm are two parts: the establishment of the algorithm model and the selection of the training set.
Historically used super-resolution algorithms: Nearest neighbor Search-Markov random field (MRF)-Neighborhood embedding
Few years:
Sparse Expression
Yang J C, Wright J, Huang T, Ma Y. Image super-resolution as sparse representation of raw image patches. In:proceedings of the IEEE Conference of computer Vision and Pattern recognition (CVPR). Anchorage, Ak:ieee, 2008.1−8

The study-based super-resolution problem is regarded as a regression problem, and the fast regression is calculated by using the sparse regression (Sparse regression) technique.
Kim K I, Kwon Y. single-image super-resolution using sparse regression and natural image prior. IEEE Transactions on Pattern analysis and Machine Intelligence, 2010,
32 (6)

A more effective dictionary (Dictionary) is proposed from the training sample concentration. Image representation with resolution independence (resolution-invariant image representation, RIIR) [67] Used for fast multilevel super-resolution image reconstruction tasks
Yang J C, Wright J, Huang T S, Ma Y. Image superresolution via sparse representation. IEEE transactions on Image processing, 2010,19 (11): 2861−2873
Wang J J, Zhu S H, Gong Y H. Resolution enhancement based on learning the sparse association of image patches. Pattern recognition Letters, 2010, 31 (1): 1−10

The method of super-resolution reconstruction based on the study of human face image is also a related research hotspot ( can be used for the study-based super-resolution of remote sensing images. )
Liu C, Shum H Y, Freeman W T. Face Hallucination:theory and practice. International Journal of Computer Vision, 2007,75 (1): 115−134
Zhang W, Cham w K learning-based face hallucination in DCT domain. In:proceedings of the IEEE Conference on computer Vision and Pattern recognition (CVPR). Anchorage, Ak:ieee, 2008. 1−8

Combining the super-resolution of the enhanced edge with the learning-based super-resolution, the reconstructed image results contain both good texture nodes and a clearer edge contour.
Sun J, Zhu J J, Tappen M F. context-constrained hallucination for Image super-resolution. In:proceedings of the IEEE Conference on computer Vision and Pattern recognition (CVPR). San Francisco, Ca:ieee, 2010. 231−238
Tai Y W, Liu S C, Brown M s, Lin S. Super resolution using edge prior and single image detail synthesis. In:proceedings of the IEEE Conference on computer Vision and Pattern recognition (CVPR). San Francisco, Ca:ieee, 2010. 2400−2407

current Super-resolution papers:
Professor Tang of the Chinese University of Hong Kong srcnn Super-resolution algorithm, is studying the relevant papers, there are other also need to check the relevant literature. There is also a super-resolution GAN based on the generation of the network is a point, take the time to see. Reference: review of super-resolution image reconstruction methods. Su Heng http://blog.sina.com.cn/s/blog_9e1e8c1301015xat.html Http://baike.baidu.com/link?url =ntgtwytifuzcv03yrl8bjpg7kxkcpwa2fvo1n8g3hxltyorqutwbdqjzd_qcpsyxbxvdlajeehzx8lsuitxzh0xhgcegruwcylethjlhhgkj_ –y7degforrcp3ctjnmhhq5t1-azxyntm6mvphu4k