In the field of film and television production, the problem of enlarging low-resolution images to high-resolution images is often involved, and sometimes the issue of adjusting non-square pixels to square pixels is also involved. This chapter discusses the problems of image amplification algorithms, pixel aspect ratio adjustment methods, and image sequence amplification and batch processing technologies.
I. Image enlargement Algorithm
There are many algorithms for image enlargement. The key is the interpolation method for unknown pixels. Below we will analyze several common algorithms in detail, and check whether the enlarged image has color distortion and whether the image details are well preserved, whether the time needed for the amplification process is allocated properly or not to compare their advantages and disadvantages. When a small image is enlarged, such as 400%, We can first use the color values of the original four adjacent pixels, find the position of the new ABCD pixel and fill the corresponding position according to the magnification, but there are a large number of pixels between them. For example, the color value of the P point is unknown and needs to be estimated. Figure 1-Distribution of four adjacent pixels of the original image figure 2-known pixel distribution after the image is enlarged 4 times
1. Nearest Neighbor (nearest neighbor) is the simplest and fastest interpolation algorithm. The method is to enlarge an unknown pixel P, converts its position to the original image, and compares it with the neighboring 4-week pixels A, B, C, and D, make the pixel value of the P point equal to the pixel value of the nearest neighboring point. For example, because the point P is closest to the point D, P = D is taken directly. This method causes obvious distortion. In a and B, the pixel value at the midpoint will suddenly jump, Which is why mosaic and sawtooth appear obviously. The only advantage of the nearest interpolation method is its high speed.
2. bilinear interpolation (bilinear interpolation) is used to convert the enlarged pixel P to the original image, and calculate the Four pixels A, B, the effect of C and D on P points (the closer the P point, the greater the value, the greater the impact) is as follows. Figure 3-The bilinear interpolation algorithm consists of three steps:
The first step is to calculate the influence of point AB on point P to obtain the value of point E. Figure 4-The linear interpolation algorithm calculates the value of linear interpolation. The color value of the pixel points between the two pixels of AB is determined to be linear, you only need to find the point at the corresponding position on the straight line. In other words, the value of any point between A and B is only related to A and B. Step 2, interpolation calculates the effect of the two points of CD on the P point to obtain the value of the F point. Step 3: Calculate the effect of EF on P points by interpolation. Because the interpolation result is continuous, the bilinear interpolation algorithm is visually better than the interpolation algorithm of the nearest vertex, but the operation speed is a little slower, it is a good compromise.
3. The dual-cube interpolation algorithm (bicubic interpolation) is similar to the bilinear interpolation algorithm. For pixel P, the influence scope is extended to 16 neighboring pixels, and interpolation is calculated based on the influence on the distance of the P point. Because the pixel value information of the P point comes from 16 neighboring points, therefore, you can obtain more detailed images, but the speed is relatively slow.
Figure 5-bilinear interpolation algorithm four neighboring points affect unknown point information
Figure 6-16 adjacent points of the dual-cube interpolation algorithm that affect the information of unknown points
However, the essential difference between the two-cube interpolation algorithm and the bilinear interpolation algorithm is that it not only expands the scope of the affected points, but also uses advanced interpolation algorithms ,. Figure 7-Nonlinear Interpolation Algorithm
Requires the value of E point between A and B, and requires the use of the pixel values of A, B around the A-1, A, B, B + 1 four points, through some nonlinear calculation, get a smooth curve to calculate the value of the E point. The so-called "double" or "secondary" means to expand the above calculation to two-dimensional space based on the influence of horizontal interpolation, and then calculate the influence of vertical interpolation. The dual-cube interpolation algorithm can produce relatively clear picture quality, but the computing workload also increases. This algorithm is most commonly used in many image processing software today, such as Photoshop, after effects, avid, Final Cut Pro, etc. In order to get better image quality, many new algorithms are emerging on the basis of the above, and they use more complex and improved interpolation methods. For example, B-spline and Mitchell interpolation algorithms aim to make the Interpolation Curve smoother and the image edge more perfect.
4. The biggest difference between the adaptive spline interpolation technology (S-spline & S-spline XL) and the preceding typical interpolation method is that s-spline adopts an adaptive technique, traditional methods always calculate the color value of unknown points based on the surrounding pixels. That is to say, the color value to be solved only depends on the position of the pixel in the image, rather than the actual pixel information of the image, the adaptive spline algorithm also considers the pixel information of the actual image. Experiments show that the image effect obtained by the S-spline algorithm is better than that obtained by the two-cube interpolation algorithm. Now the S-spline algorithm has appeared in the enhanced version S-spline XL. Compared with the new version S-spline XL algorithm, the image sharpness is further enhanced, and the object contour is clearer, the image feels more natural when the image is at the edge of the image. Ii. Practical Test of image amplification algorithms now we will test the above interpolation algorithms to see their effects. First, open the original image bird.jpg. This is an image with a resolution of 360x270. Figure 8-Original Image
We use benvista's well-known photozoom Pro 2 software for testing on PC. The CPU configuration of the computer is inter Core 2 t5500 1.66 GHz, the above five interpolation methods are used to enlarge the original image by 8 times to 5 2880X2160 images. Figure 9-Nearest point interpolation n earrst _ n eighbour
Figure 10-bilinear interpolation B ilinear
Figure 11-Dual-cube interpolation B icubic
Fig 12-adaptive spline interpolation S-S pline
Figure 13-adaptive spline enhanced interpolation S-S pline _ XL
Compare the above figures with the experimental data. I will list the effect parameters in the following table.
| Interpolation type |
Subjective feelings |
Image contour |
Overall rating |
Processing duration |
| Nearest point interpolation n earrst _ n eighbour |
Mosaic |
Not clear |
Worst |
5 seconds |
| Bilinear interpolation |
Image Blur, not sharp |
The edges are not clear, and the edges are jagged. |
Difference |
6 seconds |
| Dual-cube interpolation B icubic |
Blurred and sharp images |
The phenomenon of sawtooth has been improved. |
Compromise |
8 seconds |
| Adaptive splines, interpolation, S-S, pline |
The image is relatively clear and sharp |
The edges become clear and the serrations disappear. |
Good |
18 seconds |
| Adaptive spline reinforced S-S pline _ XL |
Clear and sharp images |
Sharp and clear edges |
Best |
20 seconds |
Apparently, in order to get the best image amplification quality, we should use adaptive spline interpolation enhancement (S-S pline _ XL) technology to enlarge the image in optional conditions. 3. pixel aspect ratio adjustment now we use the adaptive spline interpolation enhancement (S-S pline _ XL) technology, based on the 4 K equivalent resolution 35mm × 4096 of 3112 film copies, the SD-NTSC system 720x486 (720x480) resolution image was amplified. It also describes how to adjust non-square pixels to square pixels. Taking the ntsc_720 × 486 image as an example, it has a different display ratio than the 4 k film's equivalent resolution of 4096 × 3112. In order to maximize the image information, we take the mailbox mode as an example to make adjustments, and we also need to consider the change in the pixel aspect ratio of NTSC. We know that the pixel aspect ratio of the NTSC image is not, but 0.9: 1. In the square pixel display mode, when we call the NTSC image, we find that the image will be deformed, that is, the image is flattened. :
Figure 14-ntsc rectangular pixel figure 15-square pixel
Figure 16-NTSC portrait squashed in square pixel Display Mode
Now, from the benvista photozoom pro 2.3.2 software, first adjust the pixel aspect ratio deformation problem, and then adjust the image amplification parameters. Open benvista photozoom pro 2.3.2, which not only supports single image amplification, but also supports batch processing of Image Sequence frames. This version supports image formats such as TIF, PNG, TGA, and BMP, when using it, we need to generate the formats supported by the software, generally in TIF or PNG format.
1) Disable maintain aspect tatio and pre-adjust the width of the newly generated image to 720 × 0.9 = 648 to correct the deformation and Parameter Adjustment caused by changes in the pixel aspect ratio of the image. Figure 17-Pre-adjusted parameters
2) Adjust the width of the newly generated image to 4096 and the height to 486 × (4096/648) = 3072. That is, proportional amplification of the pre-generated image to the equivalent resolution of 35mm Film Size, then define the amplification of the interpolation method for S-S pline _ XL. However, due to the aspect ratio problem, when using 35mm film for recording, because of its 4 K equivalent resolution of 4 0 96X3112, the upper and lower sides of the film will leave a certain Black edge.
Iv. After the above analysis, we learned how to use benvista photozoom pro 2.3.2 software to enlarge a single-frame image, combined with its batch processing technology, the image sequence can be enlarged. Start the new batch option in the toolbar, click Add images to add Image Sequence frames, and select all frames based on the source format and the size of the image to be generated, combined with the preceding analysis data, fill in the enlarged image size and interpolation calculation mode, and create a generated path and click Run to implement batch processing of the image sequence. Figure 18-add an image sequence
After a certain period of processing, we can get a large image sequence frame.
