Photoshop oil painting effect Filter

Objective, from: http://www.cnblogs.com/hoodlum1980/archive/2011/01/15/1936078.html

[Original statement] This filter is developed by using the ps sdk, and the filter Algorithm Which of the following statements may be unknown? Filterexplorer Source code (VC 6). The main reference source of this algorithm is filters. cpp in this project. The author is Jason Waltman (18, w.l, 2001) . Another software written in C # language in China Photosprite (Version 3.0, 2006, from Lianjun Write) the algorithm of the oil painting filter should also be referenced from the former (or other Source code ). When studying this filter algorithm, I mainly refer to the C ++ of the former. Code In this article, the conceptual description of this algorithm is my understanding and interpretation. However, the efficiency of this algorithm is not high. I have greatly improved the efficiency of this algorithm. The time complexity of the template size has been improved from O (N ^ 2) to linear complexity O (n ), the constant coefficient of the complexity of the number of pixels is greatly reduced. The processing speed of the same parameter for the same test sample (A 1920*1200 pixel RGB image) is reduced from about 35 seconds to about 3 seconds, the processing speed is increased to about 10 ~ About 12 times (rough estimation ).

This article mainly publishes Photoshop oil painting effect filter (oilpaint ). The algorithm is not proposed by me. For more information, see references in this article. This filter can be seen in photosprite, a Chinese software developed by C. Someone asked me to help develop the filter in 2010. Now I have spent about a few days developing it and providing it for free.

(1) Conceptual description of the algorithm of the oil painting filter:

This is my understanding after reading the source code of filterexplorer. This filter has two parameters. One is the template radius, and the template size is (radius * 2 + 1) * (radius * 2 + 1, this algorithm is centered on the current pixel and expands the rectangular area of radius pixels. As a search range, it is called a "template" for the time being (in fact, this algorithm is not like Gaussian fuzzy, the standard template method of custom filters is only similar to the processing process, so I can implement the optimization described later ).

Another parameter is smoothness, which is actually the number of gray bins. Let's assume that the gray level/brightness of the pixel (0 ~ 255) evenly divided into smoothness intervals, each interval is called a bucket here, so that we have many buckets, which are called a bucket array (buckets) for the moment ).

This algorithm traverses each pixel in the graph. For the current position (x, y) pixels, all pixels in the template range are dimmed, that is, the image is converted into a grayscale image, then, the pixel values are further discretization, that is, the pixels in the template are put into the corresponding bucket according to the gray-scale falling range of the pixel. Find a bucket with the largest number of pixels from these buckets, and calculate the average color of all pixels in the bucket as the result value of position (x, y.

The above algorithm description is represented in the following. The intermediate image is obtained from the grayscale + discretization of the source image (equivalent to the color separation in Photoshop), and the small box represents the template. Below is the bucket array (8 buckets, namely 0 ~ The gray value of 255 is divided into eight segments ).

(2) Improve the efficiency of existing code for foreigners.

If it is not difficult to transplant the existing code to the PS filter as it is, I spent about 1 ~ In my spare time, the debugging was successful. However, when I read the source code from foreigners, I obviously felt that the efficiency of the original code was not high enough. This algorithm can be completed by traversing an image once, and processing each pixel is a constant time. Therefore, the O (n) complexity is applied to the number of pixels (image length * image width, however, the regular coefficient of the original code is large. For example, each time a pixel result is calculated, the gray level of the pixel within the template range must be re-calculated and put into the bucket, it actually produces a lot of repetitive calculations.

2.1To this end, my first improvement was to grayscale and discretize the entire image patch in PS (put into the bucket). In this way, when the patch is traversed using a template, you do not need to calculate the gray scale repeatedly and discretization it. In this way, the algorithm speed is roughly doubled (for a sample, the processing speed is increased from more than 20 seconds to about 10 seconds ).

2.2However, the speed improvement is not significant enough. So I made another more important optimization, that is, reducing the complexity of the template size from square to linear complexity. This is based on the fact that the template is moving from left to right in the current line, and the central pixel of the template (intersection of two adjacent templates) remains unchanged in the result. Only the leftmost column is removed from the template, and the rightmost column is entered into the template. Therefore, when traversing the image, we do not have to worry about the central pixels of the template. We only need to process the two edges of the template. As shown in (the radius is 2 and the template size is 5*5 pixels ):

When we reach the edge on the right side of the patch, we do not reset it to the beginning of the line as we press ENTER for a new line, but move the template down to the end of the next line, and then translate it to the left, in this way, the track of the template becomes a trajectory of a snake. After this improvement, we only need to process the two edge pixels of the template when traversing the pixels. In this way, the template size (half diameter in the parameter) is reduced from O (N ^ 2) to O (n), which greatly improves the algorithm's computing speed, combined with the optimization of 2.1, the algorithm's computing speed is improved by about 11 times (this value is only a rough estimate, and has not been tested by a large number of samples ), the optimized algorithm's processing time for large images becomes acceptable.

[Note] the reason why I can achieve this optimization is that the filter algorithm is not a standard template algorithm, and its essence is to obtain statistics within the template range, that is, the result is irrelevant to the template coordinate of the pixel. It is like we want to get information about the population and the ratio of men to women in a certain area. Therefore, we have optimized the above method.

2.3The original Code limits the radius to (1 ~ 5) Because I optimized the code, I can greatly increase the Radius Range. When I set the radius to 100, I found that the radius is too large to make sense, because we can't see what the source image is.

[Conclusion] The improved code is more skillful and challenging, including coordinate positioning between a large number of underlying pointer operations and different rectangles (input patches, output patches, templates, the code may be slightly less readable, but as long as you understand the above principles, the code is still quite readable. In addition, I also thought of improvements to this algorithm, changing the template from a rectangle to a circle, and randomly jitters the die radius and number of buckets when traversing the image, however, these improvements will invalidate the optimization mentioned in 2.2, and reduce the algorithm speed back to a lower level.

(3) The parameter dialog box uses multithreading technology to improve the efficiency of thumbnail display and avoid affecting the interaction of UI threads.

 

For more information about how to display thumbnails In the parameter dialog box, see the fourth article in the PS filter tutorial.Article. Here I will explain the improvements to UI interaction during thumbnail Update and the scaling and translation technology.

When this filter is called in Photoshop, the Parameter Setting dialog box is displayed. You can drag the slider control (trackbar, also known as slider) or enter it in the text box to change the parameters. The thumbnail updates the new parameters in real time. In the original filter implementation, I put the processing of the updated thumbnail in the same thread as the dialog box UI. In this case, the following problems will be introduced. When you drag the slider control very quickly, because the parameter changes quickly, the UI thread may be busy processing thumbnail data and be "blocked" for a short period of time, so that it cannot immediately respond to subsequent control events, that is, the slider control is not smooth enough to drag, there is a jump, frustrated, dull, and not responsive to the feedback on mouse dragging.

 

To improve this problem and avoid affecting the UI thread, I am going to put the thumbnail processing task that consumes a lot of time in a new thread to complete it, when the thread completes the thumbnail processing, the view is updated in the notification dialog box. When a trackbar is dragged, the UI thread receives a control notification at a very high frequency, as if it were a "surge, this requires that the subsequent UI events can quickly abort and exit running thread tasks.

I extract the filter algorithm as a shared function, so that the filter can be shared for actual processing and thumbnails update. The filter algorithm is required to regularly detect the "task cancellation" event during the actual use of filters by PS and during thumbnail update. For example, when the PS calls a filter, if you press the ESC key, a time-consuming filter operation will be immediately abandoned. When updating thumbnails, if a UI event surge occurs, the processing thread must be quickly aborted.

In the filter core algorithm function, I regularly check the "task cancellation" event. Because the test cancellation when the PS calls the filter is different from the test cancellation method when the thumbnail is updated, therefore, in the filter algorithm function, I added a callback function parameter (testabortproc ). In this way, when PS calls the filter for actual processing, it uses the PS built-in callback function to detect the cancellation event. when updating the thumbnail of the dialog box, I use a callback function provided by myself to detect cancellation events (this function detects a Boolean variable to determine whether a new UI event is waiting for processing ).

For processing thumbnails, I use a single thread. That is, when each new UI event arrives, it is necessary to check whether the thumbnail processing thread is running. If yes, I have a new UI event tag and then wait for the thread to exit, after the previous thread exits, I will start a new thread. In this way, there will always be only one thread for processing thumbnails, rather than opening too many threads when the UI event is continuously approaching, the advantage of doing so is that the logic is clear and easy to control, so we will not be troubled by too many threads and maintenance failures. The disadvantage is that, although the thread regularly detects cancellation events, it still takes a small amount of time to stop the thread, which may cause a tiny pause in the UI thread, but it is insignificant, this is an essential improvement over updating thumbnails In the UI thread.

After improvement, we can drag the two slide rods on the parameter dialog box at a very high speed. Although the core algorithm of this filter has a large amount of computing, we can see that the parameter dialog box still has a smooth response.

(4) scaling and shifting of thumbnails:

In fact, it is not difficult to update the thumbnail data, regardless of scaling or translation. The difficulty lies in the Translation of thumbnails, because mouse interaction is involved, which requires a very solid windows programming skills andProgramUnderstanding of underlying mechanisms.

You can drag a thumbnail in either of the following ways:

4.1 drag the result image directly.

This is divided into two methods. The first is the perfect drag effect, but at the cost of a certain amount of space and time, and encoding is also challenging. That is, the input data of the thumbnail is expanded to 9 times the size, and the result image is obtained in the memory. Only the central part of the result graph is displayed. When dragging, the thumbnail will not contain any blank part.

Another method is to take a snapshot of the current result graph (screenshot) while dragging, And Then paste the screenshot result to the corresponding position of the screen. This is more efficient, but the disadvantage is that there is a blank area next to the thumbnail when dragging. This method is often used to update a view at a high cost, such as vector graph rendering.The method I implemented in this filter belongs to this method.

4.2 drag the image as the original input image.

That is, when dragging, the image used is the original data, not the result image, which is also a compromise to reduce the cost of data update. For example, this method is used in the built-in filter Gaussian blur in Photoshop. When you drag a thumbnail, the thumbnail is displayed as the source image. The preview effect is displayed only after the mouse is released. This is economical and effective. Because the cost of raw data requested is not high, but the cost of processing a thumbnail with a filter is high.

Here is an additional technical detail. Note that because the mouse may be moved out of the customer's zone (which becomes a negative number), you cannot directly use loword (lparam) and hiword (lparam) to obtain the coordinates of the customer partition (because word is a number of unsigned characters), convert them into a number of signed symbols (short) before use ). The correct method is to use the macros get_x_lparam and get_y_lparam in the windowsx. h header file.

(5) the filter'sDownload connection(The attachment contains my PS plug-in Installation tool, which simplifies user installation ):

 

Http://files.cnblogs.com/hoodlum1980/PsPlugIn_OilPaint.rar

After installing and restarting Photoshop:

In the menu: Filter-hoodlum1980-oilpaint, call this filter.

In the menu: Help-about efficiency tool-oilpaint..., you can see about the dialog box (the appearance of the dialog box is almost the same as that of the ICO file format plug-in I developed ).

In the menu: Help-system information, you can see whether the "oilpaint" item has been loaded and its version information.

(6) Some less important supplementary notes:

6.1The size of the output patch I used is 128*128 pixels. During filter processing, each step of the progress bar on the Photoshop status bar shows that an output patch is complete. The input SMD is usually greater than or equal to the output SMD. The size of the input SMD is related to the radius in the filter parameters (the pixel distance of the template radius in four directions ).

6.2In the filter core algorithm, in order to improve the sensitivity of cancellation events, in the current row, I detect cancellation every time I process 16 pixels (pixel index in the row & 0x0f = 0x0f ). After each row is processed (In column loop), the cancellation is also detected. However, this detection frequency is slightly too frequent, and too frequent may increase the cost of function calling.

6.3Using the same image and parameters, I processed the filters, filterexplorer, and photosprite respectively, and then compared them in Photoshop. Because my algorithm is improved by referring to the source code of filterexplorer, my algorithm is equivalent to filterexplorer, but it is more efficient, so the results are identical. However, the overall effect of my filter and filterexplorer is very similar to that of photosprite, but the results are slightly different. I checked the photosprite code and found that this is caused by the difference in the grayscale algorithm of the image (when I adjusted the grayscale algorithm of photosprite to the same as that in filterexplorer, the processing result is the same ).

In photosprite, the method used for Pixel grayscale is:
Gray = (byte) (19661 * r + 38666 * g + 7209 * B)> 16 );

In filterexplorer/My developed filters, the method for Pixel grayscale is as follows:
Gray = (byte) (0.3 * r + 0.59 * g + 0.11 * B );

The grayscale method in photosprite converts floating-point multiplication to integer multiplication. The efficiency may be improved a little, but the performance is not significant here.

(7) References

7.1Filterexplorer source code.

7.2Photoshop 6.0 SDK documents.

7.3Fillred and icoformat plug-in source code, by hoodlum1980 (myself ). 

 

(8) revision history

 

8.1[H]Fixed the bug where the size of memory passed by the filter in the continue call is incorrect when the gray-scale bucket memory is allocated (the space size of the grayscale bitmap is incorrectly set.This bug is easily triggered in the following conditions: the document size is too small, or the radius parameter is very small and the smoothness parameter is very large, these factors may cause a patch to be too small. At this time, because the allocation of the gray bucket space is smaller than the actual size required, the memory may be out of bounds in the subsequent code, thus causing the PS process to terminate unexpectedly. . 

8.2 [m] adds the thumbnail zoom button and function, and optimizes the code when the zoom button is processed to reduce the flickering effect. .

8.3 [m] adds the thumbnail mouse drag function and further adjusts the code to fix the thumbnail rectangle, completely avoiding blinking During Scaling. .

8.4 [l] new feature. When you move the mouse over a thumbnail and drag and drop the mouse, the cursor turns into a hand that is stretched/grabbed. This is implemented by calling the functions in Suite pea UI hooks suite in PS, that is, the cursor inside PS is displayed on the thumbnail. .

8.5 [m] fixed the incorrect conversion of the radius parameter when you click zoom in or zoom out in the parameter dialog box. This bug makes the thumbnail display inaccurate after you click the zoom in or out button. .

8.6 [l] adjust the URL link on the dialog box to the syslink control, which can greatly simplify the Window Process Code of the dialog box. .

8.7 [l] update the plug-in installation auxiliary tool so that it can install multiple plug-ins at a time. .

8.8 [l] The computation error during thumbnail Scaling (unknown, probably because of a floating point calculation error) may make it impossible for the image close to the bottom right corner of the edge to be moved to the thumbnail view, therefore, an error buffer is added to the translation range. 2011-1-27.

8.9 [l] beautification: Change the zoom-in button of the filter parameter dialog box to directui (a new class cimgbutton is added). The interface effect is better than that of the original button control. 2011-2-14.