The realization of image pencil drawing algorithm based on "combining Sketch and Tone for Pencil Drawing Production"

one, for reference:

This paper draws on the understanding of this paper's algorithm in the summer of Csdn: Wind-blown summer's image pencil drawing algorithm, and the home page of the paper written by people such as Cewu Lu, the Chinese University of Hong Kong. The original author and blogger Wind up the summer has given code, but the code is not complete. After I read the original paper and the blogger's article, basically, the idea of the approximate algorithm was adjusted. In this paper, the details of the specific algorithm is not detailed, personal advice or first study of the original paper, the steps of the paper to first have a general understanding.

Second, the algorithm idea

1, the first need to produce a stroke structure

General idea: The original image of the gradient operation, the approximate contour-----> design of 8-direction convolution core, and according to the formula in the paper on the pixel according to the maximum value of sub-image direction classification-----> based on the formula in the paper for each direction of the pixel and corresponding convolution core convolution

A few points to note:

For the size of the convolution kernel, the paper is written in the original picture of the height and width of One-thirtieth, the author of the paper design convolution core is intended to produce a pencil drawing in a stroke of the mark, but if too large, will lead to unclear contour, convolution kernel size should be set in 3~13 between the effect better;

Depending on the input image, the original image may need to be de-noising, if the final contour is not clear, you may need histogram equalization to make the contour relatively clear.

2, followed by a tone chart

Based on the study of a large number of pencil images, the author finds that the histogram is as follows, and what we want to do is to convert the histogram of the color scale of the original picture to the following histogram, and the following histogram can be approximated by the Gaussian distribution function, the uniform distribution function and the Labourasse distribution function with weighted value (three weights added to 1). Their weights are w1:w2:w3, and after that, the SML map converts the histogram of the original image to the following histogram.

Note: The general use of w1:w2:w3 = 2:22:76, because the pencil drawing, W3, the larger the picture, but for some pictures, if too bright, some darker details will disappear, at this time can choose W1:W2:W3 = 11:37:52, also can adjust according to the final effect three parameter size, general, W3 between 52~76, W1 preferably under 10.

3. Create a tone rendering diagram from a pencil drawing background and tone chart

I am using the following pencil background chart:

This step is primarily for the understanding of the beta matrix in the following equation set, where H (x) is the above pencil background, J (x) is the tone graph generated for the second step, and now the main solution is β (x), and the final hue renders the figure t (x) = H (x). ^β (x), the purpose of this is to make the tone graph with a pencil repeated smear traces, And keep the histogram consistent with the pencil-drawing histogram:

The following is the solution formula, wind blowing summer blogger's article has a detailed solution process, the general idea is to first transform the image matrix vector, and then to solve:

Note: Depending on the final effect, gamma correction is required for background pencil images and generated tonal rendering images.

4, finally, the first step generated by the structure of the picture and the 23rd step generated by the tone rendering image directly to multiply it.

Note: Due to the fact that there are many places where matrix multiply and matrix multiplication are used in the implementation process, the Im2double function should be used to map each pixel in the image matrix to 0~1 for calculation.

5, input/Get color image VS input/get grayscale image

Input grayscale image/color Image: The input is judged on the dimension, but when the function graypencildrawing is called, the input must be grayscale image;

Output grayscale image/color Image: Output gray pencil drawing image, call function graypencildrawing, function return, return is gray pencil image, output color pencil picture, then input is definitely color image, need first color image from RGB space into YUV space, For YUV format, where Y component is color scale, UV component is hue, only need to call function graypencildrawing on Y component, and then convert YUV space into RGB space. The following are the YUV and RGB conversion formulas:

Note: The IIF else condition of the input needs to be judged.

Three, the specific code

Specific needs to change the parameters in the first code snippet is identified, you can read the code first, and then modify the relevant parameters, in addition, for each picture, the required parameters may be different, mainly based on the final effect of modifying parameters:

Code Listing 1: enter picture and background image: The code may be used as an interface code to extract the parameters that need to be modified in the paper, just modify the relevant parameters of the 4~17 line, and the direct function call can see the result. In addition, the final RGB space and YUV space are considered:

clc;clear;%============================== parameter Modification ====================================input = Imread (' input1.jpg ');  Enter original picture Dirnum = 8;% represents a convolution core with dirnum direction [M, n, src] = size (Input), KS = Floor (min (m/50, n/50) + 0.5),% of the paper is required to be high or wide 1/30, actually not so large KS = 10;% convolution kernel size, architectural pictures for 10~13, landscape flowers class picture for 3~6strokedepth = 2;% to the generated contour map multiple strokes w1 = 0.57;% Gaussian distribution weights W2 = 0.37;% evenly distributed weights W3 = 0.06;% Labourasse distribution Weights back = Rgb2gray (Imread (' pencil.jpg '));% Input background pencil Chart backdepth = 0.7;% to pencil background map pencil.jpg multiple strokes renderdepth = 0.7;%    Tudo the resulting texture rendering. If the input is a grayscale image, process it if src = = 2 Gray_out = Rgb2gray (input);    Gray_out = graypencildrawing (Gray_out, Dirnum, KS, Strokedepth, W1, W2, W3, back, backdepth, renderdepth);    Rgb_out = input;end% If a color image is entered, it is processed if src = = 3 Rgb_out = Mat2gray (input);    R = Rgb_out (:,:, 1);    G = Rgb_out (:,:, 2);    B = Rgb_out (:,:, 3);    Y = 0.299*r + 0.587*g + 0.114*b;    U = -0.147*r-0.289*g + 0.436*b;    V = 0.615*r-0.515*g-0.100*b;    y = uint8 (y. * 255); y = graypencildrawing (y, Dirnum, KS, Strokedepth, W1, W2, W3, back, BACKDEpth, renderdepth);    %yuv_out = Cat (3, Y, U, V);% according to the third dimension combination Rgb_out (:,:, 1) = Y + 1.14 * V;    Rgb_out (:,:, 2) = Y-0.39 * U-0.58 * V;    Rgb_out (:,:, 3) = Y + 2.03 * U;    Figure, Imshow (Rgb_out), title (' Color pencil chart '); Imwrite (rgb_out, ' coloredpencil.jpg '); end

Code Listing 2: the implementation of the specific call function, this step is also the first four implementation of the above algorithm ideas

function out = graypencildrawing (Input, Dirnum, KS, Strokedepth, W1, W2, W3, back, backdepth, renderdepth)%===============    ================1: Get contour map ===============================% read in single-channel picture [H, W] = size (Input);    I_gray = Input;    I_gray1 = I_gray; %input = Histeq (Input, 256);    % for some of the picture level is not clear, if the final contour is not clear, plus these two sentences%figure, imshow (Input);    I_gray = im2double (I_gray); ImX = [ABS (I_gray (:, 1: (end-1))-I_gray (:, 2:end)), Zeros (h, 1)];% rightmost add a column 0 ImY = [ABS (I_gray (1: (end-1),:)-i_gray (2:end ,:));    Zeros (1, W)];% add a line at the bottom 0 ImX = ImX. ^ 2;    ImY = ImY. ^ 2;    Imedge = sqrt (ImX + imY); The margin formula in the Thesis (1) Implements Imedge = Immultiply (Imedge, 5);    %%%%%figure, Imshow (Imedge, []), title (' Edge Extraction ');    Imwrite (Imedge, ' stroke_tmp1.jpg '); a convolution core with a horizontal direction of 1 Kerref = zeros (ks*2+1);    Kerref (ks+1,:) = 1;% the convolution cores in turn 180/dirnum degrees and use the formula in the paper to calculate 8-direction convolution, calculate the edge response = Zeros (h, W, Dirnum);        For n = 1:dirnum ker = Imrotate (Kerref, (n-1) *180/dirnum, ' bilinear ', ' crop '); Response (:,:, N) = Conv2 (Imedge,Ker, ' same '); eight-directional convolution cores on the gradient plot, and the Edge formula (2) in the thesis implements end [~, index] = max (response, [], 3);    %index is a m*n matrix, each element is the maximum ordinate number of the corresponding element in the Dirnum rectangle (the range is 1~dirnum), and the maximum value is the point direction C = Zeros (h, W, Dirnum); For N=1:dirnum C (:,:, N) = Imedge. * (index = = n); %index = = 2 is also a matrix with an element value of 0 or 1.    After multiplying the corresponding elements with Imedge, the Edge formula (3) in the thesis implements the end Spn = Zeros (h, W, Dirnum);        For n = 1:dirnum ker = Imrotate (Kerref, (n-1) *180/dirnum, ' bilinear ', ' crop '); SPN (:,:, N) = Conv2 (C (:,:, N), Ker, ' same '); % of the responses to the various aspects of the obtained direction convolution, the thesis of The Edge formula (4) to achieve end Sp = SUM (SPN, 3); % by the third dimension to the elements of this dirnum matrix each accumulate Sp = (sp-min (sp (:)))/(Max (SP (:))-min (sp (:))),%min (Sp (:)) and Max (sp (:)) respectively represent the minimum and maximum values of the elements in the matrix SP, i.e. Gray Stretch stroke = 1-sp;% due to the gradient graph background black line white, now make its background white lines dark stroke = stroke. ^ strokedepth;% Contour Stroke figure, imshow (stroke), title (' S    Troke map ');    Imwrite (Stroke, ' stroke.jpg ');    %===============================2: Get tone graph ===============================% histogram match p = zeros (1, 256);    v = zeros (1, 256);    i = 0:255; P1 = (1/9) * EXP (-(255-i)/9);    %axis ([0 255 0 1]);    P2 = zeros (1, 256);    P2 (105:225) = 1/(225-105);    P3 = (1/sqrt (2*pi*11)) * EXP (-(i-90). * (i-90)/(2.0*11*11));    p = p1*w1 + p2*w2 + p3*w3;    t = P;    %figure, Plot (imhist (I_gray)/(H*w)), title (' Primitive Histogram ');        For i = 1:h for j = 1:w V (i_gray1 (i, j) +1) = V (i_gray1 (i, J) +1) + 1;    End End v = v/(H*W);        For i = 2:256 p (i) = P (i-1) + P (i);    V (i) = V (i-1) + V (i);    End p = p/p (256);% calculates the absolute value of the difference in the gray level of the source image to the cumulative histogram of the target image scrmin = zeros (256, 256);        For x = 1:256 for y = 1:256 scrmin (x, y) = ABS (V (x)-P (y));        End end for x = 1:256 MinX = 1;        MinValue = Scrmin (x, 1);                For y = 2:256 if (MinValue > Scrmin (x, y)) MinValue = Scrmin (x, y);            MinX = y;    End end Histogrammapping (x) = MinX;    end% to the original image according to histogrammapping gray level mapping I_tone = I_gray1; For x = 1:h for y = 1:w i_tone (x, y) =Histogrammapping (I_gray1 (x, y) + 1)-1; End End%%%%%figure, subplot (211), plot (T), title (' Target histogram '),%%%%%subplot (212), Plot (Imhist (i_tone)/(H*w)), title ('    Histogram of the Hue map '); because there are some gray values in the target image that are not present in the mapping, the zigzag part of the curve close to 0 indicates that some gray values do not exist in figure;    Subplot (121), Imshow (I_gray1), title (' Original grayscale image ');    Subplot (122), Imshow (I_tone), title (' Hue map ');    Imwrite (I_tone, ' tone.jpg '); %===============================3: Texture Rendering diagram ===============================%subplot (251), Imshow (I);%    Run these 9 lines to see the background pencil chart effect A = 1;        For i = 0.2:0.2:1.8%a = Floor (i/0.2 + 0.5) + 1;        A = a + 1;        I0 = (double (back)/255). ^i);    %subplot (2, 5, a), Imshow (I0);    End I0 = ((double (back)/255). ^ backdepth);    %figure, Imshow (I0);    I0 = uint8 (I0 * 255);    Imwrite (I0, ' new_texture.jpg ');% begins calculation of P. ^β= J in β-matrix theta = 0.2;    P = im2double (Imread (' new_texture.jpg '));    J = im2double (Imread (' tone.jpg '));% initialized to vector convenient calculation p = imresize (P, [H, W]);    p = Reshape (P, h*w, 1);    Logp = log (P); Logp = SPDiags (LOGP, 0, H*w, h*w);    j = Imresize (J, [H, W]);    j = Reshape (J, h*w, 1);    LOGJ = log (J);% two gradient e = ones (h*w, 1);    Dx = Spdiags ([-E, E], [0, H], h*w, h*w);    Dy = Spdiags ([-E, E], [0, 1], h*w, h*w);% brought into solution formula calculation A = theta * (DX * dx ' + dy * dy ') + (LOGP) ' * LOGP;    b = (LOGP) ' * logj;% calculates beta beta = PCG (A, B, 1e-6, 60) in vector form, and% is converted into matrix form and stretched beta = reshape (Beta, H, W);        Beta = (beta-min (beta (:)))/(Max (Beta (:))-min (Beta (:)));    p = Reshape (P, H, W);    T = P. ^ Beta;    A = T. ^ Renderdepth;%%%%%imshow (a), title (' Tint Rendering chart ');;    Imwrite (A, ' new_tone.jpg ');    %===============================3: Structure diagram and tonal graph integration =============================== I = im2double (Imread (' stroke.jpg '));    S = A. * I;    Figure, Imshow (S), title (' Gray pencil drawing ');    Imwrite (S, ' graypencil.jpg '); out = S;end

Four, some realization effect

Original gray pencil drawing color pencil drawing

The realization of image pencil drawing algorithm based on "combining Sketch and Tone for Pencil Drawing Production"