Research on the migration algorithm of---Makeup for portrait beauty (Makeup transfer)

For the portrait of Beauty algorithm, now the beauty of the camera, play beauty makeup and other apps have done more mature, but the specific algorithm, the basic network is no can be traced, today I introduce an automatic portrait beauty algorithm----(Makeup Transfer) Makeup Migration

There are few papers related to makeup migration, as follows:

1.example-based Cosmetic Transfer

2.Makeup Transfer using Multi-example

3.A new Digtial Face Makeup method

4.An Automatic Framework For example vased virtual makeup

I mainly introduce the first article "Example-based Cosmetic Transfer", the thesis is as follows:

Note: A and A * are two inputs, B is the original image, and b* is the makeup that is migrated according to a *;

The algorithm flow in this article is as follows:

1,face Warp

To talk about A * makeup move to B, there are several conditions:

① background single;

② skin color is similar;

These two conditions are also for the final effect more natural;

First of all, face warp process requires facial feature points, so this step before the need for face detection and point alignment, to get a * and B feature points;

Then, according to the deformation algorithm, a * and B of the characteristics of the alignment, so that a * deformation to the same as B;

There are several variants of the algorithm (personal summary):

① Least-squares MLS deformation algorithm: Image deformation Using moving Least squares.

More information about the reference blog: http://blog.csdn.net/hjimce/article/details/46550001

② line-based deformation algorithm: as-rigid-as-possible Shape manipulation.

More information about the reference blog: http://blog.csdn.net/hjimce/article/details/45766321

③ Triangle Mesh affine transformation

This step is given as follows:

On the left is the original B (to avoid infringement, the eye has been processed), the middle is a makeup figure A *, the right side is face warp (I use the triangle mesh deformation);

2,cosmetic Map Calculation

The focus of this article is this step, the calculation of cosmetic Map, that is, CP, in fact, the algorithm is very simple, the formula is as follows:

Cp = ap/ap*

AP: Makeup Image A * corresponding to the original

ap*: Makeup image *

Just such a simple formula, it can be decadent as magic.

The following CP is given in this article:

3,makeup Transfer

After we get the CP, we can do the makeup migration, the algorithm is as follows:

4,others

In fact, the paper also introduces some other content, which I did not write, because my focus is the makeup migration, so the main extract this piece of content.

The paper actually also performed freckle remove freckle removal, eyebrow eye texture detail extraction, and so on, as shown below:

These things, I do not care here, in fact, is to make the final effect more natural and more realistic;

The above process is the core algorithm of this paper;

Now, the good stuff has just begun:

I put forward a few questions about this paper, which is actually a flaw in the application:

①, the paper requires three inputs (a, a *, B), an output b*

This, in practice, there is a lot of restrictions, in general, we can get a A *, that is, a good look, and then want to make their own photo B makeup, this logic is not a makeup of the original.

②, the paper requires a similar complexion, a single background

This point, the universality is too low, difficult to apply;

③, according to the logic of the paper, the mouth area is closed, can not adapt to a variety of laughter and other openings, or the effect is too poor;

Between the above three points, I have improved the algorithm:

①, according to A *, estimates of a, the estimate is a, so that only requires users to enter a good-looking, you can take their own selfie and other makeup migration;

②, according to facial feature points, to obtain a * in the skin color features, build accurate face mask, remove the background, so as to avoid the impact of the background, at the same time, the skin color conversion, a * in the color conversion to b*, so as to avoid the effect of excessive color differences;

③ acquires A * Lip color feature, b for Lip color conversion, A * skin color and lip color migration to b*, which makes use of a variety of laughter and other opening scenes;

According to the above three points, I improved the algorithm to get the following results:

Original B

Three target makeup b*

The above three makeup b* respectively correspond to the following:

Before giving a set of test diagrams:

The above effect I did make a degree of self-adaptation, so did not appear very full of servants, this is to look more natural point.

Note: I use the test map from the beauty camera and the Internet, if there is infringement please inform.

I provide a simple code call as follows:

      private void Picturebox4_click (object sender, EventArgs e) {if (picturebox1.image! = null) {              Graphics g = graphics.fromimage (Curbitmap);                                 Int[] Eyepoints = {173, 370, 177, 441, 191, 509, 212, 577, 236, 640, 267, 698, 303, 748, 347, 793, 399, 826, 465, 837, 526, 825, 575, 792, 611, 74                                 8, 643, 699, 671, 643, 695, 578, 714, 507, 727, 435, 728, 364, 214, 316, 245, 284, 285, 273, 328, 274, 370, 281, 402, 308, 363, 309, 325, 304, 287,                                 302, 251, 307, 513, 307, 544, 282, 583, 275, 623, 274, 660, 284, 688, 313, 654, 306, 620, 302, 585, 304, 549, 309, 269, 390, 282, 373, 300, 364, 323 , 361, 347, 366, 365, 380, 378, 401, 360, 406, 342, 410, 321, 412, 300, 408, 283, 401, 533, 399, 544, 378, 562, 365, 585, 359,  607, 362, 625, 371, 638, 386, 624, 398, 608, 406, 588, 410, 567,                                 408, 550, 404, 424, 394, 424, 453, 417, 512, 386, 542, 398, 580, 446, 588, 480, 588, 528, 576, 536, 539, 506, 511, 494, 452, 490, 393, 363, 653, 39                                 4, 643, 429, 637, 462, 642, 495, 636, 527, 643, 557, 654, 535, 687, 506, 713, 461, 726, 415, 715, 384, 688, 373, 656, 417, 657, 462, 661, 504, 657,                                 546, 657, 505, 676, 460, 686, 414, 676, 322, 389, 586, 387, 457, 392, 461, 502, 463, 554,              463, 588};              EYEPOINTS[2 * 50 + 1]-= 2;              Eyepoints[2 * 49 + 1]-= 3; EyepoinTs[2 * 48 + 1]-= 4;              EYEPOINTS[2 * 47 + 1]-= 3;              EYEPOINTS[2 * 46 + 1]-= 2;              Eyepoints[2 * 52 + 1]-= 1;              Eyepoints[2 * 61 + 1]-= 2;              EYEPOINTS[2 * 60 + 1]-= 3;              Eyepoints[2 * 59 + 1]-= 2;              EYEPOINTS[2 * 58 + 1]-= 1; for (int i = 0; i < 101; i++) {G.drawrectangle (new Pen (color.red, 1), New Rectangle (Eyep              OINTS[2 * I]-1, eyepoints[2 * i + 1]-1, 2, 2);              } g.dispose ();              DateTime start = DateTime.Now; Curbitmap = IP.              Softskin (Srcbitmap, New Bitmap (Startpath + "\\MakeUp\\MAP.png"), NULL, Skinratio, 30); Curbitmap = IP. Mkmakeuptransfer (Curbitmap, New Bitmap (Application.startuppath + "\\M3.              JPG "), curfacepoints, eyepoints);                  DateTime end = DateTime.Now; Label1. Text = "TC:" + (End-start).              ToString ();          pictureBox1.Image = Curbitmap; }      }

 [dllimport ("Testdemo_c.dll", CallingConvention = callingconvention.cdecl, CharSet = Charset.none, ExactSpelling = True)]        private static extern int In_pic_makeuptransfer (byte* srcdata, int width, int height, int stride, int[] Srcfacepointsall, byte* maskdata, int mwidth, int mheight, int mstride, int[] mkeypointsall);  &nbs P     Public Bitmap mkmakeuptransfer (Bitmap src, Bitmap mask, int[] Srcfacepointsall, int[] mskfacepointsall) &nbs P       {            Bitmap a = new Bitmap (SRC);        &NB Sp   int w = a.width;            int h = a.height;          &NBSP ; BitmapData srcdata = a.lockbits (new Rectangle (0, 0, A.width, a.height), Imagelockmode.readwrite, PIXELFORMAT.FORMAT32BPPARGB);            BitmapData Mskdata = mask. LockBits (New Rectangle (0, 0, mask. Width, Mask. Height), ImagelockmodE.readwrite, Pixelformat.format32bppargb);            In_pic_makeuptransfer ((byte*) Srcdata.scan0, W, H, Srcdata.stride, Srcfacepointsall, (byte*) Mskdata.scan0, mask. Width, Mask. Height, Mskdata.stride,  mskfacepointsall);            a.unlockbits (srcdata);            mask. Unlockbits (mskdata);            return a;       }

Finally, give a testDEMO: Click to open link

If the problem, please contact qq:1358009172

Research on the migration algorithm of---Makeup for portrait beauty (Makeup transfer)