How beautiful is the starry sky in Van Gogh's eyes?

.

Editor's note: Original author Piotr Wendykier, focusing on Mathematica algorithmic programming and research and development. According to local reading habits slightly deleted, good English students suggest reading the original.

Van Gogh has a painting called Starry Night, or "the Stars." I have often wondered how beautiful the vast expanse of the starry world is, or how sad it may be, when Van Gogh finishes this masterpiece in a mental hospital. It is a pity that we can only see the small square in the frame.

Recently The Cambridge University Engineering department has run an annual photo contest, "The Art of Engineering: Images from the frontiers of Technology". The second prize of the contest was finally obtained by Yarin Gal PhD from the machine learning Group, and he did something interesting: using machine learning algorithms to extend Van Gogh's famous Starry Night, as shown below.

The GAL has also built a dedicated web site to showcase this extrapolated art, which uses machine learning + image processing to extend the overall picture, because the painting often provides only a local image.

Digital patching Technology (digital inpainting) was first proposed in the 2000 SIGGRAPH convention, an article called Image repair (images inpainting). This technique is mainly designed to repair old ancient paintings, but it is also widely used in other image fields. There is also an image patching algorithm called Patchmatch, which is designed to create machine art (machine art.)

Next, let's look at a programming example that implements the extension of this masterpiece, which will use the Wolfram language. In Wolfram language, Inpaint is a built-in function, and the area of the image that needs to be patched can be assigned to three objects: image, graphic object, matrix.

Inpaint has 5 different methods (method) options for implementing different image processing algorithms: "Diffusion," "Totalvariation," "Fastmarching," "Navierstokes," and " Texturesynthesis ". One of the last methods, Texturesynthesis, is the system default, and Texturesynthesis differs from other algorithms in that it does not manipulate each color channel individually, and it does not add new pixel values. That is, each patch pixel value is obtained directly from some parts of the input image, and in the following diagram you can see clearly that using texturesynthesis can make the larger object in the image "disappear" directly.

Texturesynthesis This method is based on an improved optimal solution algorithm in p. Harrison's doctoral thesis image texture tools has been introduced. Texturesynthesis has two parameters, the first parameter is the number of adjacent pixels to compare (Neighborcount), and the second parameter is the sample rate size (Maxsamples) that is used to find the optimal image texture.

Back to Van Gogh's painting. First we import Van Gogh's Starry Night and remove the border.

Then we need to expand the image with white pixels and extend the space that can be used to fix it later.

You can then use the Texturesynthesis method to generate an optimal near-image texture to patch and extend the panorama of the image.

The effect is not bad. You can also have different extension effects by adjusting the values of Neighborcount and maxsamples. Have installed Wolfram language development software students can click here to download engineering documents, not install software students can also be in Wolfram programming cloud try.