UWP hand-drawn video creation tool technology sharing series, uwp tool technology

UWP hand-drawn video creation tool technology sharing series, uwp tool technology

First, let's talk about the original intention of writing this article.

At the beginning of the painting, I read through the UWP App code and found that there are indeed many advanced technical points, but also many problems, such as scalability, coupling, performance, and functions. So we decided to reconstruct this product from the ground up and create a brand new "come to Draw Pro". After three months of the world, this product was finally officially launched. (Make a small advertisement and search for "paint Pro" in the Windows app store. Currently, the company is positioned as a paid app, but there is a one-month trial period. If you are interested, you can ask me for free code. Here is the IT House Report: https://www.ithome.com/html/win10/324042.htm ). Based on the implementation of all the tools of the old version of "laifeng", we have implemented more Ink paint and Dial functions, better SVG parsing and drawing, and better text painting processes, insert video and gif images. The original intention of writing this article and the next series of articles is to give a comprehensive review of the technologies and problems encountered in the development of "come to Draw Pro, we hope to share this with more people interested in UWP and hand-drawn videos.

 

Now let's get down to the point and start to share the implementation methods and effects of the functions of "come to Draw Pro". A series of articles will be provided to describe each function point in detail.

1. Win2D-Basics of hand-drawn video Rendering

Win2D is a 2D graphics GPU rendering SDK Based on Windows Runtime. It can be used in UWP, Windows Phone, and Windows Runtime apps. The programming languages can be C ++, C #, or VB, I believe that students who have tried graphic rendering in UWP are familiar with Win2D. After all, I recommend the implementation method. GitHub address: Please Sample for your reference, in Windows Store can also download: https://www.microsoft.com/zh-cn/store/p/win2d-example-gallery/9nblgggxwt9f? Rtc = 1, the following is Sample:

2. parsing and drawing SVG

As you know, SVG is a vector graph format. Unlike bitmap, it is composed of XML, and node information includes path, stroke, and fill, it represents the path, path border, and filling rule of SVG respectively. Our operations on SVG are first SVG parsing and static display, and then based on the original path composition and sequence of SVG, combined with the sequence of hand-drawn habits, the total length of the path and the total length of the drawing are determined in each frame which part of the path is displayed or which part of the shape is filled. Let's take a look at the Implementation Effect in the application. The following three figures show an SVG Painting Process:

3. Drawing PNG, JPEG, BMP, and other bitmaps

PNG, JPEG, BMP, and other image formats have different compression and encoding formats. However, because they all belong to bitmaps, the components are pixels, an array of width * height length is used to store the pixel values of each pixel. The pixel values may consist of multiple formats, such as ARGB, BGR, and YUV. Because bitmap has no path, we cannot draw the creation process like SVG by default, so we chose a relatively simple rendering method: draw from one corner of the image to another, such as the upper left to the lower right corner, to maintain a stable speed and direction, this is the process of drawing.

Of course, the above is just the simplest and most crude processing method. In actual scenarios, many SVG not only contain paths, but also bitmaps. For example, process an image in PS, import it to AI, manually draw some paths, and finally export SVG. The correct SVG painting process should be based on the path in AI and the Stroke of the corresponding position in the bitmap. In processing bitmap, we can also add ways for users to outline paths and enrich the presentation methods. You can also perform automatic edge profiling and image cutting operations on bitmap, which will be shared in detail later.

 

4. Text parsing and drawing

Text is an important display and expression method in hand-drawn videos, and it is also a relatively complex process of parsing and drawing. After a font is determined, the corresponding text will be taken out of the TTF file of the corresponding font for display, and the path data representing all the edges of the text can also be obtained. However, in hand-drawn videos, the display mode is to depict the filling of the font, rather than the edge. In addition, the edge of the font itself does not have any order or regularity, but simply forms the edge of the font, therefore, we need to process the edge path as follows:

Obtain edge path data-> perform horizontal or vertical rays on points in the path point by point, record all intersection point sets-> edge path point set and Intersection Set to calculate the Intermediate Point Set-> sort the normal text of the intermediate point set in top/bottom/left to right-> sort by point distance regroup-> draw the path based on the processed point set.

5. drawing Ink and using Surface Pen and Surface Dial

Ink handwriting is widely used in Windows 10, such as plotting, writing, and hand-drawn videos. Microsoft's Windows SDK provides great InkToolbar and InkCanvas, which can be displayed with several paint brushes by default, such as ballpoint pen, pen, pencil, and highlighter. What we need to do in hand-drawn videos is to extend the static display supported by the SDK by default to support dynamic rendering and export the video support. The Ink tip, such as a ballpoint pen, can be taken out and used directly. The Ink effect of the pencil, the stroke width of the pen, and the color mixing effect of the highlighter are as follows, you need to perform separate processing on your own. We will explain in detail the processing of each paint brush later.

InkToolbar and InkCanvas support Surface Dial and Surface Pen operations by default. Our applications also provide more extended support for Surface Dial, such as color selection and paint brush selection. The following is an operation diagram of the InkToolbar and InkCanvas and the Surface Dial used in the application (from the IT house ):

 

6. Insert a video file into a hand-drawn video

You can insert videos into hand-drawn videos to enrich the representation. In common scenarios, you can add hand-drawn character materials next to a video to enable adult comments and narration. Alternatively, you can add pasters or items to a video to dynamically change content next to a video or in a video, in this way, it is richer than simply processing videos and adding stickers, which can often be seen in TV programs, such as inserting videos into hand-painted videos and turning them into a dynamic background. One of the following scenarios:

The MediaElementPlayer player of the system is used for processing in the program during operations and previewing. When exporting a video, you must independently process the inserted video frame and audio track.

 

7. video generation and export

After the above operations are completed, we need to export it into a video file, such as MP4 and WMV. This generation process mainly includes two parts: ① It is to use Win2D for background rendering operations to restore the real rendering process and speed, saves the bitmap of each frame according to the specified frame rate. ② uses a method similar to FFMpeg to save the frame sequence as a video according to the specified frame rate, simultaneously process overlapping background music and inserted audio tracks to complete video file generation.

 

By now, the basic process of parsing, rendering, operating, and exporting the main elements of hand-drawn videos has been completed. Today's article will give a general introduction to the implementation principles of each part. Later, we will give a detailed explanation of each part, especially in terms of SVG and text drawing, as well as detailed principles and implementations, and various problems encountered in the development process, or their own, or Microsoft Systems. And various paint brush implementation methods.

Thank you!

 

The addresses detailed in each of the following sections are continuously updated:

Analysis and http://www.cnblogs.com/shaomeng/p/7476480.html of SVG