A solution for adding controls to a video camera in WPF.

A solution for adding controls to a video camera in WPF.

I. Video Presentation

Some time ago, an ip camera needs to be displayed in real time in a wpf project. We should know a lot about this solution. In winform, We can discard the handle of a widget (usually panel or pictruebox) to the camera sdk to implement this function. In wpf, we can also use this solution for fast implementation.

Take the haikang platform as an example. The sdk of haikang requires us to pass a control handle. In winform, we can directly use the handle attribute of the control to obtain the handle, in wpf, we have two methods:

1. Use WindowsFormsHost as a container to nest a winform control, such as panel, inside the container. This is the same as winform;

2. directly use (HwndSource) PresentationSource. FromVisual (your wpf element). Handle; to get the Handle;

 

 

Ii. Overlay controls

Simple video display can be solved by using the one-in-one solution, but the project I took over is a program that uses the camera video for AR processing. Many icons and information bars need to be displayed on the interface, the most important thing is that the background of the entire main program is the real-time camera screen. In addition, there are various seed windows and sub-windows also need to use videos, this will inevitably involve the difficulty of overlapping controls on the video screen.

　　1. UseMicrosoft. DwayneNeed

 

Microsoft is trying to solve the problem but does not provide a perfect solution. This challenge is the famous airspace issues ). Here is a casual link from yuanyou, which mainly introduces Microsoft's current solution: Workshop.

　　

　　2. transcode the video stream and use d3d to render it as ImageSource.

This solution is troublesome and requires self-control of the transcoding process. Here is an article about this solution: Workshop (not prepared by myself): https://pan.baidu.com/s/1esabfma. the next step is demoruntime:

 

The general principle is to use the sdk callback mechanism to get the code stream frame by frame and then use direct for rendering;

In addition to more code, this solution also has the same problems as that in 1. when the control is dragged, it still gets stuck. You may not know how to solve this problem.

 

 

3. Final Solution

When I was unable to meet the ideal requirements in steps 1 and 2, I suddenly realized a problem. Why is there a airspace problem? That's because the rendering methods of winform and wpf are different. If they are unified, will the problem be solved? It can only be said that the solution is half done. If the control to be superimposed is a regular rectangle, in other words, the control without transparent channels can be used completely, however, you still need to find another method for controls with rounded corners. Let's talk about the former first. The simple code is as follows:

<wfi:WindowsFormsHost>    <wf:Panel/></wfi:WindowsFormsHost><wfi:WindowsFormsHost Width="114" Height="152">　　<ElementHost>　　　　<Border Width="114" Height="152"/>　　</ElementHost></wfi:WindowsFormsHost>

 

In fact, it is another ElementHost set in WindowsFormsHost. Now, both the video panel and the border Control above it are included in WindowsFormsHost, so the border will not be overwritten at the underlying layer during rendering.

Next, let's talk about the stacking method of the most difficult irregular control that I personally think. If you can make the WindowsFormsHost background transparent through code in the wpf application, you don't need to continue reading it, please contact me. I think this is the best solution, and the solution I will talk about may make you disdain or even laugh.

My solution is very simple, that is, to use two windows. The video is displayed separately in the bottom window and set as the main window. The top window is set to AllowsTransparency = "True", ShowInTaskbar = "False ", use showdialog instead of show when displaying the top window. This ensures that you will never be able to operate on the top window and cannot access the video window. Although popup can be used to simulate the above process for sub-Windows, popup also has many problems. By default, it is set to the top, which may cause confusion to users, for example, when two subwindows overlap, the window with a video is displayed at the bottom. However, because the video part is displayed in a popup, it is displayed in the top-Layer window, the window shuffling effect is perfectly realized. In addition, you also need to use a special offset to control the position of the popup. In view of this, I suggest you create a form specifically used to pop up information, then, the control containing the video is passed as a sub-element.

The final solution is to combine these three methods. Because the main interface is a real-time video background, any drag operation should not be delayed, so the main interface uses the No. 3 solution, if some sub-interfaces do not involve the drag operation above, you can use the No. 1 solution. The No. 2 solution can be used to be compatible with Major video platforms, for example, some video platforms do not support handle input.