OMCS tips-camera and its dynamic capabilities

Document directory

• 1. synchronous broadcast mode
• 2. asynchronous broadcast mode

When developing video chat applications, we often need to obtain camera-related information. In video chat, we may also want to have dynamic capabilities. For example, you can switch to a camera without interrupting video chat, or modify the resolution or encoding quality of the camera. OMCS provides many useful features to support the above requirements.

1. Enumeration cameras

How do we know which cameras are in the current computer?

OMCS provides a tool class OMCS. Tools. Camera to help us obtain this information. Camera has a static method GetCameras to enumerate all cameras on the current computer.

/// <Summary> /// enumerate all camera devices on the current computer. /// </Summary> public static List <CameraInformation> GetCameras ()

CameraInformation encapsulates the basic information of the camera. Its Class diagram is as follows:

You can bind the list returned by the GetCameras method to the Combox control. The effect is as follows:

By default, the index of the camera used by IMultimediaManager is 0. If the camera with an index of 0 is unavailable, or you specify to use another camera, you can save the index of the specified camera to the configuration file. When the client starts, it reads the index value from the configuration file and assigns it to the CameraDeviceIndex attribute of IMultimediaManager.

2. Obtain the resolution supported by the camera

When a camera is selected, how do we know which resolution and maximum frame rate the camera supports?

Camera provides another static method, GetCameraCapability, to obtain the capabilities of the target Camera.

/// <Summary> /// obtain the target camera's capability information (resolution, maximum frame rate) /// </summary> /// <param name = "deviceIndex"> index of the target camera </param> public static List <CameraCapability> GetCameraCapability (int deviceIndex)

The CameraCapability class encapsulates the camera's capability information. The class diagram is as follows:

You can bind the list returned by the GetCameraCapability method to the Combox control. The effect is as follows:

3. dynamically switch the camera

Let's imagine a situation where I have two available cameras installed on my computer. I am using a camera with an index of 0 to video chat with my friends. At some point, I want to switch to the camera with an index of 1 without interrupting video chat. This requirement is called dynamic camera switching.

OMCS supports dynamic camera switching, and the operation is quite simple: we only need to assign the CameraDeviceIndex attribute of IMultimediaManager to the index of the camera to switch.

The switchover is automatically performed in OMCS. After the switchover is completed in a short time, OMCS sends the video data collected by the new camera to each guest.

4. dynamically modify the Camera Resolution

I still remember that QQ video chat has this capability. We can choose to use the big window mode during video chat. This is actually to use the camera to collect videos at a higher resolution. For example, if the original acquisition resolution is 320*240, you can switch to a higher resolution of 640*480.

OMCS supports modifying the collection resolution of the camera in use without any interruption. The operation is also quite simple: we only need to set the CameraVideoSize attribute of IMultimediaManager to the target resolution size. Of course, if the resolution is not supported by the current camera, NotSupportedException will be thrown. In addition, when the encoding quality is the same, the higher the video resolution, the larger the output code stream and the larger the required bandwidth.

To avoid improper resolution settings for the CameraVideoSize attribute, we can determine whether the target Camera supports the specified resolution through the OMCS. Tools. Camera static method Support before assigning values:

/// <Summary> /// whether the target camera supports collecting the specified resolution. /// </Summary> public static bool Support (int deviceIndex, Size videoSize)

5. dynamically adjust the encoding Quality

When running the client, you can set the CameraEncodeQuality attribute of IMultimediaManager to adjust the encoding quality of the video collected by the camera in real time. The higher the encoding quality (the smaller the value of CameraEncodeQuality), the higher the bandwidth requirement; and vice versa.

Of course, as described in the previous article, if the AutoAdjustCameraEncodeQuality attribute of IMultimediaManager is set to true, CameraEncodeQuality will be automatically adjusted by OMCS to ensure voice clarity and coherence with priority, manually setting CameraEncodeQuality does not work.

If we set AutoAdjustCameraEncodeQuality to false and our application can detect real-time changes in the network status, we can manually adjust CameraEncodeQuality based on the current network status.

6. Control video output

Imagine a situation where, for some reason, I don't want attendees to see my video at some time during a video conference. After a while, I want to restore my video to its original state.

The best way to save bandwidth is to not output video frames during this period. OMCS can easily implement this control: you only need to set the OutputVideo attribute of IMultimediaManager to false to disable the output of video frames.

7. Discard video frames during Broadcast

The scenario to be imagined below is a little more complicated: during a video conference, my video will be sent to everyone at the meeting. Suppose I have established a P2P channel with each participant, all video frames are transmitted through P2P channels. When the problem arises, the quality of each P2P channel is different. Some may be fast, and some may be slow. We assume that the channel to participant A is very slow, and the channel quality to other participants is very good. Then, let's analyze the possible situations in the two Broadcast modes.

1. synchronous broadcast mode

In synchronous broadcast mode, the next video frame is sent only when a video frame is sent through all channels. In this way, the channel between A and A is slow, causing other participants to see their own videos to be stuck or incoherent.

2. asynchronous broadcast mode

In asynchronous broadcast mode, video frames are sent asynchronously on all channels, so that the slowness of Channel A does not affect other participants. OMCS adopts this mode.

However, due to the slowness of Channel A, the production speed of Video Frames is much higher than the speed at which channel A consumes video frames, this results in the need to use more memory to cache those video frames that cannot be sent. As far as the client process is concerned, the memory occupied by the client process will continue to increase, just like memory leakage.

How can this problem be solved? The solution provided by OMCS is to discard frames when the channel is busy. You can enable this scheme by setting the AllowDiscardFrameWhenBroadcast attribute of IMultimediaManager to true.

After AllowDiscardFrameWhenBroadcast is enabled, when a new video frame is sent to the corresponding participants through a P2P channel, the system checks whether the P2P channel is busy, cancel the transmission of the video frame in this channel. In this way, the wireless increase in memory is avoided. However, in this solution, participants with slow channels may encounter mosaic, card, and discontinuous video because of the discarded video frames.