HiSilicon Development Board wince platform to realize camera function

The following is the development experience of a project over a year ago:

Platform Hi3611 win Mobile6, video data through camera collection, camera drive is done by others,

The collection is 640*480 HD VGA video.

The audio data is collected by the ordinary microphone, 2 channels, 44100 sampling rate, 16 bits.

Use DirectShow and DirectDraw technology.

DirectDraw is primarily useful in previewing

DirectShow Basic Steps: Source filter-> Transform Filter-> Render Filter

The entire audio and video data acquisition process is in contact with the filter (filters).

Source Filter is a collection of sources, specifically camera and microphone, from source filter to obtain the original audio and video data.

Then in transform filter to do further processing, such as compression coding or according to their own needs to modify the video data and so on.

Render filter is the ultimate goal, such as compressed audio and video data written to the file, or preview the video data, directly displayed to the screen.

My goal is to collect the video data in accordance with H263 compression, audio in accordance with AAC compression, and then write to the MP4 file,

Because before this experience, so the earliest use of wince default WMV9 soft code, the result of less than 10 seconds compressed, the machine died.

At first thought it was their own code did not write correctly, many attempts, basically toss for a week, and finally found that the machine memory continues to fall until all eat,

Then came the clear: on the small machine, the use of soft coded compressed audio and video data, basically is not feasible, because the CPU speed limited by the small machine,

Small machines, after all, are different from PCs.

On these small machines, there is a special hardware chip, in the audio and video data compression, which is usually called the hard code.

On the market, all kinds of mobile phones and small devices that can play movies are undoubtedly integrated with hard coded chips, or they will not be able to play movies at all.

Understand this, it is better to do more, and then contact the hardware manufacturers, access to the Development Board of multimedia hard coded interface functions and related code.

So far, video coding and audio coding filter have to develop themselves, as well as write MP4 file filter to develop their own.

And I did not follow the idea of Microsoft to do, so it is written Video/audio encoder filter, but also write Mux MP4 file filter, is to make trouble for themselves,

But also write a custom filter, modify the video data (mainly to add some text information).

Very simple idea: from source filter out of the original audio and video data directly sent to their own written in a render filter,

Modify video data in render filter, hard coding, write MP4 file, send preview video data to screen: one up to finish.

Define a callback function in the render filter for the outer interface to invoke,

In the callback function, modify the video data, modify I was first to open a small rectangle bitmap, get hdc, write text,

Then the small rectangle of the RGB24 data into the YV12, and then use some algorithm to paste the original video data, the results show that the effect is good, speed is very fast.

In the callback function, while the video data directly to the memory, mainly DirectDraw credit.

When the callback function is processed, it is sent to the hard coded chip for H263 encoding,

The audio data is sent directly to encode, AAC-LC encoded, and then the 2 streams are mixed together and written to the MP4 file,

In fact, it is not too difficult to think back now, the key is not difficult.

Of course there are a lot of technical details to be solved here,

For example, audio and video synchronization problem, write file format seems not right, I am looking for another open source library,

For example, is trying to transplant the famous FFmpeg open source code to the WinCE platform.

In the preview section, I first use GDI to draw, the result can be imagined, finally using DirectDraw,

Because it is in the callback function to obtain is YV12 data, so the actual use of the overlay plane, specifically to deal with the video display of a plane.

Probably write so much, have time to the top of the processing ideas, used in Windows platform,
The Windows platform uses ffmpeg to do H263 soft coding and write MP4 files.