FFmpeg accepts socket data as the data source

I used FFMPEG to write my own player. I had a perceptual knowledge about how FFMPEG opened a file, matched Demux, extracted stream, matched decode and rendering, currently, I want to create a network interface to support playing multiple remote cameras. The initial solution was dshow to enable Camera + x264 encoding + live555 (RSTP) transmission, and found that the encoding speed is too slow, unable to meet the real-time requirements, so I changed the encoding to multiple threads and found that the speed was not improved (maybe it was a problem). Later I remembered FFMPEG (strong stuff) and paused the original solution, I will study FFMPEG again. This time, I/O is configured to read data from the I/O set by myself. I want to directly use socket to send the encoded data and then integrate the data on the client, it is transmitted to FFMPEG for decoding and output. Generally, we perform decoding according to the following steps:

1. Open the video file:
First, let's take a look at how to open a video file and get a stream from it. The first thing we need to do is initialize libavformat/libavcodec:

Av_register_all ();
This step registers all available file formats and encoders in the library so that when a file is opened, they can automatically select the corresponding file format and encoder. Note that you only need to call av_register_all () once, so try to use it in your initial code. If you want to, you can only register the individual's file format and encoding. However, you usually have to do so for no reason.

2Next, open the file:
Avformatcontext * pformatctx;
Const char * filename = "myvideo. mpg ";
// Open the video file
If (Av_open_input_file(& Pformatctx, filename, null, 0, null )! = 0)
Handle_error (); // the file cannot be opened

The last three parameters describe the file format, buffer size (size), and format parameters. By specifying null or 0, we will tell libavformat to automatically detect the file format and use the default buffer size. Replace handle_error () with an appropriate error handler in your program ().

3Next, we need to retrieve the stream information contained in the file:
// Retrieve the stream information
If (Av_find_stream_info(Pformatctx) <0)
Handle_error (); // stream information cannot be found

This step fills up the streamams field of avformatcontext with valid information. As a debugable diagnosis, we will output all this information to the standard error output, but you do not use this in an application product:
Dump_format (pformatctx, 0, filename, false );

As mentioned in the introduction, we only process video streams, not audio streams. To make it easier to understand, we simply use the first video stream we found:

Int I, videostream;
Avcodeccontext * pcodecctx;
// Find the first video stream
Videostream =-1;
For (I = 0; I <pformatctx-> nb_streams; I ++)
If (pformatctx-> streams-> codec. codec_type = codec_type_video)
{
Videostream = I;
Break;
}
If (videostream =-1)
Handle_error (); // didn't find a video stream

// Get the pointer of the Video Stream encoding Context
Pcodecctx = & pformatctx-> streams [videostream]-> codec;

Now we have a pointer to the video stream called context. But we still need to find the real encoder to open it.

4Find the video stream Decoder
Avcodec * pcodec;
Pcodec =Avcodec_find_decoder(Pcodecctx-> codec_id );
If (pcodec = NULL)
Handle_error (); // decoder not found

// Notify the decoder that we can process the truncated bit stream-ie,
// Bit stream frame boundary can be included in the package
If (pcodec-> capabilities & codec_cap_truncated)
Pcodecctx-> flags | = codec_flag_truncated;

5Enable Decoder
If (Avcodec_open(Pcodecctx, pcodec) <0)
Handle_error (); // the decoder cannot be opened (So what is "truncation bit stream "? Okay, as we can see in a moment, the data in the video stream is split into packages. Because the data size of each video frame is variable, the boundary between the two frames is not necessarily the boundary of the packet. Here, we tell the decoder that we can process bit streams .)

An important information stored in the avcodeccontext structure is the video frame rate. To allow non-integer frame rates (for example, 29.97 frames of NTSC), the rates are stored in the form of scores. the molecules are in pcodecctx-> frame_rate, and the denominator is in pcodecctx-> frame_rate_base. When using different video file test libraries, I noticed that some encoders (obviously ASF) do not seem to correctly assign values (frame_rate_base uses 1 instead of 1000 ). The following patch is provided:

// Add this sentence to correct the frame rate error generated by some encoders.
If (pcodecctx-> frame_rate> 1000 & pcodecctx-> frame_rate_base = 1)
Pcodecctx-> frame_rate_base = 1000;

Note that even if this bug is fixed in the future, leaving these words does not cause any harm. The video speed cannot exceed FPS.

6Allocate space for video frames to store decoded images:

Avframe * pframe;

Pframe =Avcodec_alloc_frame();

7. Finally, we have prepared to read data from the stream.

 

Read data

 

What we will do is read the entire video stream by reading the package, and then decode it into a frame. It is best to convert the format and save it later.

 

Int framefinished;

 

Avpacket packet;

 

I = 0;

 

While (av_read_frame (pformatctx, & Packet)> = 0 ){

 

// Is this a packet from the video stream?

 

If (packet. stream_index = videostream ){

 

// Decode Video Frame

 

Avcodec_decode_video (pcodecctx, pframe, & framefinished,

 

Packet. Data, packet. size );

 

// Did we get a video frame?

 

If (framefinished ){

 

// Convert the image from its native formatto RGB

 

Img_convert (avpicture *) pframergb, pix_fmt_rgb24,

 

(Avpicture *) pframe, pcodecctx-> pix_fmt,

 

Pcodecctx-> width, pcodecctx-> height );

 

// Save the frame to disk

 

If (++ I <= 5)

 

Saveframe (pframergb, pcodecctx-> width,

 

Pcodecctx-> height, I );

 

}

 

}

 

// Free the packet that was allocated byav_read_frame

 

Av_free_packet (& Packet );

 

}

 

This loop process is relatively simple: av_read_frame () reads a package and saves it to the avpacket struct. Note that we only applied for the structure of a package-FFMPEG applied for internal data memory for us and directed it through the packet. Data Pointer. The data can be released later through av_free_packet. Function avcodec_decode_video () converts a package to a frame. However, when decoding a packet, we may not get the required frame information. Therefore, when we get the next frame, avcodec_decode_video () sets the frame end mark framefinished for us. Finally, we use
The img_convert () function converts a frame from the original format (pcodecctx-> pix_fmt) to the RGB format. Remember, you can convert an avframe struct pointer to an avpicture struct pointer. Finally, we pass the frame and height width information to our saveframe function.

However, now I want to configure my "file source" (socket-transmitted data). Fortunately, FFMPEG not only supports matching Demux and decoder, but also provides interfaces for data sources;

First, generate your own aviocontext

Play-> m_avio_ctx = avio_alloc_context (play-> m_io_buffer,

Io_buffer_size, 0, (void *) Play, read_packet, null, seek_packet );

If (! Play-> m_io_buffer)

{

Printf ("create iocontext failed! \ N ");

Av_free (play-> m_io_buffer );

Return-1;

}

Play-> m_avio_ctx-> write_flag = 0;

 

Ret = av_probe_input_buffer (play-> m_avio_ctx, & iformat,
"", Null, 0, null );

If (Ret <0)

{

Printf ("av_probe_input_buffercall failed! \ N ");

Goto failed_flg;

}

 

// The key point is to configure Pb

Play-> m_format_ctx-> Pb = play-> m_avio_ctx;

The following is the same as before.

Ret = avformat_open_input (& Play-> m_format_ctx,
"", Iformat, null );

I don't know how long it is. I just want to write about it when the time is too busy, so I have time to add it.