Nvidia cuvid hardware solution, and read files through FFmpeg, cuvidffmpeg

Nvidia cuvid hardware solution, and read files through FFmpeg, cuvidffmpeg

Although FFmpeg has a hard solution of cuvid, but it cannot find any good information, nvidia sdk is easy to understand. Refer to FFmpeg source code to change the data acquisition of nvidia video codec sdk to FFmpeg, make up for the shortcomings that native sdks cannot use streams as data sources. The SDK version is Video_Codec_SDK_7.1.9, which can be downloaded from the NVIDIA website.

1. Modify the data source

First, some conventional FFmpeg Initialization

Bool VideoSource: init (const std: string sFileName, FrameQueue * pFrameQueue) {assert (0! = PFrameQueue); oSourceData _. hVideoParser = 0; oSourceData _. pFrameQueue = pFrameQueue; int I; AVCodec * pCodec; encode (); avformat_network_init (); pFormatCtx = avformat_alloc_context (); if (avformat_open_input (& pFormatCtx, sFileName. c_str (), NULL, NULL )! = 0) {printf ("Couldn't open input stream. \ n "); return false;} if (avformat_find_stream_info (pFormatCtx, NULL) <0) {printf (" Couldn't find stream information. \ n "); return false;} videoindex =-1; for (I = 0; I <pFormatCtx-> nb_streams; I ++) if (pFormatCtx-> streams [I]-> codec-> codec_type = AVMEDIA_TYPE_VIDEO) {videoindex = I; break;} if (videoindex =-1) {printf ("Didn't find a video stream. \ n "); return false;} pCodecCtx = pFormatCtx-> streams [videoindex]-> codec; pCodec = avcodec_find_decoder (pCodecCtx-> codec_id); if (pCodec = NULL) {printf ("Codec not found. \ n "); return false;} // Output Info ------------------------------- printf (" --------------- File Information ------------------ \ n "); av_dump_format (pFormatCtx, 0, sFileName. c_str (), 0); printf ("audio \ n"); memset (& g_stFormat, 0, sizeof (CUVIDEOFORMAT); switch (pCodecCtx-> codec_id) {case AV_CODEC_ID_H263: g_stFormat.codec = condition; break; case when: g_stFormat.codec = condition; break; case when: g_stFormat.codec = cudaVideoCodec_MPEG2; break; case when: g_stFormat.codec = break; break; case AV_CODEC_ID_VC1: g_stFormat.codec = cudaVideoCodec_VC1; break; default: return false;} // The correspondence between FFmoeg and cuvid in this region is not very definite, however, using this parameter seems to be the most reliable switch (pCodecCtx-> sw_pix_fmt) {case when: g_stFormat.chroma_format = enabled; break; case when: enabled = enabled; break; case when: g_stFormat.chroma_format = enabled; break; default: g_stFormat.chroma_format = cudaVideoChromaFormat_420; break;} // after searching for a long time, the identification space in the Field Format and frame format in FFmpeg is finally found to be scanned by line, switch (pCodecCtx-> field_order) {case av_field_sive ssive: case AV_FIELD_UNKNOWN: Unknown = true; break; default: g_stFormat.progressive_sequence = false; break ;} pCodecCtx-> centers = 1; g_stFormat.coded_width = pCodecCtx-> coded_width; g_stFormat.coded_height = pCodecCtx-> coded_height; rows = pCodecCtx-> width; rows = 0; rows = pCodecCtx-height>; dependencies = 0; if (pCodecCtx-> codec_id = AV_CODEC_ID_H264 | pCodecCtx-> codec_id = AV_CODEC_ID_HEVC) {if (pCodecCtx-> codec_id = AV_CODEC_ID_H264) hsf-bsfc = av_bitstream_filter_init ("hsf-_mp4to1_ B"); else hsf-bsfc = av_bitstream_filter_init ("hevc_mp4to1_ B");} return true ;}

A very important piece of code here is:

if (pCodecCtx->codec_id == AV_CODEC_ID_H264 || pCodecCtx->codec_id == AV_CODEC_ID_HEVC) {    if (pCodecCtx->codec_id == AV_CODEC_ID_H264)        h264bsfc = av_bitstream_filter_init("h264_mp4toannexb");    else        h264bsfc = av_bitstream_filter_init("hevc_mp4toannexb");}

Many of the code and pseudocode on the Internet say that the data source is changed to FFmpeg, but I found that the callback function of the Parser created by cuvidCreateVideoParser was not called. After some tossing, integrated with NVIDIA website, stackoverflow and FFmpeg source code, we found that we had to process H264 data to convert AVPacket into CUVIDSOURCEDATAPACKET effectively. Hsf-bsfc is defined as AVBitStreamFilterContext * hsf-bsfc = NULL;

2. Convert AVPacket to CUVIDSOURCEDATAPACKET and hand it to cuvidParseVideoData.

void VideoSource::play_thread(LPVOID lpParam){    AVPacket *avpkt;    avpkt = (AVPacket *)av_malloc(sizeof(AVPacket));    CUVIDSOURCEDATAPACKET cupkt;    int iPkt = 0;    CUresult oResult;    while (av_read_frame(pFormatCtx, avpkt) >= 0){        if (bThreadExit){            break;        }        bStarted = true;        if (avpkt->stream_index == videoindex){            cuCtxPushCurrent(g_oContext);            if (avpkt && avpkt->size) {                if (h264bsfc)                {                    av_bitstream_filter_filter(h264bsfc, pFormatCtx->streams[videoindex]->codec, NULL, &avpkt->data, &avpkt->size, avpkt->data, avpkt->size, 0);                }                cupkt.payload_size = (unsigned long)avpkt->size;                cupkt.payload = (const unsigned char*)avpkt->data;                if (avpkt->pts != AV_NOPTS_VALUE) {                    cupkt.flags = CUVID_PKT_TIMESTAMP;                    if (pCodecCtx->pkt_timebase.num && pCodecCtx->pkt_timebase.den){                        AVRational tb;                        tb.num = 1;                        tb.den = AV_TIME_BASE;                        cupkt.timestamp = av_rescale_q(avpkt->pts, pCodecCtx->pkt_timebase, tb);                    }                    else                        cupkt.timestamp = avpkt->pts;                }            }            else {                cupkt.flags = CUVID_PKT_ENDOFSTREAM;            }            oResult = cuvidParseVideoData(oSourceData_.hVideoParser, &cupkt);            if ((cupkt.flags & CUVID_PKT_ENDOFSTREAM) || (oResult != CUDA_SUCCESS)){                break;            }            iPkt++;            //printf("Succeed to read avpkt %d !\n", iPkt);            checkCudaErrors(cuCtxPopCurrent(NULL));        }        av_free_packet(avpkt);    }    oSourceData_.pFrameQueue->endDecode();    bStarted = false;}

Here, after FFmpeg reads data packets, there is an important processing for the H264 and HEVC formats, as mentioned above,

if (h264bsfc){    av_bitstream_filter_filter(h264bsfc, pFormatCtx->streams[videoindex]->codec, NULL, &avpkt->data, &avpkt->size, avpkt->data, avpkt->size, 0);}

For the meaning of this processing, see the blog http://blog.csdn.net/leixiao%1020/article/details/41067055.

In this way, through FFmpeg, CUVID can be used for streaming. I personally tried to read local files and rtsp streams. FFmpeg only needs to change the file address to the rtsp stream address to read the rtsp stream. I thought it would be complicated if I didn't use the over-stream method before.

3. A little data

This is the 20-way decoded data of the decoding process (not displayed) on GTX 1080. The 20-channel x decoding can reach a draw of 37fps, and the video card is also 6.

 

 

Source code: http://download.csdn.net/download/qq_33892166/9792997

The source code encountered a problem and could not find the cause. Code decoding on GTX 1080 and Tesla P4 works well. P4 because the drive mode is TCC mode, it can only be decoded and cannot be displayed; 1080 can be decoded and displayed. However, on the GT940M on my computer, the native SDK always reports the error CUDA_ERROR_NO_DEVICE when cuvidCreateDecoder. The driver seems to be okay. After trying CUDA demo, The CUDA operation is also normal. The query information shows that GT940M should support CUVID. Hope you can advise me if you know the reason.