Application of Linux-video streaming 4

This article focuses on the frame capture method and transfers the focus from video4linux to the network. In terms of instant transfer of network images, RTP is also the standard used by major manufacturers. In this phase, we will be able to learn how to use jrtplib
To add network functions.
Video4linux frame capture method
In the last issue of xawtv, we saw the image capture function of xawtv,
Among them, the most important part is to use video4linux for image retrieval. As long as you can write
In video4linux, it is easy to capture images. We use MMAP.
To capture images.
MMAP Retrieval Method
To illustrate how to capture images using MMAP, we do not recommend that you study xawtv directly.
The code for this part. After studying several original software codes that support MMAP image retrieval,
We recommend that you download a program named TV. Its official webpage is:
Http://effectv.sourceforge.net/index.html
Movie TV is a program designed by Japanese people. It also uses video4linux for Image Retrieval. In terms of MMAP code, Movie TV is easier to understand, at the same time, you can also learn some image processing techniques through Movie TV. TV is a video software that supports special effects,
It is an interesting program.
Main functions
Function related to frame grab in Movie TV:
Int video_grab_check (INT palette );
Int video_set_grabformat ();
Int video_grabstart ();
Int video_grabstop ();
Int video_syncframe ();
Int video_grabframe ();
These functions are defined in video. h. We will not repeat the initialization of video4linux.
For frame grab, call video_grabstart () to start Image Retrieval. The program code is as follows:
/* Start the continuous grabbing */
Int video_grabstart ()
{
VD. Frame = 0;
If (v4lgrabstart (& VD, 0) <0)
Return-1;
If (v4lgrabstart (& VD, 1) <0)
Return-1;
Return 0;
}
The main character is the v4lgrabstart () function, which is implemented in v4lutils/v4lutils. C,
The program code is as follows:
/*
* V4lgrabstart-activate MMAP capturing
*
* Vd: v4l device object
* Frame: frame number for storing captured image
*/
Int v4lgrabstart (v4ldevice * vd, int frame)
{
If (v4l_debug) fprintf (stderr, "v4lgrabstart: Grab Frame
% D. \ n ", frame );
If (vd-> framestat [frame]) {
Fprintf (stderr, "v4lgrabstart: frame % d is already used
Grab. \ n ", frame );
}
VD-> MMAP. Frame = frame;
If (IOCTL (vd-> FD, vidiocmcapture, & (vd-> MMAP) <0 ){
V4lperror ("v4lgrabstart: vidiocmcapture ");
Return-1;
}
VD-> framestat [frame] = 1;
Return 0;
}
V4lgrabstart () uses MMAP to obtain images. V4lgrabstart () is also used
IOCTL () to complete this low-level action, as in the second article in this article when implementing video4linux,
Written:
IOCTL (vd-> FD, vidiocmcapture, & (vd-> MMAP ));
The framestat field in VD mainly records the current frame status:
VD-> framestat [frame]
This field is defined in v4lutils. H, and using MMAP, we need two frames.
So framestat is declared as an array of two elements.
The complete structure definition is listed as follows:
Struct _ v4ldevice
{
Int FD;
Struct video_capability capability;
Struct video_channel [10];
Struct video_picture picture;
Struct video_clip clip;
Struct video_window window;
Struct video_capture capture;
Struct video_buffer buffer;
Struct video_mmap MMAP;
Struct video_mbuf mbuf;
Struct video_unit unit;
Unsigned char * map;
Pthread_mutex_t mutex;
Int frame;
Int framestat [2];
Int overlay;
};
Please refer back to the actual content of the second article in this article.
Is more complete. When we start grab image to one of the frames, we
To 1:
VD-> framestat [frame] = 1;
Then, use v4lsync () to wait for the frame to be retrieved and use IOCTL () to pass in vidiocsync
You can check whether the frame has been retrieved:
If (IOCTL (vd-> FD, vidiocsync, & frame) <0 ){
V4lperror ("v4lsync: vidiocsync ");
Return-1;
}
VD-> framestat [frame] = 0;
Return 0;
If the frame has been retrieved, set the frame state to 0, indicating that the current frame
There is no retrieval action, and therefore in v4lsync ()
At the beginning, we will first check this part:
If (vd-> framestat [frame] = 0 ){
Fprintf (stderr, "v4lsync: grabbing to frame % d is not started. \ n ",
Frame );
}
The v4lsync () function is also an important function. The program code is as follows:
/*
* V4lsync-Wait until MMAP capturing of the frame is finished
*
* Vd: v4l device object
* Frame: frame number
*/
Int v4lsync (v4ldevice * vd, int frame)
{
If (v4l_debug) fprintf (stderr, "v4lsync: Sync frame % d. \ n", frame );
If (vd-> framestat [frame] = 0 ){
Fprintf (stderr, "v4lsync: grabbing to frame % d is not
Started. \ n ", frame );
}
If (IOCTL (vd-> FD, vidiocsync, & frame) <0 ){
V4lperror ("v4lsync: vidiocsync ");
Return-1;
}
VD-> framestat [frame] = 0;
Return 0;
}
In Movie TV, you need to call the video_syncframe () function to wait for the frame.
Video_syncframe () calls the v4lsync () function. Video_syncframe ()
The original code of the function is as follows:
Int video_syncframe ()
{
Return v4lsyncf (& Vd );
}
Frame capture
After reading these core functions, will these functions be used in Movie TV? Expresstv
Is a video software that outputs special effects/
Each file in the directory supports a special effect independently,
Therefore, the functions we introduce are called independently by each special effect. Let's take a look at life
The main program life. C of this special effect should first be viewed by the Lifestart () function. The program code is as follows:
Int Lifestart ()
{
Screen_clear (0 );
Image_stretching_buffer_clear (0 );
Image_set_threshold_y (40 );
Field1 = field;
Field2 = field + video_area;
Clear_field ();
If (video_grabstart ())
Return-1;
Stat = 1;
Return 0;
}
After completing initialization settings, Lifestart () calls video_grabstart ()
The function starts image capture. In the lifedraw () function, call video_syncframe ()
Wait until the frame is retrieved and then output.
YUV and yiq
In terms of the PAL video standard, the main mode is YUV, which is similar to the RGB
Different. In NTSC, The yiq mode is used. For these two video and image modes,
We must also design a conversion program for YUV and RGB, yiq and RGB. YUV. C is also available in Movie TV.
The program code is responsible for conversion.
RGB Introduction
RGB uses the primary colors red, green, and blue to represent images,
After overlapping the red and blue colors, the product becomes the Magenta, and the red and green colors become the yellow ones ),
The three-color overlap is white ). The three primary colors of RGB refer to the color of light, not the color of paint,
The RGB mode is often used on the monitor, which is different from the pal or NTSC video standard.
Conversion between YUV and yiq
The conversion formula between YUV, yiq, and RGB is as follows:
Y = 0.299r + 0.587G + 0.114b
U = B ━ y
V = r y
I = 0.877 (R-Y) cos33 platinum 0.492 (B-Y) sin33
Q = 0.877 (R-Y) sin33 + 0.492 (B-Y) cos33
RGB is composed of three primary colors: R, G, and B. Similarly, YUV is composed of three elements: Y, U, and V. In pal
The conversion formula used for implementing U and V is:
U = 0.492 (B-Y)
V = 0.877 (R-Y)
Yiq can be simplified to a conversion matrix:
(Manuscript)
How to Use jrtplib
In terms of video streaming, with the procedural implementation capability of image capturing,
Complete video streaming software, such as video conferencing software,
Of course, you must add the network transfer function.
In order to transmit images over the network and implement real-time (Real-Time) functions,
We must use the RTP protocol. Here we have completed the first part of the work,
The next step is to add the network instant transfer dial function. Here, video4linux has come to an end. We will
Change the main character of video streaming to RTP for further discussion.
Add RTP protocol
Video Streaming is used to design video conferencing software,
One of the essential technical components is the RTP protocol. RTP is also VoIP
(Voice over IP) communication protocols used by related technologies.
To work with video streaming to design complete video conferencing software, we are bound to add RTP
. In this regard, we chose to use the ready-made RTP Library named ━ jrtplib.
Unlike the video4linux library, we choose to develop video4linux on our own,
However, jrtplib is a very mature project and continues to be maintained. In the future, jrtplib will join
With full support for IPv6 and multicasting, jrtplib is used to develop our video
Streaming software is the solution.
Jrtplib Introduction
The full name of RTP is real-time transport protocol, which is defined in RFC 1889 and RFC 1890,
We have briefly introduced RTP in the first article. In RFC 1889, RTP is defined:
RTP: A transport protocol for real-time applications
In RFC 1890, the description of RTP is:
RTP profile for audio and video conferences with minimal control
Now we need to use video streaming to design video conferencing software.
RTP discussion and research is one of the required lessons. For video conferencing software, RTP also provides audio
In fact, any real-time (Real-Time)-related topics cannot be separated from RTP.
Jrtplib-related projects include jvoiplib and jthread,
These two libraries are helpful for our work. This article focuses on
Jrtplib. Jrtplib implements the RTP protocol and provides easy-to-use APIs.
For software development and use. Jrtplib also supports session and can be executed on the following Platform:
MS-Windows 95,98 and NT
Linux
FreeBSD
HP-UX
Solaris
VxWorks
The official homepage of jrtplib is:
Http://lumumba.luc.ac.be/jori/jrtplib/jrtplib.html
Before using jrtplib, read the copyright notice of jrtplib.
Jrtplib's first program
After introducing the jrtplib design method, you will find that jrtplib
Very easy to use. To transmit data using the RTP protocol, you must first create a session,
The method is as follows:
# Include "rtpsession. H"
Int main (void)
{
Rtpsession sess;
Sess. Create (1, 5000 );
Return 0;
}
In the first step, we need to include rtpsession. h first:
# Include "rtpsession. H"
Next we will generate the rtpsession class objects:
Rtpsession sess;
Finally, you can create a session to complete the simplest initial action:
Sess. Create (1, 5000 );
The create () member function receives a portbase parameter, specifying the port of the session,
Initialize the timestamp and packet sequence number. Rtpsession: Create ()
The program code is as follows:
Int rtpsession: Create (INT localportbase, unsigned long localip)
{
Int status;
If (initialized)
Return err_rtp_sessionalreadyinitialized;
If (status = rtpconn. Create (localportbase, localip) <0)
Return status;
If (status = contribsources. createlocalcname () <0)
{
Rtpconn. Destroy ();
Return status;
}
Createnewssrc ();
Localinf. createsessionparameters ();
Rtcpmodule. initialize ();
Initialized = true;
Return 0;
}
Create () then an SSRC is created:
Createnewssrc ();
SSRC is the local synchronization source identifier.
Specify the target end
Next, specify the destination IP address:
Unsigned long ADDR = ntohl (inet_addr ("127.0.0.1 "));
Sess. adddestination (ADDR, 5000 );
This indicates that we want to transmit the packet to port 5000 of 127.0.0.1 (local end,
You only need to apply the settings.
Transmit RTP packets
Sess. sendpacket ("1234567890", 10, 0, false, 10 );
1234567890 indicates the string to be transmitted, and the second parameter (10) indicates the length of the transmitted string,
The third parameter is payload type, the fourth parameter is Mark flag, and the fifth parameter is
The increment unit of timestamp. In the next phase, we will work with SDL to output images,
Therefore, we will introduce how to receive RTP packets in the next phase. We Use SDL
The main purpose is to output images to the screen.
RTP packet format
RTP standards are adopted by many major manufacturers, such as Microsoft and Intel. Therefore, we need to know about them.
The low-layer Technical Department of RTP. RTP is the same as other Internet communication protocols,
There will also be a package header in the package, followed by the package information. Figure 1 shows the RTP packet header format. The entire file header is divided into 10 columns (fields ).
In rtpsession: sendpacket (), the third and fourth parameters are respectively payload type and
Mark flag, in the RTP packet header, is the payload column and m column. Payload
The column length is 7 bits, and the M column length is 1 bits.
RTP payload type
The payload type of the RTP Header specifies the encoding method of the packets. We will list five commonly used audio
Standard, and three common video standards, JPEG/H.261/H.263
We have made a brief introduction in the first article. The following table lists common payload types:
Payload
The Type encoding standard supports audio or video clock rate (HZ)
2 g.721 a 8000
4 g.723a 8000
7 LPC a 8000
9. g.722 a 8000
15 g.728 a 8000
26 JPEG v 90000
31 H.261 v 90000
34 H.263 v 90000
Linux IP stacks
For network applications, you must also have a simple understanding of Linux IP stacks,
We recommend that you study the program code of Linux kernel directly,
Of course, there are already special books for discussion:
Linux IP stacks commentary, Stephen T. satchell & H. B .j. Clifford,
Coriolisopen press, ISBN 1-57610-470-2
Linux is a network operating system, and Linux provides complete network support, including the 2.4 series.
Kernel has been added to IPv6. The communication layer functions provided by Linux kernel and module include:
Various Ethernet, Token Ring and FDDI (Fiber Distributed Data Interface)
Physical animation program
Bot programs for PPP, slip, and slip protocols
Provides IPX (Internet package exchange) communication protocols
Provide bot programs for Radio Engineering (ax25)
Provide appletalk bot programs
Body Animation programs used by other link layers
Supports router functions, including rip (Router Information Protocol) protocols.
Support for ICMP (Internet Control Message Protocol) Protocols
Support for IGMP (Internet Group Message Protocol) Protocols
Support IP protocol
Support for TCP (Transmission Control Protocol) Protocols
Support for UDP protocol
The Book of Linux IP stacks is based on basic and important TCP/IP Services,
Includes: CC, packet management, datax, and datastream.
Introduction to Linux IP stacks
For more information, see Chapter 1 of Linux IP stacks.
Before reading this book, please take a good look at the structure of this book before you can get started easily.
The second chapter introduces the background knowledge and history of TCP/IP, Including the Development Process of TCP/IP,
In principle, you only need to understand this chapter.
Chapter 3 compares TCP/IP and ISO models from an academic point of view. This chapter is more iso-oriented.
The book explains the objective of learning the ISO model theory,
It is to understand why the communication layer should be divided into these layers.