Wireless Video Surveillance System Based on arm11 + linux

Http://www.eccn.com/design_2012081614530361.htm

1 Introduction

With the popularization of wireless networks, the ARM processor's computing capability is constantly enhanced and the computer image processing technology is constantly improved. Arm-based video monitoring is becoming more and more widely used in schools, communities, hotels, internet cafes, medical care, and other fields. Traditional video surveillance systems have complex wiring, large equipment, low intelligence, and insufficient hardware and software resources. ARM embedded systems are characterized by miniaturization, small footprint, low cost, compact structure, and support for wireless networks, it makes arm11 + LINUX System with the use of to form a variety of wireless network digital monitoring systems with wide application value.

2. Overall System Design

2.1 Overall Hardware Design

The system uses the kernel of Samsung arm11 as the microprocessor. This processor is small in size and is only equivalent to the size of a 48mm * 67mm block. It also integrates a wide range of interfaces, 32-Bit Data Bus and 32-bit external address bus, srom controller, SRAM controller, NAND Flash Controller, 64 interrupt source interrupt controller, five 32-bit timers, four UART, four DMA controllers, STN and tft LCD controller, watchdog, IIS audio interface, IIC-bus interface, Two USB Host ports, one USB device port, two serial peripheral interface circuits, three SD card interfaces, camera_if
Interfaces, TV _out interfaces, MFC interfaces, two SPI and touch screen interfaces, the clock speed can reach 800 MHz, and the maximum extended bus frequency is 133mhz. in this basic, also related expansion, lead to a four-line RS-232 serial port, the serial port is used for the development of the host and the development platform of the communication; configured 1 GB nandflash, it is used to store embedded Linux operating systems, applications and data, and 128 MB of DDR memory. It is used to store running programs and data captured by cameras. It extends a Wi-Fi module, it is used to transmit video data between the development platform and the server, and supports remote video monitoring through wireless networks.

2.2 overall Software Design

The software architecture includes boot loader, operating system kernel, device driver, and Application Layer Program. The software architecture 1 is shown in.

 

Figure 1 Overall Software Architecture

After the system is powered on, run the boot loader, which initializes hardware devices, creates a memory space ing table, and directs and loads the operating system kernel, then start the embedded operating system Linux, and load some necessary drivers such as the NAND Flash Driver, LCD driver, and WiFi driver.

3 video data collection and coding design

3.1 Design of video data collection based on v4l2

In Linux, operations on video devices are implemented through video4linux2, v4l2 for short. Applications use the interface functions provided by v4l2 to perform operations on video devices. The entire video data collection process is shown in step 2.

(1) Enable the video device, int open (const char * pathname, int flags ). Call this function. If the returned value is-1, it indicates opening failed. Otherwise, it indicates the file descriptor of the opened device.

(2) Obtain device information. The IOCTL (cam_fp, vidioc_querycap, & Cap) function is used to obtain the property parameters of the device file and store them in the cap structure. cam_fp indicates the file descriptor of the open video device.

(3) Select the video input mode. The IOCTL (cam_fp, vidioc _ s_input, & Chan) function is used to set the input mode of the Video device. The Chan data structure type is v4l2_input, which is used to specify the input mode of the video.

(4) set the video frame format. The IOCTL (cam_fp, vidioc_s _ FMT, & FMT) function is used to set the video frame format. The FMT data structure type is v4l2_format, which is used to specify the video width, height, and pixel size.

(5) read video data. Use the read (cam_fp, g_yuv, YUV _ SIZE) function to store the data of a camera frame into g_yuv. The YUV _ SIZE indicates the data size of each frame.

(6) disable the video device. You can use the close (cam_fp) function to disable video devices.

 

Figure 2 video data collection process diagram.

3.2 video data h264 Encoding

To improve the video data encoding speed, the system uses h264 hard encoding mode. Hard encoding has the advantages of no CPU resources occupation and fast computing speed, so as to meet the real-time video data requirements.

The specific encoding process is shown in step 3.

(1) Create an h264 encoding structure. Call the ssbsiph264encodeinit (width, height, frame_rate, bitrate, gop_num) function. width indicates the image width, height indicates the Image Height, frame_rate indicates the frame rate, and bitrate indicates the bit rate or bit rate, gop_num indicates the maximum number of frames (B or P frames) between two adjacent key frames ).

(2) initialize the h264 encoding structure and call the ssbsiph264encode EXE (handle) function.

(3) Get the video input address, which is implemented by the ssbsiph264encodegetinbuf (handle, 0) function. This function returns the first address of the video input, which is stored in p_inbuf.

(4) input video data and call the memcpy (p_inbuf, yuv_buf, frame_size) function. p_inbuf stores the data to be encoded, yuv_buf stores the original video data, and frame_size indicates the data size.

(5) encode video data, perform h264 encoding on p_inbuf content, and call the ssbsiph264encodeexe (handle) function.

(6) output the encoded data, ssbsiph264encodegetoutbuf (handle, size). This function returns the first address of the encoded image. Size indicates the size of the encoded image.

(7) disable the hardware encoding device and call the ssbsiph264encodedeinit (handle) function.

Figure 3 diagram of h264 Encoding Process.

4. Video data transmission and display

4.1 video data transmission module design

Modern wireless communication network standards mainly include 3G (third generation mobile communication), WI-FI, Bluetooth, ZigBee (BEE) and so on. For details, see table 1.

Table 1 Basic comparison of common wireless communication network standards

Because the WI-FI has high transmission rate, many supported protocols, simple installation and setup, low cost advantages, so the wireless network standard adopted by the system is WI-FI.

4.1.1 Construction Process of WI-FI Wireless Network

(1) load the WI-FI module. Run the insmod command to load the file. here we need to load two files: helper_sd.bin and sd8686.bin. These two files can be downloaded from the official marvel website.

(2) Search for WI-FI networks. Use the ifconfig eth1 up command to open the WI-FI network interface card, and then use the iwlist eth1 scanning command to search for the WiFi network.

(3) set the IP address and subnet mask of eth1.

(4) set Essid. It is implemented through the iwconfig eth1 Essid 402 command. Essid is used to distinguish different networks.

(5) set the password. Run the iwconfig eth1 key s: your_key command. your_key indicates the logon password.

 

4.1.2 RTP-based video data transmission

RTP stands for the real-time transportprotocol, which represents a network transmission protocol. It is a common protocol for audio and video uploads [5]. RTCP and RTP work together to provide traffic control and congestion control services, which can optimize transmission efficiency with effective feedback and minimum overhead, so they are particularly suitable for transmitting real-time data, therefore, this protocol is used to transmit video data.

The system uses the RTP protocol stack provided by the open source code jrtplib. Because jrtplib encapsulates the implementation of rfc3550, it makes video data transmission easier. Because the maximum network payload of the system is 1500 bytes, the maximum RTP packet size is 1400 bytes. If the sent data is greater than 1400 bytes, the packet splitting method is used for sending, the specific transmission process is shown in Figure 4 and figure 5.

 

Figure 4 sending Process Diagram.

 

Figure 5 receiving end flow diagram.

The main sending process is as follows:

(1) Create an RTP session and set the target address. Call the create method to obtain the RTP session instance, and then call the adddestination method to set the destination IP address and destination port number.

(2) obtain the data and call the get_data () function.

(3) send data by using the sendpacket () method.

The main process of the acceptor is as follows:

(1) Create an RTP session. Call the create method to create a session instance, and set the port number while creating the session. The port number must be consistent with the sending port number.

(2) accept RTP data. Call the polldata () method of the rtpsession class to receive data.

(3) Save the RTP datagram. By creating a pointer array, which stores the RTP datagram pointer, you only need to assign the pointer that just received the RTP datagram to the pointer array, which can save the data copy time.

(4) Determine whether the receiving is complete. If not, jump to step B. Otherwise, the receiving end program exits.

4.2 video data decoding and display

Because the received data is h264 encoded, you must first decode the data before it can be displayed. On the server side, using FFMPEG. FFMPEG for video data decoding is an open-source, free, cross-platform video and audio stream solution. It is a free software.

The decoding mainly involves libavcodec library, libswscale library, and libavformat Library Under FFMPEG. The first library is a library that contains all FFMPEG audio and video codecs, and the second library is the format conversion library, the decoded data is in the yuv420 format. to display the data on a computer, the data must be in the RGB format. The function of this library is to convert the yuv420 format to the RGB format, the third library is a library that contains all the normal audio and video formats of the parser and the generator.

4.2.1 initialize the decoding thread

(1) register all file formats and codecs, and call the av_register_all () function to complete registration.

(2) set the avformatcontext struct. This struct is the input and output functions implemented during the FFMPEG format conversion process. It stores the main structure of the relevant data and is set through the av_open_input_file function.

(3) Check the video stream information. By calling the av_find_stream_info (pformatctx) function, pformatctx-> streams fills in the correct video stream information. The pformatctx type is avformatcontext.

(4) obtain the decoder context. The pcodecctx = pformatctx-> streams [videostream]-> codec, and pcodecctx pointers point to all the information about the decoder used in the stream.

(5) Open the decoder. First find the corresponding decoder through the avcodec_find_decoder function, and then call the avcodec_open function to open the decoder.

(6) applying for memory to store decoded data is implemented by calling the avcodec_alloc_frame function. Because the decoded data is in the yuv420 format, you also need to convert the data to the RGB format. Therefore, call avcodec_alloc_frame again to apply for memory to store RGB data.

(7) The requested memory is used to store raw data. Because during h264 decoding, for P frames, refer to the previous key frame or P frame, and for B frames, refer to the front and back frames, therefore, you need to store the original data. First, use avpicture_get_size to obtain the required size, and then call the av_malloc function to apply for memory space.

(8) combine frames with the newly applied memory by calling the avpicture_fill function.

(9) create a format conversion context by using img_convert_ctx = SWS _ getcontext (src_w, src_h, src_pix_fmt, dst_w, dst_h, pix_fmt_rgb24, sws_bicubic, null. Src_w indicates the width of the source image, src_h indicates the height of the source image, src_pix_fmt indicates the format of the source image, dst_w indicates the width of the target image, and dst_h indicates the height of the target image, pix_fmt_rgb24 indicates the format of the target image.

 

4.2.2 perform h264 decoding on Data

(1) Obtain a frame of data to be decoded. Because the receiving end thread has stored the received data in a pointer array, the decoding thread only needs to obtain data from the pointer data.

(2) decode the data. Call the decoding function avcodec _ decode_video (pcodecctx, pframe, & Finished, encodeddata, size) to decode the video file. The pcodecctx parameter is the pointer to the encoding context of the video stream. The pframe parameter stores the location of the decoded image. The finished parameter is used to record the number of completed frames. The encodeddata parameter is the input buffer pointer, point to the raw data to be decoded. The parameter size is the size of the input buffer.

(3) convert the decoded video data yuv420 format to RGB format, and convert the format by calling the sws_scale () function.

4.2.3 display of video data

The system uses qimage in QT to display video data. Since qimage can access a single pixel, this system saves the image when displaying the previous image, when the last frame of the image is displayed, if the pixel value is the same as that of the previous frame, you do not need to modify the value, which saves a lot of time, that is, where to change the modification, the procedure is as follows:

(1) Obtain decoded video data in RGB format.

(2) cyclically obtain the R, G, and B Components of video data.

(3) Determine whether the pixel value of the point is the same as the pixel value at the corresponding position of the previous frame. If the value is the same, jump to Step 1. Otherwise, save the pixel value.

(4) construct the color value of the pixel Based on the obtained RGB component by calling the qrgb (R, G, B) constructor.

(5) set the pixel value of the corresponding vertex, first generate the qimage class object, and then call setpixel (X, Y, RGB) of the class ). X is the X coordinate value of the image, Y is the Y coordinate value of the image, and RGB is the color value of the point.

(6) display the image. by calling the update () method, this method will trigger the painting event. Therefore, in the painting event, write the display image code to display the generated qimage object, call the drawimage () method to draw an image.

5 conclusion

The system uses the yuv420 sampling format to reduce the data volume when collecting video images. Video data encoding adopts h264 hard encoding, which greatly improves the encoding speed. During wireless network transmission, considering packet loss, encoding data is split and then sent, reducing the packet loss rate. After testing, the system collects an image taken by an ov9650 camera with a resolution of 320x240. The encoded image data is approximately 5 kb after h264, this reduces the amount of data transmitted, and supports hard encoding of 25 frames of image data per second, meeting the requirements of real-time video data encoding. For WI-FI wireless network transmission rate is generally around 11-54mbps, so the wireless network can meet the needs of real-time video transmission. This system has built a high real-time, low-cost, low-power digital wireless video monitoring platform. Based on this platform, it can build a variety of applications, such as real-time traffic monitoring, face recognition, warehouse alarm and other applications, the system has certain practical value.