Design of MPEG-4 dedicated encoder driver in Embedded Linux System

Abstract: The design of embedded video surveillance scheme has been discussed many times, but the Driving Design of Video MPEG-4 coding in Linux is rarely mentioned. This article takes the driver design of the Video Encoding Chip ime6410 as an example. It mainly discusses the implementation methods of the video Driver Based on the video for Linux (v4l) specification, including hardware interfaces, device initialization, and file operations, the internal connection between the v4l specification, video driver, and kernel is analyzed.
Keywords: Embedded Linux, video4linux, driver, ime6410

Abstract:There are pending researches on the design of embedded video surveillance system already. however, the method of design about Linux driver for MPEG-4 codec has rarely been reported. in this paper, Linux driver for video codec is discussed and an example of Driver Design for ime6410 is presented. the Implement of Driver Based on v4l interface was introduced, of which the hardware interface, device initialization, file operations etc are involved as well. the fundamental relations between v4l interface, video drivers and Linux kernel are analyzed deeply.

Key Words:Embedded Linux, video4linux, driver, ime6410

1Introduction

With the rapid development of embedded technology and streaming media technology, the video monitoring system is switching from a PC-based monitoring solution to an embedded video monitoring solution. Because the embedded digital monitoring system has the advantages of extensive control areas, complex monitoring networks, stable and reliable performance, it has more development prospects [1]. Currently, embedded video monitoring based on MPEG-2 compression standards has been widely used, even so, users and the environment of Video Monitoring increasingly demanding, it is a new goal to find a better compression method and transmission method. Based on this goal, this paper uses a dedicated MPEG-4 Encoding Chip ime6410 as an example to improve the video monitoring solution from the perspective of video compression. This section mainly describes the design and implementation of the video encoding driver in embedded Linux.

This article first introduces the hardware structure of the system, and then deeply analyzes the v4l specification. Based on this specification, ime6410 is used as an example to describe the design and development process of the driver, including chip initialization and device registration, finally, the conclusion is given.

 

2Video System Hardware Design

In this system, the MPEG-4 Encoding Chip uses ime6410. It is a single-channel audio and video compression device, and its peripherals are integrated with a variety of peripheral controllers and interfaces, such as the SDRAM Controller, I2C interface, I2S interface, host controller, and so on. The ime6410 contains Ram and can store firmware programs run by chips. According to the characteristics of the chip, design its peripheral circuit 1.

 

Figure1Hardware diagram of Video Server Encoding

Ime6410 Encoding Chip in the audio and video compression process to produce a large amount of intermediate data must be placed in a large capacity of 32-bit external SDRAM, ime6410 video interface in line with the CCIR-601 16-bit standard, external support yuv422, yuv420 sampling is used internally. External view

The decoder uses SAA7111 and supports NTSC/PAL. The on-chip registers of SAA7111 can be set through its external I2C interface. In the video server system, some arm processors have their own I2C bus, while some do not. The ime6410 compression chip has an I2C bus controller and provides access as the master, therefore, you can use ime6410 to indirectly control SAA7111. The output data volume of single-channel video encoding does not exceed 14.745 Mbps. The ARM 9 system with 16-bit bus does not carry much load.

The approximate data streams of Video Acquisition and compression are as follows: analog signals collected from cameras are transmitted to ime6410 through A/D, and ime6410 encodes and compresses video data in SDRAM in real time, and synthesize the Audio Encoding (Optional). The encoded data is sent to the 1 k fifo in the chip. When the FIFO is half full or full, the external signal nfull is generated, the arm controller can obtain the FIFO status by means of interruption or query, read the FIFO data as needed, and put it in its own ram memory for post-processing.

 

3Driver Design

The driver is between the operating system and hardware and closely related to the underlying hardware. Generally, the video for Linux (v4l) driver specification [2] is used in the video collection and Compression Systems in Linux. This module is an interface between the underlying driver and the application. It provides a set of APIs and related standards for video devices in Linux. The video encoding and related parameters can be collected or set according to the interface access control and read/write devices specified by v4l.

 

1. In-depth analysis of v4l specifications

V4l classifies video devices into four categories: [3], specifying that all video device files have the same master device number as 81. However, for different hardware types, the sub-device numbers of the special files are inconsistent, and each type of device has an independent sub-device number range. In this way, you can determine from the device number in the device file that the device belongs. V4l is associated with a specific device driver by defining some data structures and related functions. The key data structure is the video_device structure, which is defined as follows:

Struct video_device

{

Char name [32]; // device name

Int type; // device type

Int minor; // device number

Devfs_handle_t devfs_handle; // device handle

/* The following are important member pointers with deletions */

INT (* open) (struct video_device *, int mode );

Void (* close) (struct video_device *);

Long (* read) (struct video_device *, char *, unsigned long, int Noblock );

Long (* write) (struct video_device *, char *, unsigned long, int Noblock );

INT (* IOCTL) (struct video_device *, unsigned int, void *);

INT (* MMAP) (struct video_device *, const char *, unsigned long );

INT (* initialize) (struct video_device *);

};

This structure corresponds to a device controlled by the driver. The minor member represents the device ID, other open, close, read, write, IOCTL and other members all represent corresponding function pointers, which indicate operations related to a specific device. v4l registers the operation structure of another device using a unified interface, in this unified device operation structure, the driver interface function locates the struct video_device corresponding to the device file based on the device Number of the specified device file to control the specific device. To achieve this positioning, v4l defines a static array static struct video_device * video_device [256]. Each specific device occupies one of the arrays. When designing the driver, you need to use the video_register_device device registration function provided by v4l to fill in the data structure of each hardware device to be controlled in the static array. When calling this function, the specific data structure and device type of the device should be provided.

The video_register_device function is implemented in videodev. C. In fact, the main implementation of the v4l specification is in this file, and the file is located in the/driver/Media/Video/directory. Videodev. citself is also a driver module, which is usually statically loaded during kernel compilation through the menuconfig option. The initialization function module_init () of the v4l module is used to complete the registration of the gossip-type device. The process is as follows: first, register the drivers of the 81 primary device, identify the driver of each specific device by device number. After successful registration, if the proc file system exists, create a dedicated directory/proc/Video/dev/Under the/proc file directory. After the actual device file is registered, A device file [4] will be generated in this directory.

When registering an actual device and calling the registration function video_register_device, the three parameters required for registration are device struct, device type, and device number. The type determines the start address segment of the next device number. When the first device number is-1, the system automatically assigns a minimum available device number to the device, if the specified device still has the initialize function, run it here. Then run devfs_register to automatically create a device file under the/dev directory, such as video0. Mount the file operation file_operations pointer, this pointer address also points to the file pointer defined when the v4l module is loaded, and finally creates a device file under/proc/Video/dev/(as mentioned above ). Therefore, the hooks between v4l and specific driver registration are mainly implemented through the devfs_register function.

After analyzing the v4l interface specifications, according to its struct, the most important task is module initialization. The module_init system calls the video_ime_init function to complete hardware initialization and device structure filling.

 

2. Complete ime6410 and SAA7111 Initialization

First, initialize ime6410 and SAA7111 respectively. In the hardware design, set the I2C interface used by the SAA7111 register to be output through ime6410. Therefore, you must first initialize ime6410.

Unlike the General chip, the ime6410 chip needs to download the firmware program before it can work normally. Then, it initializes its on-chip registers and configures them as required parameters. According to the hardware design, first configure the External Bus Interface (EBI) of the ARM processor to meet the timing requirements of ime6410 in terms of timing, latency, and level effectiveness. Secondly, ime6410 is initialized according to Figure 2 and firmware is special for ime6410 download. After the hardware selects the host to download, the in-chip Rom checks the type and width of the external bus and waits for the host to send data to it, the method of receiving data is similar to the finite state machine written by flash: first receives the base address and offset address to be stored by the download program, and then receives a half-word (half word) data of firmware. Then, wait for a transfer command from the host port again, and start to actually store the data to the on-chip RAM. The host can query the transfer status through the Status Register and complete the download process by repeating the preceding steps, after the download is complete, the ime6410 will automatically set the version register and various encoding mode registers, and query the version register to see if the firmware download is successful.

 

Figure2Video peripheral initialization Flowchart

Firmware enables the I2C module in the chip, which is an external I2C controller for the arm controller. It uses macros to abstract I2C read/write operations in Linux. The ime6410 on-chip I2C bus only supports four-byte block write operations, which makes initialization of registers for SAA7111 troublesome, but there are not many registers to be set in SAA7111, you can complete the operation several times.

 

3. Video device registration and interrupt program Registration

The source function called by the video device registration includes a module count increase of 1, so you do not need to call the mod_inc_use_count macro count [4]. The video_device Data Structure referenced during the video registration call should be filled, and its file operation pointer should be specified through the callback function, such:

Static struct video_device ime_template =

{......

Open: video_ime_open,

Read: video_ime _ read,

IOCTL: video_ime _ ioctl,

......

};

The function of the object operation pointer callback function in this struct is defined according to actual needs. Some Function assignments in this system are listed here.

Open: the open operation enables ime6410 to work. First, the ime6410 Status Register is queried. When everything works normally, the ime6410 register starts image encoding.

Read: Read the ime6410 FIFO and read the buffer of a packet (512 bytes) each time. The first four bytes of each packet indicate the package size and type, for the application layer.

Write: ime6410 does not need to write data, so it basically does not work.

MMAP: ing video devices to buffer to the local memory cache.

IOCTL: unlike general device files, operations on video files that comply with v4l are generally set through the ioctl function to obtain various parameters and even image data. V4l lists all related operations [5]. This video encoding only supports some practical operations, such as table 1. Operations on video collection or encoding vary greatly, you need to implement their own control methods. Some parameters set through arm are directly accessed locally.

Table1Video Encoding control operations

Vidiocgpict	Obtains image parameters, including resolution, frame rate, and bit rate control.
Vidiocspict	Set Image Parameters
Vidiocgmbuf	Obtains the local ing cache parameters, mainly including the address and size.
Vidiocmcapture	Get image encoding to ing Buffer
Vidiocsync	Get the frame end mark

 

The registration of the interrupt program is called by the request_irq system. In this program, the interrupt vector number must be selected based on the hardware settings of the ARM chip. The interrupt handle points to the Interrupt Processing Program, which is executed in kernel mode. When the FIFO of ime6410 is half full or full, the ARM processor will interrupt I/O, And the Linux kernel will interrupt the handle accordingly to enter the interrupt processing program. After the program reads a packet of data, notifies upper-layer applications through the message mechanism.

The compiled driver generates an inode node file named v4lvideo0 in the/dev directory. So far, the entire driver is designed and compiled.

 

4Conclusion

This article deeply analyzes the relationship between the v4l specification and the video driver. Taking the ime6410 Encoding Chip as an example, it introduces the driver development method based on the video encoding module, A reasonable IOCTL command is selected for video compression encoding. The system calls of compiled program files follow the v4l specification, which achieves better performance in application testing. This article applies to all video driver development that comply with v4l specifications.

Innovation points of this article:

1. In Embedded Linux, a driver complying with v4l specifications is designed to make the driver universal.

2. provides a general method for driver design that complies with v4l specifications

3. It provides a reference method for the application of dedicated MPEG-4 encoder in embedded system.

 

References:

1 South ice, Li Xinghua, Jing Tao. Monitoring System Design Based on MPEG-4 standards and embedded technology [J]. Microcomputer Information, 113 --

2 Alan Cox. video4linux programming. [m] alan@redhat.com. 2000

3 Li genshen, Yan Hancheng. Driver Programming Technology for PCI interface video capture card on Linux platform [J], computer aided engineering, 2003, Vol.12 No.1: 12-19

4. Translation of Alessandro Rubini. Wei Yongming and others. Linux device drivers [M]. Beijing China Power Press, 2002

5 Yang Jihua, Yan guoping. Video Acquisition Based on Embedded Linux and S3C2410 platforms [J]. Application of Single Chip Microcomputer and embedded system, 2004, No. 11: 69-71