Development of USB camera driver in Linux

In the first half of this year, I transplanted a Linux driver to a common vimicro camera on the market to an Embedded Development Board. The development environment is:

OS: linux2.6.9

Compile: Cross 3.3.2

CPU: Intel PXA270

Camera type: vimicro zc301p + hv7131r

The driver source code used is based on the free software spca5xx. Some minor changes were made based on the source code. Record it.

USB driver architecture in Linux
The complete USB driver in Linux must include three modules: USB core, USB master controller driver, and USB device driver. The relationship between the three is as follows:

 

Where:

Linux core encapsulates APIs that support USB master controllers and USB device drivers. It abstracts hardware and masks hardware details through a series of data structures, macros, and functions, in this way, the Linux system uses an interface function like a common file handler to access USB devices, thus implementing hardware independence. "The device is a file ".

The usb hc (host controller) is fully responsible for the division of information transmission between the host and the USB, and provides a bridge between the device driver and USB core data transmission. The Real-Time Streaming Data Transmission on USB is usually implemented through the data structure of urb (USB request block). During each data transmission, the device driver establishes urb and submits it to usb hc, usb hc is responsible for completing the urb interaction with the device.

USB driver structure
Struct usb_driver spca5xx_driver

Struct video_device spca50x_template

Struct file_operations spca5xx_fops

Module initialization and uninstallation
Initialize the spca5xx_init () module; uninstall the spca5xx_disconnect () module; search, match, and configure the spca5xx_probe () physical device, and register the video device with the kernel.

System Call Interface
Spca5xx_open () is used to open and initialize the device and initialize the decoder module. spca5xx_close () is used to close the device;

Spca5xx_read () completes data reading. The main task is to transmit data from the kernel space to the process user space;

Spca5xx_mmap () is used to map the device memory to the address space of the user process;

Spca5xx_ioctl () is used to obtain file information.

Data Transmission
Spca50x_reg_write

Camera data transmission process
Information Transmission types on the USB bus can be divided into four types: control, interrupt, timing, and block. The control type is mainly used for device configuration and control; the interrupt type is mainly used for the host's periodic query of USB devices; real-time use and real-time audio and video signal transmission; block type is used for applications that require reliable transmission without strong time requirements. For cameras, real-time transmission is usually used. The USB device provides several ports for data transmission. A logical communication channel is established between the host and the port. During camera device initialization, the host communicates with port 0, and the driver detects the camera model and all port information, establish a communication pipeline between the host and the detected real-time input port, and return the image data captured by the camera in real time.

The USB Host Controller divides the bus time per second into 1024 fixed-size frames. Each frame occupies 1 ms of time bandwidth and increments accordingly, corresponding to an interactive queue at the same time. The host is responsible for allocating the image data traffic to each frame when the USB Bus is passed back. The USB Host Controller scans all frames every second, read the data packets carried by each frame in sequence to the kernel buffer. Through memory ing, the application can directly read the content of the kernel buffer in the user space and store the content in the Linux video buffer (framebuffer), so that image data is displayed on the display screen.

My experimental camera is vimicro zc301p and sensor is hv7131r. The driver transmits urb twice every second, and each urb carries 16 1023-byte packets, in this way, the size of the image data obtained per second is 32 kbyte. For the image format of rgb565, 320x240, and 16-bit color depth, the test is performed on the embedded development board of PXA270 processor. The frame speed reaches 17 frames per second, fully meets the needs of Embedded Network real-time video conferencing.

 

Several Ways to Improve the camera Quality
Dual urb
Dual-Frame Buffer
V4l protocol Improvement

 

 

Note: I recently re-analyzed the spca5xx source code. I got a rare article on the Internet. I have analyzed spca5xx in detail. I have some experiences after reading it. I will post it here to express my gratitude.

The implementation of spac5xx is written according to the driver framework of the standard USB video device (the specific driver framework can be referred to/usr/src/Linux/Drivers/USB video. c file), the entire source program consists of four main parts: device module initialization module and uninstall module, upper-layer software interface module, data transmission module. The specific module analysis is as follows:

Initialize the device module:

The driver uses an explicit module initialization and elimination function, that is, it calls module_init to initialize a module, and call the moduel-exit function when uninstalling (these two functions are supported in the 2.3.13 kernel ). The specific implementation is as follows:

1. module initialization:

Module_init (usb_spca5xx_init );

Static int _ init

Usb_spca5xx_init (void)

{

# Ifdef config_proc_fs

Proc_spca50x_create (); // create a proc Device File

# Endif/* config_proc_fs */

If (usb_register (& spca5xx_driver) <0) // register the USB driver

Return-1;

Info ("spca5xx driver % s registered", version );

Return 0;

}

2. Uninstall the module:

Module_exit (usb_spca5xx_exit );

Tatic void _ exit

Usb_spca5xx_exit (void)

{

Usb_deregister (& spca5xx_driver); // disconnect the USB driver

Info ("Driver spca5xx deregistered ");

# Ifdef config_proc_fs

Proc_spca50x_destroy (); // undo the proc Device File

# Endif/* config_proc_fs */

}

Key Data Structure: // USB drive structure, plug-and-play function implementation

Static struct usb_driver spca5xx_driver = {

"Spca5xx ",

Spca5xx_probe, // registers the device's self-Detection Function

Spca5xx_disconnect, // register the device to disconnect itself

{Null, null}

};

Use two functions to call spca5xx_probe and spca5xx_disconnect to support the plug-and-play functions of USB devices:

The specific implementation of spca5xx_probe is as follows:

Static void *

Spca5xx_probe (struct usb_device * Dev, unsigned int ifnum, const struct usb_device_id * ID)

{

Struct usb_interface_descriptor * interface; // USB device interface Descriptor

Struct usb_spca50x * spca50x; // the data structure of the physical device.

Int err_probe;

Int I;

If (Dev-> descriptor. bnumconfigurations! = 1) // check whether the device is configurable.

Goto nodevice;

If (ifnum> 0)

Goto nodevice;

Interface = & Dev-> actconfig-> interface [ifnum]. altsetting [0];

Mod_inc_use_count;

Interface = & INTF-> altsetting [0]. DESC;

If (interface-> binterfacenumber> 0)

Goto nodevice;

If (spca50x = kmalloc (sizeof (struct usb_spca50x), gfp_kernel) = NULL)

// Allocate physical address space

{

Err ("couldn't kmalloc spca50x struct ");

Goto error;

}

Memset (spca50x, 0, sizeof (struct usb_spca50x ));

Spca50x-> Dev = dev;

Spca50x-> iface = interface-> binterfacenumber;

If (err_probe = spcadetectcamera (spca50x) <0)

// Find the specific physical device, match the vendor ID and device ID (in the subroutine)

{

Err ("devices not found !! ");

Goto error;

}

Pdebug (0, "camera type % s", plist [spca50x-> cameratype]. Name

For (I = 0; I <spca50x_numframes; I ++)

Init_waitqueue_head (& spca50x-> frame [I]. WQ); // initialize the frame wait queue

Init_waitqueue_head (& spca50x-> WQ); // initialize the driver wait queue

If (! Spca50x_configure (spca50x ))

// Physical device configuration (mainly for sensor detection and graphic parameter configuration). The main idea is to write values to the control register and read back the returned values to determine the specific sensor model.

{

Spca50x-> User = 0;

Init_mutex (& spca50x-> lock); // semaphore Initialization

Init_mutex (& spca50x-> buf_lock );

Spca50x-> v4l_lock = spin_lock_unlocked;

Spca50x-> buf_state = buf_not_allocated;

}

Else

{

Err ("failed to configure camera ");

Goto error;

}

/* Init video stuff */

Spca50x-> vdev = video_device_alloc (); // The device controls block memory allocation.

If (! Spca50x-> vdev)

Goto error;

Memcpy (spca50x-> vdev, & spca50x_template, sizeof (spca50x_template ));

// Mount the system call. The system call implemented by the driver is mounted to the kernel.

Video_set_drvdata (spca50x-> vdev, spca50x );

If (video_register_device (spca50x-> vdev, vfl_type_grabber, video_nr) <0)

{// Video device registration

Err ("video_register_device failed ");

Goto error;

}

Spca50x-> present = 1;

If (spca50x-> force_rgb)

Info ("data format set to RGB ");

Spca50x-> task. Sync = 0;

Spca50x-> task. Routine = auto_bh;

Spca50x-> task. Data = spca50x;

Spca50x-> bh_requested = 0;

Mod_dec_use_count; // increase the number of modules used.

Return spca50x; // return the Data Structure

Error: // handle the error

If (spca50x-> vdev)

{

If (spca50x-> vdev-> minor =-1)

Video_device_release (spca50x-> vdev );

Else

Video_unregister_device (spca50x-> vdev );

Spca50x-> vdev = NULL;

}

If (spca50x)

{

Kfree (spca50x );

Spca50x = NULL;

}

Mod_dec_use_count;

Return NULL;

Nodevice:

Return NULL;

}

The specific implementation of spca5xx_disconnect is as follows:

Static void

Spca5xx_disconnect (struct usb_device * Dev, void * PTR)

{

Struct usb_spca50x * spca50x = (struct usb_spca50x *) PTR;

Int N;

Mod_inc_use_count; // increase the number of modules used.

If (! Spca50x)

Return;

Down (& spca50x-> lock); // reduce semaphores

Spca50x-> present = 0; // The driver is uninstalled to 0.

For (n = 0; n <spca50x_numframes; n ++) // indicates the aborting status of all frames.

Spca50x-> frame [N]. grabstate = frame_aborting;

Spca50x-> curframe =-1;

For (n = 0; n <spca50x_numframes; n ++) // wake up all waiting Processes

If (waitqueue_active (& spca50x-> frame [N]. WQ ))

Wake_up_interruptible (& spca50x-> frame [N]. WQ );

If (waitqueue_active (& spca50x-> WQ ))

Wake_up_interruptible (& spca50x-> WQ );

Spca5xx_kill_isoc (spca50x); // The subfunction terminates the transmission of the urb package.

Pdebug (3, "Disconnect kill ISOC done ");

Up (& spca50x-> lock); // increase the semaphore

While (spca50x-> User) // if there are other processes in use, process switching

Schedule ();

Down (& spca50x-> lock );

If (spca50x-> vdev)

Video_unregister_device (spca50x-> vdev); // cancel the video device

Usb_driver_release_interface (& spca5xx_driver, // port release

& Spca50x-> Dev-> actconfig->

Interface [spca50x-> iface]);

Spca50x-> Dev = NULL;

Up (& spca50x-> lock );

# Ifdef config_proc_fs

Destroy_proc_spca50x_cam (spca50x); // deregister the proc file

# Endif/* config_proc_fs */

If (spca50x &&! Spca50x-> User) // release memory space

{

Spca5xx_dealloc (spca50x );

Kfree (spca50x );

Spca50x = NULL;

}

Mod_dec_use_count; // reduce the module count

Pdebug (3, "Disconnect complete ");

}

Upper Layer software interface module:

This module uses the file_operations data structure and implements key system calls for devices based on the v4l protocol specifications, implementing the documented UNIX System Design Features of devices. As the camera head driver, the function is data collection, but not output to the camera head. Therefore, the write system is not called in the source code. Its key data structure is as follows:

Static struct video_device spca50x_template = {

. Owner = this_module,

. Name = "spca5xx USB camera ",

. Type = vid_type_capture,

. Hardware = vid_hardware_spca5xx,

. Fops = & spca5xx_fops,

};

Static struct file_operations spca5xx_fops = {

. Owner = this_module,

. Open = spca5xx_open, // OPEN function

. Release = spca5xx_close, // close Function

. Read = spca5xx_read, // READ function

. MMAP = spca5xx_mmap, // memory ing Function

. IOCTL = spca5xx_ioctl, // obtain the File Information

. Llseek = no_llseek, // file location function not implemented

};

OPEN function:

Enable and initialize the device and initialize the decoder module. The specific implementation is as follows:

Static int

Spca5xx_open (struct video_device * vdev, int flags)

{

Struct usb_spca50x * spca50x = video_get_drvdata (vdev );

Int err;

Mod_inc_use_count; // increase the module count.

Down (& spca50x-> lock );

Err =-enodev;

If (! Spca50x-> present) // check whether the device exists, whether the driver exists, or not.

Goto out;

Err =-ebusy;

If (spca50x-> User)

Goto out;

Err =-enomem;

If (spca50x_alloc (spca50x ))

Goto out;

Err = spca50x_init_source (spca50x); // initialize the sensor and decoding module. In the implementation of this function, the initialization of each DSP chip is different, the DSP chip of star micro 301p is initialized in the sub-function zc3xx_init. Its implementation method is to fill in the register value.

If (Err! = 0 ){

Pdebug (0, "dealloc error on spca50x_init_source/N ");

Up (& spca50x-> lock );

Spca5xx_dealloc (spca50x );

Goto out2;

}

Spca5xx_initdecoder (spca50x); // the initialization of the decoding module. The specific implementation of the module adopts the Huffman algorithm.

Spca5xx_setframedecoder (spca50x );

Spca50x-> User ++;

Err = spca50x_init_isoc (spca50x); // initialize the urb (USB request block) package, start the camera header, and transmit data through synchronous transmission

If (ERR)

{

Pdebug (0, "dealloc error on init_isoc/N ");

Spca50x-> User --;

Spca5xx_kill_isoc (spca50x );

Up (& spca50x-> lock );

Spca5xx_dealloc (spca50x );

Goto out2;

}

Spca50x-> brightness = spca50x_get_brghtness (spca50x) <8;

Spca50x-> whiteness = 0;

Out:

Up (& spca50x-> lock );

Out2:

If (ERR)

Mod_dec_use_count;

If (ERR)

{

Pdebug (2, "Open failed ");

}

Else

{

Pdebug (2, "Open done ");

}

Return err;

}

2. Close function:

The specific process of closing a device is as follows:

Static void

Spca5xx_close (struct video_device * vdev)

{

Struct usb_spca50x * spca50x = vdev-> priv;

Int I;

Pdebug (2, "spca50x_close ");

Down (& spca50x-> lock); // parameter settings

Spca50x-> User --;

Spca50x-> curframe =-1;

If (spca50x-> present) // present: yes or has a driver to load.

{

Spca50x_stop_isoc (spca50x); // stop camera head and send data packets

Spcacamerashutdown (spca50x); // close the camera header, which is completed by the subfunction spca50x_stop_isoc.

For (I = 0; I <spca50x_numframes; I ++) // wake up all the waiting Processes

{

If (waitqueue_active (& spca50x-> frame [I]. WQ ))

Wake_up_interruptible (& spca50x-> frame [I]. WQ );

}

If (waitqueue_active (& spca50x-> WQ ))

Wake_up_interruptible (& spca50x-> WQ );

}

Up (& spca50x-> lock );

Spca5xx_dealloc (spca50x); // reclaim memory space

Pdebug (2, "Release ressources done ");

Mod_dec_use_count;

}

READ function:

To complete data reading, the main task is to transmit data from the kernel space to the process user space.

Static long

Spca5xx_read (struct video_device * Dev, char * Buf, unsigned long

Count, int Noblock)

{

Struct usb_spca50x * spca50x = video_get_drvdata (Dev );

Int I;

Int frmx =-1;

Int RC;

Volatile struct spca50x_frame * frame;

If (down_interruptible (& spca50x-> lock) // obtain the semaphore

Return-eintr;

If (! Dev |! Buf) {// determine the device status

Up (& spca50x-> lock );

Return-efault;

}

If (! Spca50x-> Dev ){

Up (& spca50x-> lock );

Return-EIO;

}

If (! Spca50x-> streaming ){

Up (& spca50x-> lock );

Return-EIO;

}

If (rc = wait_event_interruptible (spca50x-> WQ, // sleep on the specified queue until the condition 2 is true

Spca50x-> frame [0]. grabstate = frame_done |

Spca50x-> frame [1]. grabstate = frame_done |

Spca50x-> frame [2]. grabstate = frame_done |

Spca50x-> frame [3]. grabstate = frame_done ))){

Up (& spca50x-> lock );

Return RC;

}

For (I = 0; I <spca50x_numframes; I ++) // when data arrives

If (spca50x-> frame [I]. grabstate = frame_done) // identify that the data has been

Frmx = I;

If (frmx <0)

{

Pdebug (2, "couldnt find a frame ready to be read .");

Up (& spca50x-> lock );

Return-efault;

}

Frame = & spca50x-> frame [frmx];

Pdebug (2, "Count asked: % d available: % d", (INT) count,

(INT) frame-> scanlength );

If (count> frame-> scanlength)

Count = frame-> scanlength;

If (I = copy_to_user (BUF, frame-> data, count) // copies data in the user space and kernel space

{

Pdebug (2, "Copy failed! % D bytes not copied ", I );

Up (& spca50x-> lock );

Return-efault;

}

/* Release the frame */

Frame-> grabstate = frame_ready; // The ID data is empty.

Up (& spca50x-> lock );

Return count; // return the number of copied data

}

MMAP function:

To map the device memory to the address space of the user process, the key function is remap_page_range. The specific implementation is as follows:

Static int

Spca5xx_mmap (struct video_device * Dev, const char * ADR, unsigned Long SIZE)

{

Unsigned long start = (unsigned long) ADR;

Struct usb_spca50x * spca50x = Dev-> priv;

Unsigned long page, Pos;

 

If (spca50x-> Dev = NULL)

Return-EIO;

Pdebug (4, "MMAP: % LD (% lx) bytes", size, size );

If (size>

(Spca50x_numframes * max_data_size) + page_size-1 )&~ (Page_size-1 )))

Return-einval;

If (down_interruptible (& spca50x-> lock) // obtain the semaphore

Return-eintr;

Pos = (unsigned long) spca50x-> fbuf;

While (size> 0) // implement memory ing in a loop

{

Page = k1__to_pa (POS );

If (remap_page_range (START, page, page_size, page_shared) {// implement memory ing

Up (& spca50x-> lock );

Return-eagain ;}

Start + = page_size;

Pos + = page_size;

If (size> page_size)

Size-= page_size;

Else

Size = 0;

}

Up (& spca50x-> lock); // release the semaphore

Return 0;

}

Ioctl function:

To obtain file information,

Static int

Spca5xx_ioctl (struct inode * inode, struct file * file, unsigned int cmd, unsigned long Arg)

{

Struct video_device * vdev = file-> private_data;

Struct usb_spca50x * spca50x = vdev-> priv;

Int RC;

If (down_interruptible (& spca50x-> lock) // obtain the semaphore

Return-eintr;

Rc = video_usercopy (inode, file, CMD, ARG, spca5xx_do_ioctl); // transmits information to the user process, and the key function implements spca5xx_do_ioctl.

Up (& spca50x-> lock );

Return RC;

}

The implementation of the spca5xx_do_ioctl function depends on different hardware. In order to support multiple chips, the main idea of the Implementation Program is to use copy_to_user (ARG, B, sizeof (struct video_capability) the function transmits device information to the user process.

 

Data transmission module:

The source program uses tasklet to synchronize and quickly transmit data, and uses the software decoding module on spcadecode. C to decode the graphic information. The entry point of this module is attached to the spca_open function. The specific function is spca50x_init_isoc. When the device is turned on, the synchronous transmission of data has also started, and the data is passed to the driver through the spca50x_move_data function, the driver achieves data access through polling.

Void

Outpict_do_tasklet (unsigned long PTR)

{

Int err;

Struct spca50x_frame * taskletframe = (struct spca50x_frame *) PTR;

Taskletframe-> scanlength = taskletframe-> highwater-taskletframe-> data;

Pdebug (2, "tasklet ask spcadecoder hdrwidth % d hdrheight % d method % d ",

Taskletframe-> hdrwidth, taskletframe-> hdrheight,

Taskletframe-> method );

Err = spca50x_outpicture (taskletframe); // output processed image data

If (Err! = 0)

{

Pdebug (0, "frame decoder failed (% d)", err );

Taskletframe-> grabstate = frame_error;

}

Else

{

Taskletframe-> grabstate = frame_done;

}

If (waitqueue_active (& taskletframe-> WQ) // if a process waits, wake up the waiting process

Wake_up_interruptible (& taskletframe-> WQ );

}

It is worth mentioning that spcadecode. the decoding module on C decodes the original compressed graphics data streams yyuyv, yuvy, ipv411, and ipv422 as RGB Images. However, the implementation of this decompression algorithm also depends on the compression format, ultimately, it depends on the hardware Compression Algorithm in DSP (Digital Processing Chip.

 

Iv. USB core support:

In Linux, USB core relies on the System-implemented USB core layer for underlying hardware operations. USB core provides many function interfaces for upper-layer drivers, such as usb_control_msg and usb_sndctrlpipe, the most typical use is the Read and Write Functions spca50x_reg_write and spca50x_reg_read in the source code. The specific implementation is as follows: (the implementation of spca50x_reg_write is similar to other functions)

Static int spca50x_reg_write (struct usb_device * Dev, __2010reg, __2010index,

_ 2010value)

{

Int RC;

Rc = usb_control_msg (Dev, // set the register value through the interface function provided by USB core

Usb_sndctrlpipe (Dev, 0 ),

Reg,

Usb_type_vendor | usb_recip_device,

Value, index, null, 0, timeout );

Pdebug (5, "Reg write: 0x % 02x, 0x % 02x: 0x % 02x, 0x % x", Reg, index, value, RC );

If (RC <0)

Err ("Reg write: Error % d", RC );

Return RC;

}