Kernel v4l2 framework video for linux 2 (1), v4l2linux

Kernel v4l2 framework video for linux 2 (1), v4l2linux

After reading a lot of v4l2 driver routines, I want to thoroughly study the v4l2 framework of the Linux kernel. By the way, I will record these for future reference.

Video for Linux 2

With the complexity of some video or image hardware, V4L2 drivers become more and more complex. Many hardware has multiple ics. Multiple video devices or other devices, such as DVB, ALSA, FB, I2C, and IR, are generated under/dev. Therefore, the V4L2 driver must provide audio and video synthesis and CODEC function interfaces for these hardware devices. In addition, these devices usually communicate with the CPU through multiple I2C buses, not only is the I2C bus, but others may also be used, such as SPI, 1-wire, and so on. The devices attached to these bus are called sub-devices, that is, sub-devices of V4L2 devices.

For a relatively long time, V4L2 was limited to using video_device to create V4L2 device nodes and using videobuf to process video cache. This means that in addition to creating a device, all drivers need to connect to the "sub-device" separately. This process is complex and prone to errors. The lack of such a framework makes code reuse inadequate and the driver looks bloated.

Therefore, the V4L2 framework is organized to provide some basic components, simplify driver writing through some shared function functions, and enhance code reusability, hardware drivers only need to implement related operations without having to care about interactive applications. At the same time, applications can use hardware more transparently to drive audio and video processing. In addition, this framework is constantly updated and expanded. The basic part is the v4l2API provided. However, we will not discuss the APIS provided by V4L2 and how they are used here. We will only discuss v4l2 core and Driver-related knowledge.

First, let's take a look at the several components that all v4l2 drivers must have:

-Structv4l2_device ).

-Method used to initialize and control sub-devices (structv4l2_subdev ).

-You must be able to create a device node and track the data held by the node (structvideo_device ).

-Maintain a file handle (structv4l2_fh) for each opened node ).

-Video buffer processing (videobuf or videobuf2 framework ).

The definitions of these structures are defined by the kernel above linux3.0, and the entire v4l2 framework can be snooped into from the definition. These struct types include:

Struct v4l2_device; used to describe a v4l2 device instance

Struct v4l2_subdev, used to describe a v4l2 sub-device instance

Struct video_device; used to create a device node/dev/videoX

Struct v4l2_fh; used to track file handle instances

We will discuss the videobuf and videobuf2 frameworks in the series below.

A rough graph is used to show the relationship between them, which is roughly as follows:

Device instance (v4l2_device)

| ______ Sub-device instance (v4l2_subdev)

| ______ Video device node (video_device)

| ______ File Access Control (v4l2_fh)

| ______ Video buffer processing (videobuf/videobuf2)

Now, let's analyze the definitions of these structures one by one.

1. v4l2_device

This definition is defined in linux/media/v4l2-device.h

Struct v4l2_device {

// Pointer to the device model

Struct device * dev;

# If defined (CONFIG_MEDIA_CONTROLLER)

// Pointer to a media Controller

Struct media_device * mdev;

# Endif

// Manage the two-way linked list of sub-devices. All registered sub-devices must be added to this linked list.

Struct list_head subdevs;

// Global lock

Spinlock_t lock;

// Device name

Char name [V4L2_DEVICE_NAME_SIZE];

// Notification callback function, which is usually used for sub-devices to pass events. These events can be custom events.

Void (* notify) (struct v4l2_subdev * sd, uint notification, void * arg );

// Control handle

Struct v4l2_ctrl_handler * ctrl_handler;

// The priority status of the device. Generally, there are three priorities: backend, interaction, and record.

Struct v4l2_prio_state prio;

// Mutex of ioctl operations

Struct mutex ioctl_lock;

// Reference tracing of this struct

Struct kref ref;

// Device release function

Void (* release) (struct v4l2_device * v4l2_dev );

};

To register an instance, you need to use the Function

V4l2_device_register (struct device * dev, struct v4l2_device * v4l2_dev );

This function initializes the v4l2_device structure. If dev-> driver_data is empty, v4l2_dev is assigned to driver_data.

To integrate the driver with the media device framework, you need to manually set dev-> driver_data to point to a media device structure embedded with the v4l2_device structure. This structure can be the status description of the specified driver. Call dev_set_drvdata before registering a function. You need to set these parameters before calling the register function. Similarly, if a media device exists, you must initialize the function before that, set the mdev field of v4l2_device to point to an initialized media device instance.

If v4l2_dev-> name is empty, the name of v4l2_dev will be set for the registration function based on dev-> name. If it is already set, the registration function will not be asked again. If dev is empty, you must set v4l2_dev-> name before calling the registration function. You can also use v4l2_device_set_name () to set the device instance name.

To remove registration, call the function:

V4l2_device_unregister (structv4l2_device * vd)

If it is a hot swappable device, you also need to call

V4l2_device_disconnect (structv4l2_device * vd)

To disconnect the device. Otherwise, a null pointer is generated.

Sometimes it is necessary to iterate all the devices to drive registration. This usually occurs when multiple drivers use the same hardware. For example, ivtvfb is a framebuffer driver and uses ivtv hardware, it is also a TV driver. The same situation applies to the alsa driver. So how can we complete the iteration? Take a look at the following routine:

Static int callback (struce device * dev, void * p)

{

Struct v4l2_device * vdev = dev_get_drvdata (dev );

If (vdev = NULL) return 0;

/* Do something */

Return 0;

}

Int iterate (void * p)

{

Struct device_driver * drv;

Int err = 0;

/* Find driver 'vivi' on the PCI bus */

Drv = driver_find ("vivi", & pci_bus_type );

Err = driver_for_each_device (drv, NULL, p, callback );

Put_driver (drv );

Return err;

}

Sometimes you also need to maintain a device instance count during running and define an atomic variable. If many device nodes are registered on a v4l2 device, when we remove the registered v4l2_device, the ref member will help us record the number of node registrations for v4l2_device, each call to video_register_device adds 1, and vice versa. Once this value is 0, we can call v4l2_device_unregister. If it is not a video node, manually call these two functions to count:

Void v4l2_device_get (struct v4l2_device * vd) // ref + 1

Int v4l2_device_put (struct v4l2_device * vd) // ref-1

2. v4l2_subdev

Struct v4l2_subdev {

# If defined (CONFIG_MEDIA_CONTROLLER)

// Media controller entity, and v4l2_device

Struct media_entity entity;

# Endif

Struct list_head list;

Struct module * owner;

U32 flags;

// Point to a v4l2 Device

Struct v4l2_device * v4l2_dev;

// Sub-device operation function set

Const struct v4l2_subdev_ops * ops;

// The internal operation function set of the sub-Device

Const struct v4l2_subdev_internal_ops * internal_ops;

// Control the function Processor

Struct v4l2_ctrl_handler * ctrl_handler;

// Sub-device name

Char name [V4L2_SUBDEV_NAME_SIZE];

// Group ID of the sub-Device

U32 grp_id;

// Private Data Pointer of the sub-device, generally pointing to the client of the Bus Interface

Void * dev_priv;

// The Private Data Pointer of the sub-device, usually pointing to the host of the Bus Interface

Void * host_priv;

// Device Node

Struct video_device devnode;

// Sub-device events

Unsigned int nevents;

};

Many v4l2 drivers need to communicate with sub-devices, such as audio and video synthesis, encoding, and decoding. For webcam, sub-devices are sensor and camera controllers. These are generally I2C devices, but they are not required. To provide a consistent interface for these sub-devices, the v4l2_subdev structure came into being.

Each sub-device must have a v4l2_subdev structure. This structure can be used independently or embedded into a larger structure. Generally, there is a lower-level device structure (such as i2c_client), which contains some initialization data of the device. Therefore, we recommend that v4l2_subdev-> dev_priv point to the data. You can use the function:

V4l2_set_subdevdata ()

V4l2_get_subdevdata ()

And then call v4l2_get_subdevdata () to conveniently find the actual bus-related device data from v4l2_subdev. In short, some private data can be platform-related data, or a device instance defined by itself that contains the v4l2_subdev structure.

You also need to find v4l2_subdev from a bus-related device. For example, i2c_set_clientdata (). Call this function to assign the v4l2_subdev structure pointer to the private data of i2c_client. Call i2c_get_clientdata () to obtain the pointer of v4l2_subdev. Of course, you can also operate through container_of. But since the kernel provides such an api, how can it be used?

Each v4l2_subdev contains function pointers that can be implemented by sub-devices. These functions can do many different things. They are placed in different structures according to different operation classes. The highest-level operational function set covers various operational categories. For example:

Struct v4l2_subdev_core_ops {

Int (* g_chip_ident) (struct v4l2_subdev *, struct v4l2_dbg_chip_ident *);

Int (* log_status) (struct v4l2_subdev * sd );

Int (* init) (struct v4l2_subdev * sd, u32 val );

....

};

Struct v4l2_subdev_tuner_ops {};

Struct v4l2_subdev_audio_ops {};

Struct v4l2_subdev_video_ops {};

Struct v4l2_subdev_ops {

Const struct v4l2_subdev_core_ops * core;

Const struct v4l2_subdev_tuner_ops * tuner;

Const struct v4l2_subdev_audio_ops * audio;

Const struct v4l2_subdev_video_ops * video;

};

Core operations are common to all sub-devices. Other types can be implemented based on the needs of sub-devices. For example, a video sub-device is unlikely to perform audio operations.

Now we have introduced some v4l2_subdev members and operation functions, so we can initialize them and initialize the function call:

V4l2_subdev_init (sd, & ops );

Then you need to initialize the sub-device name and owner. To integrate the media framework, we must initialize the media_entity structure and embed it into the structure of v4l2_subdev. This is achieved by calling media_entity_init.

Struct media_pad * pads = & my_sd-> pads;

Media_entity_init (& sd-> entity, npads, pads, 0 );

[Html]View plaincopy

 

 

The pads array must have been initialized before. You do not need to manually initialize the media_entity type and name, but you must initialize the revision domain if necessary. The reference parameter of Entity is automatically added or subtracted when the sub-device node is turned on or off. Do not forget to cleanup mediaentity before the sub-device is destroyed. Call media_entity_cleanup (& sd-> entity ). The knowledge about media_entity is not discussed here. We will continue to discuss the registration of v4l2_subdev.

Register the v4l2_subdev sub-device instance to the v4l2_device system. Use this function:

V4l2_device_unregister_subdev (sd)

After this function is successfully executed, subdev-> dev will point to v4l2_device. If the mdev value of v4l2_device is a non-empty value, subdev-> entity will be automatically registered as mdev. To remove the registered sub-device, call:

V4l2_device_unregister_subdev (sd)

Next, we will introduce the function calls provided by sub-devices. If you want to use the interface functions provided by sub-devices, there are two methods. The first method is to directly use the callback function in ops, but this is not recommended, generally, the second method is used to call a function:

V4l2_subdev_call (sd, o, f, arg ...)

To obtain the ID of the sub-device chip. Sd is a sub-device instance, and o is a class of sub-device operation functions. For example, it can be core/video/audio/tuner, and f is a class of function callback functions, arg is the input parameter. In addition, you can call the function callback function of all sub-devices through the v4l2 device instance. Use this function:

V4l2_device_call_all (v4l2, grp_id, o, f, arg ...)

Grp_id indicates the group ID of the sub-device. For example:

V4l2_subdev_call (sd, video, g_chip_cap, & cap );

V4l2_device_call_all (v4l2, 0, core, g_chip_id, & cap );

The former is the g_chip_cap function callback function under the video class of the sub-device sd; the latter is the g_chip_id function callback function called by the v4l2 device in the core class of all sub-devices. If grp_id is not 0, this method with the same group ID is called. The sub-device also needs to notify its v4l2 parent device of the event, which is implemented by calling the following function.

V4l2_subdev_policy (sd, notification, arg)

However, the parent device must be able to handle these events, that is, implement the notify function of v4l2_device.

In addition to the APIS exposed to the kernel through the v4l2_subdev_ops structure, v4l2 sub-devices can also be directly controlled by user programs. The device node name is v4l-subdevX created under the/dev directory so that you can directly access the sub-device by opening the device file. If a sub-device supports direct user space access, it must set the V4L2_SUBDEV_FL_HAS_DEVNODE flag before registration. After a sub-device is registered, the v4l2_device driver creates a device node for all sub-devices with this flag. This is achieved through v4l2_device_register_subdev_nodes. This device node can process a set of standard V4l2API subsets as follows:

VIDIOC_QUERYCTRL

VIDIOC_QUERYMENU

VIDIOC_G_CTRL

VIDIOC_S_CTRL

VIDIOC_G_EXT_CTRLS

VIDIOC_S_EXT_CTRLS

VIDIOC_TRY_EXT_CTRLS

All the above control calls can be completed through the core: ioctl operation. Now, we have completed the introduction of v4l2 sub-devices.

Note:

The kernel provides us with a lot of help functions, such as the v4l2_i2c sub-device driver framework, making it much easier to write such drivers. Because such drivers have many commonalities, they can be abstracted for ease of use. This abstraction is in v4l2_common.h.

The recommended method for adding v4l2_subdev to an I2C driver is to embed the v4l2_subdev structure into the state structure of the I2C device instance. If the device does not have this structure, simply create a v4l2_subdev instance. A typical state structure is like this:

Struct chipname_state {

Struct v4l2_subdev sd;

..... /* Store additional status domains here */

};

Initialize a v4l2_subdev and connect it to the i2c bus device.

V4l2_i2c_subdev_init (& state-> sd, client, subdev_ops );

This function will fill in all v4l2_subdev fields and ensure that v4l2_subdev and i2c_client can find each other. It is best to implement an inline function for mutual access between state and subdev:

Static inline struct chipname_state * to_state (struct v4l2_subdev * sd)

{

Return container_of (sd, structchipname_state, sd );

}

Then, use the following function to achieve mutual access between v4l2_subdev and i2c_client:

Struct i2c_client * client = v4l2_get_subdevdata (sd );

Struct v4l2_subdev * sd = i2c_get_clientdata (client );

Make sure that the following function is called when the subdev driver is removed:

V4l2_device_unregister_subdev (sd );

There are also some help functions that can be used:

Struct v4l2_subdev * sd = v4l2_i2c_new_subdev (v4l2_dev, adapter, "module_foo", "chipid", 0x36, NULL );

This function loads an i2c adapter and then calls i2d_new_device and creates a new i2c device based on the chipid and i2c address (0x36, subdev with the module name module_foo will be registered to v4l2_dev. For more help functions, see the v4l2-common.h file.