V4L2 Driver-Overall architecture

My UVC Open Source address: GITEE-UVC

• Character device driver Core: V4l2 itself is a character device that has all the features of a character device, exposing the interface to user space.
• V4L2 Drive Core: The main is to build a kernel in the standard video device driver framework, for video operations to provide a unified interface function.
• Platform V4L2 device driver: Under the V4L2 framework, the platform-related V4L2 drivers are implemented according to the platform's own features, including registration Video_device and V4l2_dev.
• Specific sensor driver: The main power-up, provide work clock, video image clipping, stream IO open, etc., to achieve a variety of device control methods for the upper call and register V4l2_subdev.

1 starting from the character device:

Familiar with V4L2 user space programming know, V4L2 programming is to call a series of IOCTL function to open, close, query, set up and so on v4l2 device. The V4L2 device is a character device, and its main driver is to implement a variety of IOCTL.

The overall framework of the V4L2 is as follows:

V4l2:video for Linux version 2 is a standard set of video drivers in Linux. This article is to analyze its core framework.

The implementation of this file_operations in the core of V4L2 is as follows:

static const struct file_operations v4l2_fops = {    .owner = THIS_MODULE,    .read = v4l2_read,    .write = v4l2_write,    .open = v4l2_open,    .get_unmapped_area = v4l2_get_unmapped_area,    .mmap = v4l2_mmap,    .unlocked_ioctl = v4l2_ioctl,#ifdef CONFIG_COMPAT    .compat_ioctl = v4l2_compat_ioctl32,#endif    .release = v4l2_release,    .poll = v4l2_poll,    .llseek = no_llseek,};

The V4l2_fops function is eventually bound to a Cdev and registered in the system.

v4l2_openas an example (code in kernel\drivers\media\v4l2-core ):

/* Override for the Open function */static int v4l2_open (struct inode *inode, struct file *filp) {struct VIDEO_DE Vice *vdev; int ret = 0; /* Check If the video device is available */Mutex_lock (&videodev_lock); Vdev = Video_devdata (FILP); /* Return ENODEV If the video device has already been removed. */if (Vdev = = NULL | |!video_is_registered (VDEV)) {Mutex_unlock (&videodev_lock); Return-enodev; }/* and increase the device RefCount */Video_get (VDEV); Mutex_unlock (&videodev_lock); if (Vdev->fops->open) {if (video_is_registered (Vdev))///This is the open function that calls File_operations ret = Vdev->fops->open (FILP); else ret =-enodev; } if (Vdev->debug) printk (kern_debug "%s:open (%d) \ n", Video_device_node_name (Vdev), ret); /* Decrease the refcount in case of an error */if (ret) video_put (Vdev); return ret;} 

2. Video_device structure:

The Video_device structure is used to generate the device node file in the/dev directory, exposing the interface of the operating device to the user space.

We use it video_device to manipulate and look at video_device the structure:

struct video_device{#if defined (config_media_controller) struct media_entity entity; #endif/* Device OPS */const    struct V4l2_file_operations *fops;      /* SYSFS */struct device dev;      /* v4l device */struct CDEV *cdev;      /* character device */* Set either parent or V4l2_dev if your driver uses V4l2_device */struct device *parent;   /* Device Parent */struct v4l2_device *v4l2_dev; /* V4l2_device Parent */* Control handler associated with this device node. May is NULL.    */struct V4l2_ctrl_handler *ctrl_handler; /* Vb2_queue associated with this device node. May is NULL.    */struct Vb2_queue *queue; /* Priority state. If NULL, then V4l2_dev->prio'll be used.    */struct v4l2_prio_state *prio;    /* Device Info */char name[32];   int vfl_type;    /* Device type */int vfl_dir;    /* receiver, transmitter or */*/*/* ' minor ' is set to-1 if the registration failed */int minor;    U16 Num; /* Use Bitops to SET/CLear/test Flags * * unsigned long flags;    /* attribute to differentiate multiple indices on one physical device */INT index; /* V4L2 file Handles */spinlock_t fh_lock; /* Lock for all V4L2_FHS */struct list_head fh_list;          /* List of struct V4L2_FH */int debug;        /* Activates debug level*//* Video standard VARs */v4l2_std_id tvnorms;   /* Supported TV norms */v4l2_std_id current_norm;    /* Current Tvnorm */* callbacks */void (*release) (struct video_device *vdev);    /* IOCTL callbacks */const struct V4L2_IOCTL_OPS *ioctl_ops;    Declare_bitmap (Valid_ioctls, base_vidioc_private);    /* Serialization Lock */Declare_bitmap (disable_locking, base_vidioc_private); struct mutex *lock;};

3. V4l2_device Structure:

This structure contains a very important structure v4l2_device :

struct V4l2_device {/* Dev->driver_data points to the this struct. Note:dev might was NULL if there is no parent device as was the case with e.g. ISA devices.  */struct device *dev; #if defined (config_media_controller) struct media_device *mdev; #endif/* Used to keep track    of the registered Subdevs */struct list_head Subdevs; /* lock this struct; Can be used by the driver as well if this struct was embedded into a larger struct.    */spinlock_t lock;    /* Unique device name, by default the driver name + bus ID */char name[v4l2_device_name_size]; /* Notify callback called by some sub-devices.    */void (*notify) (struct V4l2_subdev *sd, unsigned int notification, void *arg); /* The control handler. May is NULL.    */struct V4l2_ctrl_handler *ctrl_handler;    /* Device ' s priority state */struct v4l2_prio_state prio; /* BKL replacement mutex. Temporary solution only.    */struct Mutex ioctl_lock; /* Keep track of the refErences to this struct.    */struct KREF ref; /* Release function that's called when the ref count goes to 0. */void (*release) (struct v4l2_device *v4l2_dev);};

Registration and deregistration of 3.1 V4l2_device:

int v4l2_device_register(struct device*dev, struct v4l2_device *v4l2_dev)static void v4l2_device_release(struct kref *ref)

4. V4l2_subdev Structural Body

V4l2_subdev represents a sub-device that contains related properties and operations for a child device. First look at the structure prototype:

struct v4l2_subdev {          struct v4l2_device *v4l2_dev;  //指向父设备          //提供一些控制v4l2设备的接口          const struct v4l2_subdev_ops *ops;          //向V4L2框架提供的接口函数          const struct v4l2_subdev_internal_ops *internal_ops;          //subdev控制接口          struct v4l2_ctrl_handler *ctrl_handler;          /* name must be unique */          charname[V4L2_SUBDEV_NAME_SIZE];          /*subdev device node */          struct video_device *devnode;   };

Where the list field is linked to the v4l2_device structure pointed to by V4l2_dev as a linked list node, the most important member of this structure is the struct v4l2_subdev_ops *ops, which contains all the operations supported by the V4L2 device, defined as follows:

struct v4l2_subdev_ops {    const struct v4l2_subdev_core_ops  *core;   /* 通用操作合集 */    const struct v4l2_subdev_tuner_ops *tuner;  /* 调谐器操作合集 */    const struct v4l2_subdev_audio_ops *audio;  /* 音频操作合集 */    const struct v4l2_subdev_video_ops *video;  /* 视频操作合集 */};

V4l2_subdev_core_ops contains operation collections that are common to various types of devices:

struct V4l2_subdev_core_ops {int (*g_chip_ident) (struct V4l2_subdev *sd, struct v4l2_dbg_chip_ident *chip);                                      /* Get Device ID */INT (*log_status) (struct V4l2_subdev *sd);          /* Status message */INT (*s_config) (struct v4l2_subdev *sd, int irq, void *platform_data);                                   /* Set Configuration information */INT (*init) (struct V4l2_subdev *sd, u32 val);                                         /* Initialize the device */int (*LOAD_FW) (struct V4l2_subdev *sd);                                  /* Load firmware */INT (*reset) (struct V4l2_subdev *sd, u32 val);                                 /* Reset device */int (*s_gpio) (struct V4l2_subdev *sd, u32 val);            /* Set GPIO */INT (*queryctrl) (struct V4l2_subdev *sd, struct V4l2_queryctrl *qc);               /* query Device supported operation */INT (*g_ctrl) (struct V4l2_subdev *sd, struct V4l2_control *ctrl);               /* Get current command value */INT (*s_ctrl) (struct V4l2_subdev *sd, struct V4l2_control *ctrl); /* Set current command value */INT (*g_ext_ctrls)    (struct V4l2_subdev *sd, struct v4l2_ext_controls *ctrls);    /* Get External command value */INT (*s_ext_ctrls) (struct V4l2_subdev *sd, struct v4l2_ext_controls *ctrls);    /* Set the external command value */INT (*try_ext_ctrls) (struct V4l2_subdev *sd, struct v4l2_ext_controls *ctrls);            Int (*querymenu) (struct V4l2_subdev *sd, struct v4l2_querymenu *qm);                         /* Query Action menu */INT (*S_STD) (struct V4l2_subdev *sd, v4l2_std_id norm);             /* Set Data Standard */Long (*IOCTL) (struct V4l2_subdev *sd, unsigned int cmd, void *arg);       /* Handle SPECIAL commands */#ifdef config_video_adv_debug int (*g_register) (struct V4l2_subdev *sd, struct v4l2_dbg_register *reg);       /* Get Register value */INT (*s_register) (struct V4l2_subdev *sd, struct v4l2_dbg_register *reg); /* Set Register value */#endif};

The

V4l2_subdev_tuner_ops contains a collection of actions that are unique to the tuner:

struct V4l2_subdev_tuner_ops {int (*s_mode) (struct V4l2_subdev *sd, enum V4l2_tuner_type);                                    /* Set Tuner mode */INT (*s_radio) (struct V4l2_subdev *sd);   /* Set the wireless device information */INT (*s_frequency) (struct V4l2_subdev *sd, struct v4l2_frequency *freq);   /* Set frequency */INT (*g_frequency) (struct V4l2_subdev *sd, struct v4l2_frequency *freq);             /* Get frequency */INT (*g_tuner) (struct V4l2_subdev *sd, struct v4l2_tuner *vt);             /* Get tuner information */INT (*s_tuner) (struct V4l2_subdev *sd, struct v4l2_tuner *vt);     /* Set TUNER information */INT (*g_modulator) (struct V4l2_subdev *sd, struct v4l2_modulator *vm);     /* Get the amplitude modulation information */INT (*s_modulator) (struct V4l2_subdev *sd, struct v4l2_modulator *vm);      /* Set the amplitude modulation information */INT (*s_type_addr) (struct V4l2_subdev *sd, struct tuner_setup *type);   /* Install tuner */INT (*s_config) (struct V4l2_subdev *sd, const struct v4l2_priv_tun_config *config);  /* Set Configuration information */INT (*s_standby) (struct V4l2_subdev *sd);                                /* Set standard */}; 

The v4l2_subdev_audio_ops contains an operation collection that is unique to the audio section:

struct v4l2_subdev_audio_ops {    int (*s_clock_freq)(struct v4l2_subdev *sd, u32 freq);       /* 设置音频设备频率 */    int (*s_i2s_clock_freq)(struct v4l2_subdev *sd, u32 freq);   /* 设置i2s总线频率 */    int (*s_routing)(struct v4l2_subdev *sd, u32 input, u32 output, u32 config);   /* 设置音频路由 */};

The V4l2_subdev_video_ops contains a collection of actions that are unique to the video section:

struct V4l2_subdev_video_ops {int (*s_routing) (struct V4l2_subdev *sd, u32 input, u32 output, u32 config);                      /* Set the video route */INT (*s_crystal_freq) (struct V4l2_subdev *sd, u32 freq, u32 flags);   /* Set Device frequency */INT (*decode_vbi_line) (struct V4l2_subdev *sd, struct v4l2_decode_vbi_line *vbi_line);  /* Blanking Area information decoding */INT (*s_vbi_data) (struct V4l2_subdev *sd, const struct v4l2_sliced_vbi_data *vbi_data);        /* Set the blanking area data */INT (*g_vbi_data) (struct V4l2_subdev *sd, struct v4l2_sliced_vbi_data *vbi_data);    /* The hidden Data is canceled */int (*g_sliced_vbi_cap) (struct V4l2_subdev *sd, struct v4l2_sliced_vbi_cap *cap);                            Int (*s_std_output) (struct V4l2_subdev *sd, v4l2_std_id std);                               /* Set standard OUTPUT */int (*QUERYSTD) (struct V4l2_subdev *sd, v4l2_std_id *std);                              /* Query standard */INT (*g_input_status) (struct V4l2_subdev *sd, u32 *status); /* Get INPUT status */INT (*s_stream) (struct V4l2_subdev *sd,int enable);                   /* Set Data Flow */INT (*enum_fmt) (struct V4l2_subdev *sd, struct v4l2_fmtdesc *fmtdesc);                           /* Enumerate video formats */INT (*g_fmt) (struct V4l2_subdev *sd, struct v4l2_format *fmt);                         /* Get the video format */int (*try_fmt) (struct V4l2_subdev *sd, struct v4l2_format *fmt);                           /* Try to set the video format */int (*s_fmt) (struct V4l2_subdev *sd, struct v4l2_format *fmt);                         /* Set Video format */int (*cropcap) (struct V4l2_subdev *sd, struct v4l2_cropcap *cc);                           /* Video clip function */int (*g_crop) (struct V4l2_subdev *sd, struct v4l2_crop *crop);                           /* Get Clip function */int (*s_crop) (struct V4l2_subdev *sd, struct v4l2_crop *crop);                    /* Set Clip function */int (*g_parm) (struct V4l2_subdev *sd, struct v4l2_streamparm *param);                    /* Get parameters */INT (*s_parm) (struct V4l2_subdev *sd, struct v4l2_streamparm *param); /* Set parameters */INT (*enum_framesizes) (struct V4l2_subdev *sd, struct v4l2_frmsizeenum *fsize);      /* Enumerate frame sizes */int (*enum_frameintervals) (struct V4l2_subdev *sd, struct v4l2_frmivalenum *fival); /* Enumerate frame intervals */};

Registration and deregistration of 4.1 subdev

When we have implemented the V4l2_subdev that need to be implemented, we can call the following function to register the device with the V4L2 core layer:

int v4l2_device_register_subdev(struct v4l2_device*v4l2_dev, struct v4l2_subdev *sd)

When you unload a child device, you can call the following function to log off:

void v4l2_device_unregister_subdev(struct v4l2_subdev*sd)

5. Application layer specific process framework:

Let's use a picture to show it: