Video device's internal representation

video_device 할당과 release callback

`/dev` 디렉터리의 실제 장치 node는 `v4l2-dev.h`에 정의된 `video_device` 구조체로 만듭니다. 이 구조체는 `video_device_alloc()`으로 동적 할당하거나 더 큰 driver 전용 구조체 안에 포함할 수 있습니다.

동적 할당에서는 반환값이 `NULL`인지 검사하고 `vdev->release`를 `video_device_release`로 설정합니다. 기본 `video_device_release()`는 현재 할당된 메모리를 `kfree()`로 해제합니다.

구조체를 포함하는 방식에서는 `vdev->release`를 driver 전용 callback으로 설정해야 합니다. 장치의 마지막 사용자가 빠져나갈 때 callback이 호출되므로 이 callback은 반드시 지정해야 합니다.

포함된 구조체를 release 시점에 따로 처리할 필요가 없다면 아무 동작도 하지 않는 `video_device_release_empty()`를 사용할 수 있습니다.

.. SPDX-License-Identifier: GPL-2.0

Video device' s internal representation
=======================================

The actual device nodes in the ``/dev`` directory are created using the
:c:type:`video_device` struct (``v4l2-dev.h``). This struct can either be
allocated dynamically or embedded in a larger struct.

To allocate it dynamically use :c:func:`video_device_alloc`:

.. code-block:: c

        struct video_device *vdev = video_device_alloc();

        if (vdev == NULL)
                return -ENOMEM;

        vdev->release = video_device_release;

If you embed it in a larger struct, then you must set the ``release()``
callback to your own function:

.. code-block:: c

        struct video_device *vdev = &my_vdev->vdev;

        vdev->release = my_vdev_release;

The ``release()`` callback must be set and it is called when the last user
of the video device exits.

The default :c:func:`video_device_release` callback currently
just calls ``kfree`` to free the allocated memory.

There is also a :c:func:`video_device_release_empty` function that does
nothing (is empty) and should be used if the struct is embedded and there
is nothing to do when it is released.

video_device 필수 field

`video_device->v4l2_dev`에는 부모 `v4l2_device`를 지정하고, `name`에는 설명적이며 고유한 이름을 넣습니다. `vfl_dir`는 capture 장치에 `VFL_DIR_RX`, output 장치에 `VFL_DIR_TX`, memory-to-memory codec 장치에 `VFL_DIR_M2M`을 사용합니다. `VFL_DIR_RX` 값은 0이므로 보통 기본값과 같습니다.

`fops`는 `v4l2_file_operations`를 가리킵니다. ioctl 유지 관리를 단순화하려면 권장되는 `v4l2_ioctl_ops`를 `ioctl_ops`에 설정합니다. Core는 `vfl_type`과 `vfl_dir` 조합에 맞지 않는 operation을 비활성화하므로 VBI와 video node, capture와 output node가 하나의 `v4l2_ioctl_ops`를 공유할 수 있습니다.

`lock`이 `NULL`이면 driver가 모든 locking을 담당합니다. Mutex pointer를 지정하면 core가 `unlocked_ioctl` 호출 전후에 lock을 획득하고 해제합니다.

`queue`에는 장치 node와 연관된 `vb2_queue`를 지정합니다. `queue`와 `queue->lock`이 모두 `NULL`이 아니면 `VIDIOC_REQBUFS`, `CREATE_BUFS`, `QBUF`, `DQBUF`, `QUERYBUF`, `PREPARE_BUF`, `STREAMON`, `STREAMOFF` 같은 queue ioctl은 일반 `video_device->lock` 대신 `queue->lock`을 사용합니다. 이 pointer는 vb2 helper가 file handle의 queue 소유권을 확인할 때도 사용합니다.

`prio`는 `VIDIOC_G_PRIORITY`와 `VIDIOC_S_PRIORITY`를 위한 우선순위 상태를 추적합니다. `NULL`이면 부모 `v4l2_device`의 `v4l2_prio_state`를 사용하고, 장치 node 그룹마다 별도 상태가 필요하면 driver가 소유한 `v4l2_prio_state`를 지정합니다.

`dev_parent`는 `v4l2_device`가 부모 `device` 없이 등록된 특수한 경우에만 설정합니다. 예를 들어 cx88은 하나의 `v4l2_device` core를 raw video PCI 장치 cx8800과 MPEG PCI 장치 cx8802가 공유하므로 core에는 부모를 둘 수 없습니다. 하지만 각 `video_device`를 초기화할 때는 실제 부모 PCI 장치를 알 수 있으므로 올바른 PCI 장치를 지정할 수 있습니다.

You should also set these fields of :c:type:`video_device`:

- :c:type:`video_device`->v4l2_dev: must be set to the :c:type:`v4l2_device`
  parent device.

- :c:type:`video_device`->name: set to something descriptive and unique.

- :c:type:`video_device`->vfl_dir: set this to ``VFL_DIR_RX`` for capture
  devices (``VFL_DIR_RX`` has value 0, so this is normally already the
  default), set to ``VFL_DIR_TX`` for output devices and ``VFL_DIR_M2M`` for mem2mem (codec) devices.

- :c:type:`video_device`->fops: set to the :c:type:`v4l2_file_operations`
  struct.

- :c:type:`video_device`->ioctl_ops: if you use the :c:type:`v4l2_ioctl_ops`
  to simplify ioctl maintenance (highly recommended to use this and it might
  become compulsory in the future!), then set this to your
  :c:type:`v4l2_ioctl_ops` struct. The :c:type:`video_device`->vfl_type and
  :c:type:`video_device`->vfl_dir fields are used to disable ops that do not
  match the type/dir combination. E.g. VBI ops are disabled for non-VBI nodes,
  and output ops  are disabled for a capture device. This makes it possible to
  provide just one :c:type:`v4l2_ioctl_ops` struct for both vbi and
  video nodes.

- :c:type:`video_device`->lock: leave to ``NULL`` if you want to do all the
  locking  in the driver. Otherwise you give it a pointer to a struct
  ``mutex_lock`` and before the :c:type:`video_device`->unlocked_ioctl
  file operation is called this lock will be taken by the core and released
  afterwards. See the next section for more details.

- :c:type:`video_device`->queue: a pointer to the struct vb2_queue
  associated with this device node.
  If queue is not ``NULL``, and queue->lock is not ``NULL``, then queue->lock
  is used for the queuing ioctls (``VIDIOC_REQBUFS``, ``CREATE_BUFS``,
  ``QBUF``, ``DQBUF``,  ``QUERYBUF``, ``PREPARE_BUF``, ``STREAMON`` and
  ``STREAMOFF``) instead of the lock above.
  That way the :ref:`vb2 <vb2_framework>` queuing framework does not have
  to wait for other ioctls.   This queue pointer is also used by the
  :ref:`vb2 <vb2_framework>` helper functions to check for
  queuing ownership (i.e. is the filehandle calling it allowed to do the
  operation).

- :c:type:`video_device`->prio: keeps track of the priorities. Used to
  implement ``VIDIOC_G_PRIORITY`` and ``VIDIOC_S_PRIORITY``.
  If left to ``NULL``, then it will use the struct v4l2_prio_state
  in :c:type:`v4l2_device`. If you want to have a separate priority state per
  (group of) device node(s),   then you can point it to your own struct
  :c:type:`v4l2_prio_state`.

- :c:type:`video_device`->dev_parent: you only set this if v4l2_device was
  registered with ``NULL`` as the parent ``device`` struct. This only happens
  in cases where one hardware device has multiple PCI devices that all share
  the same :c:type:`v4l2_device` core.

  The cx88 driver is an example of this: one core :c:type:`v4l2_device` struct,
  but   it is used by both a raw video PCI device (cx8800) and a MPEG PCI device
  (cx8802). Since the :c:type:`v4l2_device` cannot be associated with two PCI
  devices at the same time it is setup without a parent device. But when the
  struct video_device is initialized you **do** know which parent
  PCI device to use and so you set ``dev_device`` to the correct PCI device.

ioctl dispatch와 media entity

`v4l2_ioctl_ops`를 사용한다면 `v4l2_file_operations`의 `video_device->unlocked_ioctl`을 `video_ioctl2`로 설정해야 합니다.

등록한 `v4l2_ioctl_ops` operation 중 특정 기능을 외부 조건에 따라 사용하지 않아야 할 수 있습니다. 별도 operation 구조체를 만들지 않고 기능을 끄려면 `video_register_device()` 호출 전에 `v4l2_disable_ioctl(vdev, cmd)`을 호출합니다.

`v4l2_file_operations`는 일반 `file_operations`의 부분 집합입니다. 사용되지 않는 inode 인자를 생략한다는 점이 주된 차이입니다.

Media framework와 통합할 때는 `video_device`의 `entity` field에 포함된 `media_entity`를 `media_entity_pads_init()`으로 초기화해야 합니다. 전달하는 pad 배열은 미리 초기화되어 있어야 하며, `media_entity`의 type과 name은 수동 설정할 필요가 없습니다.

Video device가 열리고 닫힐 때 entity reference는 자동으로 획득되고 해제됩니다.

If you use :c:type:`v4l2_ioctl_ops`, then you should set
:c:type:`video_device`->unlocked_ioctl to :c:func:`video_ioctl2` in your
:c:type:`v4l2_file_operations` struct.

In some cases you want to tell the core that a function you had specified in
your :c:type:`v4l2_ioctl_ops` should be ignored. You can mark such ioctls by
calling this function before :c:func:`video_register_device` is called:

        :c:func:`v4l2_disable_ioctl <v4l2_disable_ioctl>`
        (:c:type:`vdev <video_device>`, cmd).

This tends to be needed if based on external factors (e.g. which card is
being used) you want to turns off certain features in :c:type:`v4l2_ioctl_ops`
without having to make a new struct.

The :c:type:`v4l2_file_operations` struct is a subset of file_operations.
The main difference is that the inode argument is omitted since it is never
used.

If integration with the media framework is needed, you must initialize the
:c:type:`media_entity` struct embedded in the :c:type:`video_device` struct
(entity field) by calling :c:func:`media_entity_pads_init`:

.. code-block:: c

        struct media_pad *pad = &my_vdev->pad;
        int err;

        err = media_entity_pads_init(&vdev->entity, 1, pad);

The pads array must have been previously initialized. There is no need to
manually set the struct media_entity type and name fields.

A reference to the entity will be automatically acquired/released when the
video device is opened/closed.

ioctl locking

V4L core는 선택적인 locking 서비스를 제공합니다. `video_device->lock`에 mutex pointer를 지정하면 `unlocked_ioctl`이 모든 ioctl을 직렬화할 때 이 lock을 사용합니다.

Videobuf2를 사용할 때는 `video_device->queue->lock`도 지정할 수 있습니다. 이 lock이 있으면 queue ioctl은 `video_device->lock` 대신 queue lock으로 직렬화됩니다.

Queue ioctl을 별도 lock으로 분리하면 control 설정처럼 오래 걸릴 수 있는 command가 buffer dequeue를 막지 않습니다. 특히 USB webcam의 exposure 변경 중에도 `VIDIOC_DQBUF`가 불필요하게 정지하지 않게 할 수 있습니다. 두 pointer를 모두 `NULL`로 두고 driver가 locking 전체를 직접 구현할 수도 있습니다.

Videobuf2에서는 필요할 때 lock을 풀고 다시 잡는 `wait_prepare()`와 `wait_finish()` callback을 구현해야 합니다. `queue->lock`을 사용한다면 `vb2_ops_wait_prepare`와 `vb2_ops_wait_finish` helper를 사용할 수 있습니다.

Hotplug disconnect 구현은 `v4l2_device_disconnect()`를 호출하기 전에 `video_device->lock`을 잡아야 합니다. Queue lock도 사용한다면 `queue->lock`을 먼저, `video_device->lock`을 나중에 획득해야 합니다. 이 순서를 지키면 disconnect 시점에 실행 중인 ioctl이 없음을 보장할 수 있습니다.

ioctls and locking
------------------

The V4L core provides optional locking services. The main service is the
lock field in struct video_device, which is a pointer to a mutex.
If you set this pointer, then that will be used by unlocked_ioctl to
serialize all ioctls.

If you are using the :ref:`videobuf2 framework <vb2_framework>`, then there
is a second lock that you can set: :c:type:`video_device`->queue->lock. If
set, then this lock will be used instead of :c:type:`video_device`->lock
to serialize all queuing ioctls (see the previous section
for the full list of those ioctls).

The advantage of using a different lock for the queuing ioctls is that for some
drivers (particularly USB drivers) certain commands such as setting controls
can take a long time, so you want to use a separate lock for the buffer queuing
ioctls. That way your ``VIDIOC_DQBUF`` doesn't stall because the driver is busy
changing the e.g. exposure of the webcam.

Of course, you can always do all the locking yourself by leaving both lock
pointers at ``NULL``.

In the case of :ref:`videobuf2 <vb2_framework>` you will need to implement the
``wait_prepare()`` and ``wait_finish()`` callbacks to unlock/lock if applicable.
If you use the ``queue->lock`` pointer, then you can use the helper functions
:c:func:`vb2_ops_wait_prepare` and :c:func:`vb2_ops_wait_finish`.

The implementation of a hotplug disconnect should also take the lock from
:c:type:`video_device` before calling v4l2_device_disconnect. If you are also
using :c:type:`video_device`->queue->lock, then you have to first lock
:c:type:`video_device`->queue->lock followed by :c:type:`video_device`->lock.
That way you can be sure no ioctl is running when you call
:c:func:`v4l2_device_disconnect`.

Video device 등록

`video_register_device(vdev, VFL_TYPE_VIDEO, -1)`로 video device를 등록하면 character device가 생성됩니다. 등록에 실패하면 동적 `video_device`는 `video_device_release()`로, 포함된 구조체는 driver 방식으로 해제하고 오류를 반환합니다.

부모 `v4l2_device`의 `mdev`가 `NULL`이 아니면 video device entity도 media device에 자동 등록됩니다.

Type 인자는 node 종류를 결정합니다. `VFL_TYPE_VIDEO`는 `/dev/videoX`, `VFL_TYPE_VBI`는 `/dev/vbiX`, `VFL_TYPE_RADIO`는 `/dev/radioX`, `VFL_TYPE_SUBDEV`는 `/dev/v4l-subdevX`, `VFL_TYPE_SDR`은 `/dev/swradioX`, `VFL_TYPE_TOUCH`는 `/dev/v4l-touchX`를 생성합니다.

마지막 인자는 node 번호의 선택 기준입니다. 보통 `-1`을 전달해 첫 번째 빈 번호를 framework가 고르게 합니다. 특정 번호를 전달했는데 이미 사용 중이면 다음 빈 번호를 선택하고 kernel log에 경고합니다. 여러 장치를 번호 범위로 나누는 driver는 capture를 0부터, output을 16부터 시작하는 식으로 최소 번호를 지정할 수 있습니다. 지정 범위에서 실패하면 전체 범위의 첫 빈 번호를 사용합니다.

지정 번호를 얻지 못했다는 경고가 필요 없는 최소 번호 방식에는 `video_register_device_no_warn()`을 사용할 수 있습니다.

Node가 생성되면 `/sys/class/video4linux/<devX>/`에 `name`, `dev_debug`, `index` 속성이 생깁니다. `name`은 `video_device.name`, `dev_debug`는 core debugging 설정입니다. `index`는 `video_register_device()` 호출마다 증가하며 첫 등록은 0입니다. Userspace는 이 값으로 MPEG capture node에 `mpegX` 같은 udev 이름을 만들 수 있습니다.

등록 후 `vfl_type`, `minor`, `num`, `index`를 사용할 수 있습니다. 등록 실패 시 `vdev->release()`는 호출되지 않으므로 직접 메모리를 해제해야 하며, 실패한 장치를 unregister해서는 안 됩니다.

Video device registration
-------------------------

Next you register the video device with :c:func:`video_register_device`.
This will create the character device for you.

.. code-block:: c

        err = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
        if (err) {
                video_device_release(vdev); /* or kfree(my_vdev); */
                return err;
        }

If the :c:type:`v4l2_device` parent device has a not ``NULL`` mdev field,
the video device entity will be automatically registered with the media
device.

Which device is registered depends on the type argument. The following
types exist:

========================== ====================         ==============================
:c:type:`vfl_devnode_type` Device name                 Usage
========================== ====================         ==============================
``VFL_TYPE_VIDEO``         ``/dev/videoX``       for video input/output devices
``VFL_TYPE_VBI``           ``/dev/vbiX``         for vertical blank data (i.e.
                                                 closed captions, teletext)
``VFL_TYPE_RADIO``         ``/dev/radioX``       for radio tuners
``VFL_TYPE_SUBDEV``        ``/dev/v4l-subdevX``  for V4L2 subdevices
``VFL_TYPE_SDR``           ``/dev/swradioX``     for Software Defined Radio
                                                 (SDR) tuners
``VFL_TYPE_TOUCH``         ``/dev/v4l-touchX``   for touch sensors
========================== ====================         ==============================

The last argument gives you a certain amount of control over the device
node number used (i.e. the X in ``videoX``). Normally you will pass -1
to let the v4l2 framework pick the first free number. But sometimes users
want to select a specific node number. It is common that drivers allow
the user to select a specific device node number through a driver module
option. That number is then passed to this function and video_register_device
will attempt to select that device node number. If that number was already
in use, then the next free device node number will be selected and it
will send a warning to the kernel log.

Another use-case is if a driver creates many devices. In that case it can
be useful to place different video devices in separate ranges. For example,
video capture devices start at 0, video output devices start at 16.
So you can use the last argument to specify a minimum device node number
and the v4l2 framework will try to pick the first free number that is equal
or higher to what you passed. If that fails, then it will just pick the
first free number.

Since in this case you do not care about a warning about not being able
to select the specified device node number, you can call the function
:c:func:`video_register_device_no_warn` instead.

Whenever a device node is created some attributes are also created for you.
If you look in ``/sys/class/video4linux`` you see the devices. Go into e.g.
``video0`` and you will see 'name', 'dev_debug' and 'index' attributes. The
'name' attribute is the 'name' field of the video_device struct. The
'dev_debug' attribute can be used to enable core debugging. See the next
section for more detailed information on this.

The 'index' attribute is the index of the device node: for each call to
:c:func:`video_register_device()` the index is just increased by 1. The
first video device node you register always starts with index 0.

Users can setup udev rules that utilize the index attribute to make fancy
device names (e.g. '``mpegX``' for MPEG video capture device nodes).

After the device was successfully registered, then you can use these fields:

- :c:type:`video_device`->vfl_type: the device type passed to
  :c:func:`video_register_device`.
- :c:type:`video_device`->minor: the assigned device minor number.
- :c:type:`video_device`->num: the device node number (i.e. the X in
  ``videoX``).
- :c:type:`video_device`->index: the device index number.

If the registration failed, then you need to call
:c:func:`video_device_release` to free the allocated :c:type:`video_device`
struct, or free your own struct if the :c:type:`video_device` was embedded in
it. The ``vdev->release()`` callback will never be called if the registration
failed, nor should you ever attempt to unregister the device if the
registration failed.

Video device debugging

각 video, VBI, radio, swradio 장치의 `/sys/class/video4linux/<devX>/dev_debug` 속성은 file operation logging을 활성화하는 bitmask입니다.

`0x01`은 ioctl 이름과 오류를, `0x02`는 ioctl 이름·인자·오류를 기록합니다. 두 경우 모두 `VIDIOC_QBUF`와 `VIDIOC_DQBUF`는 `0x08`도 설정되어야 기록됩니다.

`0x04`는 open, release, read, write, mmap, get_unmapped_area를 기록하지만 read와 write에는 `0x08`이 함께 필요합니다. `0x08`은 read·write와 `VIDIOC_QBUF`·`VIDIOC_DQBUF`, `0x10`은 poll, `0x20`은 control operation의 오류와 메시지를 기록합니다.

video device debugging
----------------------

The 'dev_debug' attribute that is created for each video, vbi, radio or swradio
device in ``/sys/class/video4linux/<devX>/`` allows you to enable logging of
file operations.

It is a bitmask and the following bits can be set:

.. tabularcolumns:: |p{5ex}|L|

===== ================================================================
Mask  Description
===== ================================================================
0x01  Log the ioctl name and error code. VIDIOC_(D)QBUF ioctls are
      only logged if bit 0x08 is also set.
0x02  Log the ioctl name arguments and error code. VIDIOC_(D)QBUF
      ioctls are
      only logged if bit 0x08 is also set.
0x04  Log the file operations open, release, read, write, mmap and
      get_unmapped_area. The read and write operations are only
      logged if bit 0x08 is also set.
0x08  Log the read and write file operations and the VIDIOC_QBUF and
      VIDIOC_DQBUF ioctls.
0x10  Log the poll file operation.
0x20  Log error and messages in the control operations.
===== ================================================================

Video device 정리

Driver unload 또는 USB disconnect로 video device node를 제거할 때는 `video_unregister_device(vdev)`를 호출합니다. 이 함수는 sysfs node를 제거하고, 이어서 udev가 `/dev` node를 제거하게 합니다.

`video_unregister_device()`가 반환된 뒤에는 새 open이 불가능합니다. 다만 USB 장치의 node를 application이 이미 열고 있을 수 있으므로 release를 제외한 나머지 file operation은 unregister 이후 오류를 반환합니다.

Video device node의 마지막 사용자가 빠져나가면 `vdev->release()` callback이 호출되며 이곳에서 최종 정리를 수행할 수 있습니다.

Video device와 연관된 media entity를 초기화했다면 `media_entity_cleanup(&vdev->entity)`도 잊지 말아야 합니다. 이 정리는 release callback에서 수행할 수 있습니다.

Video device cleanup
--------------------

When the video device nodes have to be removed, either during the unload
of the driver or because the USB device was disconnected, then you should
unregister them with:

        :c:func:`video_unregister_device`
        (:c:type:`vdev <video_device>`);

This will remove the device nodes from sysfs (causing udev to remove them
from ``/dev``).

After :c:func:`video_unregister_device` returns no new opens can be done.
However, in the case of USB devices some application might still have one of
these device nodes open. So after the unregister all file operations (except
release, of course) will return an error as well.

When the last user of the video device node exits, then the ``vdev->release()``
callback is called and you can do the final cleanup there.

Don't forget to cleanup the media entity associated with the video device if
it has been initialized:

        :c:func:`media_entity_cleanup <media_entity_cleanup>`
        (&vdev->entity);

This can be done from the release callback.

video_device helper

`video_get_drvdata(vdev)`와 `video_set_drvdata(vdev)`로 `video_device`의 driver private data를 읽고 쓸 수 있습니다. `video_set_drvdata()`는 `video_register_device()`보다 먼저 호출해도 안전합니다.

`video_devdata(file)`는 file 구조체에 속한 `video_device`를 반환합니다. `video_drvdata(file)`는 file에서 해당 video device의 driver private data를 바로 얻는 결합 helper입니다.

`video_device`에서 부모 `v4l2_device`로 이동하려면 `vdev->v4l2_dev`를 사용합니다.

장치 node의 kernel 이름은 `video_device_node_name(vdev)`으로 얻습니다. 이 이름은 udev 같은 userspace 도구의 hint이며, 가능한 한 `video_device::num`과 `video_device::minor` field를 직접 읽는 대신 이 함수를 사용해야 합니다.

helper functions
----------------

There are a few useful helper functions:

- file and :c:type:`video_device` private data

You can set/get driver private data in the video_device struct using:

        :c:func:`video_get_drvdata <video_get_drvdata>`
        (:c:type:`vdev <video_device>`);

        :c:func:`video_set_drvdata <video_set_drvdata>`
        (:c:type:`vdev <video_device>`);

Note that you can safely call :c:func:`video_set_drvdata` before calling
:c:func:`video_register_device`.

And this function:

        :c:func:`video_devdata <video_devdata>`
        (struct file \*file);

returns the video_device belonging to the file struct.

The :c:func:`video_devdata` function combines :c:func:`video_get_drvdata`
with :c:func:`video_devdata`:

        :c:func:`video_drvdata <video_drvdata>`
        (struct file \*file);

You can go from a :c:type:`video_device` struct to the v4l2_device struct using:

.. code-block:: c

        struct v4l2_device *v4l2_dev = vdev->v4l2_dev;

- Device node name

The :c:type:`video_device` node kernel name can be retrieved using:

        :c:func:`video_device_node_name <video_device_node_name>`
        (:c:type:`vdev <video_device>`);

The name is used as a hint by userspace tools such as udev. The function
should be used where possible instead of accessing the video_device::num and
video_device::minor fields.

video_device 함수와 자료구조

`include/media/v4l2-dev.h`의 kernel-doc에서 `video_device` 함수와 자료구조의 상세 API를 이어서 제공합니다.

video_device functions and data structures
------------------------------------------

.. kernel-doc:: include/media/v4l2-dev.h