407 lines
17 KiB
ReStructuredText
407 lines
17 KiB
ReStructuredText
.. SPDX-License-Identifier: GPL-2.0
|
|
|
|
.. _vb_framework:
|
|
|
|
Videobuf Framework
|
|
==================
|
|
|
|
Author: Jonathan Corbet <corbet@lwn.net>
|
|
|
|
Current as of 2.6.33
|
|
|
|
.. note::
|
|
|
|
The videobuf framework was deprecated in favor of videobuf2. Shouldn't
|
|
be used on new drivers.
|
|
|
|
Introduction
|
|
------------
|
|
|
|
The videobuf layer functions as a sort of glue layer between a V4L2 driver
|
|
and user space. It handles the allocation and management of buffers for
|
|
the storage of video frames. There is a set of functions which can be used
|
|
to implement many of the standard POSIX I/O system calls, including read(),
|
|
poll(), and, happily, mmap(). Another set of functions can be used to
|
|
implement the bulk of the V4L2 ioctl() calls related to streaming I/O,
|
|
including buffer allocation, queueing and dequeueing, and streaming
|
|
control. Using videobuf imposes a few design decisions on the driver
|
|
author, but the payback comes in the form of reduced code in the driver and
|
|
a consistent implementation of the V4L2 user-space API.
|
|
|
|
Buffer types
|
|
------------
|
|
|
|
Not all video devices use the same kind of buffers. In fact, there are (at
|
|
least) three common variations:
|
|
|
|
- Buffers which are scattered in both the physical and (kernel) virtual
|
|
address spaces. (Almost) all user-space buffers are like this, but it
|
|
makes great sense to allocate kernel-space buffers this way as well when
|
|
it is possible. Unfortunately, it is not always possible; working with
|
|
this kind of buffer normally requires hardware which can do
|
|
scatter/gather DMA operations.
|
|
|
|
- Buffers which are physically scattered, but which are virtually
|
|
contiguous; buffers allocated with vmalloc(), in other words. These
|
|
buffers are just as hard to use for DMA operations, but they can be
|
|
useful in situations where DMA is not available but virtually-contiguous
|
|
buffers are convenient.
|
|
|
|
- Buffers which are physically contiguous. Allocation of this kind of
|
|
buffer can be unreliable on fragmented systems, but simpler DMA
|
|
controllers cannot deal with anything else.
|
|
|
|
Videobuf can work with all three types of buffers, but the driver author
|
|
must pick one at the outset and design the driver around that decision.
|
|
|
|
[It's worth noting that there's a fourth kind of buffer: "overlay" buffers
|
|
which are located within the system's video memory. The overlay
|
|
functionality is considered to be deprecated for most use, but it still
|
|
shows up occasionally in system-on-chip drivers where the performance
|
|
benefits merit the use of this technique. Overlay buffers can be handled
|
|
as a form of scattered buffer, but there are very few implementations in
|
|
the kernel and a description of this technique is currently beyond the
|
|
scope of this document.]
|
|
|
|
Data structures, callbacks, and initialization
|
|
----------------------------------------------
|
|
|
|
Depending on which type of buffers are being used, the driver should
|
|
include one of the following files:
|
|
|
|
.. code-block:: none
|
|
|
|
<media/videobuf-dma-sg.h> /* Physically scattered */
|
|
<media/videobuf-vmalloc.h> /* vmalloc() buffers */
|
|
<media/videobuf-dma-contig.h> /* Physically contiguous */
|
|
|
|
The driver's data structure describing a V4L2 device should include a
|
|
struct videobuf_queue instance for the management of the buffer queue,
|
|
along with a list_head for the queue of available buffers. There will also
|
|
need to be an interrupt-safe spinlock which is used to protect (at least)
|
|
the queue.
|
|
|
|
The next step is to write four simple callbacks to help videobuf deal with
|
|
the management of buffers:
|
|
|
|
.. code-block:: none
|
|
|
|
struct videobuf_queue_ops {
|
|
int (*buf_setup)(struct videobuf_queue *q,
|
|
unsigned int *count, unsigned int *size);
|
|
int (*buf_prepare)(struct videobuf_queue *q,
|
|
struct videobuf_buffer *vb,
|
|
enum v4l2_field field);
|
|
void (*buf_queue)(struct videobuf_queue *q,
|
|
struct videobuf_buffer *vb);
|
|
void (*buf_release)(struct videobuf_queue *q,
|
|
struct videobuf_buffer *vb);
|
|
};
|
|
|
|
buf_setup() is called early in the I/O process, when streaming is being
|
|
initiated; its purpose is to tell videobuf about the I/O stream. The count
|
|
parameter will be a suggested number of buffers to use; the driver should
|
|
check it for rationality and adjust it if need be. As a practical rule, a
|
|
minimum of two buffers are needed for proper streaming, and there is
|
|
usually a maximum (which cannot exceed 32) which makes sense for each
|
|
device. The size parameter should be set to the expected (maximum) size
|
|
for each frame of data.
|
|
|
|
Each buffer (in the form of a struct videobuf_buffer pointer) will be
|
|
passed to buf_prepare(), which should set the buffer's size, width, height,
|
|
and field fields properly. If the buffer's state field is
|
|
VIDEOBUF_NEEDS_INIT, the driver should pass it to:
|
|
|
|
.. code-block:: none
|
|
|
|
int videobuf_iolock(struct videobuf_queue* q, struct videobuf_buffer *vb,
|
|
struct v4l2_framebuffer *fbuf);
|
|
|
|
Among other things, this call will usually allocate memory for the buffer.
|
|
Finally, the buf_prepare() function should set the buffer's state to
|
|
VIDEOBUF_PREPARED.
|
|
|
|
When a buffer is queued for I/O, it is passed to buf_queue(), which should
|
|
put it onto the driver's list of available buffers and set its state to
|
|
VIDEOBUF_QUEUED. Note that this function is called with the queue spinlock
|
|
held; if it tries to acquire it as well things will come to a screeching
|
|
halt. Yes, this is the voice of experience. Note also that videobuf may
|
|
wait on the first buffer in the queue; placing other buffers in front of it
|
|
could again gum up the works. So use list_add_tail() to enqueue buffers.
|
|
|
|
Finally, buf_release() is called when a buffer is no longer intended to be
|
|
used. The driver should ensure that there is no I/O active on the buffer,
|
|
then pass it to the appropriate free routine(s):
|
|
|
|
.. code-block:: none
|
|
|
|
/* Scatter/gather drivers */
|
|
int videobuf_dma_unmap(struct videobuf_queue *q,
|
|
struct videobuf_dmabuf *dma);
|
|
int videobuf_dma_free(struct videobuf_dmabuf *dma);
|
|
|
|
/* vmalloc drivers */
|
|
void videobuf_vmalloc_free (struct videobuf_buffer *buf);
|
|
|
|
/* Contiguous drivers */
|
|
void videobuf_dma_contig_free(struct videobuf_queue *q,
|
|
struct videobuf_buffer *buf);
|
|
|
|
One way to ensure that a buffer is no longer under I/O is to pass it to:
|
|
|
|
.. code-block:: none
|
|
|
|
int videobuf_waiton(struct videobuf_buffer *vb, int non_blocking, int intr);
|
|
|
|
Here, vb is the buffer, non_blocking indicates whether non-blocking I/O
|
|
should be used (it should be zero in the buf_release() case), and intr
|
|
controls whether an interruptible wait is used.
|
|
|
|
File operations
|
|
---------------
|
|
|
|
At this point, much of the work is done; much of the rest is slipping
|
|
videobuf calls into the implementation of the other driver callbacks. The
|
|
first step is in the open() function, which must initialize the
|
|
videobuf queue. The function to use depends on the type of buffer used:
|
|
|
|
.. code-block:: none
|
|
|
|
void videobuf_queue_sg_init(struct videobuf_queue *q,
|
|
struct videobuf_queue_ops *ops,
|
|
struct device *dev,
|
|
spinlock_t *irqlock,
|
|
enum v4l2_buf_type type,
|
|
enum v4l2_field field,
|
|
unsigned int msize,
|
|
void *priv);
|
|
|
|
void videobuf_queue_vmalloc_init(struct videobuf_queue *q,
|
|
struct videobuf_queue_ops *ops,
|
|
struct device *dev,
|
|
spinlock_t *irqlock,
|
|
enum v4l2_buf_type type,
|
|
enum v4l2_field field,
|
|
unsigned int msize,
|
|
void *priv);
|
|
|
|
void videobuf_queue_dma_contig_init(struct videobuf_queue *q,
|
|
struct videobuf_queue_ops *ops,
|
|
struct device *dev,
|
|
spinlock_t *irqlock,
|
|
enum v4l2_buf_type type,
|
|
enum v4l2_field field,
|
|
unsigned int msize,
|
|
void *priv);
|
|
|
|
In each case, the parameters are the same: q is the queue structure for the
|
|
device, ops is the set of callbacks as described above, dev is the device
|
|
structure for this video device, irqlock is an interrupt-safe spinlock to
|
|
protect access to the data structures, type is the buffer type used by the
|
|
device (cameras will use V4L2_BUF_TYPE_VIDEO_CAPTURE, for example), field
|
|
describes which field is being captured (often V4L2_FIELD_NONE for
|
|
progressive devices), msize is the size of any containing structure used
|
|
around struct videobuf_buffer, and priv is a private data pointer which
|
|
shows up in the priv_data field of struct videobuf_queue. Note that these
|
|
are void functions which, evidently, are immune to failure.
|
|
|
|
V4L2 capture drivers can be written to support either of two APIs: the
|
|
read() system call and the rather more complicated streaming mechanism. As
|
|
a general rule, it is necessary to support both to ensure that all
|
|
applications have a chance of working with the device. Videobuf makes it
|
|
easy to do that with the same code. To implement read(), the driver need
|
|
only make a call to one of:
|
|
|
|
.. code-block:: none
|
|
|
|
ssize_t videobuf_read_one(struct videobuf_queue *q,
|
|
char __user *data, size_t count,
|
|
loff_t *ppos, int nonblocking);
|
|
|
|
ssize_t videobuf_read_stream(struct videobuf_queue *q,
|
|
char __user *data, size_t count,
|
|
loff_t *ppos, int vbihack, int nonblocking);
|
|
|
|
Either one of these functions will read frame data into data, returning the
|
|
amount actually read; the difference is that videobuf_read_one() will only
|
|
read a single frame, while videobuf_read_stream() will read multiple frames
|
|
if they are needed to satisfy the count requested by the application. A
|
|
typical driver read() implementation will start the capture engine, call
|
|
one of the above functions, then stop the engine before returning (though a
|
|
smarter implementation might leave the engine running for a little while in
|
|
anticipation of another read() call happening in the near future).
|
|
|
|
The poll() function can usually be implemented with a direct call to:
|
|
|
|
.. code-block:: none
|
|
|
|
unsigned int videobuf_poll_stream(struct file *file,
|
|
struct videobuf_queue *q,
|
|
poll_table *wait);
|
|
|
|
Note that the actual wait queue eventually used will be the one associated
|
|
with the first available buffer.
|
|
|
|
When streaming I/O is done to kernel-space buffers, the driver must support
|
|
the mmap() system call to enable user space to access the data. In many
|
|
V4L2 drivers, the often-complex mmap() implementation simplifies to a
|
|
single call to:
|
|
|
|
.. code-block:: none
|
|
|
|
int videobuf_mmap_mapper(struct videobuf_queue *q,
|
|
struct vm_area_struct *vma);
|
|
|
|
Everything else is handled by the videobuf code.
|
|
|
|
The release() function requires two separate videobuf calls:
|
|
|
|
.. code-block:: none
|
|
|
|
void videobuf_stop(struct videobuf_queue *q);
|
|
int videobuf_mmap_free(struct videobuf_queue *q);
|
|
|
|
The call to videobuf_stop() terminates any I/O in progress - though it is
|
|
still up to the driver to stop the capture engine. The call to
|
|
videobuf_mmap_free() will ensure that all buffers have been unmapped; if
|
|
so, they will all be passed to the buf_release() callback. If buffers
|
|
remain mapped, videobuf_mmap_free() returns an error code instead. The
|
|
purpose is clearly to cause the closing of the file descriptor to fail if
|
|
buffers are still mapped, but every driver in the 2.6.32 kernel cheerfully
|
|
ignores its return value.
|
|
|
|
ioctl() operations
|
|
------------------
|
|
|
|
The V4L2 API includes a very long list of driver callbacks to respond to
|
|
the many ioctl() commands made available to user space. A number of these
|
|
- those associated with streaming I/O - turn almost directly into videobuf
|
|
calls. The relevant helper functions are:
|
|
|
|
.. code-block:: none
|
|
|
|
int videobuf_reqbufs(struct videobuf_queue *q,
|
|
struct v4l2_requestbuffers *req);
|
|
int videobuf_querybuf(struct videobuf_queue *q, struct v4l2_buffer *b);
|
|
int videobuf_qbuf(struct videobuf_queue *q, struct v4l2_buffer *b);
|
|
int videobuf_dqbuf(struct videobuf_queue *q, struct v4l2_buffer *b,
|
|
int nonblocking);
|
|
int videobuf_streamon(struct videobuf_queue *q);
|
|
int videobuf_streamoff(struct videobuf_queue *q);
|
|
|
|
So, for example, a VIDIOC_REQBUFS call turns into a call to the driver's
|
|
vidioc_reqbufs() callback which, in turn, usually only needs to locate the
|
|
proper struct videobuf_queue pointer and pass it to videobuf_reqbufs().
|
|
These support functions can replace a great deal of buffer management
|
|
boilerplate in a lot of V4L2 drivers.
|
|
|
|
The vidioc_streamon() and vidioc_streamoff() functions will be a bit more
|
|
complex, of course, since they will also need to deal with starting and
|
|
stopping the capture engine.
|
|
|
|
Buffer allocation
|
|
-----------------
|
|
|
|
Thus far, we have talked about buffers, but have not looked at how they are
|
|
allocated. The scatter/gather case is the most complex on this front. For
|
|
allocation, the driver can leave buffer allocation entirely up to the
|
|
videobuf layer; in this case, buffers will be allocated as anonymous
|
|
user-space pages and will be very scattered indeed. If the application is
|
|
using user-space buffers, no allocation is needed; the videobuf layer will
|
|
take care of calling get_user_pages() and filling in the scatterlist array.
|
|
|
|
If the driver needs to do its own memory allocation, it should be done in
|
|
the vidioc_reqbufs() function, *after* calling videobuf_reqbufs(). The
|
|
first step is a call to:
|
|
|
|
.. code-block:: none
|
|
|
|
struct videobuf_dmabuf *videobuf_to_dma(struct videobuf_buffer *buf);
|
|
|
|
The returned videobuf_dmabuf structure (defined in
|
|
<media/videobuf-dma-sg.h>) includes a couple of relevant fields:
|
|
|
|
.. code-block:: none
|
|
|
|
struct scatterlist *sglist;
|
|
int sglen;
|
|
|
|
The driver must allocate an appropriately-sized scatterlist array and
|
|
populate it with pointers to the pieces of the allocated buffer; sglen
|
|
should be set to the length of the array.
|
|
|
|
Drivers using the vmalloc() method need not (and cannot) concern themselves
|
|
with buffer allocation at all; videobuf will handle those details. The
|
|
same is normally true of contiguous-DMA drivers as well; videobuf will
|
|
allocate the buffers (with dma_alloc_coherent()) when it sees fit. That
|
|
means that these drivers may be trying to do high-order allocations at any
|
|
time, an operation which is not always guaranteed to work. Some drivers
|
|
play tricks by allocating DMA space at system boot time; videobuf does not
|
|
currently play well with those drivers.
|
|
|
|
As of 2.6.31, contiguous-DMA drivers can work with a user-supplied buffer,
|
|
as long as that buffer is physically contiguous. Normal user-space
|
|
allocations will not meet that criterion, but buffers obtained from other
|
|
kernel drivers, or those contained within huge pages, will work with these
|
|
drivers.
|
|
|
|
Filling the buffers
|
|
-------------------
|
|
|
|
The final part of a videobuf implementation has no direct callback - it's
|
|
the portion of the code which actually puts frame data into the buffers,
|
|
usually in response to interrupts from the device. For all types of
|
|
drivers, this process works approximately as follows:
|
|
|
|
- Obtain the next available buffer and make sure that somebody is actually
|
|
waiting for it.
|
|
|
|
- Get a pointer to the memory and put video data there.
|
|
|
|
- Mark the buffer as done and wake up the process waiting for it.
|
|
|
|
Step (1) above is done by looking at the driver-managed list_head structure
|
|
- the one which is filled in the buf_queue() callback. Because starting
|
|
the engine and enqueueing buffers are done in separate steps, it's possible
|
|
for the engine to be running without any buffers available - in the
|
|
vmalloc() case especially. So the driver should be prepared for the list
|
|
to be empty. It is equally possible that nobody is yet interested in the
|
|
buffer; the driver should not remove it from the list or fill it until a
|
|
process is waiting on it. That test can be done by examining the buffer's
|
|
done field (a wait_queue_head_t structure) with waitqueue_active().
|
|
|
|
A buffer's state should be set to VIDEOBUF_ACTIVE before being mapped for
|
|
DMA; that ensures that the videobuf layer will not try to do anything with
|
|
it while the device is transferring data.
|
|
|
|
For scatter/gather drivers, the needed memory pointers will be found in the
|
|
scatterlist structure described above. Drivers using the vmalloc() method
|
|
can get a memory pointer with:
|
|
|
|
.. code-block:: none
|
|
|
|
void *videobuf_to_vmalloc(struct videobuf_buffer *buf);
|
|
|
|
For contiguous DMA drivers, the function to use is:
|
|
|
|
.. code-block:: none
|
|
|
|
dma_addr_t videobuf_to_dma_contig(struct videobuf_buffer *buf);
|
|
|
|
The contiguous DMA API goes out of its way to hide the kernel-space address
|
|
of the DMA buffer from drivers.
|
|
|
|
The final step is to set the size field of the relevant videobuf_buffer
|
|
structure to the actual size of the captured image, set state to
|
|
VIDEOBUF_DONE, then call wake_up() on the done queue. At this point, the
|
|
buffer is owned by the videobuf layer and the driver should not touch it
|
|
again.
|
|
|
|
Developers who are interested in more information can go into the relevant
|
|
header files; there are a few low-level functions declared there which have
|
|
not been talked about here. Also worthwhile is the vivi driver
|
|
(drivers/media/platform/vivi.c), which is maintained as an example of how V4L2
|
|
drivers should be written. Vivi only uses the vmalloc() API, but it's good
|
|
enough to get started with. Note also that all of these calls are exported
|
|
GPL-only, so they will not be available to non-GPL kernel modules.
|