media: Documentation: add multiplexed streams documentation
Add documentation related to multiplexed streams. Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
This commit is contained in:
Tomi Valkeinen
Mauro Carvalho Chehab
parent
837f92f070
commit
70283e99a9
|
|
@ -593,6 +593,14 @@ before calling v4l2_subdev_init_finalize():
|
|||
|
||||
This shares the driver's private mutex between the controls and the states.
|
||||
|
||||
Streams, multiplexed media pads and internal routing
|
||||
----------------------------------------------------
|
||||
|
||||
A subdevice driver can implement support for multiplexed streams by setting
|
||||
the V4L2_SUBDEV_FL_STREAMS subdev flag and implementing support for
|
||||
centrally managed subdev active state, routing and stream based
|
||||
configuration.
|
||||
|
||||
V4L2 sub-device functions and data structures
|
||||
---------------------------------------------
|
||||
|
||||
|
|
|
|||
|
|
@ -503,3 +503,141 @@ source pads.
|
|||
:maxdepth: 1
|
||||
|
||||
subdev-formats
|
||||
|
||||
Streams, multiplexed media pads and internal routing
|
||||
----------------------------------------------------
|
||||
|
||||
Commonly V4L2 subdevices support only separate video streams, that is, only a
|
||||
single stream can pass through a media link and a media pad. Thus each pad
|
||||
contains a format configuration for that single stream. In some cases a subdev
|
||||
can do stream processing and split a stream into two or compose two streams
|
||||
into one, but the inputs and outputs for the subdev are still a single stream
|
||||
per pad.
|
||||
|
||||
Some hardware, e.g. MIPI CSI-2, support multiplexed streams, that is, multiple
|
||||
data streams are transmitted on the same bus, which is represented by a media
|
||||
link connecting a transmitter source pad with a sink pad on the receiver. For
|
||||
example, a camera sensor can produce two distinct streams, a pixel stream and a
|
||||
metadata stream, which are transmitted on the multiplexed data bus, represented
|
||||
by a media link which connects the single sensor's source pad with the receiver
|
||||
sink pad. The stream-aware receiver will de-multiplex the streams received on
|
||||
the its sink pad and allows to route them individually to one of its source
|
||||
pads.
|
||||
|
||||
Subdevice drivers that support multiplexed streams are compatible with
|
||||
non-multiplexed subdev drivers, but, of course, require a routing configuration
|
||||
where the link between those two types of drivers contains only a single
|
||||
stream.
|
||||
|
||||
Understanding streams
|
||||
^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
A stream is a stream of content (e.g. pixel data or metadata) flowing through
|
||||
the media pipeline from a source (e.g. a sensor) towards the final sink (e.g. a
|
||||
receiver and demultiplexer in a SoC). Each media link carries all the enabled
|
||||
streams from one end of the link to the other, and subdevices have routing
|
||||
tables which describe how the incoming streams from sink pads are routed to the
|
||||
source pads.
|
||||
|
||||
A stream ID (often just "stream") is a media link-local identifier for a stream.
|
||||
In other words, a particular stream ID must exist on both sides of a media
|
||||
link, but another stream ID can be used for the same stream at the other side
|
||||
of the subdevice.
|
||||
|
||||
A stream at a specific point in the media pipeline is identified with the
|
||||
subdev and a (pad, stream) pair. For subdevices that do not support
|
||||
multiplexed streams the 'stream' is always 0.
|
||||
|
||||
Configuring streams
|
||||
^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
The configuration of the streams is done individually for each subdevice and
|
||||
the validity of the streams between subdevices is validated when the pipeline
|
||||
is started.
|
||||
|
||||
There are three steps in configuring the streams:
|
||||
|
||||
1) Set up links. Connect the pads between subdevices using the :ref:`Media
|
||||
Controller API <media_controller>`
|
||||
|
||||
2) Routing. The routing table for the subdevice must be set with
|
||||
:ref:`VIDIOC_SUBDEV_S_ROUTING <VIDIOC_SUBDEV_G_ROUTING>` ioctl. Note that
|
||||
setting the routing table will reset all the stream configurations in a media
|
||||
entity.
|
||||
|
||||
3) Configure streams. Each route endpoint must be configured
|
||||
with :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>`.
|
||||
|
||||
Multiplexed streams setup example
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
A simple example of a multiplexed stream setup might be as follows:
|
||||
|
||||
- Two identical sensors (Sensor A and Sensor B). Each sensor has a single source
|
||||
pad (pad 0) which carries a pixel data stream.
|
||||
|
||||
- Multiplexer bridge (Bridge). The bridge has two sink pads, connected to the
|
||||
sensors (pads 0, 1), and one source pad (pad 2), which outputs two streams.
|
||||
|
||||
- Receiver in the SoC (Receiver). The receiver has a single sink pad (pad 0),
|
||||
connected to the bridge, and two source pads (pads 1-2), going to the DMA
|
||||
engine. The receiver demultiplexes the incoming streams to the source pads.
|
||||
|
||||
- DMA Engines in the SoC (DMA Engine), one for each stream. Each DMA engine is
|
||||
connected to a single source pad in the receiver.
|
||||
|
||||
The sensors, the bridge and the receiver are modeled as V4L2 subdevices,
|
||||
exposed to userspace via /dev/v4l-subdevX device nodes. The DMA engines are
|
||||
modeled as V4L2 devices, exposed to userspace via /dev/videoX nodes.
|
||||
|
||||
To configure this pipeline, the userspace must take the following steps:
|
||||
|
||||
1) Set up media links between entities: connect the sensors to the bridge,
|
||||
bridge to the receiver, and the receiver to the DMA engines. This step does
|
||||
not differ from normal non-multiplexed media controller setup.
|
||||
|
||||
2) Configure routing.
|
||||
|
||||
.. flat-table:: Bridge routing table
|
||||
:header-rows: 1
|
||||
|
||||
* - Sink Pad/Stream
|
||||
- Source Pad/Stream
|
||||
- Routing Flags
|
||||
- Comments
|
||||
* - 0/0
|
||||
- 2/0
|
||||
- V4L2_SUBDEV_ROUTE_FL_ACTIVE
|
||||
- Pixel data stream from Sensor A
|
||||
* - 1/0
|
||||
- 2/1
|
||||
- V4L2_SUBDEV_ROUTE_FL_ACTIVE
|
||||
- Pixel data stream from Sensor B
|
||||
|
||||
.. flat-table:: Receiver routing table
|
||||
:header-rows: 1
|
||||
|
||||
* - Sink Pad/Stream
|
||||
- Source Pad/Stream
|
||||
- Routing Flags
|
||||
- Comments
|
||||
* - 0/0
|
||||
- 1/0
|
||||
- V4L2_SUBDEV_ROUTE_FL_ACTIVE
|
||||
- Pixel data stream from Sensor A
|
||||
* - 0/1
|
||||
- 2/0
|
||||
- V4L2_SUBDEV_ROUTE_FL_ACTIVE
|
||||
- Pixel data stream from Sensor B
|
||||
|
||||
3) Configure streams
|
||||
|
||||
After configuring the routing table, the next step is configuring the streams.
|
||||
This step is similar to configuring the pads in a non-multiplexed streams
|
||||
setup, with the difference that we need to configure each (pad, stream) pair
|
||||
(i.e. route endpoint) instead of just a pad.
|
||||
|
||||
A common way to accomplish this is to start from the sensors and propagate the
|
||||
configurations along the stream towards the receiver,
|
||||
using :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls to configure each
|
||||
stream endpoint in each subdev.
|
||||
|
|
|
|||
Loading…
Reference in New Issue