The vui parameters are optional. However, the vui data allows to specify
the color space of the encoded video. Write the vui parameters to make
sure that decoders are able to pick up the correct color space.
Also implement the necessary lookup functions to convert the values from
the V4L2 controls to the values specified in the hevc standard.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Currently, the driver always writes PAL as video format into the SPS of
the encoded stream.
Set the video format to the default value 5 (unspecified) and use the
color description that is already configured on the channel as color
space. This fixes the color space definition in the coded data to
reflect the configured color space of the video data that is encoded.
Add lookup functions to convert the color primaries, transfer function
and matrix coefficients from the V4L2 control values to the values
specified in the h.264 standard.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
The lookup of the h.264 or hevc values for the respective V4L2 controls
is done by the driver that uses the sps/pps generator and not in the
generator. Therefore, it is more intuitive to define these functions
directly in the header and not in the module.
Extract the functions to the headers as static inline functions.
Also simplify the function name and add kernel-doc for the hevc
functions.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
The bit rate in the SPS can be scaled by an exponent, which allows to
reduce the number of bits in the SPS in case of high bit rates.
The driver did not scale the bitrate, but used a scaling exponent of 0.
Fix this by properly calculating the scaling factor and writing the bit
rate as value and scaling factor into the SPS.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
The VCU allows to specify the QP per frame and coding unit. A buffer
that specifies the QP is passed via the ep2 field in the ENCODE_FRAME
message.
The driver currently does not support the external QP table. Simplify
the driver by not setting the ep2 fields at all.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
The encoder buffer can have a negative impact on the quality of the
encoded video.
Add a control to allow user space to disable the encoder buffer per
channel if the VCU supports the encoder buffer but the quality is not
sufficient.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
The encoder buffer serves as a cache for reference frames during the
encoding process. The encoder buffer significantly reduces the bandwidth
requirement for read accesses on the AXI ports of the VCU, but slightly
reduces the quality of the encoded video.
The encoder buffer must be configured as a whole during the firmware
initialization and later explicitly enabled for every channel that shall
use the encoder buffer.
Prior to firmware version 2019.2, it was necessary to explicitly set the
size of the encoder buffer for every channel. Since 2019.2 it is
sufficient to enable the encoder buffer and leave the rest to the
firmware. Therefore, only support the encoder buffer for firmware 2019.2
and later.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
The allegro driver must ensure that the mcu and core clocks are enabled
and set to the expected clock rate before trying to load the firmware
and to reset the MCU.
Up until now, the driver assumed that the clocks are always enabled in
the PL (programming logic), because the xlnx_vcu driver did not export
the clocks to other drivers. This has changed and by explicitly enabling
the clocks in the driver, this assumption can be dropped.
It might even be possible to disable the clocks for the encoder if the
encoder is not used. However, the behavior is not documented and it
might be necessary to reinitialize the encoder after deactivating the
clocks. Play it safe by sticking to the current behavior.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
The VCU provides information about its configuration in a dedicated
register space. These settings include, for example, the expected clock
rates and the configuration of the encoder buffer. In the device tree,
the settings are described by the "xlnx,vcu-settings" compatible.
The settings are needed to correctly configure the clocks and the
encoder buffer.
Lookup the VCU settings in the device tree and make it accessible to the
driver via a regmap.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
If the module probe finished, but the firmware initialization failed,
removing the module must not revert the firmware initialization.
Add a field to track the status of the firmware initialization and only
roll it back, if the firmware was successfully initialized.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
The mailbox is initialized after the interrupt handler is installed. As
the firmware is loaded and started even later, it should not happen that
the interrupt occurs without the mailbox being initialized.
As the Linux Driver Verification project (linuxtesting.org) keeps
reporting this as an error, add a check to ignore interrupts before the
mailbox is initialized to fix this potential null pointer dereference.
Reported-by: Yuri Savinykh <s02190703@gse.cs.msu.ru>
Reported-by: Nadezda Lutovinova <lutovinova@ispras.ru>
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
The Allegro Codec supports HEVC encoding. The messages to the MCU are
the same for H.264 and HEVC, but some options have to be changed. These
are actually only a few options.
The driver, however, must add the HEVC VPS/SPS/PPS NAL Units to the
coded stream and must properly provide the HEVC format and controls to
user space.
[hverkuil: fix warning for wrong enum type (h264 instead of hevc)]
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
These controls are specific to H.264 and shall only be activated, if the
coded format is H.264.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
The HEVC Non-VCL NAL units require more space than the H.264 Non-VCL NAL
units. Therefore, the driver needs to reserve more space in front of the
actual coded data that is written by the hardware codec.
Increase the offset that shall be used by the hardware codec from 64
bytes to 128 bytes which is as arbitrary as before.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
The maximum bitrate and the size of the CPB are dependent on the level.
As the levels differ between the different codecs, the functions to get
the maximum bitrate and CPB must be codec specific, too.
Rename the functions to make it obvious that the function are only valid
for H.264.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
The driver only supports CAVLC for H.264, but HEVC only uses CABAC. As
the driver has to explicitly tell the MCU to use CABAC for HEVC, add a
helper function to get the entropy mode.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
V4L2 specifies different controls for the QP values of different codecs.
Simplify users that just care for the QP values by providing accessor
function that return the correct control based on the currently set
codec.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
The profile and level are straight copies from the v4l2-ctrls. Avoid
duplication and directly read the value of the v4l2-ctrl.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
The cpb_size and the gop_size are straight copies of the values in the
v4l2-ctrls. To avoid this duplication, directly get the values from the
v4l2-ctrls and remove the copies.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
The cbp_size_value_minus1 in the SPS can scaled using cpb_size_scale to
reduce the number of bits necessary to encode the value. For simplicity,
the scaling was set to 1.
Restructure to the code to make it easier to drop the cbp_size from the
channel and as we are at it, also properly implement the scaling.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
As the driver now uses the v4l2-ctrls to setup the channel, there is no
need to explicitly set the bitrate. The initial bitrate is now set via
the same path as if it is set from userspace using the v4l2-ctrl.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
v4l2_ctrl_handler_setup() calls s_ctrl for all controls of the handler.
This ensures that the channel is initialized using the default values of
the v4l2-ctrls.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
There are several encoding options that are hard coded in the parameter
that is used to configure the hardware codec. However, some of the
options must be written to the SPS/PPS by the driver. Furthermore, some
of the options depend on the codec that is used (i.e. H.264 or HEVC).
Therefore, move options that depend on the codec to the channel and add
constants for options that are independent of the codec but must be
written to the SPS/PPS.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
A format change (i.e. the frame size or the codec) has multiple effects
on a channel:
- The available controls
- The limits of the available controls
- The default encoding options
To avoid scattering the changes all over the driver, add a new function
that goes over the channel and does all required adjustments.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
In order to support different codecs, the driver must support changing
the format on CAPTURE. Therefore, the driver needs to handle S_FMT on
CAPTURE.
As the driver will have a different number of formats for OUTPUT and
CAPTURE, split the check for the format index in ENUM_FMT into CAPTURE
and OUTPUT.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
The dst_handle field in the PUT_STREAM_BUFFER and ENCODE_FRAME_RESPONSE
is used to retrieve the V4L2 CAPTURE buffer. Calling it stream_id is
confusing. Call it dst_handle inspired by src_handle for the OUTPUT
buffer.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
The v4l2-m2m has various helpers for correctly handle the draining. Drop
the driver specific state machine and use the m2m helper functions.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
The stateful encoder API was finalized. Nothing is blocking the driver
from being moved out of staging.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Michael Tretter