Commit Graph

3376 Commits

Author SHA1 Message Date
Nicolas Dufresne 340ce50f75 media: hantro: Enable HOLD_CAPTURE_BUF for H.264
This is needed to optimize field decoding. Each field will be
decoded into the same capture buffer. To be able to queue multiple
buffers, we need to be able to ask the driver to hold the capture
buffer.

Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-17 10:11:19 +02:00
Nicolas Dufresne 11442b7c93 media: hantro: Add H.264 field decoding support
This adds the required code to support field decoding. While most of
the code is derived from Rockchip and VSI reference code, the
reduction of the reference list to 16 entries was found by
trial and errors. The list consists of all the references with the
opposite field parity.

The strategy is to deduplicate the reference picture that points
to the same storage (same index). The choice of opposite parity has
been made to keep the other field of the current field pair in the
list. This method may not be robust if a field was lost.

[hverkuil: fix typos in the comment before deduplicate_reflist()]
[hverkuil: document new cur_poc field]

Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-17 10:10:46 +02:00
Jonas Karlman 3630e4933d media: hantro: h264: Make dpb entry management more robust
The driver maintains stable slot locations for reference pictures. This
change makes the code more robust by using the reference_ts as key and
by marking all entries invalid right from the start.

Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-17 10:09:48 +02:00
Nicolas Dufresne 8314107009 media: hantro: Stop using H.264 parameter pic_num
The hardware expects FrameNumWrap or long_term_frame_idx. Picture
numbers are per field, and are mostly used during the memory
management process, which is done in userland. This fixes two
ITU conformance tests:

  - MR6_BT_B
  - MR8_BT_B

Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-17 10:09:10 +02:00
Nicolas Dufresne ed7bb87d3d media: rkvdec: Enable capture buffer holding for H264
In order to support interlaced video decoding, the driver must
allow holding the capture buffer so that the second field can
be decoded into it.

Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-17 10:08:52 +02:00
Nicolas Dufresne 6f32ea37c1 media: rkvdec-h264: Add field decoding support
This makes use of the new feature in the reference builder to program
up to 32 references when doing field decoding. It also signals the
parity (top or bottom) of the field to the hardware.

Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-17 10:07:49 +02:00
Jonas Karlman 5e57a860df media: rkvdec: Ensure decoded resolution fit coded resolution
Ensure decoded CAPTURE buffer resolution is larger or equal to the coded
OUTPUT buffer resolution.

Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-17 10:07:08 +02:00
Jonas Karlman f942d10dab media: rkvdec: h264: Fix reference frame_num wrap for second field
When decoding the second field in a complementary field pair the second
field is sharing the same frame_num with the first field.

Currently the frame_num for the first field is wrapped when it matches the
field being decoded, this caused issues decoding the second field in a
complementary field pair.

Fix this by using inclusive comparison: 'less than or equal'.

Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-17 10:06:49 +02:00
Jonas Karlman 77e74be830 media: rkvdec: h264: Validate and use pic width and height in mbs
The width and height in macroblocks is currently configured based on OUTPUT
buffer resolution, this works for frame pictures but can cause issues for
field pictures.

When frame_mbs_only_flag is 0 the height in mbs should be height of
the field instead of height of frame.

Validate pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1
against OUTPUT buffer resolution and use these values to configure HW.
The validation is happening in both try_ctrt() and start() since it is
otherwise possible to trick the driver during initialization by changing
the OUTPUT format after having set a valid control.

[hverkuil: when -> When (first word in a comment block)]

Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-17 10:06:31 +02:00
Nicolas Dufresne cf76bb4d5e media: rkvdec: Move H264 SPS validation in rkvdec-h264
No functional change, this moves H264 specific validation into the H264
specific code. This is in preparation of improving this validation and
reusing it when VIDIOC_STREAMON is called.

Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-17 10:06:08 +02:00
Jonas Karlman a074aa4760 media: rkvdec: h264: Fix bit depth wrap in pps packet
The luma and chroma bit depth fields in the pps packet are 3 bits wide.
8 is wrongly added to the bit depth values written to these 3 bit fields.
Because only the 3 LSB are written, the hardware was configured
correctly.

Correct this by not adding 8 to the luma and chroma bit depth value.

Fixes: cd33c83044 ("media: rkvdec: Add the rkvdec driver")
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-17 10:05:41 +02:00
Nicolas Dufresne 7ab889f09d media: rkvdec: h264: Fix dpb_valid implementation
The ref builder only provided references that are marked as valid in the
dpb. Thus the current implementation of dpb_valid would always set the
flag to 1. This is not representing missing frames (this is called
'non-existing' pictures in the spec). In some context, these non-existing
pictures still need to occupy a slot in the reference list according to
the spec.

Fixes: cd33c83044 ("media: rkvdec: Add the rkvdec driver")
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-17 10:05:13 +02:00
Nicolas Dufresne 9998943f6d media: rkvdec: Stop overclocking the decoder
While this overclock hack seems to work on some implementations
(some ChromeBooks, RockPi4) it also causes instability on other
implementations (notably LibreComputer Renegade, but there were more
reports in the LibreELEC project, where this has been removed). While
performance is indeed affected (tested with GStreamer), 4K playback
still works as long as you don't operate in lock step and keep at
least 1 frame ahead of time in the decode queue.

After discussion with ChromeOS members, it would seem that their
implementation indeed used to synchronously decode each frame, so
this hack was simply compensating for their code being less
efficient. In my opinion, this hack should not have been included
upstream.

Fixes: cd33c83044 ("media: rkvdec: Add the rkvdec driver")
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-17 10:04:35 +02:00
Nicolas Dufresne 26e4520509 media: h264: Increase reference lists size to 32
This is to accommodate support for field decoding, which splits the top
and the bottom references into the reference list.

Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-17 09:59:17 +02:00
Nicolas Dufresne 2e2c3d6c0e media: h264: Use v4l2_h264_reference for reflist
In preparation for adding field decoding support, convert the byte arrays
for reflist into array of struct v4l2_h264_reference. That struct will
allow us to mark which field of the reference picture is being referenced.

[hverkuil: top_field_order_cnt -> pic_order_count]

Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Tested-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-17 09:58:42 +02:00
Benjamin Gaignard 46c8365691 media: hantro: HEVC: unconditionnaly set pps_{cb/cr}_qp_offset values
Always set pps_cb_qp_offset and pps_cr_qp_offset values in Hantro/G2
register whatever is V4L2_HEVC_PPS_FLAG_PPS_SLICE_CHROMA_QP_OFFSETS_PRESENT
flag value.
The vendor code does the same to set these values.
This fixes conformance test CAINIT_G_SHARP_3.

Fluster HEVC score is increase by one with this patch.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-13 11:02:22 +02:00
Benjamin Gaignard 387d117695 media: hantro: HEVC: Fix reference frames management
PoC shall be int the range of -2^31 to 2^31 -1
(HEVC spec section 8.3.1 Decoding process for picture order count).
The current way to know if an entry in reference picture array is free
is to test if PoC = UNUSED_REF. Since UNUSED_REF is defined as '-1' that
could lead to decode issue if one PoC also equal '-1'.
PoC with value = '-1' exists in conformance test SLIST_B_Sony_9.

Change the way unused entries are managed in reference pictures array to
avoid using PoC to detect then.

This patch doesn't change fluster HEVC score.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-13 11:02:22 +02:00
Benjamin Gaignard d7f4149df8 media: hantro: HEVC: Fix tile info buffer value computation
Use pps->column_width_minus1[j] + 1 as value for the tile info buffer
instead of pps->column_width_minus1[j + 1].
The patch fixes DBLK_E_VIXS_2, DBLK_F_VIXS_2, DBLK_G_VIXS_2,
SAO_B_MediaTek_5, TILES_A_Cisco_2 and TILES_B_Cisco_1 tests in fluster.

Fixes: cb5dd5a0fa ("media: hantro: Introduce G2/HEVC decoder")
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-13 11:02:22 +02:00
Benjamin Gaignard 579846ec52 media: hantro: HEVC: Fix output frame chroma offset
Hantro decoder doesn't take care of the requested and aligned size
of the capture buffer.
Stop using the bitstream width/height and use capture frame size
stored in the context to get the correct values.

hantro_hevc_chroma_offset() and hantro_hevc_motion_vectors_offset()
are only used in hantro_g2_hevc_dec.c so take the opportunity
to move them here.

fluster HEVC score goes up from 77 to 85 successful tests (over 147)
with this patch.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-13 11:02:22 +02:00
Benjamin Gaignard 79c987de8b media: hantro: Use post processor scaling capacities
Hantro G2 post processor is able to down scale decoded frames
by a factor of 2, 4 or 8.
Add enum_framesizes() ops to postproc_ops structure to enumerate the
possible output sizes for a given input resolution.
For G2 post-processor use fsize->index (from 0 to 3) as power of 2
divisor. As described in v4l2 documentation return -EINVAL when scaling
down isn't possible.

fluster scores:
77/147 for HEVC
143/303 for VP9

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-13 11:02:22 +02:00
Mauro Carvalho Chehab fa1451374e media: atomisp: don't pass a pointer to a local variable
As warned by gcc 12.1:

	drivers/staging/media/atomisp/pci/runtime/rmgr/src/rmgr_vbuf.c: In function 'ia_css_rmgr_acq_vbuf':
	drivers/staging/media/atomisp/pci/runtime/rmgr/src/rmgr_vbuf.c:275:33: error: storing the address of local variable 'h' in '*handle' [-Werror=dangling-pointer=]
	  275 |                         *handle = &h;
	      |                         ~~~~~~~~^~~~
	drivers/staging/media/atomisp/pci/runtime/rmgr/src/rmgr_vbuf.c:257:40: note: 'h' declared here
	  257 |         struct ia_css_rmgr_vbuf_handle h;
	      |                                        ^
	drivers/staging/media/atomisp/pci/runtime/rmgr/src/rmgr_vbuf.c:257:40: note: 'handle' declared here
	cc1: all warnings being treated as errors

The logic uses a temporary struct to update the handler, but,
instead of copying the value to the pointer sent by the caller, it
replaces it with the content with a local variable. That's wrong, and
may lead the caller to use a weird value.

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-13 11:02:20 +02:00
Sebastian Fricke 6a5446f97b media: staging: media: rkvdec: Update TODO list
VP9 support has been added to the driver by commit f25709c4ff
("media: rkvdec: Add the VP9 backend").
And the VP9 uABI was merged with commit b88dbe38dc
("media: uapi: Add VP9 stateless decoder controls").

The remaining codec that keeps this driver in staging is HEVC.
Update the TODO list accordingly.

Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-13 11:02:19 +02:00
Sebastian Fricke 1951391137 media: staging: media: hantro: Update TODO list
VP8 has been added to the uABI by commit 363240ce1c
("media: uapi: move VP8 stateless controls out of staging")
VP9 has been added to the uABI by commit b88dbe38dc
("media: uapi: Add VP9 stateless decoder controls")
H264 has been added to the uABI by commit 46a309d275
("media: uapi: move H264 stateless controls out of staging")

The last remaining codec to be added to the uABI is HEVC.
Highlight these changes in the TODO list.

Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-13 11:02:19 +02:00
Sebastian Fricke d8f6f1c56d media: staging: media: hantro: Fix typos
Fix typos in comments within the Hantro driver.

Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-13 11:02:19 +02:00
Husni Faiz 868519693b media: av7110: fix prohibited spaces in switch statement
This patch fixes "space prohibited before that ':'" checkpatch error
in the switch statements.

Suggested-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Husni Faiz <ahamedhusni73@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-13 11:02:18 +02:00
Husni Faiz e673b25894 media: av7110: fix switch indentation
This patch fixes "switch and case should be at the same indent"
checkpatch error.

Signed-off-by: Husni Faiz <ahamedhusni73@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-05-13 11:02:18 +02:00
Chen-Yu Tsai 309373a357 media: hantro: Empty encoder capture buffers by default
The payload size for encoder capture buffers is set by the driver upon
finishing encoding each frame, based on the encoded length returned from
hardware, and whatever header and padding length used. Setting a
non-zero default serves no real purpose, and also causes issues if the
capture buffer is returned to userspace unused, confusing the
application.

Instead, always set the payload size to 0 for encoder capture buffers
when preparing them.

Fixes: 775fec6900 ("media: add Rockchip VPU JPEG encoder driver")
Fixes: 082aaecff3 ("media: hantro: Fix .buf_prepare")
Signed-off-by: Chen-Yu Tsai <wenst@chromium.org>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-04-24 08:52:45 +01:00
Chen-Yu Tsai daf3999c12 media: hantro: Implement support for encoder commands
The V4L2 stateful encoder uAPI specification requires that drivers
support the ENCODER_CMD ioctl to allow draining of buffers. This
however was not implemented, and causes issues for some userspace
applications.

Implement support for the ENCODER_CMD ioctl using v4l2-mem2mem helpers.
This is entirely based on existing code found in the vicodec test
driver.

Fixes: 775fec6900 ("media: add Rockchip VPU JPEG encoder driver")
Signed-off-by: Chen-Yu Tsai <wenst@chromium.org>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-04-24 08:52:26 +01:00
Piotr Oniszczuk 5f6bfab6da media: hantro: Add support for Hantro G1 on RK356x
RK356x has Hantro G1 video decoder capable to decode MPEG2/H.264/VP8
video formats.

This patch adds support for RK356x family in existing Hantro
video decoder kernel driver.

Tested on [1] with FFmpeg v4l2_request code taken from [2]
with MPEG2, H.642 and VP8 samples with results [3].

[1] https://github.com/warpme/minimyth2
[2] https://github.com/LibreELEC/LibreELEC.tv/blob/master/packages/multimedia/ffmpeg/patches/v4l2-request/ffmpeg-001-v4l2-request.patch
[3] https://github.com/warpme/minimyth2/blob/master/video-test-summary.txt

Signed-off-by: Piotr Oniszczuk <piotr.oniszczuk@gmail.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-04-24 08:51:44 +01:00
Kate Hsuan 3bf96b6eeb media: staging: media: ipu3: Fix AWB x_start position when rightmost stripe is used
An not calibrated x_start setting would result in an incorrect AWB grid
configuration on a sensor when only the rightmost stripe is used.

If the AWB grid coordinate is set to the rightmost stripe, for example,
x_start is greater than 640, the AWB grid will be at the rightmost edge of
the sensor. The AWB statistic will be fetched from the wrong place of the
sensor and results in an incorrect AWB estimation result. Therefore,
stripe offset should subtract from x_start to have a correct grid
configuration for the rightmost stripe.

Signed-off-by: Kate Hsuan <hpa@redhat.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-04-24 08:28:44 +01:00
Tomi Valkeinen ed647ea668 media: subdev: add subdev state locking
The V4L2 subdevs have managed without centralized locking for the state
(previously pad_config), as the try-state is supposedly safe (although I
believe two TRY ioctls for the same fd would race), and the
active-state, and its locking, is managed by the drivers internally.

We now have active-state in a centralized position, and need locking.
Strictly speaking the locking is only needed for new drivers that use
the new state, as the current drivers continue behaving as they used to.

However, active-state locking is complicated by the fact that currently
the real active-state of a subdev is split into multiple parts: the new
v4l2_subdev_state, subdev control state, and subdev's internal state.

In the future all these three states should be combined into one state
(the v4l2_subdev_state), and then a single lock for the state should be
sufficient.

But to solve the current split-state situation we need to share locks
between the three states. This is accomplished by using the same lock
management as the control handler does: we use a pointer to a mutex,
allowing the driver to override the default mutex. Thus the driver can
do e.g.:

sd->state_lock = sd->ctrl_handler->lock;

before calling v4l2_subdev_init_finalize(), resulting in sharing the
same lock between the states and the controls.

The locking model for active-state is such that any subdev op that gets
the state as a parameter expects the state to be already locked by the
caller, and expects the caller to release the lock.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-04-24 08:21:45 +01:00
Tomi Valkeinen 40aaab9d77 media: subdev: rename subdev-state alloc & free
v4l2_subdev_alloc_state() and v4l2_subdev_free_state() are not supposed
to be used by the drivers. However, we do have a few drivers that use
those at the moment, so we need to expose these functions for the time
being.

Prefix the functions with __ to mark the functions as internal.

At the same time, rename them to v4l2_subdev_state_alloc and
v4l2_subdev_state_free to match the style used for other functions like
video_device_alloc() and media_request_alloc().

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-04-24 08:16:20 +01:00
Umang Jain dbdc7237ec media: staging/intel-ipu3: Cleanup dummy buffers via helper
Use an existing helper imgu_video_nodes_exit() on imgu-video-node
initialization failure path to cleanup dummy buffers.

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-04-24 08:13:57 +01:00
Umang Jain 1ad037645a media: staging/intel-ipu3: Reset imgu_video_device sequence
Reset the sequence number of imgu_video_device on stream start.
Failing to do so results in sequence number getting incremented
for consecutive stream on/off cycles.

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-04-24 08:13:35 +01:00
Sakari Ailus 229fac6c44 media: staging: media: ipu3-imgu: Request specific firmware binary
Primarily request a specific revision of the IPU3 firmware that the driver
is known to work with,
irci_irci_ecr-master_20161208_0213_20170112_1500.bin. Some distros only
ship this while others provide a symlink called ipu3-fw.bin, which the
driver only requested previously.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-04-24 08:12:48 +01:00
Kate Hsuan 2425c81fc4 media: staging: media: ipu3: Fix AF x_start position when rightmost stripe is used
For the AF configuration, if the rightmost stripe is used, the AF scene
will be at the incorrect location of the sensor.

The AF coordinate may be set to the right part of the sensor. This
configuration would lead to x_start being greater than the
down_scaled_stripes offset and the leftmost stripe would be disabled
and only the rightmost stripe is used to control the AF coordinate. If
the x_start doesn't perform any adjustments, the AF coordinate will be
at the wrong place of the sensor since down_scaled_stripes offset
would be the new zero of the coordinate system.

In this patch, if only the rightmost stripe is used, x_start should
minus down_scaled_stripes offset to maintain its correctness of AF
scene coordinate.

Signed-off-by: Kate Hsuan <hpa@redhat.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-04-24 08:11:44 +01:00
Mauro Carvalho Chehab fc0b582c85 media: atomisp: fix bad usage at error handling logic
As warned by sparse:
	atomisp: drivers/staging/media/atomisp/pci/atomisp_acc.c:508 atomisp_acc_load_extensions() warn: iterator used outside loop: 'acc_fw'

The acc_fw interactor is used outside the loop, at the error handling
logic. On most cases, this is actually safe there, but, if
atomisp_css_set_acc_parameters() has an error, an attempt to use it
will pick an invalid value for acc_fw.

Reported-by: Hans Verkuil <hverkuil@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-03-18 05:58:35 +01:00
Mauro Carvalho Chehab 9958d30f38 media: Kconfig: cleanup VIDEO_DEV dependencies
media Kconfig has two entries associated to V4L API:
VIDEO_DEV and VIDEO_V4L2.

On Kernel 2.6.x, there were two V4L APIs, each one with its own flag.
VIDEO_DEV were meant to:
	1) enable Video4Linux and make its Kconfig options to appear;
	2) it makes the Kernel build the V4L core.

while VIDEO_V4L2 where used to distinguish between drivers that
implement the newer API and drivers that implemented the former one.

With time, such meaning changed, specially after the removal of
all V4L version 1 drivers.

At the current implementation, VIDEO_DEV only does (1): it enables
the media options related to V4L, that now has:

	menu "Video4Linux options"
		visible if VIDEO_DEV

	source "drivers/media/v4l2-core/Kconfig"
	endmenu

but it doesn't affect anymore the V4L core drivers.

The rationale is that the V4L2 core has a "soft" dependency
at the I2C bus, and now requires to select a number of other
Kconfig options:

	config VIDEO_V4L2
		tristate
		depends on (I2C || I2C=n) && VIDEO_DEV
		select RATIONAL
		select VIDEOBUF2_V4L2 if VIDEOBUF2_CORE
		default (I2C || I2C=n) && VIDEO_DEV

In the past, merging them would be tricky, but it seems that it is now
possible to merge those symbols, in order to simplify V4L dependencies.

Let's keep VIDEO_DEV, as this one is used on some make *defconfig
configurations.

Suggested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Neil Armstrong <narmstrong@baylibre.com> # for meson-vdec & meson-ge2d
Acked-by: Andrzej Pietrasiewicz <andrzejtp2010@gmail.com>
Acked-by: Łukasz Stelmach <l.stelmach@samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-03-18 05:58:35 +01:00
Mauro Carvalho Chehab c81652a4a8 Tag branch
-----BEGIN PGP SIGNATURE-----
 
 iHUEABYIAB0WIQRSrcquik9wuZrNjXJrQl33mxwedgUCYhXoBgAKCRBrQl33mxwe
 drFlAQDskgKQIoWFcIy17v6YHIClOfN8TcUkB5MK7t0eCldxGgD9HWsRtLEL5XlT
 9XLXFbNC2FjXJMNnFyIfAeelFuf0BgU=
 =VPGB
 -----END PGP SIGNATURE-----

Merge tag 'br-v5.18q' of git://linuxtv.org/hverkuil/media_tree into media_stage

Tag branch

* tag 'br-v5.18q' of git://linuxtv.org/hverkuil/media_tree:
  media: cec: seco: Drop pointless include
  media: hantro: sunxi: Fix VP9 steps
  media: imx: csis: Store pads format separately
  doc: media: Document VP9 reference_mode miss-placement
  doc: media: Document MM21 tiled format
  media: imx: imx8mq-mipi-csi2: Remove YUV422 2X8
  media: v4l2-core: Initialize h264 scaling matrix
  media: imx: imx-mipi-csis: Add output format
  media: imx: imx-mipi-csis: Add BGR888
  media: imx: imx-mipi-csis: Add RGB565_1X16
  media: imx: imx-mipi-csis: Set PIXEL_MODE for YUV422
  media: imx: imx7-media-csi: Use dual sampling for YUV 1X16
  media: imx: Rename imx7-mipi-csis.c to imx-mipi-csis.c
  media: imx: De-stage imx7-mipi-csis

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-03-07 16:27:01 +01:00
Mauro Carvalho Chehab 8268d06750 Tag branch
-----BEGIN PGP SIGNATURE-----
 
 iHUEABYIAB0WIQRSrcquik9wuZrNjXJrQl33mxwedgUCYhSuNQAKCRBrQl33mxwe
 dqOzAQCYc0iOPhTap9q7thsa5KYbyOU1A45u2jeETEz80La8mwEA4GjXUWMi2Scs
 e0BMW+hej/kjPnFHNqz4UBBXoDG0Egg=
 =TDr2
 -----END PGP SIGNATURE-----

Merge tag 'br-v5.18l' of git://linuxtv.org/hverkuil/media_tree into media_stage

Tag branch

* tag 'br-v5.18l' of git://linuxtv.org/hverkuil/media_tree: (39 commits)
  media: imx-jpeg: Set V4L2_BUF_FLAG_LAST at eos
  media: meson-ir-tx: remove incorrect doc comment
  media: vivid: use time_is_after_jiffies() instead of open coding it
  media: wl128x: use time_is_before_jiffies() instead of open coding it
  media: tda8083: use time_is_after_jiffies() instead of open coding it
  media: stv0299: use time_is_before_jiffies() instead of open coding it
  media: si21xx: use time_is_before_jiffies() instead of open coding it
  media: cedrus: h264: Fix neighbour info buffer size
  media: cx88-mpeg: clear interrupt status register before streaming video
  media: cedrus: H265: Fix neighbour info buffer size
  media: fsl-viu: use GFP_KERNEL
  media: cx18: use GFP_KERNEL
  drivers: meson: vdec: add VP9 support to GXM
  stkwebcam: add new Asus laptop to upside_down table
  media: imx-jpeg: fix a bug of accessing array out of bounds
  media: sun6i-csi: fix colorspace in sun6i_video_try_fmt()
  media: usb: go7007: s2250-board: fix leak in probe()
  media: cedrus: Add watchdog for job completion
  pixfmt-yuv-planar.rst: fix typo: 'Cr, Cr' -> 'Cr, Cb'
  media: imx-jpeg: Remove unnecessary print function dev_err()
  ...

Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-03-07 16:25:17 +01:00
Dmitry Osipenko 8bd4aaf438 media: staging: tegra-vde: De-stage driver
The TODO of tegra-vde driver has been completed, driver now supports
V4L2 stateless video decoding API. Relocate driver to drivers/media.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-03-07 16:19:43 +01:00
Dmitry Osipenko a097abd401 media: staging: tegra-vde: Bump BSEV DMA timeout
BSEV DMA timeouts if VDE is downclocked by x10. Bump the timeout to allow
DMA to complete. We don't support freq scaling yet, this is just a minor
improvement which may become useful sometime later.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-03-07 16:17:48 +01:00
Dmitry Osipenko 313db7d235 media: staging: tegra-vde: Remove legacy UAPI support
The legacy UAPI became unnecessary with the V4L stateless decoder API
support addition to the Tegra decoder driver. Remove legacy UAPI support.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-03-07 16:17:30 +01:00
Dmitry Osipenko 24d5d2bafb media: staging: tegra-vde: Support V4L stateless video decoder API
Expose Tegra video decoder as a generic V4L M2M stateless video decoder.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-03-07 16:16:41 +01:00
Dmitry Osipenko 9aa94a3164 media: staging: tegra-vde: Factor out H.264 code
Factor out H.264 hardware programming code into separate source file in a
preparation to support V4L API by the Tegra video decoder driver.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-03-07 16:14:31 +01:00
Muhammad Usama Anjum 3a6cddab2d media: imx: imx8mq-mipi_csi2: Remove unneeded code
ret is constant in imx8mq_mipi_csi_pm_suspend(). This function cannot
return error. Remove the return variable. Simplify other functions which
are using this function.

Signed-off-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
Reviewed-by: Fabio Estevam <festevam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-03-07 11:52:45 +01:00
Adam Ford cf1abc501a media: hantro: Add support for i.MX8MM Hantro-G1
The i.MX8MM has a Hantro G1 video decoder similar to the
imx8mq but lacks the post-processor present in the imx8mq.
Add support in the driver for it with the post-processing
removed.

Signed-off-by: Adam Ford <aford173@gmail.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-03-07 10:57:00 +01:00
Adam Ford 6971efb70a media: hantro: Allow i.MX8MQ G1 and G2 to run independently
The VPU in the i.MX8MQ is really the combination of Hantro G1 and
Hantro G2. With the updated vpu-blk-ctrl, the power domains system
can enable and disable them separately as well as pull them out of
reset. This simplifies the code and lets them run independently
while still retaining backwards compatibility with older device
trees for those using G1.

Signed-off-by: Adam Ford <aford173@gmail.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
2022-03-07 10:54:45 +01:00
Jernej Skrabec 17e83a0a1e media: hantro: sunxi: Fix VP9 steps
It turns out that postproc on Allwinner H6 needs width and height to be
multiple of 32.

Fixes: 86790a4fdf ("media: hantro: Add support for Allwinner H6")
Signed-off-by: Jernej Skrabec <jernej.skrabec@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
2022-02-23 08:53:08 +01:00
Jacopo Mondi 884c8bd918 media: imx: imx8mq-mipi-csi2: Remove YUV422 2X8
The 2X8 variants of MEDIA_BUS_FMT_YUYV8_2X8 does not apply to serial
busses.

Drop it and while at it also add the canonical UYVY wire format for
packed YUV422 when transmitted on the CSI-2 serial bus.

Also beautify a little the formats declaration list by putting the
opening curly brace after the comment.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
2022-02-23 08:53:08 +01:00