Recent Allwinner SoCs support >4 GiB of DRAM, so those variants of the
DMA engine support >32 bit physical addresses. This is accomplished by
placing the high bits in the "para" word in the DMA descriptor.
DMA descriptors themselves can be located at >32 bit addresses by
putting the high bits in the LSBs of the descriptor address register,
taking advantage of the required DMA descriptor alignment. However,
support for this is not really necessary, so we can avoid the
complication by allocating them from the DMA_32 zone.
Signed-off-by: Samuel Holland <samuel@sholland.org>
Acked-by: Jernej Skrabec <jernej.skrabec@gmail.com>
Link: https://lore.kernel.org/r/20220424172759.33383-4-samuel@sholland.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
This breaks on RISC-V, because dma_pool_alloc returns addresses which
are not in the linear map. Instead, plumb through the physical address
which is already known anyway.
Acked-by: Maxime Ripard <maxime@cerno.tech>
Signed-off-by: Samuel Holland <samuel@sholland.org>
Link: https://lore.kernel.org/r/20220424172759.33383-3-samuel@sholland.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Initialize slave_bw in dma_prep*() functions as the parameter is not
set for DMA_MEM_TO_MEM case in get_transfer_param(). Though the case
may never occur, initializing it avoids warning from certain static
checkers
Fixes: ee17028009 ("dmaengine: tegra: Add tegra gpcdma driver")
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Akhil R <akhilrajeev@nvidia.com>
Link: https://lore.kernel.org/r/20220426101913.43335-2-akhilrajeev@nvidia.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
At any time, a DMA transfer can be suspended to be restarted later before
the end of the DMA transfer.
In order to restart from the point where the transfer was stopped,
DMA_SxNDTR has to be read after disabling the channel by clearing the EN
bit in DMA_SxCR register, to know the number of data items already
collected.
Peripheral and/or memory addresses have to be updated in order to adjust
the address pointers.
SxNDTR register has to be updated with the remaining number of data items
to be transferred (the value read when the channel was disabled).
Then the channel can be re-enabled to resume the transfer from the point
it was suspended.
If the channel was configured in circular or double-buffer mode, the
circular or double-buffer mode must be disabled before re-enabling the
channel to be able to reconfigure SxNDTR register and re-activate circular
or double-buffer mode on next Transfer Complete interrupt where channel
will be disabled by HW. This is due to the fact that on resume, re-writing
SxNDTR register value updates internal HW auto-reload data counter, and
then it truncates all next transfers after a pause/resume sequence.
Signed-off-by: Amelie Delaunay <amelie.delaunay@foss.st.com>
Link: https://lore.kernel.org/r/20220505115611.38845-5-amelie.delaunay@foss.st.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
dmaengine framework offers device_pause and device_resume ops to pause an
on-going transfer and resume it later.
To avoid any misunderstanding with system sleep pm ops, rename pm ops into
stm32_dma_pm_suspend and stm32_dma_pm_resume.
Signed-off-by: Amelie Delaunay <amelie.delaunay@foss.st.com>
Link: https://lore.kernel.org/r/20220505115611.38845-4-amelie.delaunay@foss.st.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
stm32_dma_handle_chan_done() is called on Transfer Complete interrupt.
As DMA_SxSCR register is read in interrupt handler, pass the value as
parameter of stm32_dma_handle_chan_done(). Also return directly if
chan->desc is null to remove one ident level.
Then, stm32_dma_configure_next_sg() is doing something only if
Double-Buffer Mode (DBM) is enabled, so, check it is enabled prior calling
stm32_dma_configure_next_sg(), to remove one ident level in
stm32_dma_configure_next_sg().
Signed-off-by: Amelie Delaunay <amelie.delaunay@foss.st.com>
Link: https://lore.kernel.org/r/20220505115611.38845-3-amelie.delaunay@foss.st.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
chan->next_sg is used to know which transfer will start after the ongoing
one. It is incremented for each new transfer, either on transfer start for
non-cyclic transfers, or on transfer complete interrupt for cyclic
transfers.
For cyclic transfer, when the last item is reached, chan->next_sg must be
reinitialized to the first item.
Signed-off-by: Amelie Delaunay <amelie.delaunay@foss.st.com>
Link: https://lore.kernel.org/r/20220505115611.38845-2-amelie.delaunay@foss.st.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
One of the two DMA controllers managed by the DMAMUX can be used by the
coprocessor. It is defined in the device tree with dma-masters.
When the two DMA controllers are used by the main CPU,
dma-masters = <&dma1, &dma2>; is specified in the device tree.
When one of the controllers is used by coprocessor (so not managed by
Linux), dma-masters = <&dma1>; is specified in the device tree.
In this case, Linux driver must not reset the DMAMUX, because it could have
been configured by the coprocessor to use the second DMA controller.
count is the number of DMA controllers defined in dma-masters property.
Reset only if resets property is found and valid in device tree, and if
the two DMA controllers are under Linux control.
Signed-off-by: Amelie Delaunay <amelie.delaunay@foss.st.com>
Link: https://lore.kernel.org/r/20220504161724.123180-1-amelie.delaunay@foss.st.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Add compatible and driver_data for GPI DMA engines found in Qualcomm
SC7280. The driver_data contains ee_offset of 0x10000.
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Link: https://lore.kernel.org/r/20220421121733.1829350-1-vkoul@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
If interrupt occurs while !chan->busy, it means channel has been disabled
between the raise of the interruption and the read of status and ien, so,
spurious interrupt can be silently discarded.
Signed-off-by: Amelie Delaunay <amelie.delaunay@foss.st.com>
Link: https://lore.kernel.org/r/20220504155322.121431-4-amelie.delaunay@foss.st.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The parameter to pass back to the handler function when irq has been
requested is a struct stm32_mdma_device pointer, not a struct
stm32_mdma_chan pointer.
Even if chan is reinit later in the function, remove this wrong
initialization.
Fixes: a4ffb13c89 ("dmaengine: Add STM32 MDMA driver")
Signed-off-by: Amelie Delaunay <amelie.delaunay@foss.st.com>
Link: https://lore.kernel.org/r/20220504155322.121431-3-amelie.delaunay@foss.st.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
GISR1 was described in a not up-to-date documentation when the stm32-mdma
driver has been developed. This register has not been added in reference
manual of STM32 SoC with MDMA, which have only 32 MDMA channels.
So remove it from stm32-mdma driver.
Fixes: a4ffb13c89 ("dmaengine: Add STM32 MDMA driver")
Signed-off-by: Amelie Delaunay <amelie.delaunay@foss.st.com>
Link: https://lore.kernel.org/r/20220504155322.121431-2-amelie.delaunay@foss.st.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The generic property, used in most of the drivers and defined in generic
dma-common DT bindings, is 'dma-channels'. Switch to new property while
keeping backward compatibility.
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
Reviewed-by: Tony Lindgren <tony@atomide.com>
Link: https://lore.kernel.org/r/20220427161126.647073-4-krzysztof.kozlowski@linaro.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The generic property, used in most of the drivers and defined in generic
dma-common DT bindings, is 'dma-channels'. Switch to new property while
keeping backward compatibility.
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
Link: https://lore.kernel.org/r/20220503065407.52188-4-krzysztof.kozlowski@linaro.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The generic properties, used in most of the drivers and defined in
generic dma-common DT bindings, are 'dma-channels' and 'dma-requests'.
Switch to new properties while keeping backward compatibility.
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
Link: https://lore.kernel.org/r/20220503065407.52188-3-krzysztof.kozlowski@linaro.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
drivers/dma/tegra186-gpc-dma.c: In function ‘tegra_dma_probe’:
drivers/dma/tegra186-gpc-dma.c:1364:24: error: ‘struct iommu_fwspec’ has no member named ‘ids’
stream_id = iommu_spec->ids[0] & 0xffff;
^~
Make TEGRA186_GPC_DMA depends on IOMMU_API to fix this.
Fixes: ee17028009 ("dmaengine: tegra: Add tegra gpcdma driver")
Signed-off-by: YueHaibing <yuehaibing@huawei.com>
Link: https://lore.kernel.org/r/20220505093236.15076-1-yuehaibing@huawei.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The Renesas RZN1 DMA IP is very close to the original DW DMA IP, a DMA
router has been introduced to handle the wiring options that have been
added.
Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com>
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-By: Vinod Koul <vkoul@kernel.org>
Link: https://lore.kernel.org/r/20220427095653.91804-8-miquel.raynal@bootlin.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The Renesas RZN1 DMA IP is based on a DW core, with eg. an additional
dmamux register located in the system control area which can take up to
32 requests (16 per DMA controller). Each DMA channel can be wired to
two different peripherals.
We need two additional information from the 'dmas' property: the channel
(bit in the dmamux register) that must be accessed and the value of the
mux for this channel.
Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Reviewed-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Link: https://lore.kernel.org/r/20220427095653.91804-6-miquel.raynal@bootlin.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Calling synchronize_irq() right before free_irq() is quite useless. On one
hand the IRQ can easily fire again before free_irq() is entered, on the
other hand free_irq() itself calls synchronize_irq() internally (in a race
condition free way), before any state associated with the IRQ is freed.
Signed-off-by: Minghao Chi <chi.minghao@zte.com.cn>
Link: https://lore.kernel.org/r/20220516115412.1651772-1-chi.minghao@zte.com.cn
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>
When setting DMA_INTERRUPT capability, a callback function
dma->device_prep_dma_interrupt() is needed to support this capability.
Without setting the callback, dma_async_device_register() will fail dma
capability check.
Fixes: 4e5a4eb203 ("dmaengine: idxd: set DMA_INTERRUPT cap bit")
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/165101232637.3951447.15765792791591763119.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The generic property, used in most of the drivers and defined in generic
dma-common DT bindings, is 'dma-channels'. Switch to new property while
keeping backward compatibility.
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
Reviewed-by: Baolin Wang <baolin.wang7@gmail.com>
Link: https://lore.kernel.org/r/20220503065147.51728-3-krzysztof.kozlowski@linaro.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Skip wq irq resources freeing when wq type is not kernel since the driver
skips the irq alloction during wq enable. Add check in wq type check in
idxd_wq_free_irq() to mirror idxd_wq_request_irq().
Fixes: 63c14ae6c1 ("dmaengine: idxd: refactor wq driver enable/disable operations")
Reported-by: Tony Zu <tony.zhu@intel.com>
Tested-by: Tony Zu <tony.zhu@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/165176310726.2112428.7474366910758522079.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
pm_runtime_resume_and_get() automatically handle dev->power.usage_count
decrement on errors, so prefer using it and also implement it's error
handling.
Signed-off-by: Radhey Shyam Pandey <radhey.shyam.pandey@xilinx.com>
Link: https://lore.kernel.org/r/1652166762-18317-4-git-send-email-radhey.shyam.pandey@xilinx.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
In zynqmp_dma_alloc/free_chan_resources functions there is a
potential overflow in the below expressions.
dma_alloc_coherent(chan->dev, (2 * chan->desc_size *
ZYNQMP_DMA_NUM_DESCS),
&chan->desc_pool_p, GFP_KERNEL);
dma_free_coherent(chan->dev,(2 * ZYNQMP_DMA_DESC_SIZE(chan) *
ZYNQMP_DMA_NUM_DESCS),
chan->desc_pool_v, chan->desc_pool_p);
The arguments desc_size and ZYNQMP_DMA_NUM_DESCS were 32 bit. Though
this overflow condition is not observed but it is a potential problem
in the case of 32-bit multiplication. Hence fix it by changing the
desc_size data type to size_t.
In addition to coverity fix it also reuse ZYNQMP_DMA_DESC_SIZE macro in
dma_alloc_coherent API argument.
Addresses-Coverity: Event overflow_before_widen.
Signed-off-by: Radhey Shyam Pandey <radhey.shyam.pandey@xilinx.com>
Link: https://lore.kernel.org/r/1652166762-18317-2-git-send-email-radhey.shyam.pandey@xilinx.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
idxd_wq_device_reset_cleanup() already calls idxd_wq_disable_cleanup().
There is no need to call idxd_wq_disable_cleanup() again in
idxd_device_wqs_clear_state(). Remove redudant call from
idxd_wq_device_reset_cleanup().
Fixes: 0dcfe41e9a ("dmanegine: idxd: cleanup all device related bits after disabling device")
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/165231365717.986350.2441351765955825964.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Call idxd_wq_free_irq() in the drv_disable_wq() function before
idxd_wq_reset() is called. Otherwise the wq type is reset and the irq does
not get freed.
Fixes: 63c14ae6c1 ("dmaengine: idxd: refactor wq driver enable/disable operations")
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/165231367316.986407.11001767338124941736.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Jacob reported that with lockdep debug turned on, idxd_device_driver
removal causes kernel splat from lock assert warning for
idxd_device_wqs_clear_state(). Make sure
idxd_device_wqs_clear_state() holds the wq lock for each wq when
cleaning the wq state. Move the call outside of the device spinlock.
Reported-by: Jacob Pan <jacob.jun.pan@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/165231364426.986304.9294302800482492780.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The idxd driver always gated the pasid enabling under a single knob and
this assumption is incorrect. The pasid used for kernel operation can be
independently toggled and has no dependency on the user pasid (and vice
versa). Split the two so they are independent "enabled" flags.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/165231431746.986466.5666862038354800551.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
If a call to alloc_chrdev_region() fails, the already allocated resources
are leaking.
Add the needed error handling path to fix the leak.
Fixes: 42d279f913 ("dmaengine: idxd: add char driver to expose submission portal to userland")
Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/1b5033dcc87b5f2a953c413f0306e883e6114542.1650521591.git.christophe.jaillet@wanadoo.fr
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Use platform_irq_get() instead platform_get_resource() for IRQ resource
to fix the probe failure. platform_get_resource() fails to fetch the IRQ
resource as it might not be ready at that time.
platform_irq_get() is also the recommended way to get interrupt as it
directly gives the IRQ number and no conversion from resource is
required.
Fixes: ee17028009 ("dmaengine: tegra: Add tegra gpcdma driver")
Reported-by: Jonathan Hunter <jonathanh@nvidia.com>
Signed-off-by: Akhil R <akhilrajeev@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
Reviewed-by: Jon Hunter <jonathanh@nvidia.com>
Link: https://lore.kernel.org/r/20220505091440.12981-1-akhilrajeev@nvidia.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Move the core driver operations from wq driver to the drv_enable_wq() and
drv_disable_wq() functions. The move should reduce the wq driver's
knowledge of the core driver operations and prevent code confusion for
future wq drivers.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/165047301643.3841827.11222723219862233060.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
LKP bot reports a new warning:
Warning:
drivers/dma/ptdma/ptdma-dmaengine.c:262:1: warning: no previous prototype for 'pt_tx_status' [-Wmissing-prototypes]
pt_tx_status() should be static, so declare as such.
Reported-by: kernel test robot <lkp@intel.com>
Fixes: d965068259 ("dmaengine: PTDMA: support polled mode")
Link: https://lore.kernel.org/r/20220421052407.745637-1-vkoul@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The value parameter is a single byte, so duplicate it to the 8 byte
range that is used as the pattern.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Cc: Sinan Kaya <okaya@kernel.org>
Link: https://lore.kernel.org/r/20220301182551.883474-5-benjamin.walker@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The value passed in to .prep_dma_memset is to be treated as a single
byte repeating pattern.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Cc: Ludovic Desroches <ludovic.desroches@microchip.com>
Cc: Tudor Ambarus <tudor.ambarus@microchip.com>
Link: https://lore.kernel.org/r/20220301182551.883474-4-benjamin.walker@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The value passed in to .prep_dma_memset is to be treated as a single
byte repeating pattern.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Cc: Ludovic Desroches <ludovic.desroches@microchip.com>
Cc: Tudor Ambarus <tudor.ambarus@microchip.com>
Link: https://lore.kernel.org/r/20220301182551.883474-3-benjamin.walker@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Move the calling of request_irq() and other related irq setup code until
after the WQ is successfully enabled. This reduces the amount of
setup/teardown if the wq is not configured correctly and cannot be enabled.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/164642777730.179702.1880317757087484299.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Looks like I forgot to add DMA_INTERRUPT cap setting to the idxd driver and
dmatest is still working regardless of this mistake. Add an explicit check
of DMA_INTERRUPT capability for dmatest to make sure the DMA device being used
actually supports interrupt before the test is launched and also that the
driver is programmed correctly.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/164978679251.2361020.5856734256126725993.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Load the max_xfer_size and max_batch_size values from the values read from
registers to the shadow variables. This will allow the read-only device to
display the correct values for the sysfs attributes.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/164971507673.2201761.11244446608988838897.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Even though idxd driver has always supported interrupt, it never actually
set the DMA_INTERRUPT cap bit. Rectify this mistake so the interrupt
capability is advertised.
Reported-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/164971497859.2201379.17925303210723708961.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Set the maximum segment size, since the hardware can do transfers larger
than the default 64 KiB returned by dma_get_max_seg_size().
The maximum segment size is limited by the 24-bit transfer count field
in DMA descriptors. The number of bytes is equal to the transfer count
times the transfer size unit, which is selected by the driver based on
the DMA buffer address and length of the transfer. The size unit can be
as small as 1 byte, so set the maximum segment size to 2^24-1 bytes to
ensure the transfer count will not overflow regardless of the size unit
selected by the driver.
Signed-off-by: Aidan MacDonald <aidanmacdonald.0x0@gmail.com>
Link: https://lore.kernel.org/r/20220411153618.49876-1-aidanmacdonald.0x0@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
If the DMA_PREP_INTERRUPT flag is not provided, run in polled mode,
which significantly improves IOPS: more than twice on chunks < 4K.
Signed-off-by: Ilya Novikov <i.m.novikov@yadro.com>
Link: https://lore.kernel.org/r/20220413113733.59041-1-i.m.novikov@yadro.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
It is unnecessary to call spin_lock_bh() if you are already in a tasklet.
Signed-off-by: Yunbo Yu <yuyunbo519@gmail.com>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
Link: https://lore.kernel.org/r/20220418142021.1241558-1-yuyunbo519@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Samuel Holland