commit e8dbd6445d upstream.
>From Intel IAA spec [1], Intel IAA does not support batch processing.
Two batch related default values for IAA are incorrect in current code:
(1) The max batch size of device is set during device initialization,
that indicates batch is supported. It should be always 0 on IAA.
(2) The max batch size of work queue is set to WQ_DEFAULT_MAX_BATCH (32)
as the default value regardless of Intel DSA or IAA device during
work queue setup and cleanup. It should be always 0 on IAA.
Fix the issues by setting the max batch size of device and max batch
size of work queue to 0 on IAA device, that means batch is not
supported.
[1]: https://cdrdv2.intel.com/v1/dl/getContent/721858
【【SPR内核开发】SPR系列patch合入kernel+kvm】http://tapd.oa.com/Virtualization/prong/stories/view/1020422237869387499
--story=869387499 【SPR内核开发】SPR系列patch合入kernel+kvm
Fixes: 23084545db ("dmaengine: idxd: set max_xfer and max_batch for RO device")
Fixes: 92452a72eb ("dmaengine: idxd: set defaults for wq configs")
Fixes: bfe1d56091 ("dmaengine: idxd: Init and probe for Intel data accelerators")
Signed-off-by: Xiaochen Shen <xiaochen.shen@intel.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Fenghua Yu <fenghua.yu@intel.com>
Link: https://lore.kernel.org/r/20220930201528.18621-2-xiaochen.shen@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit opencloudos.
Bring module idxd_bus to front of idxd, otherwise idxd will register failed.
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit 3a5e5ffd0d28c0c9b71f047668baca77723b6875 Intel-BKC.
If wq is already enabled and idxd_wq_enable() is called, return 0 instead
of error since it's not a real error.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit 8866c56721b2e3f5468792015025deacde17affd Intel-BKC.
At the time when idxd_device_wqs_clear_state() gets called, which is from
device disable path, the wq may already been disabled. Given that the
driver does not actually know if the wq was previously disabled, force
cleanup the context of any available wq.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit 84a5fc5fb6891aecc3d7bd0dc520a75c8d800487 Intel-BKC.
Make sure the internal wq variables for max_xfer_size and max_batch_size
are updated from the WQCFG when loading a read-only device configuration.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit c2f10b603dc626985c6d3dcce79aee52ef95be2d Intel-BKC.
Set priv bit to 0 for read-only WQ setup when the WQ type is not kernel.
Current code does not touch the priv bit when the WQ is not kernel type.
This non-action causes problem wq reconfiguration where the WQ is
configured as a kernel type, gets disabled, and then reconfigured to be a
user WQ. The priv bit that was set by the kernel type config remains.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit af9a712e3657f6af8c282d204f19d01e405dd106 Intel-BKC.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
When using IDXD to copy HugePage/THP pages, if DMA_API_DEBUG is enabled,
DMA-API debug code would complain mapping segment size is longer than
device claimed, it use 64K as default if device doesn’t claim.
Claim DSA max_segment_size to HugePage/THP to avoid DMA-API debug code
print warning message if copy HugePage/THP pages.
Signed-off-by: Jiangbo Wu <jiangbo.wu@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit f4735eec028420bcd5ff3e0f0b955aa84c8f502c Intel-BKC.
IAX does not have batch support. So don't attempt to init the batch
structure during descriptor allocation.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit 81f5eb2b11 upstream.
Add string processing with strim() in order to remove trailing white spaces
that may be input by user.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit 81f5eb2b11 upstream.
Add string processing with strim() in order to remove trailing white spaces
that may be input by user.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit a1a86b18b59dd62c56fe597d2a7fe97f29818acc Intel-BKC.
Found below kernel bug when doing DSA DWQ live migration
[ 940.265678] BUG: kernel NULL pointer dereference, address: 0000000000000000
...
[ 940.319892] RIP: 0010:vidxd_portal_mmio_write.cold.12+0x53/0x216 [idxd_mdev]
The reason is that some critical codes missed during porting
codes from 5.12 to 5.15. Fix it.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit 098159e3d405124cbc2020d93f529dff0eb7869e Intel-BKC.
The main issue is that when we program wq with pasid of 0 but pasid_en set
for shared wq, it confuses the VDCM. Remove the WQ pasid setting during
configuration programming of the device as the driver does not have enough
information at that point to make that decision. Program the WQCFG pasid
and pasid_en at the time when it is needed. Add support in VDCM to
understand what pasid_en=1 and pasid=0 means.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit ac144d03c262a134f664e72b6d6daf6e19d8a223 Intel-BKC.
Add memcpy_sg support based on DSA batch descriptor. DSA batch
descriptor has a 'batch' field that is a set of descriptors in
the array. scatter list is filled up batch descriptor.
Co-developed-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Signed-off-by: Jiangbo Wu <jiangbo.wu@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit c9b30d5e5b33024ddf8d32705fa89ba4a6436510 Intel-BKC.
The DSA device can support a group of descriptors for submission at once.
The DSA spec calls this operation a 'batch' operation. The batch descriptor
points to an array of descriptors where the beginning of the address list
is 64-byte aligned.
Co-developed-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Signed-off-by: Jiangbo Wu <jiangbo.wu@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit opencloudos.
This reverts commit 900807990c2065f016ceef01b692e7d5750a8209.
This is no longer necessary now that all assumptions about the order of
completions have been removed from the dmaengine client API.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit a079359ae604fb3b065c1c6047457e491c65768b Intel-BKC.
This can now be supported even for devices that complete operations out
of order. Add support for directly polling transactions.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit 4e4c92933f0abc77eed2810d90034afd43ac44e6 Intel-BKC.
This is going to be packed into the cookie. It does not need to be
negative or larger than u16.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit a87a45ce5695d96a3d415f637eb878398e973ca0 Intel-BKC.
The dma_set_tx_state function will go away shortly. The two functions
are functionally equivalent.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit 08a196633ebe3fb5ddb89a6edcbef1ecd22e6c7b Intel-BKC.
Nothing uses these and they don't convey usable information.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit 29381fe8cdd2b15561a5bea9f12eee86fdfad39d Intel-BKC.
These are only used by a single driver (pxa_camera) which knows exactly
which DMA engine it is dealing with and therefore knows the behavior
of the cookie values. It can grab the value it needs directly from
the dma_chan instead. No other DMA client assumes anything about
the behavior of the cookie values, so the cookie becomes an opaque
handle after this patch.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit 20c56f51843bacdd98a6c0ca6744757880b8ae3a Intel-BKC.
This is only used by DMA providers, not DMA clients. Move it next
to the other cookie utility functions.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit caabe3641f5dbed4fe63cd366e1af91679eb59f0 Intel-BKC.
This is never actually used by any existing DMA clients. It is only
used, via dma_cookie_status, by providers.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit e918172fa7b78571a137cb10ecc17536a913864c Intel-BKC.
With the possibility of multiple wq drivers that can be bound to the wq,
the user config tool accel-config needs a way to know which wq driver to
bind to the wq. Introduce per wq driver_name sysfs attribute where the user
can indicate the driver to be bound to the wq. This allows accel-config to
just bind to the driver using wq->driver_name.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Conflicts:
drivers/dma/idxd/dma.c
include/uapi/linux/idxd.h
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit 36820eed1ae19777d268b452d79daed1fae55764 Intel-BKC.
Change where pasid is setup for DWQ WQCFG. Have the driver only set it when
WQ is being enabled.
Also remove setting of IMS pasid entry in vidxd_mmio_write() to virtual MSIXPERM
register. Add the IMS pasid configuration in set_signal() for both with
vIOMMU and without. This is where it belongs anyhow.
Also moved wq_enable_irq() to after device disable to reduce unnecessary
extra code running when the wq isn't ready.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit 3157dd0a36 upstream.
The driver currently programs the system pasid to the WQ preemptively when
system pasid is enabled. Given that a dwq will reprogram the pasid and
pasid_en bit, the programming is not necessary. Remove system pasid
programming on device config write. Add pasid programming for kernel wq
type on wq driver enable.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit 986267eb328124be087a58fb85ddd797f08b58e6 Intel-BKC.
The idxd driver always gated the pasid enabling under a single knob and
this assumption is incorrect. The pasid used for kernel operation can be
independently toggled and has no dependency on the user pasid (and vice
versa). Split the two so they are independent "enabled" flags.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit e75e0042dc577ec58b3d4e81415f1b6647daed7d Intel-BKC.
The return value ret is returned without being initialized to an error
value. This caused the failing function to return success no matter what.
Remove return value and return error directly.
Fixes: 253a70def285 ("dmaengine: idxd: Use DMA API for in-kernel PASID DMA")
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit 4b829b56c2f4ed73f9c08282f9eabb67c8ddc7b9 Intel-BKC.
User SVA is the essential feature of IDXD, enable it by default.
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit 4cc216bbb3fe83b1cded59fcd75f3c92ddf942de Intel-BKC.
The current in-kernel supervisor PASID support is based on the SVM/SVA
machinery in SVA lib. The binding between a kernel PASID and kernel
mapping has many flaws. See discussions in the link below.
This patch enables in-kernel DMA by switching from SVA lib to the
standard DMA mapping APIs. Since both DMA requests with and without
PASIDs are mapped identically, there is no change to how DMA APIs are
used after the kernel PASID is enabled.
Link: https://lore.kernel.org/linux-iommu/20210511194726.GP1002214@nvidia.com/
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit d249f9c11aad36451ae9f723b727eb9a55ddec8e Intel-BKC.
WQ reset clears all configuration. However, for devices that are RO, the
internal data structure should not be cleared. Skip when device is not
configurable (RO).
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit 684faefea881973dbada146515c3902960efed41 Intel-BKC.
Shared WQ restriction accidentally left in idxd_wq_load_config(). Not
needed when we support swq in VDCM.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit bca2a4c0ad145e938f38a3bc7bcbb23aa313a654 Intel-BKC.
NOT FOR UPSTREAM
Signed-off-by: Tom Zanussi <tom.zanussi@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit 74c8bded7efeda08bc7ce59cba3c310fb0ba0ee4 Intel-BKC.
Add a void * to idxd_wqs for user-defined context data.
Signed-off-by: Tom Zanussi <tom.zanussi@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit dbf3e023c824b280e859dc6552636a24f49f9c58 Intel-BKC.
To allow idxd sub-drivers to enable and disable wqs, both with and
without the wq mutex held, export them.
Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit d581defaa516d735db9446b9a47cdc43a5f2e49a Intel-BKC.
Add support to allow an external driver to be registered to the
dsa_bus_type and also auto-loaded.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit 1cd8e751d9 upstream.
If the device shows up as read-only configuration, skip the clearing of the
state as the context must be preserved for device re-enable after being
disabled.
Fixes: 0dcfe41e9a ("dmanegine: idxd: cleanup all device related bits after disabling device")
Reported-by: Tony Zhu <tony.zhu@intel.com>
Tested-by: Tony Zhu <tony.zhu@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/164971479479.2200566.13980022473526292759.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit 66903461ff upstream.
Block wq_max_transfer_size_store() when the device is configured as
read-only and not configurable.
Fixes: d7aad5550e ("dmaengine: idxd: add support for configurable max wq xfer size")
Reported-by: Bernice Zhang <bernice.zhang@intel.com>
Tested-by: Bernice Zhang <bernice.zhang@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/164971488154.2200913.10706665404118545941.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit 66903461ff upstream.
Block wq_max_batch_size_store() when the device is configured as read-only
and not configurable.
Fixes: e7184b159d ("dmaengine: idxd: add support for configurable max wq batch size")
Reported-by: Bernice Zhang <bernice.zhang@intel.com>
Tested-by: Bernice Zhang <bernice.zhang@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/164971493551.2201159.1942042593642155209.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit bc3452cdfc upstream.
When retries is compared to wq->enqcmds_retries each loop of idxd_enqcmds(),
wq->enqcmds_retries can potentially changed by user. Assign the value
of retries to wq->enqcmds_retries during initialization so it is the
original value set when entering the function.
Fixes: 7930d85535 ("dmaengine: idxd: add knob for enqcmds retries")
Suggested-by: Dave Hansen <dave.hansen@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/165031760154.3658664.1983547716619266558.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
commit 5d9d16e5aa upstream.
wq->enqcmds_retries is defined as unsigned int. However, retries on the
stack is defined as int. Change retries to unsigned int to compare the same
type.
Fixes: 7930d85535 ("dmaengine: idxd: add knob for enqcmds retries")
Suggested-by: Thiago Macieira <thiago.macieira@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/165031747059.3658198.6035308204505664375.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
Signed-off-by: Xinghui Li <korantli@tencent.com>
Xiaochen Shen