[ Upstream commit 6bba3c0ac5dc54737998a0982b2e272242c87e0f ]
There are two issues related to epf_ntb_epc_cleanup():
1) It should call epf_ntb_config_sspad_bar_clear()
2) The epf_ntb_bind() function should call epf_ntb_epc_cleanup()
to cleanup.
I also changed the ordering a bit. Unwinding should be done in the
mirror order from how they are allocated.
Fixes: e35f56bb03 ("PCI: endpoint: Support NTB transfer between RC and EP")
Link: https://lore.kernel.org/linux-pci/aaffbe8d-7094-4083-8146-185f4a84e8a1@moroto.mountain
Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Reviewed-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
[ Upstream commit 8e0f5a96c534f781e8c57ca30459448b3bfe5429 ]
Smatch complains about inconsistent NULL checking in vpci_scan_bus():
drivers/pci/endpoint/functions/pci-epf-vntb.c:1024 vpci_scan_bus() error: we previously assumed 'vpci_bus' could be null (see line 1021)
Instead of printing an error message and then crashing we should return
an error code and clean up.
Also the NULL check is reversed so it prints an error for success
instead of failure.
Fixes: e35f56bb03 ("PCI: endpoint: Support NTB transfer between RC and EP")
Link: https://lore.kernel.org/linux-pci/68e0f6a4-fd57-45d0-945b-0876f2c8cb86@moroto.mountain
Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Reviewed-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
[ Upstream commit aebfdfe39b9327a3077d0df8db3beb3160c9bdd0 ]
If device_register() fails in ntb_register_device(), the device name
allocated by dev_set_name() should be freed. As per the comment in
device_register(), callers should use put_device() to give up the
reference in the error path. So fix this by calling put_device() in the
error path so that the name can be freed in kobject_cleanup().
As a result of this, put_device() in the error path of
ntb_register_device() is removed and the actual error is returned.
Fixes: a1bd3baeb2 ("NTB: Add NTB hardware abstraction layer")
Signed-off-by: Yang Yingliang <yangyingliang@huawei.com>
Reviewed-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/20231201033057.1399131-1-yangyingliang@huaweicloud.com
[mani: reworded commit message]
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
[ Upstream commit 327ec5f70609cf00c6f961073c01857555c6a8eb ]
In the error path of pci_epf_mhi_edma_write() function, the DMA data
direction passed (DMA_FROM_DEVICE) doesn't match the actual direction used
for the data transfer. Fix it by passing the correct one (DMA_TO_DEVICE).
Fixes: 7b99aaadda ("PCI: epf-mhi: Add eDMA support")
Reviewed-by: Krzysztof Wilczyński <kw@linux.com>
Link: https://lore.kernel.org/r/20231214063328.40657-1-manivannan.sadhasivam@linaro.org
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
[ Upstream commit b08ded2ef2e98768d5ee5f71da8fe768b1f7774b ]
In the preparation of DMA async support, let's pass the parameters to
read_from_host() and write_to_host() APIs using mhi_ep_buf_info structure.
No functional change.
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Stable-dep-of: 327ec5f70609 ("PCI: epf-mhi: Fix the DMA data direction of dma_unmap_single()")
Signed-off-by: Sasha Levin <sashal@kernel.org>
[ Upstream commit c9501d268944d6c0475ecb3e740a084a7da9cbfe ]
The pci_epc_release() function frees "epc" so the kfree() on the next line
is a double free. Drop the redundant free.
Fixes: 7711cbb486 ("PCI: endpoint: Fix WARN() when an endpoint driver is removed")
Link: https://lore.kernel.org/r/2ce68694-87a7-4c06-b8a4-9870c891b580@moroto.mountain
Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
- Reorder struct pci_dev to avoid holes and reduce size (Christophe
JAILLET)
- Change pdev->rom_attr_enabled to single bit since it's only a boolean
value (Christophe JAILLET)
- Use struct_size() in pirq_convert_irt_table() instead of hand-writing it
(Christophe JAILLET)
- Explicitly include correct DT includes to untangle headers (Rob Herring)
- Fix a DOE race between destroy_work_on_stack() and the stack-allocated
task->work struct going out of scope in pci_doe() (Ira Weiny)
- Use pci_dev_id() when possible instead of manually composing ID from
dev->bus->number and dev->devfn (Xiongfeng Wang, Zheng Zengkai)
- Move pci_create_resource_files() declarations to linux/pci.h for alpha
build warnings (Arnd Bergmann)
- Remove unused hotplug function declarations (Yue Haibing)
- Remove unused mvebu struct mvebu_pcie.busn (Pali Rohár)
- Unexport pcie_port_bus_type (Bjorn Helgaas)
- Remove unnecessary sysfs ID local variable initialization (Bjorn Helgaas)
- Fix BAR value printk formatting to accommodate 32-bit values (Bjorn
Helgaas)
- Use consistent pointer types for config access syscall get_user() and
put_user() uses (Bjorn Helgaas)
- Simplify AER_RECOVER_RING_SIZE definition (Bjorn Helgaas)
- Simplify pci_pio_to_address() (Bjorn Helgaas)
- Simplify pci_dev_driver() (Bjorn Helgaas)
- Fix pci_bus_resetable(), pci_slot_resetable() name typos (Bjorn Helgaas)
- Fix code and doc typos and code formatting (Bjorn Helgaas)
- Tidy config space save/restore messages (Bjorn Helgaas)
* pci/misc:
PCI: Tidy config space save/restore messages
PCI: Fix code formatting inconsistencies
PCI: Fix typos in docs and comments
PCI: Fix pci_bus_resetable(), pci_slot_resetable() name typos
PCI: Simplify pci_dev_driver()
PCI: Simplify pci_pio_to_address()
PCI/AER: Simplify AER_RECOVER_RING_SIZE definition
PCI: Use consistent put_user() pointer types
PCI: Fix printk field formatting
PCI: Remove unnecessary initializations
PCI: Unexport pcie_port_bus_type
PCI: mvebu: Remove unused busn member
PCI: Remove unused function declarations
PCI/sysfs: Move declarations to linux/pci.h
PCI/P2PDMA: Use pci_dev_id() to simplify the code
PCI/IOV: Use pci_dev_id() to simplify the code
PCI/AER: Use pci_dev_id() to simplify the code
PCI: apple: Use pci_dev_id() to simplify the code
PCI/DOE: Fix destroy_work_on_stack() race
PCI: Explicitly include correct DT includes
x86/PCI: Use struct_size() in pirq_convert_irt_table()
PCI: Change pdev->rom_attr_enabled to single bit
PCI: Reorder pci_dev fields to reduce holes
For transfers below 4K, let's use iATU since using eDMA for such small
transfers is inefficient.
This is mainly because setting up an eDMA transfer and waiting for
completion adds some latency. This latency is negligible for large
transfers but not for the smaller ones.
With using iATU, there is an increase in ~50Mbps throughput on both MHI
UL (Uplink) and DL (Downlink) channels.
[kwilczynski: commit log]
Link: https://lore.kernel.org/linux-pci/20230717065459.14138-7-manivannan.sadhasivam@linaro.org
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Add support for Qualcomm Snapdragon SM8450 SoC to the EPF driver. SM8450
has the dedicated PID (0x0306) and supports eDMA. Currently, it has no
fixed PCI class, so it is being advertised as "PCI_CLASS_OTHERS".
Link: https://lore.kernel.org/linux-pci/20230717065459.14138-6-manivannan.sadhasivam@linaro.org
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Add support for Embedded DMA (eDMA) available in the DesignWare PCIe IP
to transfer the MHI buffers between the host and the endpoint. The eDMA
use helps achieve greater throughput as the transfers are offloaded from
CPUs.
For differentiating the iATU and eDMA APIs, the pci_epf_mhi_{read/write}
APIs are renamed to pci_epf_mhi_iatu_{read/write} and separate eDMA
specific APIs pci_epf_mhi_edma_{read/write} are introduced.
Platforms that require eDMA support can pass the MHI_EPF_USE_DMA flag
through pci_epf_mhi_ep_info.
[kwilczynski: commit log]
Link: https://lore.kernel.org/linux-pci/20230717065459.14138-5-manivannan.sadhasivam@linaro.org
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
The DT of_device.h and of_platform.h date back to the separate
of_platform_bus_type before it as merged into the regular platform bus. As
part of that merge prepping Arm DT support 13 years ago, they "temporarily"
include each other. They also include platform_device.h and of.h. As a
result, there's a pretty much random mix of those include files used
throughout the tree. In order to detangle these headers and replace the
implicit includes with struct declarations, users need to explicitly
include the correct includes.
Link: https://lore.kernel.org/r/20230714174827.4061572-1-robh@kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
As ntb_register_device() don't handle error of device_register(),
if ntb_register_device() returns error in pci_vntb_probe(), name of kobject
which is allocated in dev_set_name() called in device_add() is leaked.
As comment of device_add() says, it should call put_device() to drop the
reference count that was set in device_initialize()
when it fails, so the name can be freed in kobject_cleanup().
Signed-off-by: ruanjinjie <ruanjinjie@huawei.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
Add PCI Endpoint driver for the Qualcomm MHI (Modem Host Interface) bus.
The driver implements the MHI function over PCI in the endpoint device such
as SDX55 modem. The MHI endpoint function driver acts as a controller
driver for the MHI Endpoint stack and carries out all PCI related
functionality.
Link: https://lore.kernel.org/r/20230602114756.36586-9-manivannan.sadhasivam@linaro.org
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
Add support to notify the EPF device about the Bus Master Enable (BME)
event received by the EPC device from the Root complex.
Link: https://lore.kernel.org/r/20230602114756.36586-6-manivannan.sadhasivam@linaro.org
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
Add support to notify the EPF device about the linkdown event from the EPC
device.
Link: https://lore.kernel.org/r/20230602114756.36586-5-manivannan.sadhasivam@linaro.org
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
When the EPC is started or stopped multiple times from configfs, just
return -EALREADY. There is no need to call the EPC start/stop functions
in those cases.
Link: https://lore.kernel.org/r/20230602114756.36586-4-manivannan.sadhasivam@linaro.org
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
Currently, the EPF probe function doesn't get the device ID argument needed
to correctly identify the device table ID of the EPF device.
When multiple entries are added to the "struct pci_epf_device_id" table,
the probe function needs to identify the correct one. This is achieved by
modifying the pci_epf_match_id() function to return the match ID pointer
and passing it to the driver's probe function.
pci_epf_device_match() function can return bool based on the return value
of pci_epf_match_id().
Link: https://lore.kernel.org/r/20230602114756.36586-3-manivannan.sadhasivam@linaro.org
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
Both pci_epc_raise_irq() and pci_epc_map_msi_irq() APIs expect the
MSI/MSI-X vectors to start from 1 but it is not documented. Add the range
info to the kdoc of the APIs to make it clear.
Link: https://lore.kernel.org/r/20230602114756.36586-2-manivannan.sadhasivam@linaro.org
Fixes: 5e8cb40338 ("PCI: endpoint: Add EP core layer to enable EP controller and EP functions")
Fixes: 87d5972e47 ("PCI: endpoint: Add pci_epc_ops to map MSI IRQ")
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
pci_epf_type_add_cfs() should not be called with an unbound EPF device,
that is, an epf device with epf->driver not set. For such case, replace the
NULL return in pci_epf_type_add_cfs() with a clear ERR_PTR(-ENODEV) pointer
error return.
Link: https://lore.kernel.org/r/20230515074348.595704-2-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivami <manivannan.sadhasivam@linaro.org>
In pci_epf_test_init_dma_chan() epf_test->dma_chan_rx is assigned from
dma_request_channel() with DMA_DEV_TO_MEM as filter.dma_mask.
However, in pci_epf_test_data_transfer() if the dir is DMA_DEV_TO_MEM,
epf->dma_chan_rx should be used but instead we are using
epf_test->dma_chan_tx.
Fix it.
Link: https://lore.kernel.org/r/20230412063447.2841177-1-yoshihiro.shimoda.uh@renesas.com
Fixes: 8353813c88 ("PCI: endpoint: Enable DMA tests for endpoints with DMA capabilities")
Tested-by: Kunihiko Hayashi <hayashi.kunihiko@socionext.com>
Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
In pci_epf_test_print_rate(), instead of open coding a reduction loop to
allow for a division by a 32-bits ns value, simply use div64_u64() to
calculate the transfer rate. To match the printed unit of KB/s, this
calculation divides the rate by 1000 instead of 1024 (that would be KiB/s
unit).
Change the format of the results printed by pci_epf_test_print_rate() to be
more compact without the double new line. Also use dev_info() instead of
pr_info().
Link: https://lore.kernel.org/r/20230415023542.77601-14-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
There is no need to have each read, write and copy test functions check
for the FLAG_USE_DMA flag against the DMA support status indicated by
epf_test->dma_supported. Move this test to the command handler function
pci_epf_test_cmd_handler() to check once for all cases.
Link: https://lore.kernel.org/r/20230415023542.77601-13-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Each of the test functions pci_epf_test_write(), pci_epf_test_read() and
pci_epf_test_copy() return an int result which is used by
pci_epf_test_cmd_handler() to set a success or error bit in the request
status.
In the spirit of keeping the processing of each test case self-contained
within its own test function, move the request status field update from
pci_epf_test_cmd_handler() to each of these test functions and change these
functions declaration to returning void.
Link: https://lore.kernel.org/r/20230415023542.77601-12-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Command codes are never combined together as flags into a single value.
Thus we can replace the series of "if" tests in pci_epf_test_cmd_handler()
with a cleaner switch-case statement. This also allows checking that we
got a valid command and print an error message if we did not.
Link: https://lore.kernel.org/r/20230415023542.77601-11-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
The pci-epf-test driver uses the test register BAR memory directly to get
and execute a test registers set by the RC side and defined using a struct
pci_epf_test_reg. This direct use relies on using the register BAR address
as a pointer to a struct pci_epf_test_reg to execute the test case and to
send back the test result through the status field of struct
pci_epf_test_reg. In practice, the status field is always updated before an
interrupt is raised in pci_epf_test_raise_irq(), to ensure that the RC side
sees the updated status when receiving an interrupt.
However, such assignment direct access does not ensure that changes to the
status register make it to memory, and so visible to the host, before an
interrupt is raised, thus potentially resulting in the RC host not seeing
the correct status result for a test.
Avoid this potential problem by using READ_ONCE()/WRITE_ONCE() when
accessing the command and status fields of a pci_epf_test_reg structure.
This ensure that a test start (pci_epf_test_cmd_handler() function) and
completion (with the function pci_epf_test_raise_irq()) achieve a correct
synchronization with the MMIO register accesses on the RC host.
Link: https://lore.kernel.org/r/20230415023542.77601-10-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
For the commands COMMAND_RAISE_LEGACY_IRQ, COMMAND_RAISE_MSI_IRQ and
COMMAND_RAISE_MSIX_IRQ, the function pci_epf_test_cmd_handler()
sets the STATUS_IRQ_RAISED status flag and calls the epc function
pci_epc_raise_irq() directly. However, this is also exactly what the
pci_epf_test_raise_irq() function does. Avoid duplicating these
operations by directly using pci_epf_test_raise_irq() for the IRQ test
commands. It is OK to do so as the host side endpoint test driver always
set the correct IRQ type for the IRQ test commands.
At the same time, move the IRQ number check done for the
COMMAND_RAISE_MSI_IRQ and COMMAND_RAISE_MSIX_IRQ commands
to pci_epf_test_raise_irq(), to also check the IRQ number requested
by the host for other test commands.
This significantly simplifies pci_epf_test_cmd_handler().
Link: https://lore.kernel.org/r/20230415023542.77601-9-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Change the interface of the function pci_epf_test_raise_irq() to directly
pass a pointer to the struct pci_epf_test_reg defining the test being
executed. This avoids the need for grabbing this pointer using the register
BAR address and simplifies the call sites as the IRQ type and IRQ numbers
do not have to be passed as arguments.
Link: https://lore.kernel.org/r/20230415023542.77601-8-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
The function pci_epf_test_cmd_handler() uses the register BAR address as a
pointer to a struct pci_epf_test_reg to determine the command sent by the
host and to execute the test function accordingly. There is no need for
doing this assignment again in each of the read, write and copy test
functions. We can simply pass the reg pointer as an argument to the
functions pci_epf_test_write(), pci_epf_test_read() and
pci_epf_test_copy().
Link: https://lore.kernel.org/r/20230415023542.77601-7-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Instead of an open coded call to the tx_submit() operation of struct
dma_async_tx_descriptor, use the helper function dmaengine_submit().
No functional change is introduced with this.
Link: https://lore.kernel.org/r/20230415023542.77601-6-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
pci_epf_test_data_transfer() and pci_epf_test_dma_callback() are not
handling DMA transfer completion correctly, leading to completion
notifications to the RC side that are too early. This problem can be
detected when the RC side is running an IOMMU with messages such as:
pci-endpoint-test 0000:0b:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x001c address=0xfff00000 flags=0x0000]
When running the pcitest.sh tests: the address used for a previous
test transfer generates the above error while the next test transfer is
running.
Fix this by testing the DMA transfer status in pci_epf_test_dma_callback()
and notifying the completion only when the transfer status is DMA_COMPLETE
or DMA_ERROR. Furthermore, in pci_epf_test_data_transfer(), be paranoid and
check again the transfer status and always call dmaengine_terminate_sync()
before returning.
Link: https://lore.kernel.org/r/20230415023542.77601-5-dlemoal@kernel.org
Fixes: 8353813c88 ("PCI: endpoint: Enable DMA tests for endpoints with DMA capabilities")
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Cc: stable@vger.kernel.org
Reinitialize the transfer_complete DMA transfer completion before calling
tx_submit(), to avoid seeing the DMA transfer complete before the
completion is initialized, thus potentially losing the completion
notification.
Link: https://lore.kernel.org/r/20230415023542.77601-4-dlemoal@kernel.org
Fixes: 8353813c88 ("PCI: endpoint: Enable DMA tests for endpoints with DMA capabilities")
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Cc: stable@vger.kernel.org
pci_epf_type_add_cfs() is called only from pci_ep_cfs_add_type_group() in
drivers/pci/endpoint/pci-ep-cfs.c, so there is no need to export this
function. Move its code from pci-epf-core.c to pci-ep-cfs.c as a static
function.
Link: https://lore.kernel.org/r/20230415023542.77601-3-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
A PCI endpoint function driver can define function specific attributes
under its function configfs directory using the add_cfs() endpoint driver
operation. This is done by tying up the mkdir operation for the function
configfs directory to a call to the add_cfs() operation. However, there
are no checks preventing the user from repeatedly creating function
specific attribute directories with different names, resulting in the same
endpoint specific attributes group being added multiple times, which also
result in an invalid reference counting for the attribute groups. E.g.,
using the pci-epf-ntb function driver as an example, the user creates the
function as follows:
$ modprobe pci-epf-ntb
$ cd /sys/kernel/config/pci_ep/functions/pci_epf_ntb
$ mkdir func0
$ tree func0
func0/
|-- baseclass_code
|-- cache_line_size
|-- ...
`-- vendorid
$ mkdir func0/attrs
$ tree func0
func0/
|-- attrs
| |-- db_count
| |-- mw1
| |-- mw2
| |-- mw3
| |-- mw4
| |-- num_mws
| `-- spad_count
|-- baseclass_code
|-- cache_line_size
|-- ...
`-- vendorid
At this point, the function can be started by linking the EP controller.
However, if the user mistakenly creates again a directory:
$ mkdir func0/attrs2
$ tree func0
func0/
|-- attrs
| |-- db_count
| |-- mw1
| |-- mw2
| |-- mw3
| |-- mw4
| |-- num_mws
| `-- spad_count
|-- attrs2
| |-- db_count
| |-- mw1
| |-- mw2
| |-- mw3
| |-- mw4
| |-- num_mws
| `-- spad_count
|-- baseclass_code
|-- cache_line_size
|-- ...
`-- vendorid
The endpoint function specific attributes are duplicated and cause a crash
when the endpoint function device is torn down:
refcount_t: addition on 0; use-after-free.
WARNING: CPU: 2 PID: 834 at lib/refcount.c:25 refcount_warn_saturate+0xc8/0x144
CPU: 2 PID: 834 Comm: rmdir Not tainted 6.3.0-rc1 #1
Hardware name: Pine64 RockPro64 v2.1 (DT)
pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
...
Call trace:
refcount_warn_saturate+0xc8/0x144
config_item_get+0x7c/0x80
configfs_rmdir+0x17c/0x30c
vfs_rmdir+0x8c/0x204
do_rmdir+0x158/0x184
__arm64_sys_unlinkat+0x64/0x80
invoke_syscall+0x48/0x114
...
Fix this by modifying pci_epf_cfs_work() to execute the new function
pci_ep_cfs_add_type_group() which itself calls pci_epf_type_add_cfs() to
obtain the function specific attribute group and the group name (directory
name) from the endpoint function driver. If the function driver defines an
attribute group, pci_ep_cfs_add_type_group() then proceeds to register this
group using configfs_register_group(), thus automatically exposing the
function type specific configfs attributes to the user. E.g.:
$ modprobe pci-epf-ntb
$ cd /sys/kernel/config/pci_ep/functions/pci_epf_ntb
$ mkdir func0
$ tree func0
func0/
|-- baseclass_code
|-- cache_line_size
|-- ...
|-- pci_epf_ntb.0
| |-- db_count
| |-- mw1
| |-- mw2
| |-- mw3
| |-- mw4
| |-- num_mws
| `-- spad_count
|-- primary
|-- ...
`-- vendorid
With this change, there is no need for the user to create or delete
directories in the endpoint function attributes directory. The
pci_epf_type_group_ops group operations are thus removed.
Also update the documentation for the pci-epf-ntb and pci-epf-vntb function
drivers to reflect this change, removing the explanations showing the need
to manually create the sub-directory for the function specific attributes.
Link: https://lore.kernel.org/r/20230415023542.77601-2-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
vNTB driver and NTB driver have same Kconfig prompt. Changed to make it
distinguishable.
Link: https://lore.kernel.org/r/20230202103832.2038286-1-mie@igel.co.jp
Fixes: e35f56bb03 ("PCI: endpoint: Support NTB transfer between RC and EP")
Signed-off-by: Shunsuke Mie <mie@igel.co.jp>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
The module pointer in class_create() never actually did anything, and it
shouldn't have been requred to be set as a parameter even if it did
something. So just remove it and fix up all callers of the function in
the kernel tree at the same time.
Cc: "Rafael J. Wysocki" <rafael@kernel.org>
Acked-by: Benjamin Tissoires <benjamin.tissoires@redhat.com>
Link: https://lore.kernel.org/r/20230313181843.1207845-4-gregkh@linuxfoundation.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
- Convert dra7xx to threaded IRQ handler (Manivannan Sadhasivam)
- Move tegra194 dw_pcie_ep_linkup() to threaded IRQ handler (Manivannan
Sadhasivam)
- Add a separate lock for the endpoint pci_epf list to avoid deadlock
while running callbacks (Manivannan Sadhasivam)
- Use callbacks instead of notifier chains to signal events from EPC to EPF
drivers (Manivannan Sadhasivam)
- Use link_up() callback in place of LINK_UP notifier (Manivannan
Sadhasivam)
* pci/endpoint:
PCI: endpoint: Use link_up() callback in place of LINK_UP notifier
PCI: endpoint: Use callback mechanism for passing events from EPC to EPF
PCI: endpoint: Use a separate lock for protecting epc->pci_epf list
PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler
PCI: dra7xx: Use threaded IRQ handler for "dra7xx-pcie-main" IRQ
Since 8b41fc4454 ("kbuild: create modules.builtin without
Makefile.modbuiltin or tristate.conf"), MODULE_LICENSE declarations are
used to identify modules. As a consequence, MODULE_LICENSE in non-modules
causes modprobe to misidentify the object file as a module when it is not,
and modprobe might succeed rather than failing with a suitable error
message.
For tristate modules that can be either built-in or loaded at runtime,
modprobe succeeds in both cases:
# modprobe ext4
[exit status zero if CONFIG_EXT4_FS=y or =m]
For boolean modules like the Standard Hot Plug Controller driver (shpchp)
that cannot be loaded at runtime, modprobe should always fail like this:
# modprobe shpchp
modprobe: FATAL: Module shpchp not found in directory /lib/modules/...
[exit status non-zero regardless of CONFIG_HOTPLUG_PCI_SHPC]
but prior to this commit, shpchp_core.c contained MODULE_LICENSE, so
"modprobe shpchp" silently succeeded when it should have failed.
Remove MODULE_LICENSE in files that cannot be built as modules.
[bhelgaas: commit log, squash]
Suggested-by: Luis Chamberlain <mcgrof@kernel.org>
Link: https://lore.kernel.org/r/20230216152410.4312-1-nick.alcock@oracle.com/
Signed-off-by: Nick Alcock <nick.alcock@oracle.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Cc: Luis Chamberlain <mcgrof@kernel.org>
Cc: Hitomi Hasegawa <hasegawa-hitomi@fujitsu.com>
Cc: Rob Herring <robh@kernel.org>
Cc: Lorenzo Pieralisi <lpieralisi@kernel.org>
As a part of the transition towards callback mechanism for signalling the
events from EPC to EPF, let's use the link_up() callback in the place of
the LINK_UP notifier. This also removes the notifier support completely
from the PCI endpoint framework.
Link: https://lore.kernel.org/linux-pci/20230124071158.5503-6-manivannan.sadhasivam@linaro.org
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Acked-by: Kishon Vijay Abraham I <kishon@kernel.org>
Instead of using the notifiers for passing the events from EPC to EPF,
let's introduce a callback based mechanism where the EPF drivers can
populate relevant callbacks for EPC events they want to subscribe.
The use of notifiers in kernel is not recommended if there is a real link
between the sender and receiver, like in this case. Also, the existing
atomic notifier forces the notification functions to be in atomic context
while the caller may be in non-atomic context. For instance, the two
in-kernel users of the notifiers, pcie-qcom and pcie-tegra194, both are
calling the notifier functions in non-atomic context (from threaded IRQ
handlers). This creates a sleeping in atomic context issue with the
existing EPF_TEST driver that calls the EPC APIs that may sleep.
For all these reasons, let's get rid of the notifier chains and use the
simple callback mechanism for signalling the events from EPC to EPF
drivers. This preserves the context of the caller and avoids the latency
of going through a separate interface for triggering the notifications.
As a first step of the transition, the core_init() callback is introduced
in this commit, that'll replace the existing CORE_INIT notifier used for
signalling the init complete event from EPC.
During the occurrence of the event, EPC will go over the list of EPF
drivers attached to it and will call the core_init() callback if available.
Link: https://lore.kernel.org/linux-pci/20230124071158.5503-5-manivannan.sadhasivam@linaro.org
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Acked-by: Kishon Vijay Abraham I <kishon@kernel.org>
The EPC controller maintains a list of EPF drivers added to it. For
protecting this list against the concurrent accesses, the epc->lock
(used for protecting epc_ops) has been used so far. Since there were
no users trying to use epc_ops and modify the pci_epf list simultaneously,
this was not an issue.
But with the addition of callback mechanism for passing the events, this
will be a problem. Because the pci_epf list needs to be iterated first
for getting hold of the EPF driver and then the relevant event specific
callback needs to be called for the driver.
If the same epc->lock is used, then it will result in a deadlock scenario.
For instance,
...
mutex_lock(&epc->lock);
list_for_each_entry(epf, &epc->pci_epf, list) {
epf->event_ops->core_init(epf);
|
|-> pci_epc_set_bar();
|
|-> mutex_lock(&epc->lock) # DEADLOCK
...
So to fix this issue, use a separate lock called "list_lock" for
protecting the pci_epf list against the concurrent accesses. This lock
will also be used by the callback mechanism.
Link: https://lore.kernel.org/linux-pci/20230124071158.5503-4-manivannan.sadhasivam@linaro.org
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>
8e4bfbe644 ("PCI: endpoint: pci-epf-vntb: fix error handle in
epf_ntb_mw_bar_init()") added a "num_mws" parameter to
epf_ntb_mw_bar_clear() but failed to add kernel-doc for num_mws.
Add kernel-doc for num_mws on epf_ntb_mw_bar_clear().
Fixes: 8e4bfbe644 ("PCI: endpoint: pci-epf-vntb: fix error handle in epf_ntb_mw_bar_init()")
Link: https://lore.kernel.org/r/20230103024907.293853-1-yangyingliang@huawei.com
Signed-off-by: Yang Yingliang <yangyingliang@huawei.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Use epf_db[i] dereference instead of readl() because epf_db is
in memory allocated by dma_alloc_coherent(), not I/O.
Remove useless/duplicated readl() in the process.
Link: https://lore.kernel.org/r/20221102141014.1025893-7-Frank.Li@nxp.com
Signed-off-by: Frank Li <frank.li@nxp.com>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Dan Carpenter