Add a mechanism to identify standard PCIe regulators in the DT, allocate
them, and turn them on before the rest of the bus is scanned during
pci_host_probe().
The allocated structure that contains the regulators is stored in the port
driver dev.driver_data field. Here is a point-by-point of how and when
this mechanism is activated:
If:
-- PCIe RC driver sets pci_ops {add,remove)_bus to
pci_subdev_regulators_{add,remove}_bus during its probe.
-- There is a DT node "RB" under the host bridge DT node.
-- During the RC driver's pci_host_probe() the add_bus callback
is invoked where (bus->parent && pci_is_root_bus(bus->parent)
is true
Then:
-- A struct subdev_regulators structure will be allocated and
assigned to bus->dev.driver_data.
-- regulator_bulk_{get,enable} will be invoked on &bus->dev
and the former will search for and process any
vpcie{12v,3v3,3v3aux}-supply properties that reside in node "RB".
-- The regulators will be turned off/on for any unbind/bind operations.
-- The regulators will be turned off/on for any suspend/resumes, but
only if the RC driver handles this on its own. This will appear
in a later commit for the pcie-brcmstb.c driver.
The unabridged reason for doing this is as follows. We would like the
Broadcom STB PCIe root complex driver (and others) to be able to turn
off/on regulators[1] that provide power to endpoint[2] devices. Typically,
the drivers of these endpoint devices are stock Linux drivers that are not
aware that these regulator(s) exist and must be turned on for the driver to
be probed. The simple solution of course is to turn these regulators on at
boot and keep them on. However, this solution does not satisfy at least
three of our usage modes:
1. For example, one customer uses multiple PCIe controllers, but wants
the ability to, by script invoking and unbind, turn any or all of them
and their subdevices off to save power, e.g. when in battery mode.
2. Another example is when a watchdog script discovers that an endpoint
device is in an unresponsive state and would like to unbind, power
toggle, and re-bind just the PCIe endpoint and controller.
3. Of course we also want power turned off during suspend mode. However,
some endpoint devices may be able to "wake" during suspend and we need
to recognise this case and veto the nominal act of turning off its
regulator. Such is the case with Wake-on-LAN and Wake-on-WLAN support
where the PCIe endpoint device needs to be kept powered on in order to
receive network packets and wake the system.
In all of these cases it is advantageous for the PCIe controller to govern
the turning off/on the regulators needed by the endpoint device. The first
two cases can be done by simply unbinding and binding the PCIe controller,
if the controller has control of these regulators.
[1] These regulators typically govern the actual power supply to the
endpoint chip. Sometimes they may be the official PCIe socket
power -- such as 3.3v or aux-3.3v. Sometimes they are truly
the regulator(s) that supply power to the EP chip.
[2] The 99% configuration of our boards is a single endpoint device
attached to the PCIe controller. I use the term endpoint but it could
possibly mean a switch as well.
Link: https://lore.kernel.org/r/20220106160332.2143-6-jim2101024@gmail.com
Signed-off-by: Jim Quinlan <jim2101024@gmail.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
We need to take some code in brcm_pcie_setup() and put it in a new function
brcm_pcie_linkup(). In future commits the brcm_pcie_linkup() function will
be called indirectly by pci_host_probe() as opposed to the host driver
invoking it directly.
Some code that was executed after the PCIe linkup is now placed so that it
executes prior to linkup, since this code has to run prior to the
invocation of pci_host_probe().
Link: https://lore.kernel.org/r/20220106160332.2143-5-jim2101024@gmail.com
Signed-off-by: Jim Quinlan <jim2101024@gmail.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Do at least a dev_err() on some calls to reset_control_rearm() and
brcm_phy_stop(). In some cases it may not make sense to return this error
value "above" as doing so will cause more trouble than is warranted.
Link: https://lore.kernel.org/r/20220106160332.2143-2-jim2101024@gmail.com
Signed-off-by: Jim Quinlan <jim2101024@gmail.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Florian Fainelli <f.fainelli@gmail.com>
Define the legacy MSI interrupt bitmask as well as the non-legacy interrupt
bitmask using GENMASK and then use them in brcm_msi_set_regs() in place of
__GENMASK().
Link: https://lore.kernel.org/r/20211122190459.3189616-1-f.fainelli@gmail.com
Reported-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com>
The 'used' field of 'struct brcm_msi' is used as a bitmap. Declare it with
DECLARE_BITMAP() and adjust users accordingly.
This fixes a harmless Coverity warning about array vs singleton usage.
This bitmap can be used for either legacy or MSI interrupts, which require
a size of BRCM_INT_PCI_MSI_LEGACY_NR or BRCM_INT_PCI_MSI_NR respectively.
Add a BUILD_BUG_ON() to ensure it is large enough.
Suggested-by: Krzysztof Wilczynski <kw@linux.com>
Addresses-Coverity: "Out-of-bounds access (ARRAY_VS_SINGLETON)"
Link: https://lore.kernel.org/r/e6d9da2112aab2939d1507b90962d07bfd735b4c.1636273671.git.christophe.jaillet@wanadoo.fr
Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
Add arm64 Hyper-V vPCI support by implementing the arch specific
interfaces. Introduce an IRQ domain and chip specific to Hyper-v vPCI that
is based on SPIs. The IRQ domain parents itself to the arch GIC IRQ domain
for basic vector management.
[bhelgaas: squash in fix from Yang Li <yang.lee@linux.alibaba.com>:
https://lore.kernel.org/r/20220112003324.62755-1-yang.lee@linux.alibaba.com]
Link: https://lore.kernel.org/r/1641411156-31705-3-git-send-email-sunilmut@linux.microsoft.com
Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Michael Kelley <mikelley@microsoft.com>
Encapsulate arch dependencies in Hyper-V vPCI through a set of
arch-dependent interfaces. Adding these arch specific interfaces will
allow for an implementation for other architectures, such as arm64.
There are no functional changes expected from this patch.
Link: https://lore.kernel.org/r/1641411156-31705-2-git-send-email-sunilmut@linux.microsoft.com
Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Boqun Feng <boqun.feng@gmail.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Michael Kelley <mikelley@microsoft.com>
Use down_read_nested() and down_write_nested() when taking the
ctrl->reset_lock rw-sem, passing the number of PCIe hotplug controllers in
the path to the PCI root bus as lock subclass parameter.
This fixes the following false-positive lockdep report when unplugging a
Lenovo X1C8 from a Lenovo 2nd gen TB3 dock:
pcieport 0000:06:01.0: pciehp: Slot(1): Link Down
pcieport 0000:06:01.0: pciehp: Slot(1): Card not present
============================================
WARNING: possible recursive locking detected
5.16.0-rc2+ #621 Not tainted
--------------------------------------------
irq/124-pciehp/86 is trying to acquire lock:
ffff8e5ac4299ef8 (&ctrl->reset_lock){.+.+}-{3:3}, at: pciehp_check_presence+0x23/0x80
but task is already holding lock:
ffff8e5ac4298af8 (&ctrl->reset_lock){.+.+}-{3:3}, at: pciehp_ist+0xf3/0x180
other info that might help us debug this:
Possible unsafe locking scenario:
CPU0
----
lock(&ctrl->reset_lock);
lock(&ctrl->reset_lock);
*** DEADLOCK ***
May be due to missing lock nesting notation
3 locks held by irq/124-pciehp/86:
#0: ffff8e5ac4298af8 (&ctrl->reset_lock){.+.+}-{3:3}, at: pciehp_ist+0xf3/0x180
#1: ffffffffa3b024e8 (pci_rescan_remove_lock){+.+.}-{3:3}, at: pciehp_unconfigure_device+0x31/0x110
#2: ffff8e5ac1ee2248 (&dev->mutex){....}-{3:3}, at: device_release_driver+0x1c/0x40
stack backtrace:
CPU: 4 PID: 86 Comm: irq/124-pciehp Not tainted 5.16.0-rc2+ #621
Hardware name: LENOVO 20U90SIT19/20U90SIT19, BIOS N2WET30W (1.20 ) 08/26/2021
Call Trace:
<TASK>
dump_stack_lvl+0x59/0x73
__lock_acquire.cold+0xc5/0x2c6
lock_acquire+0xb5/0x2b0
down_read+0x3e/0x50
pciehp_check_presence+0x23/0x80
pciehp_runtime_resume+0x5c/0xa0
device_for_each_child+0x45/0x70
pcie_port_device_runtime_resume+0x20/0x30
pci_pm_runtime_resume+0xa7/0xc0
__rpm_callback+0x41/0x110
rpm_callback+0x59/0x70
rpm_resume+0x512/0x7b0
__pm_runtime_resume+0x4a/0x90
__device_release_driver+0x28/0x240
device_release_driver+0x26/0x40
pci_stop_bus_device+0x68/0x90
pci_stop_bus_device+0x2c/0x90
pci_stop_and_remove_bus_device+0xe/0x20
pciehp_unconfigure_device+0x6c/0x110
pciehp_disable_slot+0x5b/0xe0
pciehp_handle_presence_or_link_change+0xc3/0x2f0
pciehp_ist+0x179/0x180
This lockdep warning is triggered because with Thunderbolt, hotplug ports
are nested. When removing multiple devices in a daisy-chain, each hotplug
port's reset_lock may be acquired recursively. It's never the same lock, so
the lockdep splat is a false positive.
Because locks at the same hierarchy level are never acquired recursively, a
per-level lockdep class is sufficient to fix the lockdep warning.
The choice to use one lockdep subclass per pcie-hotplug controller in the
path to the root-bus was made to conserve class keys because their number
is limited and the complexity grows quadratically with number of keys
according to Documentation/locking/lockdep-design.rst.
Link: https://lore.kernel.org/linux-pci/20190402021933.GA2966@mit.edu/
Link: https://lore.kernel.org/linux-pci/de684a28-9038-8fc6-27ca-3f6f2f6400d7@redhat.com/
Link: https://lore.kernel.org/r/20211217141709.379663-1-hdegoede@redhat.com
Link: https://bugzilla.kernel.org/show_bug.cgi?id=208855
Reported-by: "Theodore Ts'o" <tytso@mit.edu>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Lukas Wunner <lukas@wunner.de>
Cc: stable@vger.kernel.org
Some devices have an erratum such that they only support DWORD accesses to
some registers. E.g., this Bayhub O2 device ([VID:DID] = [0x1217:0x8621])
only supports DWORD accesses to LTR latency registers and L1 PM substates
control registers:
https://github.com/rajatxjain/public_shared/blob/main/OZ711LV2_appnote.pdf
The L1 PM substate control registers are DWORD sized, and hence their
access in the kernel is already DWORD sized, so we don't need to do
anything for them.
However, the LTR registers being WORD sized, are in need of a solution.
Convert the WORD sized accesses to these registers into DWORD sized
accesses while saving and restoring them.
Link: https://lore.kernel.org/r/20211222012105.3438916-1-rajatja@google.com
Signed-off-by: Rajat Jain <rajatja@google.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Per PCIe r5, sec 7.5.1.2.4, a device must not claim accesses to its
Expansion ROM unless both the Memory Space Enable and the Expansion ROM
Enable bit are set. But apparently some Intel I210 NICs don't work
correctly if the ROM BAR overlaps another BAR, even if the Expansion ROM is
disabled.
Michael reported that on a Kontron SMARC-sAL28 ARM64 system with U-Boot
v2021.01-rc3, the ROM BAR overlaps BAR 3, and networking doesn't work at
all:
BAR 0: 0x40000000 (32-bit, non-prefetchable) [size=1M]
BAR 3: 0x40200000 (32-bit, non-prefetchable) [size=16K]
ROM: 0x40200000 (disabled) [size=1M]
NETDEV WATCHDOG: enP2p1s0 (igb): transmit queue 0 timed out
Hardware name: Kontron SMARC-sAL28 (Single PHY) on SMARC Eval 2.0 carrier (DT)
igb 0002:01:00.0 enP2p1s0: Reset adapter
Previously, pci_std_update_resource() wrote the assigned ROM address to the
BAR only when the ROM was enabled. This meant that the I210 ROM BAR could
be left with an address assigned by firmware, which might overlap with
other BARs.
Quirk these I210 devices so pci_std_update_resource() always writes the
assigned address to the ROM BAR, whether or not the ROM is enabled.
Link: https://lore.kernel.org/r/20211223163754.GA1267351@bhelgaas
Link: https://lore.kernel.org/r/20201230185317.30915-1-michael@walle.cc
Link: https://bugzilla.kernel.org/show_bug.cgi?id=211105
Reported-by: Michael Walle <michael@walle.cc>
Tested-by: Michael Walle <michael@walle.cc>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
The current brcmstb driver works for Arm and Arm64. A few things are
modified here for us to support MIPs as well.
o There are four outbound range register groups and each directs a window
of up to 128MB. Even though there are four 128MB DT "ranges" in the
bmips PCIe DT node, these ranges are contiguous and are collapsed into
a single range by the OF range parser. Now the driver assumes a single
range -- for MIPs only -- and splits it back into 128MB sizes.
o For bcm7425, the config space accesses must be 32-bit reads or
writes. In addition, the 4k config space register array is missing
and not used.
o The registers for the upper 32-bits of the outbound window address do
not exist.
o Burst size must be set to 256 (this refers to an internal bus).
Signed-off-by: Jim Quinlan <jim2101024@gmail.com>
Acked-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
- Remove device_add_properties() which does not work correctly if
software nodes holding additional device properties are shared
or reused (Heikki Krogerus).
- Fix nargs_prop property handling for software nodes (Clément Léger).
- Update documentation of ACPI device properties (Sakari Ailus).
- Update the handling of graph properties in the generic framework
to match the DT case (Sakari Ailus).
- Update software nodes entry in MAINTAINERS (Andy Shevchenko).
-----BEGIN PGP SIGNATURE-----
iQJGBAABCAAwFiEE4fcc61cGeeHD/fCwgsRv/nhiVHEFAmHcg1ASHHJqd0Byand5
c29ja2kubmV0AAoJEILEb/54YlRx66YP/Ar0GTqI6iE+6WrF2B00ftHFq11PVTo9
DNiYSJiV60FiaoIANd+59QfC1i2erDrUmRFvZ+Kip4rQG9UomY4Tx7QYw3jzCj8t
yA5JJhN5kYphs0+GigdNVLkN56zwK6Cd759+GVCXQW9KkPryxhjGJeNNRthlXf8s
Bs+4Gp3NOanl5z4LdulkjyoFMDr6kIjotcN2j8+7RkgWc4VIS3OYlFZf5dLi/i1O
cqSkJgikRA40ZuNtsRRVGPOSGoPAPaAFZJY4j/gYx6sAhB9UQi/Xe4iQaXvsEWM2
NlC+X2D48SiTkb1M1QpM0nJ5N/txRp+/FrMiAagBWCNer4lFA42ibyCd6dhADnIE
lMLnbKaUqB3exCBP/BQdYDi+ypKZf88E0zX6OoZfvHj0uQV5KwOgUbbhckpLkI/j
WZQwm/qtLGqpxW6N+IfBRwBBwPkXePep3CG37twfyVp4IXk+hm+ipMQ1dZmxwNKZ
q9o9Iwv35KEbX6nR8psE7GCm6znYeoFPDx8GEjEDh9nfIpt2bFSYy3rw82wghkgq
EeBD/irNS3PbFXyt8cV/cDjctgbG9SumOA5B6Iicq9y5PSnUNjKi49DGZAsqq8g/
I2c5IFBzYwnKk4z1wCRshba9jTF2sH0NTGVvPnfWv/vCtrvURXwBxTa9z8jtuMtw
E44VNKDhYWu2
=/oIe
-----END PGP SIGNATURE-----
Merge tag 'devprop-5.17-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm
Pull device properties framework updates from Rafael Wysocki:
"These update the handling of software nodes and graph properties, and
the MAINTAINERS entry for the former.
Specifics:
- Remove device_add_properties() which does not work correctly if
software nodes holding additional device properties are shared or
reused (Heikki Krogerus).
- Fix nargs_prop property handling for software nodes (Clément
Léger).
- Update documentation of ACPI device properties (Sakari Ailus).
- Update the handling of graph properties in the generic framework to
match the DT case (Sakari Ailus).
- Update software nodes entry in MAINTAINERS (Andy Shevchenko)"
* tag 'devprop-5.17-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm:
software node: Update MAINTAINERS data base
software node: fix wrong node passed to find nargs_prop
device property: Drop fwnode_graph_get_remote_node()
device property: Use fwnode_graph_for_each_endpoint() macro
device property: Implement fwnode_graph_get_endpoint_count()
Documentation: ACPI: Update references
Documentation: ACPI: Fix data node reference documentation
device property: Fix documentation for FWNODE_GRAPH_DEVICE_DISABLED
device property: Fix fwnode_graph_devcon_match() fwnode leak
device property: Remove device_add_properties() API
driver core: Don't call device_remove_properties() from device_del()
PCI: Convert to device_create_managed_software_node()
Fix a number of misspelled words, and while at it, correct two phrases used
to indicate a status of an operation where words used have been cleverly
truncated and thus always trigger a spellchecking error while performing a
static code analysis over the PCI tree.
[bhelgaas: reverse sense of quirk ternary]
Link: https://lore.kernel.org/r/20220107225942.121484-1-kw@linux.com
Signed-off-by: Krzysztof Wilczyński <kw@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Get "syscon" pcie_mode and pcie_id offset from the argument of
"ti,syscon-pcie-id" and "ti,syscon-pcie-mode" phandle respectively.
Previously a subnode to "syscon" node was added which has the
exact memory mapped address of pcie_mode and pcie_id but now the
offset of pcie_mode and pcie_id within "syscon" is now being passed
as argument to "ti,syscon-pcie-id" and "ti,syscon-pcie-mode" phandle.
If the offset is not provided in "ti,syscon-pcie-id"/"ti,syscon-pcie-mode",
the full memory mapped address of pcie_ctrl is used in order to maintain
old DT compatibility.
Similar change for J721E is as discussed in [1]
[1] -> http://lore.kernel.org/r/CAL_JsqKiUcO76bo1GoepWM1TusJWoty_BRy2hFSgtEVMqtrvvQ@mail.gmail.com
Link: https://lore.kernel.org/r/20211126083119.16570-3-kishon@ti.com
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Described in PCIe CEM specification sections 2.2 (PERST# Signal) and
2.2.1 (Initial Power-Up (G3 to S0)). The deassertion of PERST# should
be delayed 100ms (TPVPERL) for the power and clock to become stable.
Link: https://lore.kernel.org/r/20211227133110.14500-1-qizhong.cheng@mediatek.com
Signed-off-by: qizhong cheng <qizhong.cheng@mediatek.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Pali Rohár <pali@kernel.org>
Now when driver uses devm_pci_remap_iospace() function, it is possible
implement ->remove() callback for unbinding device from driver.
Implement mvebu_pcie_remove() callback with proper cleanup phase, drop
driver's suppress_bind_attrs flag and switch type of CONFIG_PCI_MVEBU
option from bool to tristate.
This allows to compile pci-mvebu.c driver as loadable module pci-mvebu.ko
with ability to unload it.
Link: https://lore.kernel.org/r/20211126144307.7568-3-pali@kernel.org
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Add support for this VMD device which supports the bus restriction mode.
The feature that turns off vector 0 for MSI-X remapping is also enabled.
Link: https://lore.kernel.org/r/20211217231211.46018-1-francisco.munoz.ruiz@linux.intel.com
Signed-off-by: Karthik L Gopalakrishnan <karthik.l.gopalakrishnan@intel.com>
Signed-off-by: Francisco Munoz <francisco.munoz.ruiz@linux.intel.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Jon Derrick <jonathan.derrick@linux.dev>
The current Layerscape PCIe driver directly uses the physical layer
LTSSM code to check the link-up state, which treats the > L0 states
as link-up. This is not correct, since there is not explicit map
between link-up state and LTSSM. So this patch changes to use the
DWC common link-up check function.
Link: https://lore.kernel.org/r/20211224094000.8513-1-Zhiqiang.Hou@nxp.com
Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Rob Herring <robh@kernel.org>
The only usage of pci_ep_ops is to assign its address to the ops field
in the dw_pcie_ep struct which is a pointer to const struct dw_pcie_ep_ops.
Make it const to allow the compiler to put it in read-only memory.
Link: https://lore.kernel.org/r/20211204220316.88655-1-rikard.falkeborn@gmail.com
Signed-off-by: Rikard Falkeborn <rikard.falkeborn@gmail.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Krzysztof Wilczyński <kw@linux.com>
When Samsung PCIe Gen4 NVMe is connected to Intel ADL VMD, the
combination causes AER message flood and drags the system performance
down.
The issue doesn't happen when VMD mode is disabled in BIOS, since AER
isn't enabled by acpi_pci_root_create() . When VMD mode is enabled, AER
is enabled regardless of _OSC:
[ 0.410076] acpi PNP0A08:00: _OSC: platform does not support [AER]
...
[ 1.486704] pcieport 10000:e0:06.0: AER: enabled with IRQ 146
Since VMD is an aperture to regular PCIe root ports, honor ACPI _OSC to
disable PCIe features accordingly to resolve the issue.
Suggested-by: Rafael J. Wysocki <rafael@kernel.org>
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=215027
Link: https://lore.kernel.org/r/20211203031541.1428904-1-kai.heng.feng@canonical.com
Signed-off-by: Kai-Heng Feng <kai.heng.feng@canonical.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Armada XP and new hardware supports access to DEVCAP2, DEVCTL2 and LNKCTL2
configuration registers of PCIe core via PCIE_CAP_PCIEXP. So export them
via emulated software root bridge.
Pre-XP hardware does not support these registers and returns zeros.
Link: https://lore.kernel.org/r/20211125124605.25915-16-pali@kernel.org
Fixes: 1f08673eef ("PCI: mvebu: Convert to PCI emulated bridge config space")
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
PME Status bit in Root Status Register (PCIE_RC_RTSTA_OFF) is read-only and
can be cleared only by writing 0b to the Interrupt Cause RW0C register
(PCIE_INT_CAUSE_OFF).
Link: https://lore.kernel.org/r/20211125124605.25915-15-pali@kernel.org
Fixes: 1f08673eef ("PCI: mvebu: Convert to PCI emulated bridge config space")
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Comment in Armada 370 functional specification is misleading.
PCI_EXP_DEVCTL_*RE bits are supported and configures receiving of error
interrupts.
Link: https://lore.kernel.org/r/20211125124605.25915-14-pali@kernel.org
Fixes: 1f08673eef ("PCI: mvebu: Convert to PCI emulated bridge config space")
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Hardware supports PCIe Hot Reset via PCIE_CTRL_OFF register. Use it for
implementing PCI_BRIDGE_CTL_BUS_RESET bit of PCI_BRIDGE_CONTROL register on
emulated bridge.
With this change the function pci_reset_secondary_bus() starts working and
can reset connected PCIe card.
Link: https://lore.kernel.org/r/20211125124605.25915-13-pali@kernel.org
Fixes: 1f08673eef ("PCI: mvebu: Convert to PCI emulated bridge config space")
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
It looks like that mvebu PCIe controller has for each PCIe link fully
independent PCIe host bridge and so every PCIe Root Port is isolated not
only on its own bus but also isolated from each others. But in past device
tree structure was defined to put all PCIe Root Ports (as PCI Bridge
devices) into one root bus 0 and this bus is emulated by pci-mvebu.c
driver.
Probably reason for this decision was incorrect understanding of PCIe
topology of these Armada SoCs and also reason of misunderstanding how is
PCIe controller generating Type 0 and Type 1 config requests (it is fully
different compared to other drivers). Probably incorrect setup leaded to
very surprised things like having PCIe Root Port (PCI Bridge device, with
even incorrect Device Class set to Memory Controller) and the PCIe device
behind the Root Port on the same PCI bus, which obviously was needed to
somehow hack (as these two devices cannot be in reality on the same bus).
Properly set mvebu local bus number and mvebu local device number based on
PCI Bridge secondary bus number configuration. Also correctly report
configured secondary bus number in config space. And explain in driver
comment why this setup is correct.
Link: https://lore.kernel.org/r/20211125124605.25915-12-pali@kernel.org
Fixes: 1f08673eef ("PCI: mvebu: Convert to PCI emulated bridge config space")
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
The default value of Class Code of this bridge corresponds to a Memory
controller, though. This is probably relict from the past when old
Marvell/Galileo PCI-based controllers were used as standalone PCI device
for connecting SDRAM or workaround for PCs with broken BIOS. Details are
in commit 36de23a4c5 ("MIPS: Cobalt: Explain GT64111 early PCI fixup").
Change the Class Code to correspond to a PCI Bridge.
Add comment explaining this change.
Link: https://lore.kernel.org/r/20211125124605.25915-11-pali@kernel.org
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
This driver operates only in Root Complex mode, so ensure that hardware is
properly configured in Root Complex mode.
Link: https://lore.kernel.org/r/20211125124605.25915-10-pali@kernel.org
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Properly propagate failure from mvebu_pcie_add_windows() function back to
the caller mvebu_pci_bridge_emul_base_conf_write() and correctly updates
PCI_IO_BASE, PCI_MEM_BASE and PCI_IO_BASE_UPPER16 registers on error.
On error set base value higher than limit value which indicates that
address range is disabled. When IO is unsupported then let IO registers
zeroed as required by PCIe base specification.
Link: https://lore.kernel.org/r/20211125124605.25915-9-pali@kernel.org
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
PCI IO type bits are already initialized in mvebu_pci_bridge_emul_init()
function and only when IO support is enabled. These type bits are read-only
and pci-bridge-emul.c code already does not allow to modify them from upper
layers.
When IO support is disabled then all IO registers should be read-only and
return zeros. Therefore do not modify PCI IO type bits in
mvebu_pci_bridge_emul_base_conf_write() callback.
Link: https://lore.kernel.org/r/20211125124605.25915-8-pali@kernel.org
Fixes: 1f08673eef ("PCI: mvebu: Convert to PCI emulated bridge config space")
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
According to PCI specifications bits [0:2] of Command Register, this should
be by default disabled on reset. So explicitly disable these bits at early
beginning of driver initialization.
Also remove code which unconditionally enables all 3 bits and let kernel
code (via pci_set_master() function) to handle bus mastering of PCI Bridge
via emulated PCI_COMMAND on emulated bridge.
Adjust existing functions mvebu_pcie_handle_iobase_change() and
mvebu_pcie_handle_membase_change() to handle PCI_IO_BASE and PCI_MEM_BASE
registers correctly even when bus mastering on emulated bridge is disabled.
Link: https://lore.kernel.org/r/20211125124605.25915-7-pali@kernel.org
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Interrupt support on mvebu emulated bridges is not implemented yet.
So properly indicate return value to callers that they cannot request
interrupts from emulated bridge.
Link: https://lore.kernel.org/r/20211125124605.25915-6-pali@kernel.org
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Function mvebu_pcie_hw_rd_conf() does not handle invalid size. So correctly
set read value to all-ones and return appropriate error return value
PCIBIOS_BAD_REGISTER_NUMBER like in mvebu_pcie_hw_wr_conf() function.
Link: https://lore.kernel.org/r/20211125124605.25915-5-pali@kernel.org
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Driver cannot handle PCI bridges at non-zero function address. So add
appropriate check. Currently all in-tree kernel DTS files set PCI bridge
function to zero.
Link: https://lore.kernel.org/r/20211125124605.25915-4-pali@kernel.org
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Function pci_bridge_emul_init() may fail so correctly check for errors.
Link: https://lore.kernel.org/r/20211125124605.25915-3-pali@kernel.org
Fixes: 1f08673eef ("PCI: mvebu: Convert to PCI emulated bridge config space")
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Some mvebu ports do not have to be initialized. So skip these uninitialized
mvebu ports in every port iteration function to prevent access to unmapped
memory or dereferencing NULL pointers. Uninitialized mvebu port has base
address set to NULL.
Link: https://lore.kernel.org/r/20211125124605.25915-2-pali@kernel.org
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
The sparse checker complains about converting pointers between address
spaces. We correctly stored an __iomem pointer in struct spear13xx_pcie,
but discarded the __iomem when extracting app_base, causing one warning.
Then we passed the non-__iomem pointer to writel(), which expects an
__iomem pointer, causing another warning.
Add the appropriate annotations.
The sparse warnings look like this:
$ make C=2 drivers/pci/controller/
drivers/pci/controller/dwc/pcie-spear13xx.c:72:54: warning: incorrect type in initializer (different address spaces)
drivers/pci/controller/dwc/pcie-spear13xx.c:72:54: expected struct pcie_app_reg *app_reg
drivers/pci/controller/dwc/pcie-spear13xx.c:72:54: got void [noderef] __iomem *app_base
drivers/pci/controller/dwc/pcie-spear13xx.c:78:26: warning: incorrect type in argument 2 (different address spaces)
drivers/pci/controller/dwc/pcie-spear13xx.c:78:26: expected void volatile [noderef] __iomem *addr
drivers/pci/controller/dwc/pcie-spear13xx.c:78:26: got unsigned int *
Link: https://lore.kernel.org/r/20211223213749.1314142-3-helgaas@kernel.org
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Pratyush Anand <pratyush.anand@gmail.com>
The sparse checker complains about converting pointers between address
spaces. The pci_config_window.priv pointer is a generic void *, but
hisi_pcie_map_bus() needs a void __iomem *.
This isn't a problem in other drivers because they store the __iomem
pointer in a driver struct. Add a trivial struct hisi_pcie to avoid the
warning.
The sparse warning looks like this:
$ make C=2 drivers/pci/controller/
drivers/pci/controller/dwc/pcie-hisi.c:61:37: warning: incorrect type in initializer (different address spaces)
drivers/pci/controller/dwc/pcie-hisi.c:61:37: expected void [noderef] __iomem *reg_base
drivers/pci/controller/dwc/pcie-hisi.c:61:37: got void *priv
Link: https://lore.kernel.org/r/20211223213749.1314142-2-helgaas@kernel.org
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Zhou Wang <wangzhou1@hisilicon.com>
Rename struct xilinx_cpm_pcie_port to xilinx_cpm_pcie to match the
convention of <driver>_pcie. No functional change intended.
Link: https://lore.kernel.org/r/20211223011054.1227810-24-helgaas@kernel.org
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Michal Simek <michal.simek@xilinx.com>
Rename struct xilinx_pcie_port to xilinx_pcie to match the convention of
<driver>_pcie. No functional change intended.
Link: https://lore.kernel.org/r/20211223011054.1227810-23-helgaas@kernel.org
Signed-off-by: Fan Fei <ffclaire1224@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Michal Simek <michal.simek@xilinx.com>
Rename struct xgene_pcie_port to xgene_pcie to match the convention of
<driver>_pcie. No functional change intended.
Link: https://lore.kernel.org/r/20211223011054.1227810-22-helgaas@kernel.org
Signed-off-by: Fan Fei <ffclaire1224@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Toan Le <toan@os.amperecomputing.com>
Rename struct uniphier_pcie_priv to uniphier_pcie to match the convention
of <driver>_pcie. No functional change intended.
[bhelgaas: drop extraneous uniphier_pcie_irq_ack() from patch]
Link: https://lore.kernel.org/r/20211223011054.1227810-21-helgaas@kernel.org
Signed-off-by: Fan Fei <ffclaire1224@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Kunihiko Hayashi <hayashi.kunihiko@socionext.com>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Rename tegra_pcie_dw to tegra194_pcie to match the convention of
<driver>_pcie. No functional change intended.
[bhelgaas: rename functions similarly]
Link: https://lore.kernel.org/r/20211223011054.1227810-20-helgaas@kernel.org
Signed-off-by: Fan Fei <ffclaire1224@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Thierry Reding <thierry.reding@gmail.com>
Cc: Jonathan Hunter <jonathanh@nvidia.com>
Rename struct rcar_pci_priv to rcar_pci to match the convention of
<driver>_pci. No functional change intended.
Link: https://lore.kernel.org/r/20211223011054.1227810-19-helgaas@kernel.org
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Marek Vasut <marek.vasut+renesas@gmail.com>
Cc: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Rename mt7621_pci_* structs and functions to mt7621_pcie_* for consistency
with the rest of the file.
Link: https://lore.kernel.org/r/20211223011054.1227810-18-helgaas@kernel.org
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Sergio Paracuellos <sergio.paracuellos@gmail.com>
Cc: Matthias Brugger <matthias.bgg@gmail.com>
Rename struct mc_port to mc_pcie to match the convention of <driver>_pcie.
No functional change intended.
Link: https://lore.kernel.org/r/20211223011054.1227810-16-helgaas@kernel.org
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Daire McNamara <daire.mcnamara@microchip.com>
Rename struct mtk_pcie_port to mtk_gen3_pcie to match the convention of
<driver>_pcie. No functional change intended.
Link: https://lore.kernel.org/r/20211223011054.1227810-15-helgaas@kernel.org
Signed-off-by: Fan Fei <ffclaire1224@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Ryder Lee <ryder.lee@mediatek.com>
Cc: Jianjun Wang <jianjun.wang@mediatek.com>
Rename iproc_pcie_pltfm_* to iproc_pltfm_pcie_* for consistency with other
drivers. No functional change intended.
Link: https://lore.kernel.org/r/20211223011054.1227810-13-helgaas@kernel.org
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Ray Jui <rjui@broadcom.com>
Cc: Scott Branden <sbranden@broadcom.com>
Rename iproc_pcie_bcma_* to iproc_bcma_pcie_* for consistency with other
drivers. No functional change intended.
Link: https://lore.kernel.org/r/20211223011054.1227810-12-helgaas@kernel.org
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Ray Jui <rjui@broadcom.com>
Cc: Scott Branden <sbranden@broadcom.com>
Rename struct intel_pcie_port to intel_pcie to match the convention of
<driver>_pcie. No functional change intended.
Link: https://lore.kernel.org/r/20211223011054.1227810-11-helgaas@kernel.org
Signed-off-by: Fan Fei <ffclaire1224@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Rahul Tanwar <rtanwar@maxlinear.com>
The struct cdns_pcie already contains the struct device for the j721e PCIe
controller. There's no need to store another copy in struct j721e_pcie.
Remove the redundant copy from struct j721e_pcie.
Link: https://lore.kernel.org/r/20211223011054.1227810-10-helgaas@kernel.org
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Kishon Vijay Abraham I <kishon@ti.com>
Cc: Tom Joseph <tjoseph@cadence.com>
of_device_get_match_data() returns "void *", so no cast is needed when
assigning the result to a pointer type. Drop the unnecessary cast.
Link: https://lore.kernel.org/r/20211223011054.1227810-9-helgaas@kernel.org
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Kishon Vijay Abraham I <kishon@ti.com>
Cc: Tom Joseph <tjoseph@cadence.com>
The kirin driver only needs the device data, not the whole struct
of_device_id. Use of_device_get_match_data() instead of of_match_device().
No functional change intended.
[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20211223011054.1227810-8-helgaas@kernel.org
Signed-off-by: Fan Fei <ffclaire1224@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Xiaowei Song <songxiaowei@hisilicon.com>
Cc: Binghui Wang <wangbinghui@hisilicon.com>
The keystone driver only needs the device data, not the whole struct
of_device_id. Use of_device_get_match_data() instead of of_match_device().
No functional change intended.
[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20211223011054.1227810-7-helgaas@kernel.org
Signed-off-by: Fan Fei <ffclaire1224@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Kishon Vijay Abraham I <kishon@ti.com>
The dra7xx driver only needs the device data, not the whole struct
of_device_id. Use of_device_get_match_data() instead of of_match_device().
No functional change intended.
[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20211223011054.1227810-6-helgaas@kernel.org
Signed-off-by: Fan Fei <ffclaire1224@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Kishon Vijay Abraham I <kishon@ti.com>
The designware-plat driver only needs the device data, not the whole struct
of_device_id. Use of_device_get_match_data() instead of of_match_device().
No functional change intended.
[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20211223011054.1227810-5-helgaas@kernel.org
Signed-off-by: Fan Fei <ffclaire1224@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
The cadence driver only needs the device data, not the whole struct
of_device_id. Use of_device_get_match_data() instead of of_match_device().
No functional change intended.
[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20211223011054.1227810-4-helgaas@kernel.org
Signed-off-by: Fan Fei <ffclaire1224@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Tom Joseph <tjoseph@cadence.com>
The artpec6 driver only needs the device data, not the whole struct
of_device_id. Use of_device_get_match_data() instead of of_match_device().
No functional change intended.
[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20211223011054.1227810-3-helgaas@kernel.org
Signed-off-by: Fan Fei <ffclaire1224@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Jesper Nilsson <jesper.nilsson@axis.com>
The altera driver only needs the device data, not the whole struct
of_device_id. Use of_device_get_match_data() instead of of_match_device().
No functional change intended.
[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20211223011054.1227810-2-helgaas@kernel.org
Signed-off-by: Fan Fei <ffclaire1224@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Joyce Ooi <joyce.ooi@intel.com>
There are currently two ways to create a set of sysfs files for a
kobj_type: through the default_attrs field, and the default_groups field.
Move the PCI slot code to use the default_groups field which has been the
preferred way since aa30f47cf6 ("kobject: Add support for default
attribute groups to kobj_type") so that we can soon get rid of the obsolete
default_attrs field.
Link: https://lore.kernel.org/r/20211228135722.381023-1-gregkh@linuxfoundation.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
i.MX8MM PCIe works mostly like the i.MX8MQ one, but has a different PHY
and allows to output the internal PHY reference clock via the refclk pad.
Add the i.MX8MM PCIe support based on the standalone PHY driver.
Link: https://lore.kernel.org/r/1640312885-31142-2-git-send-email-hongxing.zhu@nxp.com
Tested-by: Marcel Ziswiler <marcel.ziswiler@toradex.com>
Tested-by: Tim Harvey <tharvey@gateworks.com>
Signed-off-by: Richard Zhu <hongxing.zhu@nxp.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Tim Harvey <tharvey@gateworks.com>
is restored to its reset state
- Mask MSI-X vectors late on the init path in order to handle
out-of-spec Marvell NVME devices which apparently look at the MSI-X mask
even when MSI-X is disabled
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEzv7L6UO9uDPlPSfHEsHwGGHeVUoFAmG/PCUACgkQEsHwGGHe
VUpAkg//WZNlzHyovL4EslM/E2UPq23ALYGfK36td4b9G2+9PvnOZMeQ6hM8rOeN
pMcwpSBvOs3hlWTWjhDLKWZwkxFBNi53Fj2YvxDFUFj771jFNIpvoIY47z2AB03E
FnNUQHJNUe5clxARNc3RTlLtMURGghyfY7xwd/3UQcr6xfma7bZhiraUUpy/vUTn
29QhR4/iSNIqZdA1gzapDwnlRjvr3qcFe/Xa7tfqrs3xOKFGIRcHTpVMdhB4iBON
7JCZr/lKiw/HiJzXHgNG1SmLR6QyGVLshga5tMoswzKqoIDHbW7szBUFMe816/rP
M5jfwkiUznvmrcSzqs12K5whLSRPXT2IFRegtcuA8bjNowwZqPWVEaY+z1s2iHeO
u2WZnsaAHrqQqgZIR2bP6x1cex2bRERDEeUq2tj3A9EiTzEjI5OT66/cjeThiVDT
b8J/vJEYiNBnqEI5REM1oajYLDO71H4SHxjZqfSGRxgHBwEcVry8b39QbglQPZ0V
87+2IhDcLfKcCQehCLb5Qd5PqFZMS4NM1KlcVjrt8O9Y01A0qIor9ZVPcA38R5cA
nPo5EAALVX5z22KbnMmKgaL3APARsh9NM2wW0F5699irL2EgiOEWW+ENKOzycd8f
dzFLT0UrZ6S07IiANAc4mhMpI2cIRuVJGDKJuWVOwn47V1L37ys=
=Kdqk
-----END PGP SIGNATURE-----
Merge tag 'irq_urgent_for_v5.16_rc6' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull irq fixes from Borislav Petkov:
- Clear the PCI_MSIX_FLAGS_MASKALL bit too on the error path so that it
is restored to its reset state
- Mask MSI-X vectors late on the init path in order to handle
out-of-spec Marvell NVME devices which apparently look at the MSI-X
mask even when MSI-X is disabled
* tag 'irq_urgent_for_v5.16_rc6' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
PCI/MSI: Clear PCI_MSIX_FLAGS_MASKALL on error
PCI/MSI: Mask MSI-X vectors only on success
The recent cleanup of pci_irq_get_affinity() broke the function for
PCI/MSI-X and indices > 0. Only the MSI descriptor for PCI/MSI has more
than one affinity mask which can be retrieved via the MSI index.
PCI/MSI-X has one descriptor per vector and each has a single affinity
mask.
Use index 0 when accessing the affinity mask in the MSI descriptor when
MSI-X is enabled.
Fixes: f482359001 ("PCI/MSI: Simplify pci_irq_get_affinity()")
Reported-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Link: https://lore.kernel.org/r/87v8zm9pmd.ffs@tglx
Replace the about to vanish iterators and make use of the filtering. Take
the descriptor lock around the iterators.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Michael Kelley <mikelley@microsoft.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20211206210748.629363944@linutronix.de
Use the new iterator functions which pave the way for dynamically extending
MSI-X vectors.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Michael Kelley <mikelley@microsoft.com>
Tested-by: Nishanth Menon <nm@ti.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20211206210748.142603657@linutronix.de
Set the domain info flag which tells the core code to free the MSI
descriptors from msi_domain_free_irqs() and add an explicit call to the
core function into the legacy code.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Michael Kelley <mikelley@microsoft.com>
Tested-by: Nishanth Menon <nm@ti.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20211206210748.089085131@linutronix.de
Simplify the allocation of MSI descriptors by using msi_add_msi_desc()
which moves the storage handling to core code and prepares for dynamic
extension of the MSI-X vector space.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Michael Kelley <mikelley@microsoft.com>
Tested-by: Nishanth Menon <nm@ti.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20211206210748.035348646@linutronix.de
To prepare for dynamic extension of MSI-X vectors, protect the MSI
operations for MSI and MSI-X. This requires to move the invocation of
irq_create_affinity_masks() out of the descriptor lock section to avoid
reverse lock ordering vs. CPU hotplug lock as some callers of the PCI/MSI
allocation interfaces already hold it.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Michael Kelley <mikelley@microsoft.com>
Tested-by: Nishanth Menon <nm@ti.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20211206210747.982292705@linutronix.de
Replace open coded MSI descriptor chasing and use the proper accessor
functions instead.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Michael Kelley <mikelley@microsoft.com>
Tested-by: Nishanth Menon <nm@ti.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Link: https://lore.kernel.org/r/20211210221814.900929381@linutronix.de
Use msi_get_vector() and handle the return value to be compatible.
No functional change intended.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Michael Kelley <mikelley@microsoft.com>
Tested-by: Nishanth Menon <nm@ti.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Link: https://lore.kernel.org/r/20211210221814.841243231@linutronix.de
Provide a domain info flag which makes the core code check for a contiguous
MSI-X index on allocation. That's simpler than checking it at some other
domain callback in architecture code.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20211210221814.662401116@linutronix.de
The usage of msi_desc::pci::entry_nr is confusing at best. It's the index
into the MSI[X] descriptor table.
Use msi_desc::msi_index which is shared between all MSI incarnations
instead of having a PCI specific storage for no value.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Michael Kelley <mikelley@microsoft.com>
Tested-by: Nishanth Menon <nm@ti.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20211210221814.602911509@linutronix.de
Set the domain info flag which makes the core code handle sysfs groups and
put an explicit invocation into the legacy code.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Michael Kelley <mikelley@microsoft.com>
Tested-by: Nishanth Menon <nm@ti.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20211210221814.048612053@linutronix.de
Allocate MSI device data on first use, i.e. when a PCI driver invokes one
of the PCI/MSI enablement functions.
Add a wrapper function to ensure that the ordering vs. pcim_msi_release()
is correct.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Michael Kelley <mikelley@microsoft.com>
Tested-by: Nishanth Menon <nm@ti.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Link: https://lore.kernel.org/r/87r1adrdje.ffs@tglx
The MSI core will introduce runtime allocation of MSI related data. This
data will be devres managed and has to be set up before enabling
PCI/MSI[-X]. This would introduce an ordering issue vs. pcim_release().
The setup order is:
pcim_enable_device()
devres_alloc(pcim_release...);
...
pci_irq_alloc()
msi_setup_device_data()
devres_alloc(msi_device_data_release, ...)
and once the device is released these release functions are invoked in the
opposite order:
msi_device_data_release()
...
pcim_release()
pci_disable_msi[x]()
which is obviously wrong, because pci_disable_msi[x]() requires the MSI
data to be available to tear down the MSI[-X] interrupts.
Remove the MSI[-X] teardown from pcim_release() and add an explicit action
to be installed on the attempt of enabling PCI/MSI[-X].
This allows the MSI core data allocation to be ordered correctly in a
subsequent step.
Reported-by: Nishanth Menon <nm@ti.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Michael Kelley <mikelley@microsoft.com>
Tested-by: Nishanth Menon <nm@ti.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Link: https://lore.kernel.org/r/87tuf9rdoj.ffs@tglx
There are quite some places which retrieve the first MSI descriptor to
evaluate whether the setup is for MSI or MSI-X. That's required because
pci_dev::msi[x]_enabled is only set when the setup completed successfully.
There is no real reason why msi[x]_enabled can't be set at the beginning of
the setup sequence and cleared in case of a failure.
Implement that so the MSI descriptor evaluations can be converted to simple
property queries.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Michael Kelley <mikelley@microsoft.com>
Tested-by: Nishanth Menon <nm@ti.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Link: https://lore.kernel.org/r/20211210221813.250049810@linutronix.de
Since pci_alloc_p2pmem() has already called rcu_read_lock(), we're in an
RCU read-side critical section and don't need to take the lock again. Use
percpu_ref_tryget_live_rcu() instead of percpu_ref_tryget_live() to save a
few cycles.
[bhelgaas: commit log]
Link: https://lore.kernel.org/r/ab80164f4d5b32f9e6240aa4863c3a147ff9c89f.1635974126.git.christophe.jaillet@wanadoo.fr
Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Krzysztof Wilczyński <kw@linux.com>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
Like other SATA controller chips in the Marvell 88SE91xx series, the
Marvell 88SE9125 has the same DMA requester ID hardware bug that prevents
it from working under IOMMU. Add it to the list of devices that need the
quirk.
Without this patch, device initialization fails with DMA errors:
ata8: softreset failed (1st FIS failed)
DMAR: DRHD: handling fault status reg 2
DMAR: [DMA Write NO_PASID] Request device [03:00.1] fault addr 0xfffc0000 [fault reason 0x02] Present bit in context entry is clear
DMAR: DRHD: handling fault status reg 2
DMAR: [DMA Read NO_PASID] Request device [03:00.1] fault addr 0xfffc0000 [fault reason 0x02] Present bit in context entry is clear
After applying the patch, the controller can be successfully initialized:
ata8: SATA link up 1.5 Gbps (SStatus 113 SControl 330)
ata8.00: ATAPI: PIONEER BD-RW BDR-207M, 1.21, max UDMA/100
ata8.00: configured for UDMA/100
scsi 7:0:0:0: CD-ROM PIONEER BD-RW BDR-207M 1.21 PQ: 0 ANSI: 5
Link: https://lore.kernel.org/r/YahpKVR+McJVDdkD@work
Reported-by: Sam Bingner <sam@bingner.com>
Tested-by: Sam Bingner <sam@bingner.com>
Tested-by: Yifeng Li <tomli@tomli.me>
Signed-off-by: Yifeng Li <tomli@tomli.me>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Krzysztof Wilczyński <kw@linux.com>
Cc: stable@vger.kernel.org
PCI_MSIX_FLAGS_MASKALL is set in the MSI-X control register at MSI-X
interrupt setup time. It's cleared on success, but the error handling path
only clears the PCI_MSIX_FLAGS_ENABLE bit.
That's incorrect as the reset state of the PCI_MSIX_FLAGS_MASKALL bit is
zero. That can be observed via lspci:
Capabilities: [b0] MSI-X: Enable- Count=67 Masked+
Clear the bit in the error path to restore the reset state.
Fixes: 438553958b ("PCI/MSI: Enable and mask MSI-X early")
Reported-by: Stefan Roese <sr@denx.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Stefan Roese <sr@denx.de>
Cc: linux-pci@vger.kernel.org
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Michal Simek <michal.simek@xilinx.com>
Cc: Marek Vasut <marex@denx.de>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/r/87tufevoqx.ffs@tglx
Masking all unused MSI-X entries is done to ensure that a crash kernel
starts from a clean slate, which correponds to the reset state of the
device as defined in the PCI-E specificion 3.0 and later:
Vector Control for MSI-X Table Entries
--------------------------------------
"00: Mask bit: When this bit is set, the function is prohibited from
sending a message using this MSI-X Table entry.
...
This bit’s state after reset is 1 (entry is masked)."
A Marvell NVME device fails to deliver MSI interrupts after trying to
enable MSI-X interrupts due to that masking. It seems to take the MSI-X
mask bits into account even when MSI-X is disabled.
While not specification compliant, this can be cured by moving the masking
into the success path, so that the MSI-X table entries stay in device reset
state when the MSI-X setup fails.
[ tglx: Move it into the success path, add comment and amend changelog ]
Fixes: aa8092c1d1f1 ("PCI/MSI: Mask all unused MSI-X entries")
Signed-off-by: Stefan Roese <sr@denx.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-pci@vger.kernel.org
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Michal Simek <michal.simek@xilinx.com>
Cc: Marek Vasut <marex@denx.de>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/r/20211210161025.3287927-1-sr@denx.de
The driver is not ready yet to be compiled as a module since it depends
on some symbols not exported on MIPS. We have the following current
problems:
Building mips:allmodconfig ... failed
--------------
Error log:
ERROR: modpost: missing MODULE_LICENSE() in drivers/pci/controller/pcie-mt7621.o
ERROR: modpost: "mips_cm_unlock_other" [drivers/pci/controller/pcie-mt7621.ko] undefined!
ERROR: modpost: "mips_cpc_base" [drivers/pci/controller/pcie-mt7621.ko] undefined!
ERROR: modpost: "mips_cm_lock_other" [drivers/pci/controller/pcie-mt7621.ko] undefined!
ERROR: modpost: "mips_cm_is64" [drivers/pci/controller/pcie-mt7621.ko] undefined!
ERROR: modpost: "mips_gcr_base" [drivers/pci/controller/pcie-mt7621.ko] undefined!
Temporarily move from 'tristate' to 'bool' until a better solution is
ready.
Also RALINK is redundant because SOC_MT7621 already depends on it.
Hence, simplify condition.
Fixes: 2bdd5238e7 ("PCI: mt7621: Add MediaTek MT7621 PCIe host controller driver").
Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com>
Reviewed-and-Tested-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
-----BEGIN PGP SIGNATURE-----
iQJIBAABCgAyFiEEgMe7l+5h9hnxdsnuWYigwDrT+vwFAmGzf4IUHGJoZWxnYWFz
QGdvb2dsZS5jb20ACgkQWYigwDrT+vwjcBAAiWel9P5H947jR9sTbz4ya6wH1biD
k2w97VDa65DyH/LBJSgNwmblnXs7yIUuGTd+mRq9bhlpE8CQi9BfeCehP1vCfTeQ
JtMH62dW8KBLkvIHU83H1SSZZNKQgDn7hqUsrrMa0HD+Z+ovbuQYp4M1Oh6xRAEM
TTBTKb0KivA8bFwvtgj/mu7K7sVJH+cVMilD9ABoVeGmCWfUSO48ovEjWB+vmBFs
UyTCU5CUg/FkjvVmZTOv5GY4EL83FA9Jdtzy8inRA+hSWY6ImXHTzmQlAzvA+Rkv
k344ZQM9GNvbvwKfBa9iW2g+B2y/OJXafGoVL0NBUcj/eiY5dnAX0/tZHvx0aXFy
G1Txy2utaG2MSkfZzchEKbRvS0tV7kiFiTmqp9lNmffTZiP72k4+kFJHQC5AzvZb
O7Ce/XSQifQ1Z3f5B+Ymx6EOgKYJUaWO9B1U1KF0EKGMe5GB0TBiXh/tS2EmV1O8
1hkUJm032Bbf1Bv5R6BLdgKVz4I3UsqmGKH5gg3blyylAQ1oHsioaUKeV6iHSq40
u9rNZaKGC3SweYZVISNE1uoII4qzEgLOHggHpZvWxhQy35cFBz8ZsNfLwBD3/8z9
UfFuLSLHjx+hv3Ev5mgDWH1mzAlzyq5KkDT0bodBix07s5mviDH+57yyw3JtHUuL
F+tMrYjUHK9ArC8=
=BWkT
-----END PGP SIGNATURE-----
Merge tag 'pci-v5.16-fixes-2' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci
Pull PCI fixes from Bjorn Helgaas:
- Revert emulation of Marvell Armada A3720 expansion ROM because it
doesn't work as expected (Marek Behún)
- Assert PERST# in Apple M1 driver to fix initialization when booting
from bootloaders using PCIe, such as U-Boot (Marc Zyngier)
- Describe PERST# as active low in Apple T8103 DT and update driver to
match (Marc Zyngier)
* tag 'pci-v5.16-fixes-2' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci:
PCI: apple: Fix PERST# polarity
arm64: dts: apple: t8103: Mark PCIe PERST# polarity active low in DT
PCI: apple: Follow the PCIe specifications when resetting the port
Revert "PCI: aardvark: Fix support for PCI_ROM_ADDRESS1 on emulated bridge"
The functions get_max_adapter_speed() and get_bus_name() in ibmphp_core.c
are commented-out and the fields .get_max_adapter_speed and
.get_bus_name_status are removed from struct hotplug_slot_ops in
pci_hotplug.h. Remove the commented-out functions.
Link: https://lore.kernel.org/r/20211209213618.20522-1-makvihas@gmail.com
Signed-off-by: Vihas Mak <makvihas@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
The irqdomain code already returns the information. Move the loop to the
legacy code.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Link: https://lore.kernel.org/r/20211206210225.101336873@linutronix.de
Get rid of yet another irqdomain callback and let the core code return the
already available information of how many descriptors could be allocated.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: Bjorn Helgaas <bhelgaas@google.com> # PCI
Link: https://lore.kernel.org/r/20211206210225.046615302@linutronix.de
It's only required for PCI/MSI. So no point in having it in every struct
device.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20211206210224.925241961@linutronix.de
Unmapping the MSI-X base mapping in the loops which allocate/free MSI
descriptors is daft and in the way of allowing runtime expansion of MSI-X
descriptors.
Store the mapping in struct pci_dev and free it after freeing the MSI-X
descriptors.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20211206210224.871651518@linutronix.de
Move the irqdomain specific code into its own file.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20211206210224.817754783@linutronix.de
Split out the non irqdomain code into its own file.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20211206210224.763574089@linutronix.de
These functions are required even when CONFIG_PCI_MSI is not set. Move them
to their own file.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20211206210224.710137730@linutronix.de
msi.c is getting larger and really could do with a splitup. Move it into
its own directory to prepare for that.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20211206210224.655043033@linutronix.de
No need to walk the descriptors and check for each one whether the entries
pointer function argument is NULL. Do it once.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20211206210224.600351129@linutronix.de
Get rid of the pile of unneeded includes which accumulated over time.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20211206210224.539281124@linutronix.de
Make arch_restore_msi_irqs() return a boolean which indicates whether the
core code should restore the MSI message or not. Get rid of the indirection
in x86.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com> # PCI
Link: https://lore.kernel.org/r/20211206210224.485668098@linutronix.de
The unnamed struct sucks and is in the way of further cleanups. Stick the
PCI related MSI data into a real data structure and cleanup all users.
No functional change.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: Kalle Valo <kvalo@codeaurora.org>
Link: https://lore.kernel.org/r/20211206210224.374863119@linutronix.de
Jim Quinlan