Commit Graph

1861 Commits

Author SHA1 Message Date
Eric Auger e5b5234a36 iommu: Rename iommu_dm_regions into iommu_resv_regions
We want to extend the callbacks used for dm regions and
use them for reserved regions. Reserved regions can be
- directly mapped regions
- regions that cannot be iommu mapped (PCI host bridge windows, ...)
- MSI regions (because they belong to another address space or because
  they are not translated by the IOMMU and need special handling)

So let's rename the struct and also the callbacks.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Acked-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Tomasz Nowicki <tomasz.nowicki@caviumnetworks.com>
Tested-by: Tomasz Nowicki <tomasz.nowicki@caviumnetworks.com>
Tested-by: Bharat Bhushan <bharat.bhushan@nxp.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2017-01-23 11:48:16 +00:00
Robin Murphy fdbe574eb6 iommu/dma: Allow MSI-only cookies
IOMMU domain users such as VFIO face a similar problem to DMA API ops
with regard to mapping MSI messages in systems where the MSI write is
subject to IOMMU translation. With the relevant infrastructure now in
place for managed DMA domains, it's actually really simple for other
users to piggyback off that and reap the benefits without giving up
their own IOVA management, and without having to reinvent their own
wheel in the MSI layer.

Allow such users to opt into automatic MSI remapping by dedicating a
region of their IOVA space to a managed cookie, and extend the mapping
routine to implement a trivial linear allocator in such cases, to avoid
the needless overhead of a full-blown IOVA domain.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Tomasz Nowicki <tomasz.nowicki@caviumnetworks.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Tested-by: Eric Auger <eric.auger@redhat.com>
Tested-by: Tomasz Nowicki <tomasz.nowicki@caviumnetworks.com>
Tested-by: Bharat Bhushan <bharat.bhushan@nxp.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2017-01-23 11:48:16 +00:00
Robin Murphy 14b4dbafa7 Revert "iommu/arm-smmu: Set PRIVCFG in stage 1 STEs"
This reverts commit df5e1a0f2a2d779ad467a691203bcbc74d75690e.

Now that proper privileged mappings can be requested via IOMMU_PRIV,
unconditionally overriding the incoming PRIVCFG becomes the wrong thing
to do, so stop it.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2017-01-19 15:56:21 +00:00
Sricharan R e19898077c iommu/arm-smmu: Set privileged attribute to 'default' instead of 'unprivileged'
Currently the driver sets all the device transactions privileges
to UNPRIVILEGED, but there are cases where the iommu masters wants
to isolate privileged supervisor and unprivileged user.
So don't override the privileged setting to unprivileged, instead
set it to default as incoming and let it be controlled by the pagetable
settings.

Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Sricharan R <sricharan@codeaurora.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2017-01-19 15:56:20 +00:00
Mitchel Humpherys 737c85ca1c arm64/dma-mapping: Implement DMA_ATTR_PRIVILEGED
The newly added DMA_ATTR_PRIVILEGED is useful for creating mappings that
are only accessible to privileged DMA engines.  Implement it in
dma-iommu.c so that the ARM64 DMA IOMMU mapper can make use of it.

Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Tested-by: Robin Murphy <robin.murphy@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2017-01-19 15:56:19 +00:00
Robin Murphy 5baf1e9d0b iommu/io-pgtable-arm-v7s: Add support for the IOMMU_PRIV flag
The short-descriptor format also allows privileged-only mappings, so
let's wire it up.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Tested-by: Sricharan R <sricharan@codeaurora.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2017-01-19 15:56:18 +00:00
Jeremy Gebben e7468a23da iommu/io-pgtable-arm: add support for the IOMMU_PRIV flag
Allow the creation of privileged mode mappings, for stage 1 only.

Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Tested-by: Robin Murphy <robin.murphy@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2017-01-19 15:56:18 +00:00
Robin Murphy 797a8b4d76 iommu: Handle default domain attach failure
We wouldn't normally expect ops->attach_dev() to fail, but on IOMMUs
with limited hardware resources, or generally misconfigured systems,
it is certainly possible. We report failure correctly from the external
iommu_attach_device() interface, but do not do so in iommu_group_add()
when attaching to the default domain. The result of failure there is
that the device, group and domain all get left in a broken,
part-configured state which leads to weird errors and misbehaviour down
the line when IOMMU API calls sort-of-but-don't-quite work.

Check the return value of __iommu_attach_device() on the default domain,
and refactor the error handling paths to cope with its failure and clean
up correctly in such cases.

Fixes: e39cb8a3aa ("iommu: Make sure a device is always attached to a domain")
Reported-by: Punit Agrawal <punit.agrawal@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-01-17 16:49:24 +01:00
Marek Szyprowski fff2fd1a9e iommu/exynos: Properly release device from the default domain in ->remove
IOMMU core doesn't detach device from the default domain before calling
->iommu_remove_device, so check that and do the proper cleanup or
warn if device is still attached to non-default domain.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-01-10 15:01:21 +01:00
Marek Szyprowski 0bd5a0c77a iommu/exynos: Ensure that SYSMMU is added only once to its master device
This patch prepares Exynos IOMMU driver for deferred probing
support. Once it gets added, of_xlate() callback might be called
more than once for the same SYSMMU controller and master device
(for example it happens when masters device driver fails with
EPROBE_DEFER). This patch adds a check, which ensures that SYSMMU
controller is added to its master device (owner) only once.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-01-10 15:01:21 +01:00
Marek Szyprowski 0d6d3da46a iommu/exynos: Fix warnings from DMA-debug
Add a simple checks for dma_map_single() return value to make DMA-debug
checker happly. Exynos IOMMU on Samsung Exynos SoCs always use device,
which has linear DMA mapping ops (dma address is equal to physical memory
address), so no failures are returned from dma_map_single().

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-01-10 15:01:21 +01:00
Marek Szyprowski ec5d241b5f iommu/exynos: Improve page fault debug message
Add master device name to default IOMMU fault message to make easier to
find which device triggered the fault. While at it, move printing some
information (like page table base and first level entry addresses) to
dev_dbg(), because those are typically not very useful for typical device
driver user/developer not equipped with hardware debugging tools.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-01-10 15:01:21 +01:00
Rafael J. Wysocki 99e8ccd383 iommu/amd: Fix error code path in early_amd_iommu_init()
Prevent early_amd_iommu_init() from leaking memory mapped via
acpi_get_table() if check_ivrs_checksum() returns an error.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-01-10 14:57:34 +01:00
Linus Torvalds 65cdc405b3 IOMMU Fixes for Linux v4.10-rc2
Three fixes queued up:
 
 	* Fix an issue with command buffer overflow handling in the AMD
 	  IOMMU driver
 
 	* Add an additional context entry flush to the Intel VT-d driver
 	  to make sure any old context entry from kdump copying is
 	  flushed out of the cache
 
 	* Correct the encoding of the PASID table size in the Intel VT-d
 	  driver
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2
 
 iQIcBAABAgAGBQJYb6+0AAoJECvwRC2XARrjDh4QAN6frtp8eu+klRWL2/l3JdZA
 Iht+HfwC6RCq7Rl8fqqjcfyFXImb+QazfKnyFriH/RxohV9PD5LqLVSmlngoficb
 vuas7ZQB1+fUUz4pPkJOEium5QhVn+RFHgnJPke685EsASVb9/QJBswzrhAvAM0X
 RYli/hZ2+MpWeQdmF8pWgO0Dt5ker4bT+9AnYyz8IUJgyQeqZPucoPwksnjvouJz
 och0IWvfNH3OGRIM5MDpJ5h9W7fdxVL5tS0KJ+NyXLBSqEr7fmKtJgwwxSIOJUIN
 7J+1ASu/fHvegCVG9eNELEy9RPKCp/Ju6mgw4AtVDHDG5IRKKX9T/ab+6d3PIPE5
 jZxycerGip5NV4TieK9abk9mw4I+NOyKn4OdyC6ZrGwF73rdqNG7mljLlFE39HR8
 Qvua1bMhykcpKwSJpb52InbYEFkb09Rn6/AVDSpby9WnDai2wdcn+YFtgZrnYevD
 e3HoO+X1VrmSRlTxG6yPvPmcYhCZxl7W+duzPYd9Ff4aS4VA24xgfMs5w1dsu3Tt
 hb/BqMmQ4bSUQqaemj2JpJqVBRlXQc5pkFFtQtdrAO6xWyrQXcjZemzHRBMAmtZP
 xI8ksD5L9QRbg/KTKcYNFSQ6WVZMBV2dTXWhEycLkGokGDXGCis0BF6IRXTwJNQU
 9cLOXtL9hyBBm7tKn199
 =wit6
 -----END PGP SIGNATURE-----

Merge tag 'iommu-fixes-v4.10-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu

Pull IOMMU fixes from Joerg Roedel:
 "Three fixes queued up:

   - fix an issue with command buffer overflow handling in the AMD IOMMU
     driver

   - add an additional context entry flush to the Intel VT-d driver to
     make sure any old context entry from kdump copying is flushed out
     of the cache

   - correct the encoding of the PASID table size in the Intel VT-d
     driver"

* tag 'iommu-fixes-v4.10-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu:
  iommu/amd: Fix the left value check of cmd buffer
  iommu/vt-d: Fix pasid table size encoding
  iommu/vt-d: Flush old iommu caches for kdump when the device gets context mapped
2017-01-06 10:49:36 -08:00
Rafael J. Wysocki 696c7f8e03 ACPI / DMAR: Avoid passing NULL to acpi_put_table()
Linus reported that commit 174cc7187e "ACPICA: Tables: Back port
acpi_get_table_with_size() and early_acpi_os_unmap_memory() from
Linux kernel" added a new warning on his desktop system:

 ACPI Warning: Table ffffffff9fe6c0a0, Validation count is zero before decrement

which turns out to come from the acpi_put_table() in
detect_intel_iommu().

This happens if the DMAR table is not present in which case NULL is
passed to acpi_put_table() which doesn't check against that and
attempts to handle it regardless.

For this reason, check the pointer passed to acpi_put_table()
before invoking it.

Reported-by: Linus Torvalds <torvalds@linux-foundation.org>
Tested-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Fixes: 6b11d1d677 ("ACPI / osl: Remove acpi_get_table_with_size()/early_acpi_os_unmap_memory() users")
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2017-01-05 15:10:52 +01:00
Geliang Tang eba484b51b iommu/iova: Use rb_entry()
To make the code clearer, use rb_entry() instead of container_of() to
deal with rbtree.

Signed-off-by: Geliang Tang <geliangtang@gmail.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-01-04 15:35:19 +01:00
Huang Rui 432abf68a7 iommu/amd: Fix the left value check of cmd buffer
The generic command buffer entry is 128 bits (16 bytes), so the offset
of tail and head pointer should be 16 bytes aligned and increased with
0x10 per command.

When cmd buf is full, head = (tail + 0x10) % CMD_BUFFER_SIZE.

So when left space of cmd buf should be able to store only two
command, we should be issued one COMPLETE_WAIT additionally to wait
all older commands completed. Then the left space should be increased
after IOMMU fetching from cmd buf.

So left check value should be left <= 0x20 (two commands).

Signed-off-by: Huang Rui <ray.huang@amd.com>
Fixes: ac0ea6e92b ('x86/amd-iommu: Improve handling of full command buffer')
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-01-04 15:22:22 +01:00
Jacob Pan 65ca7f5f7d iommu/vt-d: Fix pasid table size encoding
Different encodings are used to represent supported PASID bits
and number of PASID table entries.
The current code assigns ecap_pss directly to extended context
table entry PTS which is wrong and could result in writing
non-zero bits to the reserved fields. IOMMU fault reason
11 will be reported when reserved bits are nonzero.
This patch converts ecap_pss to extend context entry pts encoding
based on VT-d spec. Chapter 9.4 as follows:
 - number of PASID bits = ecap_pss + 1
 - number of PASID table entries = 2^(pts + 5)
Software assigned limit of pasid_max value is also respected to
match the allocation limitation of PASID table.

cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
cc: Ashok Raj <ashok.raj@intel.com>
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
Tested-by: Mika Kuoppala <mika.kuoppala@intel.com>
Fixes: 2f26e0a9c9 ('iommu/vt-d: Add basic SVM PASID support')
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-01-04 15:18:57 +01:00
Xunlei Pang aec0e86172 iommu/vt-d: Flush old iommu caches for kdump when the device gets context mapped
We met the DMAR fault both on hpsa P420i and P421 SmartArray controllers
under kdump, it can be steadily reproduced on several different machines,
the dmesg log is like:
HP HPSA Driver (v 3.4.16-0)
hpsa 0000:02:00.0: using doorbell to reset controller
hpsa 0000:02:00.0: board ready after hard reset.
hpsa 0000:02:00.0: Waiting for controller to respond to no-op
DMAR: Setting identity map for device 0000:02:00.0 [0xe8000 - 0xe8fff]
DMAR: Setting identity map for device 0000:02:00.0 [0xf4000 - 0xf4fff]
DMAR: Setting identity map for device 0000:02:00.0 [0xbdf6e000 - 0xbdf6efff]
DMAR: Setting identity map for device 0000:02:00.0 [0xbdf6f000 - 0xbdf7efff]
DMAR: Setting identity map for device 0000:02:00.0 [0xbdf7f000 - 0xbdf82fff]
DMAR: Setting identity map for device 0000:02:00.0 [0xbdf83000 - 0xbdf84fff]
DMAR: DRHD: handling fault status reg 2
DMAR: [DMA Read] Request device [02:00.0] fault addr fffff000 [fault reason 06] PTE Read access is not set
hpsa 0000:02:00.0: controller message 03:00 timed out
hpsa 0000:02:00.0: no-op failed; re-trying

After some debugging, we found that the fault addr is from DMA initiated at
the driver probe stage after reset(not in-flight DMA), and the corresponding
pte entry value is correct, the fault is likely due to the old iommu caches
of the in-flight DMA before it.

Thus we need to flush the old cache after context mapping is setup for the
device, where the device is supposed to finish reset at its driver probe
stage and no in-flight DMA exists hereafter.

I'm not sure if the hardware is responsible for invalidating all the related
caches allocated in the iommu hardware before, but seems not the case for hpsa,
actually many device drivers have problems in properly resetting the hardware.
Anyway flushing (again) by software in kdump kernel when the device gets context
mapped which is a quite infrequent operation does little harm.

With this patch, the problematic machine can survive the kdump tests.

CC: Myron Stowe <myron.stowe@gmail.com>
CC: Joseph Szczypek <jszczype@redhat.com>
CC: Don Brace <don.brace@microsemi.com>
CC: Baoquan He <bhe@redhat.com>
CC: Dave Young <dyoung@redhat.com>
Fixes: 091d42e43d ("iommu/vt-d: Copy translation tables from old kernel")
Fixes: dbcd861f25 ("iommu/vt-d: Do not re-use domain-ids from the old kernel")
Fixes: cf484d0e69 ("iommu/vt-d: Mark copied context entries")
Signed-off-by: Xunlei Pang <xlpang@redhat.com>
Tested-by: Don Brace <don.brace@microsemi.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2017-01-04 15:14:04 +01:00
Linus Torvalds 9be962d525 More ACPI updates for v4.10-rc1
- Move some Linux-specific functionality to upstream ACPICA and
    update the in-kernel users of it accordingly (Lv Zheng).
 
  - Drop a useless warning (triggered by the lack of an optional
    object) from the ACPI namespace scanning code (Zhang Rui).
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2
 
 iQIcBAABCAAGBQJYW90kAAoJEILEb/54YlRxS78P/RZwSOwkgdowIMGOlO7Hz5Xz
 6Psjfc3QdGhVGOau4F3Irg7HIDsPaHE5+xijwujWFK0wh89tknrChrFDrOueND6K
 rZ4w72lYROQhtQl4bu9yXZ2TswP3I5aYERDPyqKfDAi+SaQBpZUtmWdoh0I/JN2M
 svyuzxoRlNglgp2GWPnnnKHFe6plEeZgKjgoGjPAufDHakqYvP0cQwY3i4+PpuYh
 prXb4Xr4fFvAG2MhM9ciT02ewrQJcYUgVj5bnuSBmMu0y0zUBJ1kK7abiQlf+lTy
 /BCqQbllhHrrXQD0zkqi2oBFqWL4Bnj9r+5zj03KmBsfoz3u+zXr5CApDei8Yc06
 gGIZXX9aCqBirahkpsMFYPEQVPoPFCek0slXSyF5xKjCWYyocwgUtHnB87uIiSFO
 jCeSsWwT5IO9HnbsTf6x4xDWBdY6LOJgJyDirIGcNSUX+q4tfgn/LFrgN9Ck4M2g
 xkZn8e8H7u09ACX9l6dHZ1PCHlg7bWKeH6gcqdo5R4NOVeZxt9YDIz13ERsG3D17
 JZvdrAbBdC1fXfHnOLBj/BRy+TFvieWOC+kHBTAnPQlnkr7NjXXv/vgWU8flVL/z
 kxlR2U2lD2XarG/b8OHvYxO2k/TDLRenN0IqqFmYbmyhH0dHYKGGBLY4TetAXKNL
 N4EouS+xUBulCP2XOI8N
 =55TG
 -----END PGP SIGNATURE-----

Merge tag 'acpi-extra-4.10-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm

Pull more ACPI updates from Rafael Wysocki:
 "Here are new versions of two ACPICA changes that were deferred
  previously due to a problem they had introduced, two cleanups on top
  of them and the removal of a useless warning message from the ACPI
  core.

  Specifics:

   - Move some Linux-specific functionality to upstream ACPICA and
     update the in-kernel users of it accordingly (Lv Zheng)

   - Drop a useless warning (triggered by the lack of an optional
     object) from the ACPI namespace scanning code (Zhang Rui)"

* tag 'acpi-extra-4.10-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm:
  ACPI / osl: Remove deprecated acpi_get_table_with_size()/early_acpi_os_unmap_memory()
  ACPI / osl: Remove acpi_get_table_with_size()/early_acpi_os_unmap_memory() users
  ACPICA: Tables: Allow FADT to be customized with virtual address
  ACPICA: Tables: Back port acpi_get_table_with_size() and early_acpi_os_unmap_memory() from Linux kernel
  ACPI: do not warn if _BQC does not exist
2016-12-22 10:19:32 -08:00
Rafael J. Wysocki c8e008e2a6 Merge branches 'acpica' and 'acpi-scan'
* acpica:
  ACPI / osl: Remove deprecated acpi_get_table_with_size()/early_acpi_os_unmap_memory()
  ACPI / osl: Remove acpi_get_table_with_size()/early_acpi_os_unmap_memory() users
  ACPICA: Tables: Allow FADT to be customized with virtual address
  ACPICA: Tables: Back port acpi_get_table_with_size() and early_acpi_os_unmap_memory() from Linux kernel

* acpi-scan:
  ACPI: do not warn if _BQC does not exist
2016-12-22 14:34:24 +01:00
Lv Zheng 6b11d1d677 ACPI / osl: Remove acpi_get_table_with_size()/early_acpi_os_unmap_memory() users
This patch removes the users of the deprectated APIs:
 acpi_get_table_with_size()
 early_acpi_os_unmap_memory()
The following APIs should be used instead of:
 acpi_get_table()
 acpi_put_table()

The deprecated APIs are invented to be a replacement of acpi_get_table()
during the early stage so that the early mapped pointer will not be stored
in ACPICA core and thus the late stage acpi_get_table() won't return a
wrong pointer. The mapping size is returned just because it is required by
early_acpi_os_unmap_memory() to unmap the pointer during early stage.

But as the mapping size equals to the acpi_table_header.length
(see acpi_tb_init_table_descriptor() and acpi_tb_validate_table()), when
such a convenient result is returned, driver code will start to use it
instead of accessing acpi_table_header to obtain the length.

Thus this patch cleans up the drivers by replacing returned table size with
acpi_table_header.length, and should be a no-op.

Reported-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Lv Zheng <lv.zheng@intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2016-12-21 02:36:38 +01:00
Linus Torvalds a9a16a6d13 IOMMU Updates for Linux v4.10
These changes include:
 
 	* Support for the ACPI IORT table on ARM systems and patches to
 	  make the ARM-SMMU driver make use of it
 
 	* Conversion of the Exynos IOMMU driver to device dependency
 	  links and implementation of runtime pm support based on that
 	  conversion
 
 	* Update the Mediatek IOMMU driver to use the new
 	  struct device->iommu_fwspec member
 
 	* Implementation of dma_map/unmap_resource in the generic ARM
 	  dma-iommu layer
 
 	* A number of smaller fixes and improvements all over the place
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2
 
 iQIcBAABAgAGBQJYUskLAAoJECvwRC2XARrjGnQQALfaIJffrh+5vjs08h3J86TF
 lUEvxu1/5PGCwXEWbIH+NQh4pMgBuOL0exa6sR55CqtHPkUKMndv+sVXWUkCrl9Z
 Q4jro+IuPxbHHdQvULjdqn1gtvPvBXN/OFCbQsfj6G+GblAZCIn0OtkKm2MqdGa4
 xHPTAhW+QAt7X8O7PoMsvCNa0loZJx05ToIPbhN23U0H12TwU4aHaMg13+KSGgz/
 wEXGDYDU8EnGubcM/JEpy84qywRU1D+TDwzaIFT2sy7Ewhmgz6rSf+SOa+vn4u5z
 G9xhc6b8Vq+2ZKr1j28pPdrCYMRpQxBWru7t2rcdV0Xa+J1JEPKJfyEPxeldDeXJ
 4bqld/D/0nq7MFvGL73qff+oiU2qOJ1VnPrQO8SbHyueeFd4axMYlPsU7GeDWqoz
 LX1gKuHE1t2GqsNgn/dLouyJGDKjyYzZbio0HuPHGjegkx+dkTXEM0yzLtZJG8hw
 cZSlYijrRJeEROpM8/5BGid3BOAIx8qlOXO/QzI41e0KnQikkgQBgr1L2+Ki8ZaB
 mNs/S/BqDr6LSUP5ArQHlZ0wu/pk1ehwYkmzp9j7Ui1jGdSWwbqw7KkLDXd0pL4g
 NMhsprJLYlnY2GUdrbcDOEWHS9Ex0vRsPKNWoqCggzg/8h8cQmuiDZO346vhlvq/
 ORMwDO7LNZuPHqDM0WA+
 =EYLL
 -----END PGP SIGNATURE-----

Merge tag 'iommu-updates-v4.10' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu

Pull IOMMU updates from Joerg Roedel:
 "These changes include:

   - support for the ACPI IORT table on ARM systems and patches to make
     the ARM-SMMU driver make use of it

   - conversion of the Exynos IOMMU driver to device dependency links
     and implementation of runtime pm support based on that conversion

   - update the Mediatek IOMMU driver to use the new struct
     device->iommu_fwspec member

   - implementation of dma_map/unmap_resource in the generic ARM
     dma-iommu layer

   - a number of smaller fixes and improvements all over the place"

* tag 'iommu-updates-v4.10' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (44 commits)
  ACPI/IORT: Make dma masks set-up IORT specific
  iommu/amd: Missing error code in amd_iommu_init_device()
  iommu/s390: Drop duplicate header pci.h
  ACPI/IORT: Introduce iort_iommu_configure
  ACPI/IORT: Add single mapping function
  ACPI/IORT: Replace rid map type with type mask
  iommu/arm-smmu: Add IORT configuration
  iommu/arm-smmu: Split probe functions into DT/generic portions
  iommu/arm-smmu-v3: Add IORT configuration
  iommu/arm-smmu-v3: Split probe functions into DT/generic portions
  ACPI/IORT: Add support for ARM SMMU platform devices creation
  ACPI/IORT: Add node match function
  ACPI: Implement acpi_dma_configure
  iommu/arm-smmu-v3: Convert struct device of_node to fwnode usage
  iommu/arm-smmu: Convert struct device of_node to fwnode usage
  iommu: Make of_iommu_set/get_ops() DT agnostic
  ACPI/IORT: Add support for IOMMU fwnode registration
  ACPI/IORT: Introduce linker section for IORT entries probing
  ACPI: Add FWNODE_ACPI_STATIC fwnode type
  iommu/arm-smmu: Set SMTNMB_TLBEN in ACR to enable caching of bypass entries
  ...
2016-12-15 12:24:14 -08:00
Linus Torvalds e71c3978d6 Merge branch 'smp-hotplug-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull smp hotplug updates from Thomas Gleixner:
 "This is the final round of converting the notifier mess to the state
  machine. The removal of the notifiers and the related infrastructure
  will happen around rc1, as there are conversions outstanding in other
  trees.

  The whole exercise removed about 2000 lines of code in total and in
  course of the conversion several dozen bugs got fixed. The new
  mechanism allows to test almost every hotplug step standalone, so
  usage sites can exercise all transitions extensively.

  There is more room for improvement, like integrating all the
  pointlessly different architecture mechanisms of synchronizing,
  setting cpus online etc into the core code"

* 'smp-hotplug-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (60 commits)
  tracing/rb: Init the CPU mask on allocation
  soc/fsl/qbman: Convert to hotplug state machine
  soc/fsl/qbman: Convert to hotplug state machine
  zram: Convert to hotplug state machine
  KVM/PPC/Book3S HV: Convert to hotplug state machine
  arm64/cpuinfo: Convert to hotplug state machine
  arm64/cpuinfo: Make hotplug notifier symmetric
  mm/compaction: Convert to hotplug state machine
  iommu/vt-d: Convert to hotplug state machine
  mm/zswap: Convert pool to hotplug state machine
  mm/zswap: Convert dst-mem to hotplug state machine
  mm/zsmalloc: Convert to hotplug state machine
  mm/vmstat: Convert to hotplug state machine
  mm/vmstat: Avoid on each online CPU loops
  mm/vmstat: Drop get_online_cpus() from init_cpu_node_state/vmstat_cpu_dead()
  tracing/rb: Convert to hotplug state machine
  oprofile/nmi timer: Convert to hotplug state machine
  net/iucv: Use explicit clean up labels in iucv_init()
  x86/pci/amd-bus: Convert to hotplug state machine
  x86/oprofile/nmi: Convert to hotplug state machine
  ...
2016-12-12 19:25:04 -08:00
Joerg Roedel 1465f48146 Merge branches 'arm/mediatek', 'arm/smmu', 'x86/amd', 's390', 'core' and 'arm/exynos' into next 2016-12-06 17:32:16 +01:00
Anna-Maria Gleixner 21647615db iommu/vt-d: Convert to hotplug state machine
Install the callbacks via the state machine.

Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: iommu@lists.linux-foundation.org
Cc: rt@linutronix.de
Cc: David Woodhouse <dwmw2@infradead.org>
Link: http://lkml.kernel.org/r/20161126231350.10321-14-bigeasy@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2016-12-02 00:52:37 +01:00
Joerg Roedel ac1d35659b Merge branch 'for-joerg/arm-smmu/updates' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into arm/smmu 2016-11-30 15:35:31 +01:00
Dan Carpenter 24c790fbf5 iommu/amd: Missing error code in amd_iommu_init_device()
We should set "ret" to -EINVAL if iommu_group_get() fails.

Fixes: 55c99a4dc5 ("iommu/amd: Use iommu_attach_group()")
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-29 17:39:44 +01:00
Geliang Tang 37bad55b78 iommu/s390: Drop duplicate header pci.h
Drop duplicate header pci.h from s390-iommu.c.

Signed-off-by: Geliang Tang <geliangtang@gmail.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-29 17:37:58 +01:00
Lorenzo Pieralisi d6fcd3b149 iommu/arm-smmu: Add IORT configuration
In ACPI based systems, in order to be able to create platform
devices and initialize them for ARM SMMU components, the IORT
kernel implementation requires a set of static functions to be
used by the IORT kernel layer to configure platform devices for
ARM SMMU components.

Add static configuration functions to the IORT kernel layer for
the ARM SMMU components, so that the ARM SMMU driver can
initialize its respective platform device by relying on the IORT
kernel infrastructure and by adding a corresponding ACPI device
early probe section entry.

Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Tomasz Nowicki <tn@semihalf.com>
Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
Tested-by: Tomasz Nowicki <tn@semihalf.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Joerg Roedel <joro@8bytes.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-11-29 15:57:47 +00:00
Lorenzo Pieralisi bbb8a1848f iommu/arm-smmu: Split probe functions into DT/generic portions
Current ARM SMMU probe functions intermingle HW and DT probing
in the initialization functions to detect and programme the ARM SMMU
driver features. In order to allow probing the ARM SMMU with other
firmwares than DT, this patch splits the ARM SMMU init functions into
DT and HW specific portions so that other FW interfaces (ie ACPI) can
reuse the HW probing functions and skip the DT portion accordingly.

This patch implements no functional change, only code reshuffling.

Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Tomasz Nowicki <tn@semihalf.com>
Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
Tested-by: Tomasz Nowicki <tn@semihalf.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Hanjun Guo <hanjun.guo@linaro.org>
Cc: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-11-29 15:57:46 +00:00
Lorenzo Pieralisi e4dadfa812 iommu/arm-smmu-v3: Add IORT configuration
In ACPI bases systems, in order to be able to create platform
devices and initialize them for ARM SMMU v3 components, the IORT
kernel implementation requires a set of static functions to be
used by the IORT kernel layer to configure platform devices for
ARM SMMU v3 components.

Add static configuration functions to the IORT kernel layer for
the ARM SMMU v3 components, so that the ARM SMMU v3 driver can
initialize its respective platform device by relying on the IORT
kernel infrastructure and by adding a corresponding ACPI device
early probe section entry.

Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Tomasz Nowicki <tn@semihalf.com>
Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
Tested-by: Tomasz Nowicki <tn@semihalf.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Joerg Roedel <joro@8bytes.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-11-29 15:57:46 +00:00
Lorenzo Pieralisi 2985b5210f iommu/arm-smmu-v3: Split probe functions into DT/generic portions
Current ARM SMMUv3 probe functions intermingle HW and DT probing in the
initialization functions to detect and programme the ARM SMMU v3 driver
features. In order to allow probing the ARM SMMUv3 with other firmwares
than DT, this patch splits the ARM SMMUv3 init functions into DT and HW
specific portions so that other FW interfaces (ie ACPI) can reuse the HW
probing functions and skip the DT portion accordingly.

This patch implements no functional change, only code reshuffling.

Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Tomasz Nowicki <tn@semihalf.com>
Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
Tested-by: Tomasz Nowicki <tn@semihalf.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Hanjun Guo <hanjun.guo@linaro.org>
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Joerg Roedel <joro@8bytes.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-11-29 15:57:45 +00:00
Lorenzo Pieralisi 778de07453 iommu/arm-smmu-v3: Convert struct device of_node to fwnode usage
Current ARM SMMU v3 driver rely on the struct device.of_node pointer for
device look-up and iommu_ops retrieval.

In preparation for ACPI probing enablement, convert the driver to use
the struct device.fwnode member for device and iommu_ops look-up so that
the driver infrastructure can be used also on systems that do not
associate an of_node pointer to a struct device (eg ACPI), making the
device look-up and iommu_ops retrieval firmware agnostic.

Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Tomasz Nowicki <tn@semihalf.com>
Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
Tested-by: Tomasz Nowicki <tn@semihalf.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Hanjun Guo <hanjun.guo@linaro.org>
Cc: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-11-29 15:57:44 +00:00
Lorenzo Pieralisi ce9babe5f6 iommu/arm-smmu: Convert struct device of_node to fwnode usage
Current ARM SMMU driver rely on the struct device.of_node pointer for
device look-up and iommu_ops retrieval.

In preparation for ACPI probing enablement, convert the driver to use
the struct device.fwnode member for device and iommu_ops look-up so that
the driver infrastructure can be used also on systems that do not
associate an of_node pointer to a struct device (eg ACPI), making the
device look-up and iommu_ops retrieval firmware agnostic.

Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Tomasz Nowicki <tn@semihalf.com>
Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
Tested-by: Tomasz Nowicki <tn@semihalf.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Hanjun Guo <hanjun.guo@linaro.org>
Cc: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-11-29 15:57:43 +00:00
Lorenzo Pieralisi e4f10ffe4c iommu: Make of_iommu_set/get_ops() DT agnostic
The of_iommu_{set/get}_ops() API is used to associate a device
tree node with a specific set of IOMMU operations. The same
kernel interface is required on systems booting with ACPI, where
devices are not associated with a device tree node, therefore
the interface requires generalization.

The struct device fwnode member represents the fwnode token associated
with the device and the struct it points at is firmware specific;
regardless, it is initialized on both ACPI and DT systems and makes an
ideal candidate to use it to associate a set of IOMMU operations to a
given device, through its struct device.fwnode member pointer, paving
the way for representing per-device iommu_ops (ie an iommu instance
associated with a device).

Convert the DT specific of_iommu_{set/get}_ops() interface to
use struct device.fwnode as a look-up token, making the interface
usable on ACPI systems and rename the data structures and the
registration API so that they are made to represent their usage
more clearly.

Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Tomasz Nowicki <tn@semihalf.com>
Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
Tested-by: Tomasz Nowicki <tn@semihalf.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Hanjun Guo <hanjun.guo@linaro.org>
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Joerg Roedel <joro@8bytes.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-11-29 15:57:43 +00:00
Nipun Gupta 6eb18d4a2b iommu/arm-smmu: Set SMTNMB_TLBEN in ACR to enable caching of bypass entries
The SMTNMB_TLBEN in the Auxiliary Configuration Register (ACR) provides an
option to enable the updation of TLB in case of bypass transactions due to
no stream match in the stream match table. This reduces the latencies of
the subsequent transactions with the same stream-id which bypasses the SMMU.
This provides a significant performance benefit for certain networking
workloads.

With this change substantial performance improvement of ~9% is observed with
DPDK l3fwd application (http://dpdk.org/doc/guides/sample_app_ug/l3_forward.html)
on NXP's LS2088a platform.

Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Nipun Gupta <nipun.gupta@nxp.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-11-29 15:57:41 +00:00
Bhumika Goyal dfed5f01e2 iommu/io-pgtable-arm: Use const and __initconst for iommu_gather_ops structures
Check for iommu_gather_ops structures that are only stored in the tlb
field of an io_pgtable_cfg structure. The tlb field is of type
const struct iommu_gather_ops *, so iommu_gather_ops structures
having this property can be declared as const. Also, replace __initdata
with __initconst.

Acked-by: Julia Lawall <julia.lawall@lip6.fr>
Signed-off-by: Bhumika Goyal <bhumirks@gmail.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-11-29 15:57:41 +00:00
Bhumika Goyal ca297aad17 iommu/arm-smmu: Constify iommu_gather_ops structures
Check for iommu_gather_ops structures that are only stored in the tlb
field of an io_pgtable_cfg structure. The tlb field is of type
const struct iommu_gather_ops *, so iommu_gather_ops structures
having this property can be declared as const.

Acked-by: Julia Lawall <julia.lawall@lip6.fr>
Signed-off-by: Bhumika Goyal <bhumirks@gmail.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-11-29 15:57:40 +00:00
Bhumika Goyal 5896f3a3a1 iommu/arm-smmu: Constify iommu_gather_ops structures
Check for iommu_gather_ops structures that are only stored in the tlb
field of an io_pgtable_cfg structure. The tlb field is of type
const struct iommu_gather_ops *, so iommu_gather_ops structures
having this property can be declared as const.

Acked-by: Julia Lawall <julia.lawall@lip6.fr>
Signed-off-by: Bhumika Goyal <bhumirks@gmail.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-11-29 15:57:40 +00:00
Kefeng Wang 4ae8a5c528 iommu/io-pgtable-arm: Use for_each_set_bit to simplify the code
We can use for_each_set_bit() to simplify the code slightly in the
ARM io-pgtable self tests.

Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-11-29 15:57:40 +00:00
Linus Torvalds 105ecadc6d Merge git://git.infradead.org/intel-iommu
Pull IOMMU fixes from David Woodhouse:
 "Two minor fixes.

  The first fixes the assignment of SR-IOV virtual functions to the
  correct IOMMU unit, and the second fixes the excessively large (and
  physically contiguous) PASID tables used with SVM"

* git://git.infradead.org/intel-iommu:
  iommu/vt-d: Fix PASID table allocation
  iommu/vt-d: Fix IOMMU lookup for SR-IOV Virtual Functions
2016-11-27 08:24:46 -08:00
David Woodhouse 9101704429 iommu/vt-d: Fix PASID table allocation
Somehow I ended up with an off-by-three error in calculating the size of
the PASID and PASID State tables, which triggers allocations failures as
those tables unfortunately have to be physically contiguous.

In fact, even the *correct* maximum size of 8MiB is problematic and is
wont to lead to allocation failures. Since I have extracted a promise
that this *will* be fixed in hardware, I'm happy to limit it on the
current hardware to a maximum of 0x20000 PASIDs, which gives us 1MiB
tables — still not ideal, but better than before.

Reported by Mika Kuoppala <mika.kuoppala@linux.intel.com> and also by
Xunlei Pang <xlpang@redhat.com> who submitted a simpler patch to fix
only the allocation (and not the free) to the "correct" limit... which
was still problematic.

Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Cc: stable@vger.kernel.org
2016-11-19 09:42:35 -08:00
Robin Murphy 62280cf2e8 iommu/iova: Extend cached node lookup condition
When searching for a free IOVA range, we optimise the tree traversal
by starting from the cached32_node, instead of the last node, when
limit_pfn is equal to dma_32bit_pfn. However, if limit_pfn happens to
be smaller, then we'll go ahead and start from the top even though
dma_32bit_pfn is still a more suitable upper bound. Since this is
clearly a silly thing to do, adjust the lookup condition appropriately.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-15 12:39:52 +01:00
Robin Murphy 5d1d43b0f6 iommu/mediatek: Fix M4Uv1 group refcounting
For each subsequent device assigned to the m4u_group after its initial
allocation, we need to take an additional reference. Otherwise, the
caller of iommu_group_get_for_dev() will inadvertently remove the
reference taken by iommu_group_add_device(), and the group will be
freed prematurely if any device is removed.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-15 12:25:33 +01:00
Robin Murphy 3a8d40b6ce iommu/mediatek: Fix M4Uv2 group refcounting
For each subsequent device assigned to the m4u_group after its initial
allocation, we need to take an additional reference. Otherwise, the
caller of iommu_group_get_for_dev() will inadvertently remove the
reference taken by iommu_group_add_device(), and the group will be
freed prematurely if any device is removed.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-15 12:25:33 +01:00
Robin Murphy f2f101f6bc iommu/amd: Fix group refcounting
If acpihid_device_group() finds an existing group for the relevant
devid, it should be taking an additional reference on that group.
Otherwise, the caller of iommu_group_get_for_dev() will inadvertently
remove the reference taken by iommu_group_add_device(), and the group
will be freed prematurely if any device is removed.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-15 12:25:33 +01:00
Robin Murphy e1b44cbe7b iommu/arm-smmu: Fix group refcounting
When arm_smmu_device_group() finds an existing group due to Stream ID
aliasing, it should be taking an additional reference on that group.
Otherwise, the caller of iommu_group_get_for_dev() will inadvertently
remove the reference taken by iommu_group_add_device(), and the group
will be freed prematurely if any device is removed.

Reported-by: Sricharan R <sricharan@codeaurora.org>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-15 12:25:33 +01:00
Robin Murphy 13f59a78c6 iommu: Allow taking a reference on a group directly
iommu_group_get_for_dev() expects that the IOMMU driver's device_group
callback return a group with a reference held for the given device.
Whilst allocating a new group is fine, and pci_device_group() correctly
handles reusing an existing group, there is no general means for IOMMU
drivers doing their own group lookup to take additional references on an
existing group pointer without having to also store device pointers or
resort to elaborate trickery.

Add an IOMMU-driver-specific function to fill the hole.

Acked-by: Sricharan R <sricharan@codeaurora.org>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-15 12:25:33 +01:00
Marek Szyprowski 2f5f44f205 iommu/exynos: Use device dependency links to control runtime pm
This patch uses recently introduced device dependency links to track the
runtime pm state of the master's device. The goal is to let SYSMMU
controller device's runtime PM to follow the runtime PM state of the
respective master's device. This way each SYSMMU controller is active
only when its master's device is active and can properly restore or save
its state instead on runtime PM transition of master's device.
This approach replaces old behavior, when SYSMMU controller was set to
runtime active once after attaching to the master device. In the new
approach SYSMMU controllers no longer prevents respective power domains
to be turned off when master's device is not being used.

This patch reduces total power consumption of idle system, because most
power domains can be finally turned off. For example, on Exynos 4412
based Odroid U3 this patch reduces power consuption from 136mA to 130mA
at 5V (by 4.4%).

The dependency links also enforce proper order of suspending/restoring
devices during system sleep transition, so there is no more need to use
LATE_SYSTEM_SLEEP_PM_OPS-based workaround for ensuring that SYSMMUs are
suspended after their master devices.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-14 17:11:59 +01:00
Marek Szyprowski 9b265536c2 iommu/exynos: Add runtime pm support
This patch adds runtime pm implementation, which is based on previous
suspend/resume code. SYSMMU controller is now being enabled/disabled mainly
from the runtime pm callbacks. System sleep callbacks relies on generic
pm_runtime_force_suspend/pm_runtime_force_resume helpers. To ensure
internal state consistency, additional lock for runtime pm transitions
was introduced.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-14 17:11:59 +01:00
Marek Szyprowski e11723000f iommu/exynos: Rework and fix internal locking
This patch reworks locking in the exynos_iommu_attach/detach_device
functions to ensure that all entries of the sysmmu_drvdata and
exynos_iommu_owner structure are updated under the respective spinlocks,
while runtime pm functions are called without any spinlocks held.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-14 17:11:59 +01:00
Marek Szyprowski 92798b4566 iommu/exynos: Set master device once on boot
To avoid possible races, set master device pointer in each SYSMMU
controller once on boot. Suspend/resume callbacks now properly relies on
the configured iommu domain to enable or disable SYSMMU controller.
While changing the code, also update the sleep debug messages and make
them conditional.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-14 17:11:59 +01:00
Marek Szyprowski 47a574fffc iommu/exynos: Simplify internal enable/disable functions
Remove remaining leftovers of the ref-count related code in the
__sysmmu_enable/disable functions inline __sysmmu_enable/disable_nocount
to them. Suspend/resume callbacks now checks if master device is set for
given SYSMMU controller instead of relying on the activation count.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-14 17:11:59 +01:00
Marek Szyprowski b0d4c861a9 iommu/exynos: Remove dead code
__sysmmu_enable/disable functions were designed to do ref-count based
operations, but current code always calls them only once, so the code for
checking the conditions and invalid conditions can be simply removed
without any influence to the driver operation.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-14 17:11:59 +01:00
Marek Szyprowski e7689f0dc4 iommu/exynos: Remove excessive, useless debug
Remove excessive, useless debug about skipping TLB invalidation, which
is a normal situation when more aggressive power management is enabled.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-14 17:11:59 +01:00
Robin Murphy 51f8cc9e81 iommu/dma: Implement dma_{map,unmap}_resource()
With the new dma_{map,unmap}_resource() functions added to the DMA API
for the benefit of cases like slave DMA, add suitable implementations to
the arsenal of our generic layer. Since cache maintenance should not be
a concern, these can both be standalone callback implementations without
the need for arch code wrappers.

CC: Joerg Roedel <joro@8bytes.org>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-14 16:58:36 +01:00
Marek Szyprowski 1a0d8dac51 iommu/exynos: Add support for page access protection bits
This patch add support for page access protection bits. Till now this
feature was disabled and Exynos SYSMMU always mapped pages as read/write.
Now page access bits are set according to the protection bits provided
in iommu_map(), so Exynos SYSMMU is able to detect incorrect access to
mapped pages. Exynos SYSMMU earlier than v5 doesn't support write-only
mappings, so pages with such protection bits are mapped as read/write.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-14 15:59:43 +01:00
Lucas Stach ebcfa28439 iommu/amd: Tell kmemleak about the irq_remap_table
This will get rid of a lot false positives caused by kmemleak being
unaware of the irq_remap_table. Based on a suggestion from Catalin Marinas.

Signed-off-by: Lucas Stach <dev@lynxeye.de>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-10 13:16:45 +01:00
Russell King 00c7c81f7b iommu/mediatek: Convert DT component matching to component_match_add_release()
Convert DT component matching to use component_match_add_release().

Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-10 12:25:16 +01:00
Robin Murphy 84672f1926 iommu/mediatek: Convert M4Uv1 to iommu_fwspec
Our per-device data consists of the M4U instance and firmware-provided
list of LARB IDs, which is a perfect fit for the generic iommu_fwspec
machinery. Use that directly instead of the custom archdata code - while
we can't rely on the of_xlate() mechanism to initialise things until the
32-bit ARM DMA code learns about groups and default domains, it still
results in a reasonable simplification overall.

CC: Honghui Zhang <honghui.zhang@mediatek.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Tested-by: Honghui Zhang <honghui.zhang@mediatek.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-10 11:59:32 +01:00
Robin Murphy 58f0d1d536 iommu/mediatek: Convert M4Uv2 to iommu_fwspec
Our per-device data consists of the M4U instance and firmware-provided
list of LARB IDs, which is a perfect fit for the generic iommu_fwspec
machinery. Use that directly as a simpler alternative to the custom
archdata code.

CC: Yong Wu <yong.wu@mediatek.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Tested-by: Yong Wu <yong.wu@mediatek.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-10 11:59:32 +01:00
Joerg Roedel bea64033dd iommu/vt-d: Fix dead-locks in disable_dmar_iommu() path
It turns out that the disable_dmar_iommu() code-path tried
to get the device_domain_lock recursivly, which will
dead-lock when this code runs on dmar removal. Fix both
code-paths that could lead to the dead-lock.

Fixes: 55d940430a ('iommu/vt-d: Get rid of domain->iommu_lock')
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-08 15:08:26 +01:00
Robin Murphy 8c82d6ec5a iommu/arm-smmu: Fix out-of-bounds dereference
When we iterate a master's config entries, what we generally care
about is the entry's stream map index, rather than the entry index
itself, so it's nice to have the iterator automatically assign the
former from the latter. Unfortunately, booting with KASAN reveals
the oversight that using a simple comma operator results in the
entry index being dereferenced before being checked for validity,
so we always access one element past the end of the fwspec array.

Flip things around so that the check always happens before the index
may be dereferenced.

Fixes: adfec2e709 ("iommu/arm-smmu: Convert to iommu_fwspec")
Reported-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-08 14:52:41 +01:00
Robin Murphy 3c117b5435 iommu/arm-smmu: Check that iommu_fwspecs are ours
We seem to have forgotten to check that iommu_fwspecs actually belong to
us before we go ahead and dereference their private data. Oops.

Fixes: 021bb8420d ("iommu/arm-smmu: Wire up generic configuration support")
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-08 14:46:41 +01:00
Robin Murphy ec615f43d3 iommu/arm-smmu: Don't inadvertently reject multiple SMMUv3s
We now delay installing our per-bus iommu_ops until we know an SMMU has
successfully probed, as they don't serve much purpose beforehand, and
doing so also avoids fights between multiple IOMMU drivers in a single
kernel. However, the upshot of passing the return value of bus_set_iommu()
back from our probe function is that if there happens to be more than
one SMMUv3 device in a system, the second and subsequent probes will
wind up returning -EBUSY to the driver core and getting torn down again.

Avoid re-setting ops if ours are already installed, so that any genuine
failures stand out.

Fixes: 08d4ca2a67 ("iommu/arm-smmu: Support non-PCI devices with SMMUv3")
CC: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
CC: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-08 14:46:41 +01:00
Robin Murphy fba4f8e5c4 iommu/arm-smmu: Work around ARM DMA configuration
The 32-bit ARM DMA configuration code predates the IOMMU core's default
domain functionality, and instead relies on allocating its own domains
and attaching any devices using the generic IOMMU binding to them.
Unfortunately, it does this relatively early on in the creation of the
device, before we've seen our add_device callback, which leads us to
attempt to operate on a half-configured master.

To avoid a crash, check for this situation on attach, but refuse to
play, as there's nothing we can do. This at least allows VFIO to keep
working for people who update their 32-bit DTs to the generic binding,
albeit with a few (innocuous) warnings from the DMA layer on boot.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-11-08 14:46:41 +01:00
Ashok Raj 1c387188c6 iommu/vt-d: Fix IOMMU lookup for SR-IOV Virtual Functions
The VT-d specification (§8.3.3) says:
    ‘Virtual Functions’ of a ‘Physical Function’ are under the scope
    of the same remapping unit as the ‘Physical Function’.

The BIOS is not required to list all the possible VFs in the scope
tables, and arguably *shouldn't* make any attempt to do so, since there
could be a huge number of them.

This has been broken basically for ever — the VF is never going to match
against a specific unit's scope, so it ends up being assigned to the
INCLUDE_ALL IOMMU. Which was always actually correct by coincidence, but
now we're looking at Root-Complex integrated devices with SR-IOV support
it's going to start being wrong.

Fix it to simply use pci_physfn() before doing the lookup for PCI devices.

Cc: stable@vger.kernel.org
Signed-off-by: Sainath Grandhi <sainath.grandhi@intel.com>
Signed-off-by: Ashok Raj <ashok.raj@intel.com>
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
2016-10-30 05:32:51 -06:00
Linus Torvalds 56e520c7a0 IOMMU Updates for Linux v4.9
Including:
 
 	* Support for interrupt virtualization in the AMD IOMMU driver.
 	  These patches were shared with the KVM tree and are already
 	  merged through that tree.
 
 	* Generic DT-binding support for the ARM-SMMU driver. With this
 	  the driver now makes use of the generic DMA-API code. This
 	  also required some changes outside of the IOMMU code, but
 	  these are acked by the respective maintainers.
 
 	* More cleanups and fixes all over the place.
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2
 
 iQIcBAABAgAGBQJX/PmUAAoJECvwRC2XARrjyloP/1hymxXC2yXZ4EIBTHSO5X+c
 jSJaGTIbQAdQDpllscSNJ0Au43L3vGtJcHo4JqwEERNlwLsU82LH7QJhq+q1La/b
 5cPaY5gI3E++qxQt8umuZJAIUQthFYrfGoS5lJc5t5r/d8iVsLWbW4VkR19/1o7A
 4/Uz7ETmi9VVy8Hkvumx+PQ0VHJet381KB7ud9LU5Spim0En2AAGwZXLMkmxXd2W
 uDQ+O1rlDVc2/ka3+GmfZEml5EASWRqS/MTNoU/ZbQGYWKCWygXbuiqt6gLudWjx
 dCR1Knh68b0gN6k/QAj8XY/1gkfmZ3YkfS0AHIMLYTFRT51BuxOrkXrBdkYnWEBv
 UirmaiV87SlR1j83yb3ZmjpBPvd2sGWYFDqY1P0riLutjGUS6zycWWs13olvbfbz
 SFrH7PT7JPQGYprI1oVn4ihszjN1NZ4+Gj7QBhyFW6FtvqTzmaFVsMOlDIeg1FwR
 k8cOzov4NG33Bp4IpsHK8e0/qV6K3oJOiOQgCyQp9kPKK+UWv9v9+HaEA7npJuRV
 c+lTE6j3G4LjEoVybkqm8TiPKxTMVNjUjgA3kwB2yNkCQT7hTCNYIAFrtfCYjYdo
 B1dnFE7feVqtoimnu2qvkVs59hWlF7Hc3RRHoBxMmO8DLwl9n2OcmoQIeCTsviss
 i9aNwC9bzBs+Hd3X/psB
 =1hFE
 -----END PGP SIGNATURE-----

Merge tag 'iommu-updates-v4.9' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu

Pull IOMMU updates from Joerg Roedel:

 - support for interrupt virtualization in the AMD IOMMU driver. These
   patches were shared with the KVM tree and are already merged through
   that tree.

 - generic DT-binding support for the ARM-SMMU driver. With this the
   driver now makes use of the generic DMA-API code. This also required
   some changes outside of the IOMMU code, but these are acked by the
   respective maintainers.

 - more cleanups and fixes all over the place.

* tag 'iommu-updates-v4.9' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (40 commits)
  iommu/amd: No need to wait iommu completion if no dte irq entry change
  iommu/amd: Free domain id when free a domain of struct dma_ops_domain
  iommu/amd: Use standard bitmap operation to set bitmap
  iommu/amd: Clean up the cmpxchg64 invocation
  iommu/io-pgtable-arm: Check for v7s-incapable systems
  iommu/dma: Avoid PCI host bridge windows
  iommu/dma: Add support for mapping MSIs
  iommu/arm-smmu: Set domain geometry
  iommu/arm-smmu: Wire up generic configuration support
  Docs: dt: document ARM SMMU generic binding usage
  iommu/arm-smmu: Convert to iommu_fwspec
  iommu/arm-smmu: Intelligent SMR allocation
  iommu/arm-smmu: Add a stream map entry iterator
  iommu/arm-smmu: Streamline SMMU data lookups
  iommu/arm-smmu: Refactor mmu-masters handling
  iommu/arm-smmu: Keep track of S2CR state
  iommu/arm-smmu: Consolidate stream map entry state
  iommu/arm-smmu: Handle stream IDs more dynamically
  iommu/arm-smmu: Set PRIVCFG in stage 1 STEs
  iommu/arm-smmu: Support non-PCI devices with SMMUv3
  ...
2016-10-11 12:52:41 -07:00
Linus Torvalds 6218590bcb KVM updates for v4.9-rc1
All architectures:
   Move `make kvmconfig` stubs from x86;  use 64 bits for debugfs stats.
 
 ARM:
   Important fixes for not using an in-kernel irqchip; handle SError
   exceptions and present them to guests if appropriate; proxying of GICV
   access at EL2 if guest mappings are unsafe; GICv3 on AArch32 on ARMv8;
   preparations for GICv3 save/restore, including ABI docs; cleanups and
   a bit of optimizations.
 
 MIPS:
   A couple of fixes in preparation for supporting MIPS EVA host kernels;
   MIPS SMP host & TLB invalidation fixes.
 
 PPC:
   Fix the bug which caused guests to falsely report lockups; other minor
   fixes; a small optimization.
 
 s390:
   Lazy enablement of runtime instrumentation; up to 255 CPUs for nested
   guests; rework of machine check deliver; cleanups and fixes.
 
 x86:
   IOMMU part of AMD's AVIC for vmexit-less interrupt delivery; Hyper-V
   TSC page; per-vcpu tsc_offset in debugfs; accelerated INS/OUTS in
   nVMX; cleanups and fixes.
 -----BEGIN PGP SIGNATURE-----
 
 iQEcBAABCAAGBQJX9iDrAAoJEED/6hsPKofoOPoIAIUlgojkb9l2l1XVDgsXdgQL
 sRVhYSVv7/c8sk9vFImrD5ElOPZd+CEAIqFOu45+NM3cNi7gxip9yftUVs7wI5aC
 eDZRWm1E4trDZLe54ZM9ThcqZzZZiELVGMfR1+ZndUycybwyWzafpXYsYyaXp3BW
 hyHM3qVkoWO3dxBWFwHIoO/AUJrWYkRHEByKyvlC6KPxSdBPSa5c1AQwMCoE0Mo4
 K/xUj4gBn9eMelNhg4Oqu/uh49/q+dtdoP2C+sVM8bSdquD+PmIeOhPFIcuGbGFI
 B+oRpUhIuntN39gz8wInJ4/GRSeTuR2faNPxMn4E1i1u4LiuJvipcsOjPfe0a18=
 =fZRB
 -----END PGP SIGNATURE-----

Merge tag 'kvm-4.9-1' of git://git.kernel.org/pub/scm/virt/kvm/kvm

Pull KVM updates from Radim Krčmář:
 "All architectures:
   - move `make kvmconfig` stubs from x86
   - use 64 bits for debugfs stats

  ARM:
   - Important fixes for not using an in-kernel irqchip
   - handle SError exceptions and present them to guests if appropriate
   - proxying of GICV access at EL2 if guest mappings are unsafe
   - GICv3 on AArch32 on ARMv8
   - preparations for GICv3 save/restore, including ABI docs
   - cleanups and a bit of optimizations

  MIPS:
   - A couple of fixes in preparation for supporting MIPS EVA host
     kernels
   - MIPS SMP host & TLB invalidation fixes

  PPC:
   - Fix the bug which caused guests to falsely report lockups
   - other minor fixes
   - a small optimization

  s390:
   - Lazy enablement of runtime instrumentation
   - up to 255 CPUs for nested guests
   - rework of machine check deliver
   - cleanups and fixes

  x86:
   - IOMMU part of AMD's AVIC for vmexit-less interrupt delivery
   - Hyper-V TSC page
   - per-vcpu tsc_offset in debugfs
   - accelerated INS/OUTS in nVMX
   - cleanups and fixes"

* tag 'kvm-4.9-1' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (140 commits)
  KVM: MIPS: Drop dubious EntryHi optimisation
  KVM: MIPS: Invalidate TLB by regenerating ASIDs
  KVM: MIPS: Split kernel/user ASID regeneration
  KVM: MIPS: Drop other CPU ASIDs on guest MMU changes
  KVM: arm/arm64: vgic: Don't flush/sync without a working vgic
  KVM: arm64: Require in-kernel irqchip for PMU support
  KVM: PPC: Book3s PR: Allow access to unprivileged MMCR2 register
  KVM: PPC: Book3S PR: Support 64kB page size on POWER8E and POWER8NVL
  KVM: PPC: Book3S: Remove duplicate setting of the B field in tlbie
  KVM: PPC: BookE: Fix a sanity check
  KVM: PPC: Book3S HV: Take out virtual core piggybacking code
  KVM: PPC: Book3S: Treat VTB as a per-subcore register, not per-thread
  ARM: gic-v3: Work around definition of gic_write_bpr1
  KVM: nVMX: Fix the NMI IDT-vectoring handling
  KVM: VMX: Enable MSR-BASED TPR shadow even if APICv is inactive
  KVM: nVMX: Fix reload apic access page warning
  kvmconfig: add virtio-gpu to config fragment
  config: move x86 kvm_guest.config to a common location
  arm64: KVM: Remove duplicating init code for setting VMID
  ARM: KVM: Support vgic-v3
  ...
2016-10-06 10:49:01 -07:00
Linus Torvalds e46cae4418 Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux
Pull s390 updates from Martin Schwidefsky:
 "The new features and main improvements in this merge for v4.9

   - Support for the UBSAN sanitizer

   - Set HAVE_EFFICIENT_UNALIGNED_ACCESS, it improves the code in some
     places

   - Improvements for the in-kernel fpu code, in particular the overhead
     for multiple consecutive in kernel fpu users is recuded

   - Add a SIMD implementation for the RAID6 gen and xor operations

   - Add RAID6 recovery based on the XC instruction

   - The PCI DMA flush logic has been improved to increase the speed of
     the map / unmap operations

   - The time synchronization code has seen some updates

  And bug fixes all over the place"

* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux: (48 commits)
  s390/con3270: fix insufficient space padding
  s390/con3270: fix use of uninitialised data
  MAINTAINERS: update DASD maintainer
  s390/cio: fix accidental interrupt enabling during resume
  s390/dasd: add missing \n to end of dev_err messages
  s390/config: Enable config options for Docker
  s390/dasd: make query host access interruptible
  s390/dasd: fix panic during offline processing
  s390/dasd: fix hanging offline processing
  s390/pci_dma: improve lazy flush for unmap
  s390/pci_dma: split dma_update_trans
  s390/pci_dma: improve map_sg
  s390/pci_dma: simplify dma address calculation
  s390/pci_dma: remove dma address range check
  iommu/s390: simplify registration of I/O address translation parameters
  s390: migrate exception table users off module.h and onto extable.h
  s390: export header for CLP ioctl
  s390/vmur: fix irq pointer dereference in int handler
  s390/dasd: add missing KOBJ_CHANGE event for unformatted devices
  s390: enable UBSAN
  ...
2016-10-04 14:05:52 -07:00
Sebastian Ott bb2b7ffbc4 iommu/s390: simplify registration of I/O address translation parameters
When a new function is attached to an iommu domain we need to register
I/O address translation parameters. Since commit
69eea95c ("s390/pci_dma: fix DMA table corruption with > 4 TB main memory")
start_dma and end_dma correctly describe the range of usable I/O addresses.

Simplify the code by using these values directly.

Signed-off-by: Sebastian Ott <sebott@linux.vnet.ibm.com>
Reviewed-by: Gerald Schaefer <gerald.schaefer@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-09-22 13:42:28 +02:00
Joerg Roedel 13a0825918 Merge branches 'x86/amd', 'x86/vt-d', 'arm/exynos', 'arm/mediatek', 'arm/renesas' and 'arm/smmu' into next 2016-09-20 13:27:09 +02:00
Baoquan He 09284b9c4a iommu/amd: No need to wait iommu completion if no dte irq entry change
This is a clean up. In get_irq_table() only if DTE entry is changed
iommu_completion_wait() need be called. Otherwise no need to do it.

Signed-off-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-20 11:54:11 +02:00
Baoquan He c3db901c54 iommu/amd: Free domain id when free a domain of struct dma_ops_domain
The current code missed freeing domain id when free a domain of
struct dma_ops_domain.

Signed-off-by: Baoquan He <bhe@redhat.com>
Fixes: ec487d1a11 ('x86, AMD IOMMU: add domain allocation and deallocation functions')
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-19 15:49:01 +02:00
Baoquan He 5c87f62dd1 iommu/amd: Use standard bitmap operation to set bitmap
It will be more readable and safer than the old setting.

Signed-off-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-19 15:49:01 +02:00
Baoquan He 134414fffc iommu/amd: Clean up the cmpxchg64 invocation
Change it as it's designed for and keep it consistent with other
places.

Signed-off-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-19 15:49:01 +02:00
Robin Murphy 82db33dc5e iommu/io-pgtable-arm: Check for v7s-incapable systems
On machines with no 32-bit addressable RAM whatsoever, we shouldn't
even touch the v7s format as it's never going to work.

Fixes: e5fc9753b1 ("iommu/io-pgtable: Add ARMv7 short descriptor support")
Reported-by: Eric Auger <eric.auger@redhat.com>
Tested-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:23 +01:00
Robin Murphy fade1ec055 iommu/dma: Avoid PCI host bridge windows
With our DMA ops enabled for PCI devices, we should avoid allocating
IOVAs which a host bridge might misinterpret as peer-to-peer DMA and
lead to faults, corruption or other badness. To be safe, punch out holes
for all of the relevant host bridge's windows when initialising a DMA
domain for a PCI device.

CC: Marek Szyprowski <m.szyprowski@samsung.com>
CC: Inki Dae <inki.dae@samsung.com>
Reported-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:22 +01:00
Robin Murphy 44bb7e243b iommu/dma: Add support for mapping MSIs
When an MSI doorbell is located downstream of an IOMMU, attaching
devices to a DMA ops domain and switching on translation leads to a rude
shock when their attempt to write to the physical address returned by
the irqchip driver faults (or worse, writes into some already-mapped
buffer) and no interrupt is forthcoming.

Address this by adding a hook for relevant irqchip drivers to call from
their compose_msi_msg() callback, to swizzle the physical address with
an appropriatly-mapped IOVA for any device attached to one of our DMA
ops domains.

Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:22 +01:00
Robin Murphy 455eb7d34a iommu/arm-smmu: Set domain geometry
For non-aperture-based IOMMUs, the domain geometry seems to have become
the de-facto way of indicating the input address space size. That is
quite a useful thing from the users' perspective, so let's do the same.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:22 +01:00
Robin Murphy 021bb8420d iommu/arm-smmu: Wire up generic configuration support
With everything else now in place, fill in an of_xlate callback and the
appropriate registration to plumb into the generic configuration
machinery, and watch everything just work.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:21 +01:00
Robin Murphy adfec2e709 iommu/arm-smmu: Convert to iommu_fwspec
In the final step of preparation for full generic configuration support,
swap our fixed-size master_cfg for the generic iommu_fwspec. For the
legacy DT bindings, the driver simply gets to act as its own 'firmware'.
Farewell, arbitrary MAX_MASTER_STREAMIDS!

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:20 +01:00
Robin Murphy 588888a739 iommu/arm-smmu: Intelligent SMR allocation
Stream Match Registers are one of the more awkward parts of the SMMUv2
architecture; there are typically never enough to assign one to each
stream ID in the system, and configuring them such that a single ID
matches multiple entries is catastrophically bad - at best, every
transaction raises a global fault; at worst, they go *somewhere*.

To address the former issue, we can mask ID bits such that a single
register may be used to match multiple IDs belonging to the same device
or group, but doing so also heightens the risk of the latter problem
(which can be nasty to debug).

Tackle both problems at once by replacing the simple bitmap allocator
with something much cleverer. Now that we have convenient in-memory
representations of the stream mapping table, it becomes straightforward
to properly validate new SMR entries against the current state, opening
the door to arbitrary masking and SMR sharing.

Another feature which falls out of this is that with IDs shared by
separate devices being automatically accounted for, simply associating a
group pointer with the S2CR offers appropriate group allocation almost
for free, so hook that up in the process.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:20 +01:00
Robin Murphy d3097e3930 iommu/arm-smmu: Add a stream map entry iterator
We iterate over the SMEs associated with a master config quite a lot in
various places, and are about to do so even more. Let's wrap the idiom
in a handy iterator macro before the repetition gets out of hand.

Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:20 +01:00
Robin Murphy d6fc5d9776 iommu/arm-smmu: Streamline SMMU data lookups
Simplify things somewhat by stashing our arm_smmu_device instance in
drvdata, so that it's readily available to our driver model callbacks.
Then we can excise the private list entirely, since the driver core
already has a perfectly good list of SMMU devices we can use in the one
instance we actually need to. Finally, make a further modest code saving
with the relatively new of_device_get_match_data() helper.

Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:19 +01:00
Robin Murphy f80cd885fc iommu/arm-smmu: Refactor mmu-masters handling
To be able to support the generic bindings and handle of_xlate() calls,
we need to be able to associate SMMUs and stream IDs directly with
devices *before* allocating IOMMU groups. Furthermore, to support real
default domains with multi-device groups we also have to handle domain
attach on a per-device basis, as the "whole group at a time" assumption
fails to properly handle subsequent devices added to a group after the
first has already triggered default domain creation and attachment.

To that end, use the now-vacant dev->archdata.iommu field for easy
config and SMMU instance lookup, and unify config management by chopping
down the platform-device-specific tree and probing the "mmu-masters"
property on-demand instead. This may add a bit of one-off overhead to
initially adding a new device, but we're about to deprecate that binding
in favour of the inherently-more-efficient generic ones anyway.

For the sake of simplicity, this patch does temporarily regress the case
of aliasing PCI devices by losing the duplicate stream ID detection that
the previous per-group config had. Stay tuned, because we'll be back to
fix that in a better and more general way momentarily...

Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:19 +01:00
Robin Murphy 8e8b203eab iommu/arm-smmu: Keep track of S2CR state
Making S2CRs first-class citizens within the driver with a high-level
representation of their state offers a neat solution to a few problems:

Firstly, the information about which context a device's stream IDs are
associated with is already present by necessity in the S2CR. With that
state easily accessible we can refer directly to it and obviate the need
to track an IOMMU domain in each device's archdata (its earlier purpose
of enforcing correct attachment of multi-device groups now being handled
by the IOMMU core itself).

Secondly, the core API now deprecates explicit domain detach and expects
domain attach to move devices smoothly from one domain to another; for
SMMUv2, this notion maps directly to simply rewriting the S2CRs assigned
to the device. By giving the driver a suitable abstraction of those
S2CRs to work with, we can massively reduce the overhead of the current
heavy-handed "detach, free resources, reallocate resources, attach"
approach.

Thirdly, making the software state hardware-shaped and attached to the
SMMU instance once again makes suspend/resume of this register group
that much simpler to implement in future.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:18 +01:00
Robin Murphy 1f3d5ca430 iommu/arm-smmu: Consolidate stream map entry state
In order to consider SMR masking, we really want to be able to validate
ID/mask pairs against existing SMR contents to prevent stream match
conflicts, which at best would cause transactions to fault unexpectedly,
and at worst lead to silent unpredictable behaviour. With our SMMU
instance data holding only an allocator bitmap, and the SMR values
themselves scattered across master configs hanging off devices which we
may have no way of finding, there's essentially no way short of digging
everything back out of the hardware. Similarly, the thought of power
management ops to support suspend/resume faces the exact same problem.

By massaging the software state into a closer shape to the underlying
hardware, everything comes together quite nicely; the allocator and the
high-level view of the data become a single centralised state which we
can easily keep track of, and to which any updates can be validated in
full before being synchronised to the hardware itself.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:18 +01:00
Robin Murphy 21174240e4 iommu/arm-smmu: Handle stream IDs more dynamically
Rather than assuming fixed worst-case values for stream IDs and SMR
masks, keep track of whatever implemented bits the hardware actually
reports. This also obviates the slightly questionable validation of SMR
fields in isolation - rather than aborting the whole SMMU probe for a
hardware configuration which is still architecturally valid, we can
simply refuse masters later if they try to claim an unrepresentable ID
or mask (which almost certainly implies a DT error anyway).

Acked-by: Will Deacon <will.deacon@arm.com>
Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:18 +01:00
Robin Murphy 95fa99aa40 iommu/arm-smmu: Set PRIVCFG in stage 1 STEs
Implement the SMMUv3 equivalent of d346180e70 ("iommu/arm-smmu: Treat
all device transactions as unprivileged"), so that once again those
pesky DMA controllers with their privileged instruction fetches don't
unexpectedly fault in stage 1 domains due to VMSAv8 rules.

Acked-by: Will Deacon <will.deacon@arm.com>
Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:17 +01:00
Robin Murphy 08d4ca2a67 iommu/arm-smmu: Support non-PCI devices with SMMUv3
With the device <-> stream ID relationship suitably abstracted and
of_xlate() hooked up, the PCI dependency now looks, and is, entirely
arbitrary. Any bus using the of_dma_configure() mechanism will work,
so extend support to the platform and AMBA buses which do just that.

Acked-by: Will Deacon <will.deacon@arm.com>
Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:17 +01:00
Robin Murphy 8f78515425 iommu/arm-smmu: Implement of_xlate() for SMMUv3
Now that we can properly describe the mapping between PCI RIDs and
stream IDs via "iommu-map", and have it fed it to the driver
automatically via of_xlate(), rework the SMMUv3 driver to benefit from
that, and get rid of the current misuse of the "iommus" binding.

Since having of_xlate wired up means that masters will now be given the
appropriate DMA ops, we also need to make sure that default domains work
properly. This necessitates dispensing with the "whole group at a time"
notion for attaching to a domain, as devices which share a group get
attached to the group's default domain one by one as they are initially
probed.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:16 +01:00
Robin Murphy dc87a98db7 iommu/arm-smmu: Fall back to global bypass
Unlike SMMUv2, SMMUv3 has no easy way to bypass unknown stream IDs,
other than allocating and filling in the entire stream table with bypass
entries, which for some configurations would waste *gigabytes* of RAM.
Otherwise, all transactions on unknown stream IDs will simply be aborted
with a C_BAD_STREAMID event.

Rather than render the system unusable in the case of an invalid DT,
avoid enabling the SMMU altogether such that everything bypasses
(though letting the explicit disable_bypass option take precedence).

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:16 +01:00
Robin Murphy 57f98d2f61 iommu: Introduce iommu_fwspec
Introduce a common structure to hold the per-device firmware data that
most IOMMU drivers need to keep track of. This enables us to configure
much of that data from common firmware code, and consolidate a lot of
the equivalent implementations, device look-up tables, etc. which are
currently strewn across IOMMU drivers.

This will also be enable us to address the outstanding "multiple IOMMUs
on the platform bus" problem by tweaking IOMMU API calls to prefer
dev->fwspec->ops before falling back to dev->bus->iommu_ops, and thus
gracefully handle those troublesome systems which we currently cannot.

As the first user, hook up the OF IOMMU configuration mechanism. The
driver-defined nature of DT cells means that we still need the drivers
to translate and add the IDs themselves, but future users such as the
much less free-form ACPI IORT will be much simpler and self-contained.

CC: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Suggested-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:15 +01:00
Robin Murphy b996444cf3 iommu/of: Handle iommu-map property for PCI
Now that we have a way to pick up the RID translation and target IOMMU,
hook up of_iommu_configure() to bring PCI devices into the of_xlate
mechanism and allow them IOMMU-backed DMA ops without the need for
driver-specific handling.

Reviewed-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:15 +01:00
Will Deacon 8ded2909e2 iommu/arm-smmu: Disable interrupts whilst holding the cmdq lock
The cmdq lock is taken whenever we issue commands into the command queue,
which can occur in IRQ context (as a result of unmap) or in process
context (as a result of a threaded IRQ handler or device probe).

This can lead to a theoretical deadlock if the interrupt handler
performing the unmap hits whilst the lock is taken, so explicitly use
the {irqsave,irqrestore} spin_lock accessors for the cmdq lock.

Tested-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:14 +01:00
Jean-Philippe Brucker bcfced1580 iommu/arm-smmu: Fix polling of command queue
When the SMMUv3 driver attempts to send a command, it adds an entry to the
command queue. This is a circular buffer, where both the producer and
consumer have a wrap bit. When producer.index == consumer.index and
producer.wrap == consumer.wrap, the list is empty. When producer.index ==
consumer.index and producer.wrap != consumer.wrap, the list is full.

If the list is full when the driver needs to add a command, it waits for
the SMMU to consume one command, and advance the consumer pointer. The
problem is that we currently rely on "X before Y" operation to know if
entries have been consumed, which is a bit fiddly since it only makes
sense when the distance between X and Y is less than or equal to the size
of the queue. At the moment when the list is full, we use "Consumer before
Producer + 1", which is out of range and returns a value opposite to what
we expect: when the queue transitions to not full, we stay in the polling
loop and time out, printing an error.

Given that the actual bug was difficult to determine, simplify the polling
logic by relying exclusively on queue_full and queue_empty, that don't
have this range constraint. Polling the queue is now straightforward:

* When we want to add a command and the list is full, wait until it isn't
  full and retry.
* After adding a sync, wait for the list to be empty before returning.

Suggested-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:13 +01:00
Robin Murphy 6070529beb iommu/arm-smmu: Support v7s context format
Fill in the last bits of machinery required to drive a stage 1 context
bank in v7 short descriptor format. By default we'll prefer to use it
only when the CPUs are also using the same format, such that we're
guaranteed that everything will be strictly 32-bit.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:13 +01:00
Jean-Philippe Brucker b4163fb333 iommu/arm-smmu: Fix event queues synchronization
SMMUv3 only sends interrupts for event queues (EVTQ and PRIQ) when they
transition from empty to non-empty. At the moment, if the SMMU adds new
items to a queue before the event thread finished consuming a previous
batch, the driver ignores any new item. The queue is then stuck in
non-empty state and all subsequent events will be lost.

As an example, consider the following flow, where (P, C) is the SMMU view
of producer/consumer indices, and (p, c) the driver view.

						P C | p c
  1. SMMU appends a PPR to the PRI queue,	1 0 | 0 0
          sends an MSI
  2. PRIQ handler is called.			1 0 | 1 0
  3. SMMU appends a PPR to the PRI queue.	2 0 | 1 0
  4. PRIQ thread removes the first element.	2 1 | 1 1

  5. PRIQ thread believes that the queue is empty, goes into idle
     indefinitely.

To avoid this, always synchronize the producer index and drain the queue
once before leaving an event handler. In order to prevent races on the
local producer index, move all event queue handling into the threads.

Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:12 +01:00
Peng Fan e2d42311ff iommu/arm-smmu: Drop devm_free_irq when driver detach
There is no need to call devm_free_irq when driver detach.
devres_release_all which is called after 'drv->remove' will
release all managed resources.

Signed-off-by: Peng Fan <van.freenix@gmail.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-16 09:34:12 +01:00
Joerg Roedel 4bf5beef57 iommu/amd: Don't put completion-wait semaphore on stack
The semaphore used by the AMD IOMMU to signal command
completion lived on the stack until now, which was safe as
the driver busy-waited on the semaphore with IRQs disabled,
so the stack can't go away under the driver.

But the recently introduced vmap-based stacks break this as
the physical address of the semaphore can't be determinded
easily anymore. The driver used the __pa() macro, but that
only works in the direct-mapping. The result were
Completion-Wait timeout errors seen by the IOMMU driver,
breaking system boot.

Since putting the semaphore on the stack is bad design
anyway, move the semaphore into 'struct amd_iommu'. It is
protected by the per-iommu lock and now in the direct
mapping again. This fixes the Completion-Wait timeout errors
and makes AMD IOMMU systems boot again with vmap-based
stacks enabled.

Reported-by: Borislav Petkov <bp@alien8.de>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2016-09-15 11:28:19 +02:00
Paolo Bonzini 4174879b2e Merge branch 'x86/amd-avic' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu into HEAD
Merge IOMMU bits for virtualization of interrupt injection into
virtual machines.
2016-09-05 16:00:24 +02:00
Joerg Roedel 1c5ebba95b iommu/vt-d: Make sure RMRRs are mapped before domain goes public
When a domain is allocated through the get_valid_domain_for_dev
path, it will be context-mapped before the RMRR regions are
mapped in the page-table. This opens a short time window
where device-accesses to these regions fail and causing DMAR
faults.

Fix this by mapping the RMRR regions before the domain is
context-mapped.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-05 13:00:28 +02:00
Joerg Roedel 76208356a0 iommu/vt-d: Split up get_domain_for_dev function
Split out the search for an already existing domain and the
context mapping of the device to the new domain.

This allows to map possible RMRR regions into the domain
before it is context mapped.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-05 13:00:28 +02:00
Shawn Lin b1e2afca63 iommu/ipmmu-vmsa: Fix wrong error handle of ipmmu_add_device
Let's fix the error handle of ipmmu_add_device
when failing to find utlbs, otherwise we take a
risk of pontential memleak.

Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-05 12:53:58 +02:00
Joerg Roedel 368d06cd53 Merge branch 'x86/amd-avic' into x86/amd 2016-09-05 12:43:16 +02:00
Fabian Frederick 395adae450 iommu/amd: Remove AMD_IOMMU_STATS
Commit e85e8f69ce ("iommu/amd: Remove statistics code")
removed that configuration.

Also remove function definition (suggested by Joerg Roedel)

Signed-off-by: Fabian Frederick <fabf@skynet.be>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-05 12:42:27 +02:00
Suravee Suthikulpanit d98de49a53 iommu/amd: Enable vAPIC interrupt remapping mode by default
Introduce struct iommu_dev_data.use_vapic flag, which IOMMU driver
uses to determine if it should enable vAPIC support, by setting
the ga_mode bit in the device's interrupt remapping table entry.

Currently, it is enabled for all pass-through device if vAPIC mode
is enabled.

Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-05 12:41:46 +02:00
Suravee Suthikulpanit b9fc6b56f4 iommu/amd: Implements irq_set_vcpu_affinity() hook to setup vapic mode for pass-through devices
This patch implements irq_set_vcpu_affinity() function to set up interrupt
remapping table entry with vapic mode for pass-through devices.

In case requirements for vapic mode are not met, it falls back to set up
the IRTE in legacy mode.

Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-05 12:41:46 +02:00
Suravee Suthikulpanit 8dbea3fd7b iommu/amd: Introduce amd_iommu_update_ga()
Introduces a new IOMMU API, amd_iommu_update_ga(), which allows
KVM (SVM) to update existing posted interrupt IOMMU IRTE when
load/unload vcpu.

Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-05 12:41:46 +02:00
Suravee Suthikulpanit bd6fcefc66 iommu/amd: Adding GALOG interrupt handler
This patch adds AMD IOMMU guest virtual APIC log (GALOG) handler.
When IOMMU hardware receives an interrupt targeting a blocking vcpu,
it creates an entry in the GALOG, and generates an interrupt to notify
the AMD IOMMU driver.

At this point, the driver processes the log entry, and notify the SVM
driver via the registered iommu_ga_log_notifier function.

Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-05 12:41:46 +02:00
Suravee Suthikulpanit 8bda0cfbdc iommu/amd: Detect and initialize guest vAPIC log
This patch adds support to detect and initialize IOMMU Guest vAPIC log
(GALOG). By default, it also enable GALog interrupt to notify IOMMU driver
when GA Log entry is created.

Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-05 12:41:46 +02:00
Suravee Suthikulpanit 77bdab46f0 iommu/amd: Add support for multiple IRTE formats
This patch enables support for the new 128-bit IOMMU IRTE format,
which can be used for both legacy and vapic interrupt remapping modes.
It replaces the existing operations on IRTE, which can only support
the older 32-bit IRTE format, with calls to the new struct amd_irt_ops.

It also provides helper functions for setting up, accessing, and
updating interrupt remapping table entries in different mode.

Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-05 12:41:46 +02:00
Suravee Suthikulpanit 880ac60e25 iommu/amd: Introduce interrupt remapping ops structure
Currently, IOMMU support two interrupt remapping table entry formats,
32-bit (legacy) and 128-bit (GA). The spec also implies that it might
support additional modes/formats in the future.

So, this patch introduces the new struct amd_irte_ops, which allows
the same code to work with different irte formats by providing hooks
for various operations on an interrupt remapping table entry.

Suggested-by: Joerg Roedel <joro@8bytes.org>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-05 12:41:46 +02:00
Suravee Suthikulpanit a38180bd36 iommu/amd: Move and introduce new IRTE-related unions and structures
Move existing unions and structs for accessing/managing IRTE to a proper
header file. This is mainly to simplify variable declarations in subsequent
patches.

Besides, this patch also introduces new struct irte_ga for the new
128-bit IRTE format.

Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-05 12:41:46 +02:00
Suravee Suthikulpanit 3928aa3f57 iommu/amd: Detect and enable guest vAPIC support
This patch introduces a new IOMMU driver parameter, amd_iommu_guest_ir,
which can be used to specify different interrupt remapping mode for
passthrough devices to VM guest:
    * legacy: Legacy interrupt remapping (w/ 32-bit IRTE)
    * vapic : Guest vAPIC interrupt remapping (w/ GA mode 128-bit IRTE)

Note that in vapic mode, it can also supports legacy interrupt remapping
for non-passthrough devices with the 128-bit IRTE.

Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-09-05 12:41:46 +02:00
Joerg Roedel 4df36185bb Merge branch 'for-joerg/arm-smmu/fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into iommu/fixes 2016-08-22 12:33:56 +02:00
Will Deacon 5bc0a11664 iommu/arm-smmu: Don't BUG() if we find aborting STEs with disable_bypass
The disable_bypass cmdline option changes the SMMUv3 driver to put down
faulting stream table entries by default, as opposed to bypassing
transactions from unconfigured devices.

In this mode of operation, it is entirely expected to see aborting
entries in the stream table if and when we come to installing a valid
translation, so don't trigger a BUG() as a result of misdiagnosing these
entries as stream table corruption.

Cc: <stable@vger.kernel.org>
Fixes: 48ec83bcbc ("iommu/arm-smmu: Add initial driver support for ARM SMMUv3 devices")
Tested-by: Robin Murphy <robin.murphy@arm.com>
Reported-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-08-19 09:48:15 +01:00
Will Deacon 3714ce1d66 iommu/arm-smmu: Disable stalling faults for all endpoints
Enabling stalling faults can result in hardware deadlock on poorly
designed systems, particularly those with a PCI root complex upstream of
the SMMU.

Although it's not really Linux's job to save hardware integrators from
their own misfortune, it *is* our job to stop userspace (e.g. VFIO
clients) from hosing the system for everybody else, even if they might
already be required to have elevated privileges.

Given that the fault handling code currently executes entirely in IRQ
context, there is nothing that can sensibly be done to recover from
things like page faults anyway, so let's rip this code out for now and
avoid the potential for deadlock.

Cc: <stable@vger.kernel.org>
Fixes: 48ec83bcbc ("iommu/arm-smmu: Add initial driver support for ARM SMMUv3 devices")
Reported-by: Matt Evans <matt.evans@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-08-19 09:48:01 +01:00
Will Deacon aea2037e0d iommu/arm-smmu: Fix CMDQ error handling
In the unlikely event of a global command queue error, the ARM SMMUv3
driver attempts to convert the problematic command into a CMD_SYNC and
resume the command queue. Unfortunately, this code is pretty badly
broken:

  1. It uses the index into the error string table as the CMDQ index,
     so we probably read the wrong entry out of the queue

  2. The arguments to queue_write are the wrong way round, so we end up
     writing from the queue onto the stack.

These happily cancel out, so the kernel is likely to stay alive, but
the command queue will probably fault again when we resume.

This patch fixes the error handling code to use the correct queue index
and write back the CMD_SYNC to the faulting entry.

Cc: <stable@vger.kernel.org>
Fixes: 48ec83bcbc ("iommu/arm-smmu: Add initial driver support for ARM SMMUv3 devices")
Reported-by: Diwakar Subraveti <Diwakar.Subraveti@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-08-19 09:43:25 +01:00
Robin Murphy e633fc7a13 iommu/io-pgtable-arm-v7s: Fix attributes when splitting blocks
Due to the attribute bits being all over the place in the different
types of short-descriptor PTEs, when remapping an existing entry, e.g.
splitting a section into pages, we take the approach of decomposing
the PTE attributes back to the IOMMU API flags to start from scratch.

On inspection, though, the existing code seems to have got the read-only
bit backwards and ignored the XN bit. How embarrassing...

Fortunately the primary user so far, the Mediatek IOMMU, both never
splits blocks (because it only serves non-overlapping DMA API calls) and
also ignores permissions anyway, but let's put things right before any
future users trip up.

Cc: <stable@vger.kernel.org>
Fixes: e5fc9753b1 ("iommu/io-pgtable: Add ARMv7 short descriptor support")
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-08-19 09:40:16 +01:00
Robin Murphy c987ff0d3c iommu/dma: Respect IOMMU aperture when allocating
Where a device driver has set a 64-bit DMA mask to indicate the absence
of addressing limitations, we still need to ensure that we don't
allocate IOVAs beyond the actual input size of the IOMMU. The reported
aperture is the most reliable way we have of inferring that input
address size, so use that to enforce a hard upper limit where available.

Fixes: 0db2e5d18f ("iommu: Implement common IOMMU ops for DMA mapping")
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-08-10 12:02:02 +02:00
Robin Murphy 3ec60043f7 iommu/dma: Don't put uninitialised IOVA domains
Due to the limitations of having to wait until we see a device's DMA
restrictions before we know how we want an IOVA domain initialised,
there is a window for error if a DMA ops domain is allocated but later
freed without ever being used. In that case, init_iova_domain() was
never called, so calling put_iova_domain() from iommu_put_dma_cookie()
ends up trying to take an uninitialised lock and crashing.

Make things robust by skipping the call unless the IOVA domain actually
has been initialised, as we probably should have done from the start.

Fixes: 0db2e5d18f ("iommu: Implement common IOMMU ops for DMA mapping")
Cc: stable@vger.kernel.org
Reported-by: Nate Watterson <nwatters@codeaurora.org>
Reviewed-by: Nate Watterson <nwatters@codeaurora.org>
Tested-by: Nate Watterson <nwatters@codeaurora.org>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Tested-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-08-09 17:31:39 +02:00
Amitoj Kaur Chawla 423595e89d iommu/exynos: Fix error handling for of_platform_device_create
of_platform_device_create returns NULL on error so an IS_ERR test is
incorrect here and a NULL check is required.

The Coccinelle semantic patch used to make this change is as follows:
@@
expression e;
@@

  e = of_platform_device_create(...);
if(
-    IS_ERR(e)
+    !e
    )
    {
  <+...
  return
- PTR_ERR(e)
+ -ENODEV
  ;
  ...+>
  }

Signed-off-by: Amitoj Kaur Chawla <amitoj1606@gmail.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-08-09 17:27:48 +02:00
Wei Yongjun 517abe495d iommu/amd: Fix error return code in irq_remapping_alloc()
Fix to return a negative error code from the alloc_irq_index() error
handling case instead of 0, as done elsewhere in this function.

Signed-off-by: Wei Yongjun <weiyj.lk@gmail.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-08-09 17:18:45 +02:00
Wei Yongjun a5604f260e iommu/amd: Fix non static symbol warning
Fixes the following sparse warning:

drivers/iommu/amd_iommu.c:106:1: warning:
 symbol '__pcpu_scope_flush_queue' was not declared. Should it be static?

Signed-off-by: Wei Yongjun <weiyj.lk@gmail.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-08-09 17:18:45 +02:00
Joerg Roedel 9a8a5dcf20 iommu/mediatek: Mark static functions in headers inline
This was an oversight while merging these functions. Fix it.

Cc: Honghui Zhang <honghui.zhang@mediatek.com>
Fixes: 9ca340c98c ('iommu/mediatek: move the common struct into header file')
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-08-09 15:46:46 +02:00
Krzysztof Kozlowski 00085f1efa dma-mapping: use unsigned long for dma_attrs
The dma-mapping core and the implementations do not change the DMA
attributes passed by pointer.  Thus the pointer can point to const data.
However the attributes do not have to be a bitfield.  Instead unsigned
long will do fine:

1. This is just simpler.  Both in terms of reading the code and setting
   attributes.  Instead of initializing local attributes on the stack
   and passing pointer to it to dma_set_attr(), just set the bits.

2. It brings safeness and checking for const correctness because the
   attributes are passed by value.

Semantic patches for this change (at least most of them):

    virtual patch
    virtual context

    @r@
    identifier f, attrs;

    @@
    f(...,
    - struct dma_attrs *attrs
    + unsigned long attrs
    , ...)
    {
    ...
    }

    @@
    identifier r.f;
    @@
    f(...,
    - NULL
    + 0
     )

and

    // Options: --all-includes
    virtual patch
    virtual context

    @r@
    identifier f, attrs;
    type t;

    @@
    t f(..., struct dma_attrs *attrs);

    @@
    identifier r.f;
    @@
    f(...,
    - NULL
    + 0
     )

Link: http://lkml.kernel.org/r/1468399300-5399-2-git-send-email-k.kozlowski@samsung.com
Signed-off-by: Krzysztof Kozlowski <k.kozlowski@samsung.com>
Acked-by: Vineet Gupta <vgupta@synopsys.com>
Acked-by: Robin Murphy <robin.murphy@arm.com>
Acked-by: Hans-Christian Noren Egtvedt <egtvedt@samfundet.no>
Acked-by: Mark Salter <msalter@redhat.com> [c6x]
Acked-by: Jesper Nilsson <jesper.nilsson@axis.com> [cris]
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch> [drm]
Reviewed-by: Bart Van Assche <bart.vanassche@sandisk.com>
Acked-by: Joerg Roedel <jroedel@suse.de> [iommu]
Acked-by: Fabien Dessenne <fabien.dessenne@st.com> [bdisp]
Reviewed-by: Marek Szyprowski <m.szyprowski@samsung.com> [vb2-core]
Acked-by: David Vrabel <david.vrabel@citrix.com> [xen]
Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> [xen swiotlb]
Acked-by: Joerg Roedel <jroedel@suse.de> [iommu]
Acked-by: Richard Kuo <rkuo@codeaurora.org> [hexagon]
Acked-by: Geert Uytterhoeven <geert@linux-m68k.org> [m68k]
Acked-by: Gerald Schaefer <gerald.schaefer@de.ibm.com> [s390]
Acked-by: Bjorn Andersson <bjorn.andersson@linaro.org>
Acked-by: Hans-Christian Noren Egtvedt <egtvedt@samfundet.no> [avr32]
Acked-by: Vineet Gupta <vgupta@synopsys.com> [arc]
Acked-by: Robin Murphy <robin.murphy@arm.com> [arm64 and dma-iommu]
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-08-04 08:50:07 -04:00
Linus Torvalds dd9671172a IOMMU Updates for Linux v4.8
In the updates:
 
 	* Big endian support and preparation for defered probing for the
 	  Exynos IOMMU driver
 
 	* Simplifications in iommu-group id handling
 
 	* Support for Mediatek generation one IOMMU hardware
 
 	* Conversion of the AMD IOMMU driver to use the generic IOVA
 	  allocator. This driver now also benefits from the recent
 	  scalability improvements in the IOVA code.
 
 	* Preparations to use generic DMA mapping code in the Rockchip
 	  IOMMU driver
 
 	* Device tree adaption and conversion to use generic page-table
 	  code for the MSM IOMMU driver
 
 	* An iova_to_phys optimization in the ARM-SMMU driver to greatly
 	  improve page-table teardown performance with VFIO
 
 	* Various other small fixes and conversions
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2
 
 iQIcBAABAgAGBQJXl3e+AAoJECvwRC2XARrjMIgP/1Mm9qIfcaAxKY4ByqbVfrH8
 313PO6rpwUhhywUmnf/1F/x+JbuLv8MmRXfSc106mdB1rq9NXpkORYKrqVxs0cSq
 6u6TzZWbF6WN1ipqXxDITNFBSy7u97K1VuFaKyYFfLbg8xrkcdkMZJ7BqM2xIEdk
 rnRKcfHo6wsmCXJ6InsUPmKAqU6AfMewZTGjO+v77Gce0rZEbsJ8n7BRKC9vO2bc
 akvN2W+zzEUSyhbuyYQBG+agpmC5GJvz4u+6QvAP5sxTWfAsnwAoPpP4xxR+/KjT
 eicHlja4v0YK6Hr4AJaMxoKfKIrCdqpWm0D2tg/edyWZCeg98AW/w7/s0I8OD3ao
 Otj6IqC8nPk0pYciOeEPQ7aqPbvKAqU2FYWt7lWamrdr98u2R3p2nXGl0KthoAj6
 JqzrCZXvBS7sj1IPLlGpj939yvbKbjpE0p7y1qhI1VEBXoBWFNvlKydkYx76BTGK
 F6paGVqn2Zwy00AqAsylTEkvIK063zwShZ6nPqz4bMdVlgzjrjCzdDecjfbHr8Ic
 6D2oCwyF+RJ8qw+Ecm9EmWFik80sgb+iUTeeYEXNf+YzLYt5McIj7fi3N+sUPel3
 YJ4S4x0sIpgUZZ1i+rOo8ZPAFHRU6SRPYV+ewaeYKrMt+Un5dTn9SddpqrJdbiUu
 YrF36BaQjc123IRGKrSd
 =xiS2
 -----END PGP SIGNATURE-----

Merge tag 'iommu-updates-v4.8' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu

Pull IOMMU updates from Joerg Roedel:

 - big-endian support and preparation for defered probing for the Exynos
   IOMMU driver

 - simplifications in iommu-group id handling

 - support for Mediatek generation one IOMMU hardware

 - conversion of the AMD IOMMU driver to use the generic IOVA allocator.
   This driver now also benefits from the recent scalability
   improvements in the IOVA code.

 - preparations to use generic DMA mapping code in the Rockchip IOMMU
   driver

 - device tree adaption and conversion to use generic page-table code
   for the MSM IOMMU driver

 - an iova_to_phys optimization in the ARM-SMMU driver to greatly
   improve page-table teardown performance with VFIO

 - various other small fixes and conversions

* tag 'iommu-updates-v4.8' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (59 commits)
  iommu/amd: Initialize dma-ops domains with 3-level page-table
  iommu/amd: Update Alias-DTE in update_device_table()
  iommu/vt-d: Return error code in domain_context_mapping_one()
  iommu/amd: Use container_of to get dma_ops_domain
  iommu/amd: Flush iova queue before releasing dma_ops_domain
  iommu/amd: Handle IOMMU_DOMAIN_DMA in ops->domain_free call-back
  iommu/amd: Use dev_data->domain in get_domain()
  iommu/amd: Optimize map_sg and unmap_sg
  iommu/amd: Introduce dir2prot() helper
  iommu/amd: Implement timeout to flush unmap queues
  iommu/amd: Implement flush queue
  iommu/amd: Allow NULL pointer parameter for domain_flush_complete()
  iommu/amd: Set up data structures for flush queue
  iommu/amd: Remove align-parameter from __map_single()
  iommu/amd: Remove other remains of old address allocator
  iommu/amd: Make use of the generic IOVA allocator
  iommu/amd: Remove special mapping code for dma_ops path
  iommu/amd: Pass gfp-flags to iommu_map_page()
  iommu/amd: Implement apply_dm_region call-back
  iommu/amd: Create a list of reserved iova addresses
  ...
2016-08-01 07:25:10 -04:00
Linus Torvalds f64d6e2aaa DeviceTree update for 4.8:
- Removal of most of_platform_populate() calls in arch code. Now the DT
 core code calls it in the default case and platforms only need to call
 it if they have special needs.
 
 - Use pr_fmt on all the DT core print statements.
 
 - CoreSight binding doc improvements to block name descriptions.
 
 - Add dt_to_config script which can parse dts files and list
 corresponding kernel config options.
 
 - Fix memory leak hit with a PowerMac DT.
 
 - Correct a bunch of STMicro compatible strings to use the correct
 vendor prefix.
 
 - Fix DA9052 PMIC binding doc to match what is actually used in dts
 files.
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQIcBAABAgAGBQJXm9KcAAoJEPr7XbWNvGHDRT4QAIIIOSB4AWHardnMLROgGge9
 aOQKZ/05O9feOcxYKe8FkQbcH+IujJjrUL+yrRD36yGQPAyBP21gtcmmfrkCcwFM
 kH915f/JbGvXpfwEf8dcarHhzYH6FFJiQGduPpWfwSSWynx+xq5EKPwCqYzMg8bN
 SExxt7vUx1MKFOExZ0K8BNCo8VMVLUWQoJ1DNeJDuL25Op4EU3i2l1HQNYV/3XDk
 BSA3x7Lw3GjrWEH20VWYn2Azq1OFLY+E2FC2lnG4nbkk5X8dZbUH9PR1Sk7uTQDj
 uxTjWe59NBpliCxKSAbMbTAU/WwSB1pJ0I+zDJBiQsdFT+nb5F4zOrs3qSKHa/A9
 Rv6AC8k5gdSMrDB1dOspfF2vWvOOInXgNV4/Kza0D92mbCpwyUuF+vhE6rfcMrZU
 OiD7rj2/fvO7Y9fUAhrp6zrfrOfH9B1Z9vS+940AlK96YwPE2+J0SA2vBxR/wg8H
 7fj4Ud5X+SFisXWQhh5Wlv0W9o6e7C7fsi8vpkQ7gufmezLFWVnJKsUfQaxGEwhG
 Hkhm9kuSHHMd+6dEnn2756DnNfJAtQv6rSR0/QR4Lf9y5L4dvR3kAQIci8X/nx4P
 sIk+IJWGZG6wziZq59hh+SO6HEqdSNuvh+5sbR0iUimdE/1HsDBdPiocXf/r8iwK
 NY9nGeZPRrXmFgdpoZfm
 =wLMr
 -----END PGP SIGNATURE-----

Merge tag 'devicetree-for-4.8' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux

Pull DeviceTree updates from Rob Herring:

 - remove most of_platform_populate() calls in arch code.  Now the DT
   core code calls it in the default case and platforms only need to
   call it if they have special needs

 - use pr_fmt on all the DT core print statements

 - CoreSight binding doc improvements to block name descriptions

 - add dt_to_config script which can parse dts files and list
   corresponding kernel config options

 - fix memory leak hit with a PowerMac DT

 - correct a bunch of STMicro compatible strings to use the correct
   vendor prefix

 - fix DA9052 PMIC binding doc to match what is actually used in dts
   files

* tag 'devicetree-for-4.8' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux: (35 commits)
  documentation: da9052: Update regulator bindings names to match DA9052/53 DTS expectations
  xtensa: Partially Revert "xtensa: Remove unnecessary of_platform_populate with default match table"
  xtensa: Fix build error due to missing include file
  MIPS: ath79: Add missing include file
  Fix spelling errors in Documentation/devicetree
  ARM: dts: fix STMicroelectronics compatible strings
  powerpc/dts: fix STMicroelectronics compatible strings
  Documentation: dt: i2c: use correct STMicroelectronics vendor prefix
  scripts/dtc: dt_to_config - kernel config options for a devicetree
  of: fdt: mark unflattened tree as detached
  of: overlay: add resolver error prints
  coresight: document binding acronyms
  Documentation/devicetree: document cavium-pip rx-delay/tx-delay properties
  of: use pr_fmt prefix for all console printing
  of/irq: Mark initialised interrupt controllers as populated
  of: fix memory leak related to safe_name()
  Revert "of/platform: export of_default_bus_match_table"
  of: unittest: use of_platform_default_populate() to populate default bus
  memory: omap-gpmc: use of_platform_default_populate() to populate default bus
  bus: uniphier-system-bus: use of_platform_default_populate() to populate default bus
  ...
2016-07-30 11:32:01 -07:00
Linus Torvalds 194dc870a5 Add braces to avoid "ambiguous ‘else’" compiler warnings
Some of our "for_each_xyz()" macro constructs make gcc unhappy about
lack of braces around if-statements inside or outside the loop, because
the loop construct itself has a "if-then-else" statement inside of it.

The resulting warnings look something like this:

  drivers/gpu/drm/i915/i915_debugfs.c: In function ‘i915_dump_lrc’:
  drivers/gpu/drm/i915/i915_debugfs.c:2103:6: warning: suggest explicit braces to avoid ambiguous ‘else’ [-Wparentheses]
     if (ctx != dev_priv->kernel_context)
        ^

even if the code itself is fine.

Since the warning is fairly easy to avoid by adding a braces around the
if-statement near the for_each_xyz() construct, do so, rather than
disabling the otherwise potentially useful warning.

(The if-then-else statements used in the "for_each_xyz()" constructs are
designed to be inherently safe even with no braces, but in this case
it's quite understandable that gcc isn't really able to tell that).

This finally leaves the standard "allmodconfig" build with just a
handful of remaining warnings, so new and valid warnings hopefully will
stand out.

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-07-27 20:03:31 -07:00
Kirill A. Shutemov dcddffd41d mm: do not pass mm_struct into handle_mm_fault
We always have vma->vm_mm around.

Link: http://lkml.kernel.org/r/1466021202-61880-8-git-send-email-kirill.shutemov@linux.intel.com
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-07-26 16:19:19 -07:00
Joerg Roedel f360d3241f Merge branches 'x86/amd', 'x86/vt-d', 'arm/exynos', 'arm/mediatek', 'arm/msm', 'arm/rockchip', 'arm/smmu' and 'core' into next 2016-07-26 16:02:37 +02:00
Joerg Roedel ffec219770 iommu/amd: Initialize dma-ops domains with 3-level page-table
A two-level page-table can map up to 1GB of address space.
With the IOVA allocator now in use, the allocated addresses
are often more closely to 4G, which requires the address
space to be increased much more often. Avoid that by using a
three-level page-table by default.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-26 15:56:39 +02:00
Joerg Roedel 3254de6bf7 iommu/amd: Update Alias-DTE in update_device_table()
Not doing so might cause IO-Page-Faults when a device uses
an alias request-id and the alias-dte is left in a lower
page-mode which does not cover the address allocated from
the iova-allocator.

Fixes: 492667dacc ('x86/amd-iommu: Remove amd_iommu_pd_table')
Cc: stable@vger.kernel.org
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-26 15:55:57 +02:00
Wei Yang 5c365d18a7 iommu/vt-d: Return error code in domain_context_mapping_one()
In 'commit <55d940430ab9> ("iommu/vt-d: Get rid of domain->iommu_lock")',
the error handling path is changed a little, which makes the function
always return 0.

This path fixes this.

Signed-off-by: Wei Yang <richard.weiyang@gmail.com>
Fixes: 55d940430a ('iommu/vt-d: Get rid of domain->iommu_lock')
Cc: stable@vger.kernel.org # v4.3+
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-14 10:26:30 +02:00
Joerg Roedel b3311b061d iommu/amd: Use container_of to get dma_ops_domain
This is better than storing an extra pointer in struct
protection_domain, because this pointer can now be removed
from the struct.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-14 10:21:57 +02:00
Joerg Roedel 281e8ccbff iommu/amd: Flush iova queue before releasing dma_ops_domain
Before a dma_ops_domain can be freed, we need to make sure
it is not longer referenced by the flush queue. So empty the
queue before a dma_ops_domain can be freed.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-14 10:21:16 +02:00
Joerg Roedel cda7005ba2 iommu/amd: Handle IOMMU_DOMAIN_DMA in ops->domain_free call-back
This domain type is not yet handled in the
iommu_ops->domain_free() call-back. Fix that.

Fixes: 0bb6e243d7 ('iommu/amd: Support IOMMU_DOMAIN_DMA type allocation')
Cc: stable@vger.kernel.org # v4.2+
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-14 10:21:16 +02:00
Joerg Roedel d26592a93d iommu/amd: Use dev_data->domain in get_domain()
Using the cached value is much more efficient than calling
into the IOMMU core code.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-14 10:21:15 +02:00
Joerg Roedel 80187fd39d iommu/amd: Optimize map_sg and unmap_sg
Optimize these functions so that they need only one call
into the address alloctor. This also saves a couple of
io-tlb flushes in the unmap_sg path.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-14 10:21:07 +02:00
Joerg Roedel f37f7f33d5 iommu/amd: Introduce dir2prot() helper
This function converts dma_data_direction to
iommu-protection flags. This will be needed on multiple
places in the code, so this will save some code.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-13 12:48:35 +02:00
Joerg Roedel bb279475db iommu/amd: Implement timeout to flush unmap queues
In case the queue doesn't fill up, we flush the TLB at least
10ms after the unmap happened to make sure that the TLB is
cleaned up.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-13 12:48:35 +02:00
Joerg Roedel b1516a1465 iommu/amd: Implement flush queue
With the flush queue the IOMMU TLBs will not be flushed at
every dma-ops unmap operation. The unmapped ranges will be
queued and flushed at once, when the queue is full. This
makes unmapping operations a lot faster (on average) and
restores the performance of the old address allocator.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-13 12:48:35 +02:00
Joerg Roedel f1eae7c580 iommu/amd: Allow NULL pointer parameter for domain_flush_complete()
If domain == NULL is passed to the function, it will queue a
completion-wait command on all IOMMUs in the system.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-13 12:48:35 +02:00
Joerg Roedel c5b5da9c79 iommu/amd: Set up data structures for flush queue
The flush queue is the equivalent to defered-flushing in the
Intel VT-d driver. This patch sets up the data structures
needed for this.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-13 12:48:35 +02:00
Joerg Roedel bda350dbdb iommu/amd: Remove align-parameter from __map_single()
This parameter is not required anymore because the
iova-allocations are always aligned to its size.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-13 12:48:06 +02:00
Joerg Roedel 2d4c515bf0 iommu/amd: Remove other remains of old address allocator
There are other remains in the code from the old allocatore.
Remove them all.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-13 12:48:06 +02:00
Joerg Roedel 256e4621c2 iommu/amd: Make use of the generic IOVA allocator
Remove the old address allocation code and make use of the
generic IOVA allocator that is also used by other dma-ops
implementations.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-13 12:46:05 +02:00
Joerg Roedel 518d9b4503 iommu/amd: Remove special mapping code for dma_ops path
Use the iommu-api map/unmap functions instead. This will be
required anyway when IOVA code is used for address
allocation.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-13 12:46:05 +02:00
Joerg Roedel b911b89b6d iommu/amd: Pass gfp-flags to iommu_map_page()
Make this function ready to be used in the DMA-API path.
Reorder parameters a bit while at it.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-13 12:46:05 +02:00
Joerg Roedel 8d54d6c8b8 iommu/amd: Implement apply_dm_region call-back
It is used to reserve the dm-regions in the iova-tree.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-13 12:46:05 +02:00
Joerg Roedel 81cd07b9c9 iommu/amd: Create a list of reserved iova addresses
Put the MSI-range, the HT-range and the MMIO ranges of PCI
devices into that range, so that these addresses are not
allocated for DMA.

Copy this address list into every created dma_ops_domain.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-13 12:46:05 +02:00
Joerg Roedel 307d5851ef iommu/amd: Allocate iova_domain for dma_ops_domain
Use it later for allocating the IO virtual addresses.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-13 12:46:05 +02:00
Joerg Roedel a72c422544 iommu/amd: Select IOMMU_IOVA for AMD IOMMU
Include the generic IOVA code to make use of it in the AMD
IOMMU driver too.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-13 12:46:05 +02:00
Joerg Roedel 33b21a6b20 iommu: Add apply_dm_region call-back to iommu-ops
This new call-back will be used by the iommu driver to do
reserve the given dm_region in its iova space before the
mapping is created.

The call-back is temporary until the dma-ops implementation
is part of the common iommu code.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-13 12:46:05 +02:00
Joerg Roedel b548e786ce iommu/amd: Init unity mappings only for dma_ops domains
The default domain for a device might also be
identity-mapped. In this case the kernel would crash when
unity mappings are defined for the device. Fix that by
making sure the domain is a dma_ops domain.

Fixes: 0bb6e243d7 ('iommu/amd: Support IOMMU_DOMAIN_DMA type allocation')
Cc: stable@vger.kernel.org # v4.2+
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-13 12:35:24 +02:00
Heiner Kallweit feccf398db iommu: Simplify and fix ida handling
Ida handling can be much simplified by using the ida_simple_.. functions.

This change also fixes the bug that previously checking for errors
returned by ida_get_new() was incomplete.
ida_get_new() can return errors other than EAGAIN, e.g. ENOSPC.
This case wasn't handled.

Signed-off-by: Heiner Kallweit <hkallweit1@gmail.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-13 12:28:55 +02:00
Heiner Kallweit e38d1f1312 iommu: Simplify init function
iommu_group_ida and iommu_group_mutex can be initialized statically.
There's no need to do this dynamically in the init function.

Signed-off-by: Heiner Kallweit <hkallweit1@gmail.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-13 12:28:55 +02:00
Nadav Amit 452014d2b4 iommu/vt-d: Remove unnecassary qi clflushes
According to the manual: "Hardware access to ...  invalidation queue ...
are always coherent."

Remove unnecassary clflushes accordingly.

Signed-off-by: Nadav Amit <namit@vmware.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-13 12:06:35 +02:00
Joerg Roedel c3928e7515 Merge branch 'for-joerg/arm-smmu/updates' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into arm/smmu 2016-07-12 13:26:59 +02:00
Peng Fan bee1400445 iommu/arm-smmu: Use devm_request_irq and devm_free_irq
Use devm_request_irq to simplify error handling path,
when probe smmu device.

Also devm_{request|free}_irq when init or destroy domain context.

Signed-off-by: Peng Fan <van.freenix@gmail.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-07-06 18:31:51 +01:00
Joerg Roedel 522e5cb76d iommu/amd: Fix unity mapping initialization race
There is a race condition in the AMD IOMMU init code that
causes requested unity mappings to be blocked by the IOMMU
for a short period of time. This results on boot failures
and IO_PAGE_FAULTs on some machines.

Fix this by making sure the unity mappings are installed
before all other DMA is blocked.

Fixes: aafd8ba0ca ('iommu/amd: Implement add_device and remove_device')
Cc: stable@vger.kernel.org # v4.2+
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-06 18:04:55 +02:00
Aaron Campbell 0caa7616a6 iommu/vt-d: Fix infinite loop in free_all_cpu_cached_iovas
Per VT-d spec Section 10.4.2 ("Capability Register"), the maximum
number of possible domains is 64K; indeed this is the maximum value
that the cap_ndoms() macro will expand to.  Since the value 65536
will not fix in a u16, the 'did' variable must be promoted to an
int, otherwise the test for < 65536 will always be true and the
loop will never end.

The symptom, in my case, was a hung machine during suspend.

Fixes: 3bd4f9112f ("iommu/vt-d: Fix overflow of iommu->domains array")
Signed-off-by: Aaron Campbell <aaron@monkey.org>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-07-04 13:34:52 +02:00
Will Deacon 7c6d90e2bb iommu/io-pgtable-arm: Fix iova_to_phys for block entries
The implementation of iova_to_phys for the long-descriptor ARM
io-pgtable code always masks with the granule size when inserting the
low virtual address bits into the physical address determined from the
page tables. In cases where the leaf entry is found before the final
level of table (i.e. due to a block mapping), this results in rounding
down to the bottom page of the block mapping. Consequently, the physical
address range batching in the vfio_unmap_unpin is defeated and we end
up taking the long way home.

This patch fixes the problem by masking the virtual address with the
appropriate mask for the level at which the leaf descriptor is located.
The short-descriptor code already gets this right, so no change is
needed there.

Cc: <stable@vger.kernel.org>
Reported-by: Robin Murphy <robin.murphy@arm.com>
Tested-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-07-01 14:04:37 +01:00
Wei Chen 112c898b59 iommu/arm-smmu: request pcie devices to enable ACS
The PCIe ACS capability will affect the layout of iommu groups.
Generally speaking, if the path from root port to the PCIe device
is ACS enabled, the iommu will create a single iommu group for this
PCIe device. If all PCIe devices on the path are ACS enabled then
Linux can determine this path is ACS enabled.

Linux use two PCIe configuration registers to determine the ACS
status of PCIe devices:
ACS Capability Register and ACS Control Register.

The first register is used to check the implementation of ACS function
of a PCIe device, the second register is used to check the enable status
of ACS function. If one PCIe device has implemented and enabled the ACS
function then Linux will determine this PCIe device enabled ACS.

From the Chapter:6.12 of PCI Express Base Specification Revision 3.1a,
we can find that when a PCIe device implements ACS function, the enable
status is set to disabled by default and can be enabled by ACS-aware
software.

ACS will affect the iommu groups topology, so, the iommu driver is
ACS-aware software. This patch adds a call to pci_request_acs() to the
arm-smmu driver to enable the ACS function in PCIe devices that support
it, when they get probed.

Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Wei Chen <Wei.Chen@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-07-01 14:04:36 +01:00
Shunqian Zheng a93db2f22b iommu/rockchip: Prepare to support generic DMA mapping
Set geometry for allocated domains and fix .domain_alloc() callback to
work with IOMMU_DOMAIN_DMA domain type, which is used for implicit
domains on ARM64.

Signed-off-by: Shunqian Zheng <zhengsq@rock-chips.com>
Signed-off-by: Tomasz Figa <tfiga@chromium.org>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-27 14:50:08 +02:00
Shunqian Zheng 4f0aba6767 iommu/rockchip: Use DMA API to manage coherency
Use DMA API instead of architecture internal functions like
__cpuc_flush_dcache_area() etc.

The biggest difficulty here is that dma_map and _sync calls require some
struct device, while there is no real 1:1 relation between an IOMMU
domain and some device. To overcome this, a simple platform device is
registered for each allocated IOMMU domain.

With this patch, this driver can be used on both ARM and ARM64
platforms, such as RK3288 and RK3399 respectively.

Signed-off-by: Shunqian Zheng <zhengsq@rock-chips.com>
Signed-off-by: Tomasz Figa <tfiga@chromium.org>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-27 14:50:08 +02:00
Shunqian Zheng 3d08f434bd iommu/rockchip: Fix allocation of bases array in driver probe
In .probe(), devm_kzalloc() is called with size == 0 and works only
by luck, due to internal behavior of the allocator and the fact
that the proper allocation size is small. Let's use proper value for
calculating the size.

Fixes: cd6438c5f8 ("iommu/rockchip: Reconstruct to support multi slaves")

Signed-off-by: Shunqian Zheng <zhengsq@rock-chips.com>
Signed-off-by: Tomasz Figa <tfiga@chromium.org>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-27 14:50:08 +02:00
Simon Xue e6d0f4737c iommu/rockchip: Add map_sg callback for rk_iommu_ops
The iommu_dma_alloc() in iommu/dma-iommu.c calls iommu_map_sg()
that requires the callback iommu_ops .map_sg(). Adding the
default_iommu_map_sg() to Rockchip IOMMU accordingly.

Signed-off-by: Simon Xue <xxm@rock-chips.com>
Signed-off-by: Shunqian Zheng <xxm@rock-chips.com>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Signed-off-by: Tomasz Figa <tfiga@chromium.org>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-27 14:50:08 +02:00
Simon Xue fec3b217ff iommu/rockchip: Fix devm_{request,free}_irq parameter
Even though the IOMMU shares IRQ with its master, the struct device
passed to {request,free}_irq is supposed to represent the device that is
signalling the interrupt. This patch makes the driver use IOMMU device
instead of master's device to make things clear.

Signed-off-by: Simon Xue <xxm@rock-chips.com>
Signed-off-by: Shunqian Zheng <zhengsq@rock-chips.com>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Signed-off-by: Tomasz Figa <tfiga@chromium.org>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-27 14:50:08 +02:00
Nicolas Iooss 6082ee72e9 iommu/amd: Initialize devid variable before using it
Commit 2a0cb4e2d4 ("iommu/amd: Add new map for storing IVHD dev entry
type HID") added a call to DUMP_printk in init_iommu_from_acpi() which
used the value of devid before this variable was initialized.

Fixes: 2a0cb4e2d4 ('iommu/amd: Add new map for storing IVHD dev entry type HID')
Signed-off-by: Nicolas Iooss <nicolas.iooss_linux@m4x.org>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-27 13:24:46 +02:00
Jan Niehusmann 3bd4f9112f iommu/vt-d: Fix overflow of iommu->domains array
The valid range of 'did' in get_iommu_domain(*iommu, did)
is 0..cap_ndoms(iommu->cap), so don't exceed that
range in free_all_cpu_cached_iovas().

The user-visible impact of the out-of-bounds access is the machine
hanging on suspend-to-ram. It is, in fact, a kernel panic, but due
to already suspended devices, that's often not visible to the user.

Fixes: 22e2f9fa63 ("iommu/vt-d: Use per-cpu IOVA caching")
Signed-off-by: Jan Niehusmann <jan@gondor.com>
Tested-By: Marius Vlad <marius.c.vlad@intel.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-27 13:21:37 +02:00
Chris Wilson 583248e662 iommu/iova: Disable preemption around use of this_cpu_ptr()
Between acquiring the this_cpu_ptr() and using it, ideally we don't want
to be preempted and work on another CPU's private data. this_cpu_ptr()
checks whether or not preemption is disable, and get_cpu_ptr() provides
a convenient wrapper for operating on the cpu ptr inside a preemption
disabled critical section (which currently is provided by the
spinlock).

[  167.997877] BUG: using smp_processor_id() in preemptible [00000000] code: usb-storage/216
[  167.997940] caller is debug_smp_processor_id+0x17/0x20
[  167.997945] CPU: 7 PID: 216 Comm: usb-storage Tainted: G     U          4.7.0-rc1-gfxbench-RO_Patchwork_1057+ #1
[  167.997948] Hardware name: Hewlett-Packard HP Pro 3500 Series/2ABF, BIOS 8.11 10/24/2012
[  167.997951]  0000000000000000 ffff880118b7f9c8 ffffffff8140dca5 0000000000000007
[  167.997958]  ffffffff81a3a7e9 ffff880118b7f9f8 ffffffff8142a927 0000000000000000
[  167.997965]  ffff8800d499ed58 0000000000000001 00000000000fffff ffff880118b7fa08
[  167.997971] Call Trace:
[  167.997977]  [<ffffffff8140dca5>] dump_stack+0x67/0x92
[  167.997981]  [<ffffffff8142a927>] check_preemption_disabled+0xd7/0xe0
[  167.997985]  [<ffffffff8142a947>] debug_smp_processor_id+0x17/0x20
[  167.997990]  [<ffffffff81507e17>] alloc_iova_fast+0xb7/0x210
[  167.997994]  [<ffffffff8150c55f>] intel_alloc_iova+0x7f/0xd0
[  167.997998]  [<ffffffff8151021d>] intel_map_sg+0xbd/0x240
[  167.998002]  [<ffffffff810e5efd>] ? debug_lockdep_rcu_enabled+0x1d/0x20
[  167.998009]  [<ffffffff81596059>] usb_hcd_map_urb_for_dma+0x4b9/0x5a0
[  167.998013]  [<ffffffff81596d19>] usb_hcd_submit_urb+0xe9/0xaa0
[  167.998017]  [<ffffffff810cff2f>] ? mark_held_locks+0x6f/0xa0
[  167.998022]  [<ffffffff810d525c>] ? __raw_spin_lock_init+0x1c/0x50
[  167.998025]  [<ffffffff810e5efd>] ? debug_lockdep_rcu_enabled+0x1d/0x20
[  167.998028]  [<ffffffff815988f3>] usb_submit_urb+0x3f3/0x5a0
[  167.998032]  [<ffffffff810d0082>] ? trace_hardirqs_on_caller+0x122/0x1b0
[  167.998035]  [<ffffffff81599ae7>] usb_sg_wait+0x67/0x150
[  167.998039]  [<ffffffff815dc202>] usb_stor_bulk_transfer_sglist.part.3+0x82/0xd0
[  167.998042]  [<ffffffff815dc29c>] usb_stor_bulk_srb+0x4c/0x60
[  167.998045]  [<ffffffff815dc42e>] usb_stor_Bulk_transport+0x17e/0x420
[  167.998049]  [<ffffffff815dcf32>] usb_stor_invoke_transport+0x242/0x540
[  167.998052]  [<ffffffff810e5efd>] ? debug_lockdep_rcu_enabled+0x1d/0x20
[  167.998058]  [<ffffffff815dba19>] usb_stor_transparent_scsi_command+0x9/0x10
[  167.998061]  [<ffffffff815de518>] usb_stor_control_thread+0x158/0x260
[  167.998064]  [<ffffffff815de3c0>] ? fill_inquiry_response+0x20/0x20
[  167.998067]  [<ffffffff815de3c0>] ? fill_inquiry_response+0x20/0x20
[  167.998071]  [<ffffffff8109ddfa>] kthread+0xea/0x100
[  167.998078]  [<ffffffff817ac6af>] ret_from_fork+0x1f/0x40
[  167.998081]  [<ffffffff8109dd10>] ? kthread_create_on_node+0x1f0/0x1f0

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=96293
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: iommu@lists.linux-foundation.org
Cc: linux-kernel@vger.kernel.org
Fixes: 9257b4a206 ('iommu/iova: introduce per-cpu caching to iova allocation')
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-27 13:07:45 +02:00
Kefeng Wang bb8e15d604 of: iommu: make of_iommu_init() postcore_initcall_sync
The of_iommu_init() is called multiple times by arch code,
make it postcore_initcall_sync, then we can drop relevant
calls fully.

Note, the IOMMUs should have a chance to perform some basic
initialisation before we start adding masters to them. So
postcore_initcall_sync is good choice, it ensures of_iommu_init()
called before of_platform_populate.

Acked-by: Rich Felker <dalias@libc.org>
Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Rich Felker <dalias@libc.org>
Cc: Rob Herring <robh+dt@kernel.org>
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
Acked-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Rob Herring <robh@kernel.org>
2016-06-23 14:57:40 -05:00
Sricharan R 1cb13f7832 iommu/msm: Remove driver BROKEN
Now that the driver is DT adapted, bus_set_iommu gets called only
when on compatible matching. So the driver should not break multiplatform
builds now. So remove the BROKEN config.

Signed-off-by: Sricharan R <sricharan@codeaurora.org>
Tested-by: Archit Taneja <architt@codeaurora.org>
Tested-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-21 13:57:13 +02:00
Sricharan R c9220fbd77 iommu/msm: use generic ARMV7S short descriptor pagetable ops
This iommu uses the armv7 short descriptor format. So use the
generic ARMV7S pagetable ops instead of rewriting the same stuff
in the driver.

Signed-off-by: Sricharan R <sricharan@codeaurora.org>
Tested-by: Archit Taneja <architt@codeaurora.org>
Tested-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-21 13:57:13 +02:00
Sricharan R f78ebca8ff iommu/msm: Add support for generic master bindings
This adds the xlate callback which gets invoked during
device registration from DT. The master devices gets added
through this.

Signed-off-by: Sricharan R <sricharan@codeaurora.org>
Tested-by: Archit Taneja <architt@codeaurora.org>
Tested-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-21 13:57:13 +02:00
Sricharan R f7f125ef0b iommu/msm: Move the contents from msm_iommu_dev.c to msm_iommu.c
There are only two functions left in msm_iommu_dev.c. Move it to
msm_iommu.c and delete the file.

Signed-off-by: Sricharan R <sricharan@codeaurora.org>
Tested-by: Archit Taneja <architt@codeaurora.org>
Tested-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-21 13:57:13 +02:00
Sricharan R 109bd48ea2 iommu/msm: Add DT adaptation
The driver currently works based on platform data. Remove this
and add support for DT. A single master can have multiple ports
connected to more than one iommu.

                      master
                        |
                        |
                        |
           ------------------------
           |                      |
         IOMMU0                 IOMMU1
           |                      |
      ctx0   ctx1            ctx0   ctx1

This association of master and iommus/contexts were previously
represented by platform data parent/child device details. The client
drivers were responsible for programming all of the iommus/contexts
for the device. Now while adapting to generic DT bindings we maintain the
list of iommus, contexts that each master domain is connected to and
program all of them on attach/detach.

Signed-off-by: Sricharan R <sricharan@codeaurora.org>
Tested-by: Archit Taneja <architt@codeaurora.org>
Tested-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-21 13:56:00 +02:00
Ben Dooks 6ae5343c26 iommu/exynos: update to use iommu big-endian
Add initial support for big endian by always writing the pte
in le32. Note, revisit if hardware capable of doing big endian
fetches.

Signed-off-by: Ben Dooks <ben.dooks@codethink.co.uk>
Acked-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-21 11:59:03 +02:00
Joerg Roedel 131bc8ebb4 iommu/mediatek: Make mtk_iommu_pm_ops static
The symbol exists elsewhere already, so that is fails to
link if the symbol is non-static.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-21 11:54:06 +02:00
Honghui Zhang b17336c55d iommu/mediatek: add support for mtk iommu generation one HW
Mediatek SoC's M4U has two generations of HW architcture. Generation one
uses flat, one layer pagetable, and was shipped with ARM architecture, it
only supports 4K size page mapping. MT2701 SoC uses this generation one
m4u HW. Generation two uses the ARM short-descriptor translation table
format for address translation, and was shipped with ARM64 architecture,
MT8173 uses this generation two m4u HW. All the two generation iommu HW
only have one iommu domain, and all its iommu clients share the same
iova address.

These two generation m4u HW have slit different register groups and
register offset, but most register names are the same. This patch add iommu
support for mediatek SoC mt2701.

Signed-off-by: Honghui Zhang <honghui.zhang@mediatek.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-21 11:36:19 +02:00
Honghui Zhang 9ca340c98c iommu/mediatek: move the common struct into header file
Move the struct defines of mtk iommu into a new header files for
common use.

Signed-off-by: Honghui Zhang <honghui.zhang@mediatek.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-21 11:36:19 +02:00
Honghui Zhang 9fec79df89 iommu/mediatek: Do not call of_node_put in mtk_iommu_of_xlate
The device_node will be released in of_iommu_configure, it may be double
released if call of_node_put in mtk_iommu_of_xlate.

Signed-off-by: Honghui Zhang <honghui.zhang@mediatek.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-21 11:36:19 +02:00
Bhaktipriya Shridhar cf7513e759 iommu/amd: Remove create_workqueue
alloc_workqueue replaces deprecated create_workqueue().

A dedicated workqueue has been used since the workitem (viz
&fault->work), is involved in IO page-fault handling.
WQ_MEM_RECLAIM has been set to guarantee forward progress under memory
pressure, which is a requirement here.
Since there are only a fixed number of work items, explicit concurrency
limit is unnecessary.

Signed-off-by: Bhaktipriya Shridhar <bhaktipriya96@gmail.com>
Acked-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-21 11:13:44 +02:00
Joerg Roedel a4c34ff1c0 iommu/vt-d: Enable QI on all IOMMUs before setting root entry
This seems to be required on some X58 chipsets on systems
with more than one IOMMU. QI does not work until it is
enabled on all IOMMUs in the system.

Reported-by: Dheeraj CVR <cvr.dheeraj@gmail.com>
Tested-by: Dheeraj CVR <cvr.dheeraj@gmail.com>
Fixes: 5f0a7f7614 ('iommu/vt-d: Make root entry visible for hardware right after allocation')
Cc: stable@vger.kernel.org
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-17 11:29:48 +02:00
Roland Dreier ffb2d1eb88 iommu/vt-d: Don't reject NTB devices due to scope mismatch
On a system with an Intel PCIe port configured as an NTB device, iommu
initialization fails with

    DMAR: Device scope type does not match for 0000:80:03.0

This is because the DMAR table reports this device as having scope 2
(ACPI_DMAR_SCOPE_TYPE_BRIDGE):

    [0A0h 0160   1]      Device Scope Entry Type : 02
    [0A1h 0161   1]                 Entry Length : 08
    [0A2h 0162   2]                     Reserved : 0000
    [0A4h 0164   1]               Enumeration ID : 00
    [0A5h 0165   1]               PCI Bus Number : 80

    [0A6h 0166   2]                     PCI Path : 03,00

but the device has a type 0 PCI header:

    80:03.0 Bridge [0680]: Intel Corporation Device [8086:2f0d] (rev 02)
    00: 86 80 0d 2f 00 00 10 00 02 00 80 06 10 00 80 00
    10: 0c 00 c0 00 c0 38 00 00 0c 00 00 00 80 38 00 00
    20: 00 00 00 c8 00 00 10 c8 00 00 00 00 86 80 00 00
    30: 00 00 00 00 60 00 00 00 00 00 00 00 ff 01 00 00

VT-d works perfectly on this system, so there's no reason to bail out
on initialization due to this apparent scope mismatch.  Use the class
0x0680 ("Other bridge device") as a heuristic for allowing DMAR
initialization for non-bridge PCI devices listed with scope bridge.

Signed-off-by: Roland Dreier <roland@purestorage.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-15 15:24:29 +02:00
Marek Szyprowski 96f6655700 iommu/exynos: Prepare for deferred probe support
Register iommu_ops at the end of successful probe instead of doing that
unconditionally. This makes Exynos IOMMU driver ready for deferred probe
caused by not-yet-available clocks.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-15 13:59:58 +02:00
Marek Szyprowski fecc49db88 iommu/exynos: Prepare clocks when needed, not in driver probe
Make clock preparation together with clk_enable(). This way inactive
SYSMMU controllers will not keep clocks prepared all the time.
This change allows more fine graded power management in the future.
All the code assumes that clock management doesn't fail, so guard
clock_prepare_enable() it with BUG_ON().

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-15 13:59:58 +02:00
Marek Szyprowski 01324ab2c9 iommu/exynos: Fix master clock management for inactive SYSMMU
If SYSMMU controller is not active, there is no point in enabling master's
clock just for doing the the of internal state. This patch moves enabling
that clock to the block which actually does the register access.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-15 13:59:58 +02:00
Marek Szyprowski 0c2b063f18 iommu/exynos: Return proper errors from getting clocks
This patch reworks driver probe code to propagate error codes from
clk_get() operation. This will allow to properly handle deferred probe
in the future.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-15 13:59:58 +02:00
Wei Yang 86f004c77c iommu/vt-d: Reduce extra first level entry in iommu->domains
In commit <8bf478163e69> ("iommu/vt-d: Split up iommu->domains array"), it
it splits iommu->domains in two levels. Each first level contains 256
entries of second level. In case of the ndomains is exact a multiple of
256, it would have one more extra first level entry for current
implementation.

This patch refines this calculation to reduce the extra first level entry.

Signed-off-by: Wei Yang <richard.weiyang@gmail.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-15 13:36:58 +02:00
Marek Szyprowski b54b874fba iommu/exynos: Suppress unbinding to prevent system failure
Removal of IOMMU driver cannot be done reliably, so Exynos IOMMU driver
doesn't support this operation. It is essential for system operation, so
it makes sense to prevent unbinding by disabling bind/unbind sysfs
feature for SYSMMU controller driver to avoid kernel ops or trashing
memory caused by such operation.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
CC: stable@vger.kernel.org  # v4.2+
Reviewed-by: Krzysztof Kozlowski <k.kozlowski@samsung.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-15 13:27:55 +02:00
John Keeping ae8a7910fb iommu/rockchip: Fix zap cache during device attach
rk_iommu_command() takes a struct rk_iommu and iterates over the slave
MMUs, so this is doubly wrong in that we're passing in the wrong pointer
and talking to MMUs that we shouldn't be.

Fixes: cd6438c5f8 ("iommu/rockchip: Reconstruct to support multi slaves")
Cc: stable@vger.kernel.org
Signed-off-by: John Keeping <john@metanate.com>
Tested-by: Heiko Stuebner <heiko@sntech.de>
Reviewed-by: Heiko Stuebner <heiko@sntech.de>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-15 12:03:00 +02:00
Wan Zongshun 0076cd3d06 iommu/amd: Set AMD iommu callbacks for platform bus driver
AMD has more drivers will use ACPI to platform bus driver later,
all those devices need iommu support, for example: eMMC driver.

For latest AMD eMMC controller, it will utilize sdhci-acpi.c driver,
which will rely on platform bus to match device and driver, where we
will set 'dev' of struct platform_device as map_sg parameter passing
to iommu driver for DMA request, so the iommu-ops are needed on the
platform bus.

Signed-off-by: Wan Zongshun <Vincent.Wan@amd.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-15 12:00:10 +02:00
Jean-Philippe Brucker 9aeb26cfc2 iommu/arm-smmu: Wire up map_sg for arm-smmu-v3
The map_sg callback is missing from arm_smmu_ops, but is required by
iommu.h. Similarly to most other IOMMU drivers, connect it to
default_iommu_map_sg.

Cc: <stable@vger.kernel.org>
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-06-13 11:00:59 +02:00
Arnd Bergmann 287980e49f remove lots of IS_ERR_VALUE abuses
Most users of IS_ERR_VALUE() in the kernel are wrong, as they
pass an 'int' into a function that takes an 'unsigned long'
argument. This happens to work because the type is sign-extended
on 64-bit architectures before it gets converted into an
unsigned type.

However, anything that passes an 'unsigned short' or 'unsigned int'
argument into IS_ERR_VALUE() is guaranteed to be broken, as are
8-bit integers and types that are wider than 'unsigned long'.

Andrzej Hajda has already fixed a lot of the worst abusers that
were causing actual bugs, but it would be nice to prevent any
users that are not passing 'unsigned long' arguments.

This patch changes all users of IS_ERR_VALUE() that I could find
on 32-bit ARM randconfig builds and x86 allmodconfig. For the
moment, this doesn't change the definition of IS_ERR_VALUE()
because there are probably still architecture specific users
elsewhere.

Almost all the warnings I got are for files that are better off
using 'if (err)' or 'if (err < 0)'.
The only legitimate user I could find that we get a warning for
is the (32-bit only) freescale fman driver, so I did not remove
the IS_ERR_VALUE() there but changed the type to 'unsigned long'.
For 9pfs, I just worked around one user whose calling conventions
are so obscure that I did not dare change the behavior.

I was using this definition for testing:

 #define IS_ERR_VALUE(x) ((unsigned long*)NULL == (typeof (x)*)NULL && \
       unlikely((unsigned long long)(x) >= (unsigned long long)(typeof(x))-MAX_ERRNO))

which ends up making all 16-bit or wider types work correctly with
the most plausible interpretation of what IS_ERR_VALUE() was supposed
to return according to its users, but also causes a compile-time
warning for any users that do not pass an 'unsigned long' argument.

I suggested this approach earlier this year, but back then we ended
up deciding to just fix the users that are obviously broken. After
the initial warning that caused me to get involved in the discussion
(fs/gfs2/dir.c) showed up again in the mainline kernel, Linus
asked me to send the whole thing again.

[ Updated the 9p parts as per Al Viro  - Linus ]

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Cc: Andrzej Hajda <a.hajda@samsung.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Link: https://lkml.org/lkml/2016/1/7/363
Link: https://lkml.org/lkml/2016/5/27/486
Acked-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org> # For nvmem part
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-05-27 15:26:11 -07:00
Linus Torvalds 2566278551 Merge git://git.infradead.org/intel-iommu
Pull intel IOMMU updates from David Woodhouse:
 "This patchset improves the scalability of the Intel IOMMU code by
  resolving two spinlock bottlenecks and eliminating the linearity of
  the IOVA allocator, yielding up to ~5x performance improvement and
  approaching 'iommu=off' performance"

* git://git.infradead.org/intel-iommu:
  iommu/vt-d: Use per-cpu IOVA caching
  iommu/iova: introduce per-cpu caching to iova allocation
  iommu/vt-d: change intel-iommu to use IOVA frame numbers
  iommu/vt-d: avoid dev iotlb logic for domains with no dev iotlbs
  iommu/vt-d: only unmap mapped entries
  iommu/vt-d: correct flush_unmaps pfn usage
  iommu/vt-d: per-cpu deferred invalidation queues
  iommu/vt-d: refactoring of deferred flush entries
2016-05-27 13:49:24 -07:00
Linus Torvalds e7f44b65b5 Devicetree for 4.7:
- Rewrite of the unflattening code to avoid recursion and lessen the
   stack usage.
 
 - Rewrite of the phandle args parsing code to get rid of the fixed args
   size. This is needed for IOMMU code.
 
 - Sync to latest dtc which adds more dts style checking. These warnings
   are enabled with "W=1" compiles.
 
 - Tegra documentation updates related to the above warnings.
 
 - A bunch of spelling and other doc fixes.
 
 - Various vendor prefix additions.
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQIcBAABAgAGBQJXP3OZAAoJEPr7XbWNvGHDEUAQAJLbR9Js7RENPGX/u0NSJNaJ
 yQhyNVsz/BkAWFfWT6YEfyNnDY0UcRs2N9RHb+z65TsX1jvJJxLRDLRfz+rExfiZ
 cA1RJaF77kPOdA0eZapJIzPvAf97Zik+nzKLsqUPUSYaIzghV5rN6aR2AjXN5AYv
 TMQP41NwNQkxfO5I+NOssEB8IBH+DlAzg0LYXw8wNsAJc8o+DgEQjU8cxCqR0NgE
 SbpbJNF8tRXEJckZRC+Q7Gyn2J7VglmM/5VTFbBBwgIly2lcLADPVuX/Z6hZE3OH
 K7mhNWBu61vI5lU6u7q64ePeb63j+Ut/RR0tTPgjsLg0Qg0ue+6iZ66S9ZHEicbU
 wT4A/hjSImvZoQGGMrtUF5HGcaoMHGLGFgFc/Ouox8OQflntQBzuEx/gOQpPXcIT
 vdwITNW8/OGV3rgtmRO9mbdSZiAHPsydoTkIl+Ucod3nTrlEEOwgQARYO+2CfSRj
 sknndj26Kf+0n0tSv2d4JAEdEozp2ZPyfiAfpPXW74jOmOxeswUb3Kxx8YMwhCEl
 +s96rm1vtpNmJzXtuPV3eB0TydWMQ/3NXN6XOS7qEN/5y1AbQqKEoIyJOziBQMxe
 c9Eh/YSjsm4uw3Q0wHOI3s4hTwWfuBmwpIsANJVKrRbftPF58bMUBYU/44ReTtA8
 iMsrqJpnSCcAyS8doWRY
 =s+gW
 -----END PGP SIGNATURE-----

Merge tag 'devicetree-for-4.7' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux

Pull devicetree updates from Rob Herring:

 - Rewrite of the unflattening code to avoid recursion and lessen the
   stack usage.

 - Rewrite of the phandle args parsing code to get rid of the fixed args
   size.  This is needed for IOMMU code.

 - Sync to latest dtc which adds more dts style checking.  These
   warnings are enabled with "W=1" compiles.

 - Tegra documentation updates related to the above warnings.

 - A bunch of spelling and other doc fixes.

 - Various vendor prefix additions.

* tag 'devicetree-for-4.7' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux: (52 commits)
  devicetree: Add Creative Technology vendor id
  gpio: dt-bindings: add ibm,ppc4xx-gpio binding
  of/unittest: Remove unnecessary module.h header inclusion
  drivers/of: Fix build warning in populate_node()
  drivers/of: Fix depth when unflattening devicetree
  of: dynamic: changeset prop-update revert fix
  drivers/of: Export of_detach_node()
  drivers/of: Return allocated memory from of_fdt_unflatten_tree()
  drivers/of: Specify parent node in of_fdt_unflatten_tree()
  drivers/of: Rename unflatten_dt_node()
  drivers/of: Avoid recursively calling unflatten_dt_node()
  drivers/of: Split unflatten_dt_node()
  of: include errno.h in of_graph.h
  of: document refcount incrementation of of_get_cpu_node()
  Documentation: dt: soc: fix spelling mistakes
  Documentation: dt: power: fix spelling mistake
  Documentation: dt: pinctrl: fix spelling mistake
  Documentation: dt: opp: fix spelling mistake
  Documentation: dt: net: fix spelling mistakes
  Documentation: dt: mtd: fix spelling mistake
  ...
2016-05-20 14:51:34 -07:00
Linus Torvalds e0fb1b3639 IOMMU Updates for Linux v4.7
The updates include:
 
 	* Rate limiting for the VT-d fault handler
 
 	* Remove statistics code from the AMD IOMMU driver. It is unused
 	  and should be replaced by something more generic if needed
 
 	* Per-domain pagesize-bitmaps in IOMMU core code to support
 	  systems with different types of IOMMUs
 
 	* Support for ACPI devices in the AMD IOMMU driver
 
 	* 4GB mode support for Mediatek IOMMU driver
 
 	* ARM-SMMU updates from Will Deacon:
 
 		- Support for 64k pages with SMMUv1 implementations
 		  (e.g MMU-401)
 
 		- Remove open-coded 64-bit MMIO accessors
 
 		- Initial support for 16-bit VMIDs, as supported by some
 		  ThunderX SMMU implementations
 
 		- A couple of errata workarounds for silicon in the
 		  field
 
 	* Various fixes here and there
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2
 
 iQIcBAABAgAGBQJXPeM1AAoJECvwRC2XARrjA2QP/2Cz+pVkpQCuvhAse57eN4rB
 wWXKTjqSFZ4PcA3Vu5yvX6XMv15g46xXFJAhf2spE5//8+xgFfYBgkBRpnqu1brw
 SL6f8A912MnfMRgWqcdKkJNeHbiN0kOvcIQv1J8GNfciqMiyYFhiLP6fFiRmWR/F
 XDBjUeFZ5+Uwf1BAGqw0cVPexeakEbsLHUGqxFsh5g2T4i43aHzO2HJT3IdwWHDt
 F2ivs8gNFGBeJEyzhW8TD0rOEEyHAnM3N18qPEU9+dD0UmjnTQPymEZSbsGW5d4j
 Cn40QYlA+Zmbwgx6LaDVChzQyRJu6O3uvFThyRviiYKCri/Nc9cUT4vHsFGU4MXb
 1d3bqrgzaw7vw31BN7S1Py3MV+WpVnEYjFm2O+hW28OjtSpm6ZvbI8wc0rF4UT/I
 KgL0gSeA8tp25uVISM+ktpIrObYsAcoCz8nvurpDv2AGkKRzhyoSze0Jg43rusD8
 BH7iFWu1LRPlulTGlrHMtNmbZeEApUPbObcQAOcrBOj9vjuFaZ8qduZmB+hwS2iV
 p9atn+54LmGO0LuzqsGrhApIeXTeTZSrGyjlbUADWBJlTw8Xyk/CR39Wf3m/Xmpr
 DiJ/5oa8SKQtNbwvbScn1+sInNWP/pH/JgnRO3Yvqth8HWF/DlpzNj5XxAB8czwr
 qjk9WjpEXun50ocPFQeS
 =jpPD
 -----END PGP SIGNATURE-----

Merge tag 'iommu-updates-v4.7' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu

Pull IOMMU updates from Joerg Roedel:
 "The updates include:

   - rate limiting for the VT-d fault handler

   - remove statistics code from the AMD IOMMU driver.  It is unused and
     should be replaced by something more generic if needed

   - per-domain pagesize-bitmaps in IOMMU core code to support systems
     with different types of IOMMUs

   - support for ACPI devices in the AMD IOMMU driver

   - 4GB mode support for Mediatek IOMMU driver

   - ARM-SMMU updates from Will Deacon:
      - support for 64k pages with SMMUv1 implementations (e.g MMU-401)
      - remove open-coded 64-bit MMIO accessors
      - initial support for 16-bit VMIDs, as supported by some ThunderX
        SMMU implementations
      - a couple of errata workarounds for silicon in the field

   - various fixes here and there"

* tag 'iommu-updates-v4.7' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (44 commits)
  iommu/arm-smmu: Use per-domain page sizes.
  iommu/amd: Remove statistics code
  iommu/dma: Finish optimising higher-order allocations
  iommu: Allow selecting page sizes per domain
  iommu: of: enforce const-ness of struct iommu_ops
  iommu: remove unused priv field from struct iommu_ops
  iommu/dma: Implement scatterlist segment merging
  iommu/arm-smmu: Clear cache lock bit of ACR
  iommu/arm-smmu: Support SMMUv1 64KB supplement
  iommu/arm-smmu: Decouple context format from kernel config
  iommu/arm-smmu: Tidy up 64-bit/atomic I/O accesses
  io-64-nonatomic: Add relaxed accessor variants
  iommu/arm-smmu: Work around MMU-500 prefetch errata
  iommu/arm-smmu: Convert ThunderX workaround to new method
  iommu/arm-smmu: Differentiate specific implementations
  iommu/arm-smmu: Workaround for ThunderX erratum #27704
  iommu/arm-smmu: Add support for 16 bit VMID
  iommu/amd: Move get_device_id() and friends to beginning of file
  iommu/amd: Don't use IS_ERR_VALUE to check integer values
  iommu/amd: Signedness bug in acpihid_device_group()
  ...
2016-05-19 17:07:04 -07:00
Linus Torvalds 7afd16f882 PCI changes for the v4.7 merge window:
Enumeration
     Refine PCI support check in pcibios_init() (Adrian-Ken Rueegsegger)
     Provide common functions for ECAM mapping (Jayachandran C)
     Allow all PCIe services on non-ACPI host bridges (Jon Derrick)
     Remove return values from pcie_port_platform_notify() and relatives (Jon Derrick)
     Widen portdrv service type from 4 bits to 8 bits (Keith Busch)
     Add Downstream Port Containment portdrv service type (Keith Busch)
     Add Downstream Port Containment driver (Keith Busch)
 
   Resource management
     Identify Enhanced Allocation (EA) BAR Equivalent resources in sysfs (Alex Williamson)
     Supply CPU physical address (not bus address) to iomem_is_exclusive() (Bjorn Helgaas)
     alpha: Call iomem_is_exclusive() for IORESOURCE_MEM, but not IORESOURCE_IO (Bjorn Helgaas)
     Mark Broadwell-EP Home Agent 1 as having non-compliant BARs (Prarit Bhargava)
     Disable all BAR sizing for devices with non-compliant BARs (Prarit Bhargava)
     Move PCI I/O space management from OF to PCI core code (Tomasz Nowicki)
 
   PCI device hotplug
     acpiphp_ibm: Avoid uninitialized variable reference (Dan Carpenter)
     Use cached copy of PCI_EXP_SLTCAP_HPC bit (Lukas Wunner)
 
   Virtualization
     Mark Intel i40e NIC INTx masking as broken (Alex Williamson)
     Reverse standard ACS vs device-specific ACS enabling (Alex Williamson)
     Work around Intel Sunrise Point PCH incorrect ACS capability (Alex Williamson)
 
   IOMMU
     Add pci_add_dma_alias() to abstract implementation (Bjorn Helgaas)
     Move informational printk to pci_add_dma_alias() (Bjorn Helgaas)
     Add support for multiple DMA aliases (Jacek Lawrynowicz)
     Add DMA alias quirk for mic_x200_dma (Jacek Lawrynowicz)
 
   Thunderbolt
     Fix double free of drom buffer (Andreas Noever)
     Add Intel Thunderbolt device IDs (Lukas Wunner)
     Fix typos and magic number (Lukas Wunner)
     Support 1st gen Light Ridge controller (Lukas Wunner)
 
   Generic host bridge driver
     Use generic ECAM API (Jayachandran C)
 
   Cavium ThunderX host bridge driver
     Don't clobber read-only bits in bridge config registers (David Daney)
     Use generic ECAM API (Jayachandran C)
 
   Freescale i.MX6 host bridge driver
     Use enum instead of bool for variant indicator (Andrey Smirnov)
     Implement reset sequence for i.MX6+ (Andrey Smirnov)
     Factor out ref clock enable (Bjorn Helgaas)
     Add initial imx6sx support (Christoph Fritz)
     Add reset-gpio-active-high boolean property to DT (Petr Štetiar)
     Add DT property for link gen, default to Gen1 (Tim Harvey)
     dts: Specify imx6qp version of PCIe core (Andrey Smirnov)
     dts: Fix PCIe reset GPIO polarity on Toradex Apalis Ixora (Petr Štetiar)
 
   Marvell Armada host bridge driver
     add DT binding for Marvell Armada 7K/8K PCIe controller (Thomas Petazzoni)
     Add driver for Marvell Armada 7K/8K PCIe controller (Thomas Petazzoni)
 
   Marvell MVEBU host bridge driver
     Constify mvebu_pcie_pm_ops structure (Jisheng Zhang)
     Use SET_NOIRQ_SYSTEM_SLEEP_PM_OPS for mvebu_pcie_pm_ops (Jisheng Zhang)
 
   Microsoft Hyper-V host bridge driver
     Report resources release after stopping the bus (Vitaly Kuznetsov)
     Add explicit barriers to config space access (Vitaly Kuznetsov)
 
   Renesas R-Car host bridge driver
     Select PCI_MSI_IRQ_DOMAIN (Arnd Bergmann)
 
   Synopsys DesignWare host bridge driver
     Remove incorrect RC memory base/limit configuration (Gabriele Paoloni)
     Move Root Complex setup code to dw_pcie_setup_rc() (Jisheng Zhang)
 
   TI Keystone host bridge driver
     Add error IRQ handler (Murali Karicheri)
     Remove unnecessary goto statement (Murali Karicheri)
 
   Miscellaneous
     Fix spelling errors (Colin Ian King)
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQIcBAABAgAGBQJXPdMKAAoJEFmIoMA60/r8ofUP/j0zyzn24f0xY1wLeGJ8geB9
 6nHk1QdkPqwCiXZahEcnA5HMlFCl/ciWjjsoCqeMlvS6NXkX13KGcc1UGZszelTs
 68bFhyBKqcoMn0it53vBjBXnkfA64PmlxwY/T1ADulxL8amFOCpjjBruZ8pxJ/U7
 r6uHvhxUxHCRF7hMmpNN+V5XWXWCFFkPJZvxOTkglaxkbdnhZ0h0Xz9p9liUvjPH
 mBE72E3WUjiGogXGoLAPDclz1NI6rhRVUyTRcQ8EWaOwitV3OqMuDpAwoWH62ZZJ
 iorCkQk2/eKfN6OA6UgZh4loauAty0FeoZDX7ZVftQr52IpAzRUVx1oAq0J7u4ga
 KRX37mlK/53UcMZyv9Lz2kw4KjaLLELiInzcF+w3Bbov4UhY4/sL5uh9eNMFvSUU
 iZuY+GFlceL0P6wZuVKU5U8td/CyBr3f5vY/3htxuYHE1xJq4FkL92JpWRCvwpVr
 YdCzocscw73Yn8ZMplt8DX2fyabN7HyGezbQISrDDGY6T0ZDsRRKc6FFAt4xF+ta
 JJ+bcY8OcXtxGw6SXtrscL7vNXdR7Zg1HBSa8Sl/CopCdW9zs0VdwgFoxgORcWDT
 mphIgt57DMzaiUUaV8FRQz0mSLixnAcCEfGjVbAEEw3SP5ZChGfS3EknKb/CPRyk
 TD6I3pXTBhTWXd8aS113
 =68Iz
 -----END PGP SIGNATURE-----

Merge tag 'pci-v4.7-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci

Pull PCI updates from Bjorn Helgaas:
 "Enumeration:
   - Refine PCI support check in pcibios_init() (Adrian-Ken Rueegsegger)
   - Provide common functions for ECAM mapping (Jayachandran C)
   - Allow all PCIe services on non-ACPI host bridges (Jon Derrick)
   - Remove return values from pcie_port_platform_notify() and relatives (Jon Derrick)
   - Widen portdrv service type from 4 bits to 8 bits (Keith Busch)
   - Add Downstream Port Containment portdrv service type (Keith Busch)
   - Add Downstream Port Containment driver (Keith Busch)

  Resource management:
   - Identify Enhanced Allocation (EA) BAR Equivalent resources in sysfs (Alex Williamson)
   - Supply CPU physical address (not bus address) to iomem_is_exclusive() (Bjorn Helgaas)
   - alpha: Call iomem_is_exclusive() for IORESOURCE_MEM, but not IORESOURCE_IO (Bjorn Helgaas)
   - Mark Broadwell-EP Home Agent 1 as having non-compliant BARs (Prarit Bhargava)
   - Disable all BAR sizing for devices with non-compliant BARs (Prarit Bhargava)
   - Move PCI I/O space management from OF to PCI core code (Tomasz Nowicki)

  PCI device hotplug:
   - acpiphp_ibm: Avoid uninitialized variable reference (Dan Carpenter)
   - Use cached copy of PCI_EXP_SLTCAP_HPC bit (Lukas Wunner)

  Virtualization:
   - Mark Intel i40e NIC INTx masking as broken (Alex Williamson)
   - Reverse standard ACS vs device-specific ACS enabling (Alex Williamson)
   - Work around Intel Sunrise Point PCH incorrect ACS capability (Alex Williamson)

  IOMMU:
   - Add pci_add_dma_alias() to abstract implementation (Bjorn Helgaas)
   - Move informational printk to pci_add_dma_alias() (Bjorn Helgaas)
   - Add support for multiple DMA aliases (Jacek Lawrynowicz)
   - Add DMA alias quirk for mic_x200_dma (Jacek Lawrynowicz)

  Thunderbolt:
   - Fix double free of drom buffer (Andreas Noever)
   - Add Intel Thunderbolt device IDs (Lukas Wunner)
   - Fix typos and magic number (Lukas Wunner)
   - Support 1st gen Light Ridge controller (Lukas Wunner)

  Generic host bridge driver:
   - Use generic ECAM API (Jayachandran C)

  Cavium ThunderX host bridge driver:
   - Don't clobber read-only bits in bridge config registers (David Daney)
   - Use generic ECAM API (Jayachandran C)

  Freescale i.MX6 host bridge driver:
   - Use enum instead of bool for variant indicator (Andrey Smirnov)
   - Implement reset sequence for i.MX6+ (Andrey Smirnov)
   - Factor out ref clock enable (Bjorn Helgaas)
   - Add initial imx6sx support (Christoph Fritz)
   - Add reset-gpio-active-high boolean property to DT (Petr Štetiar)
   - Add DT property for link gen, default to Gen1 (Tim Harvey)
   - dts: Specify imx6qp version of PCIe core (Andrey Smirnov)
   - dts: Fix PCIe reset GPIO polarity on Toradex Apalis Ixora (Petr Štetiar)

  Marvell Armada host bridge driver:
   - add DT binding for Marvell Armada 7K/8K PCIe controller (Thomas Petazzoni)
   - Add driver for Marvell Armada 7K/8K PCIe controller (Thomas Petazzoni)

  Marvell MVEBU host bridge driver:
   - Constify mvebu_pcie_pm_ops structure (Jisheng Zhang)
   - Use SET_NOIRQ_SYSTEM_SLEEP_PM_OPS for mvebu_pcie_pm_ops (Jisheng Zhang)

  Microsoft Hyper-V host bridge driver:
   - Report resources release after stopping the bus (Vitaly Kuznetsov)
   - Add explicit barriers to config space access (Vitaly Kuznetsov)

  Renesas R-Car host bridge driver:
   - Select PCI_MSI_IRQ_DOMAIN (Arnd Bergmann)

  Synopsys DesignWare host bridge driver:
   - Remove incorrect RC memory base/limit configuration (Gabriele Paoloni)
   - Move Root Complex setup code to dw_pcie_setup_rc() (Jisheng Zhang)

  TI Keystone host bridge driver:
   - Add error IRQ handler (Murali Karicheri)
   - Remove unnecessary goto statement (Murali Karicheri)

  Miscellaneous:
   - Fix spelling errors (Colin Ian King)"

* tag 'pci-v4.7-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci: (48 commits)
  PCI: Disable all BAR sizing for devices with non-compliant BARs
  x86/PCI: Mark Broadwell-EP Home Agent 1 as having non-compliant BARs
  PCI: Identify Enhanced Allocation (EA) BAR Equivalent resources in sysfs
  PCI, of: Move PCI I/O space management to PCI core code
  PCI: generic, thunder: Use generic ECAM API
  PCI: Provide common functions for ECAM mapping
  PCI: hv: Add explicit barriers to config space access
  PCI: Use cached copy of PCI_EXP_SLTCAP_HPC bit
  PCI: Add Downstream Port Containment driver
  PCI: Add Downstream Port Containment portdrv service type
  PCI: Widen portdrv service type from 4 bits to 8 bits
  PCI: designware: Remove incorrect RC memory base/limit configuration
  PCI: hv: Report resources release after stopping the bus
  ARM: dts: imx6qp: Specify imx6qp version of PCIe core
  PCI: imx6: Implement reset sequence for i.MX6+
  PCI: imx6: Use enum instead of bool for variant indicator
  PCI: thunder: Don't clobber read-only bits in bridge config registers
  thunderbolt: Fix double free of drom buffer
  PCI: rcar: Select PCI_MSI_IRQ_DOMAIN
  PCI: armada: Add driver for Marvell Armada 7K/8K PCIe controller
  ...
2016-05-19 13:10:54 -07:00
Joerg Roedel 6c0b43df74 Merge branches 'arm/io-pgtable', 'arm/rockchip', 'arm/omap', 'x86/vt-d', 'ppc/pamu', 'core' and 'x86/amd' into next 2016-05-09 19:39:17 +02:00
Robin Murphy d546635731 iommu/arm-smmu: Use per-domain page sizes.
Now that we can accurately reflect the context format we choose for each
domain, do that instead of imposing the global lowest-common-denominator
restriction and potentially ending up with nothing. We currently have a
strict 1:1 correspondence between domains and context banks, so we don't
need to entertain the possibility of multiple formats _within_ a domain.

Signed-off-by: Will Deacon <will.deacon@arm.com>
[rm: split from original patch, added SMMUv3]
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-05-09 19:38:39 +02:00
Joerg Roedel e85e8f69ce iommu/amd: Remove statistics code
The statistics are not really used for anything and should
be replaced by generic and per-device statistic counters.
Remove the code for now.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-05-09 16:58:37 +02:00
Robin Murphy 3b6b7e19e3 iommu/dma: Finish optimising higher-order allocations
Now that we know exactly which page sizes our caller wants to use in the
given domain, we can restrict higher-order allocation attempts to just
those sizes, if any, and avoid wasting any time or effort on other sizes
which offer no benefit. In the same vein, this also lets us accommodate
a minimum order greater than 0 for special cases.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Tested-by: Yong Wu <yong.wu@mediatek.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-05-09 15:33:29 +02:00
Robin Murphy d16e0faab9 iommu: Allow selecting page sizes per domain
Many IOMMUs support multiple page table formats, meaning that any given
domain may only support a subset of the hardware page sizes presented in
iommu_ops->pgsize_bitmap. There are also certain use-cases where the
creator of a domain may want to control which page sizes are used, for
example to force the use of hugepage mappings to reduce pagetable walk
depth.

To this end, add a per-domain pgsize_bitmap to represent the subset of
page sizes actually in use, to make it possible for domains with
different requirements to coexist.

Signed-off-by: Will Deacon <will.deacon@arm.com>
[rm: hijacked and rebased original patch with new commit message]
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-05-09 15:33:29 +02:00
Robin Murphy 53c92d7933 iommu: of: enforce const-ness of struct iommu_ops
As a set of driver-provided callbacks and static data, there is no
compelling reason for struct iommu_ops to be mutable in core code, so
enforce const-ness throughout.

Acked-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-05-09 15:33:29 +02:00
Joerg Roedel 9a51f3fc77 Merge branch 'arm/smmu' into core 2016-05-09 15:33:17 +02:00
Robin Murphy 809eac54cd iommu/dma: Implement scatterlist segment merging
Stop wasting IOVA space by over-aligning scatterlist segments for a
theoretical worst-case segment boundary mask, and instead take the real
limits into account to merge consecutive segments wherever appropriate,
so our callers can benefit from getting back nicely simplified lists.

This also represents the last piece of functionality wanted by users of
the current arch/arm implementation, thus brings us a small step closer
to converting that over to the common code.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-05-09 12:17:49 +02:00
Joerg Roedel 8801561ce0 Merge branch 'for-joerg/arm-smmu/updates' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into arm/smmu 2016-05-09 12:03:37 +02:00
Ingo Molnar 1fb48f8e54 Linux 4.6-rc6
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQEcBAABAgAGBQJXJoi6AAoJEHm+PkMAQRiGYKIIAIcocIV48DpGAHXFuSZbzw5D
 rp9EbE5TormtddPz1J1zcqu9tl5H8tfxS+LvHqRaDXqQkbb0BWKttmEpKTm9mrH8
 kfGNW8uwrEgTMMsar54BypAMMhHz4ITsj3VQX5QLSC5j6wixMcOmQ+IqH0Bwt3wr
 Y5JXDtZRysI1GoMkSU7/qsQBjC7aaBa5VzVUiGvhV8DdvPVFQf73P89G1vzMKqb5
 HRWbH4ieu6/mclLvW9N2QKGMHQntlB+9m2kG9nVWWbBSDxpAotwqQZFh3D52MBUy
 6DH/PNgkVyDhX4vfjua0NrmXdwTfKxLWGxe4dZ8Z+JZP5c6pqWlClIPBCkjHj50=
 =KLSM
 -----END PGP SIGNATURE-----

Merge tag 'v4.6-rc6' into x86/asm, to refresh the tree

Signed-off-by: Ingo Molnar <mingo@kernel.org>
2016-05-05 08:35:00 +02:00
Peng Fan 3ca3712a42 iommu/arm-smmu: Clear cache lock bit of ACR
According MMU-500r2 TRM, section 3.7.1 Auxiliary Control registers,
You can modify ACTLR only when the ACR.CACHE_LOCK bit is 0.

So before clearing ARM_MMU500_ACTLR_CPRE of each context bank,
need clear CACHE_LOCK bit of ACR register first.

Since CACHE_LOCK bit is only present in MMU-500r2 onwards,
need to check the major number of IDR7.

Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Peng Fan <van.freenix@gmail.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-05-03 18:23:04 +01:00
Robin Murphy b7862e3559 iommu/arm-smmu: Support SMMUv1 64KB supplement
The 64KB Translation Granule Supplement to the SMMUv1 architecture
allows an SMMUv1 implementation to support 64KB pages for stage 2
translations, using a constrained VMSAv8 descriptor format limited
to 40-bit addresses. Now that we can freely mix and match context
formats, we can actually handle having 4KB pages via an AArch32
context but 64KB pages via an AArch64 context, so plumb it in.

It is assumed that any implementations will have hardware capabilities
matching the format constraints, thus obviating the need for excessive
sanity-checking; this is the case for MMU-401, the only ARM Ltd.
implementation.

CC: Eric Auger <eric.auger@linaro.org>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-05-03 18:23:03 +01:00
Robin Murphy 7602b87106 iommu/arm-smmu: Decouple context format from kernel config
The way the driver currently forces an AArch32 or AArch64 context format
based on the kernel config and SMMU architecture version is suboptimal,
in that it makes it very hard to support oddball mix-and-match cases
like the SMMUv1 64KB supplement, or situations where the reduced table
depth of an AArch32 short descriptor context may be desirable under an
AArch64 kernel. It also only happens to work on current implementations
which do support all the relevant formats.

Introduce an explicit notion of context format, so we can manage that
independently and get rid of the inflexible #ifdeffery.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-05-03 18:23:03 +01:00
Robin Murphy f9a05f05b1 iommu/arm-smmu: Tidy up 64-bit/atomic I/O accesses
With {read,write}q_relaxed now able to fall back to the common
nonatomic-hi-lo helper, make use of that so that we don't have to
open-code our own. In the process, also convert the other remaining
split accesses, and repurpose the custom accessor to smooth out the
couple of troublesome instances where we really want to avoid
nonatomic writes (and a 64-bit access is unnecessary in the 32-bit
context formats we would use on a 32-bit CPU).

This paves the way for getting rid of some of the assumptions currently
baked into the driver which make it really awkward to use 32-bit context
formats with SMMUv2 under a 64-bit kernel.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-05-03 18:23:03 +01:00
Robin Murphy f0cfffc48c iommu/arm-smmu: Work around MMU-500 prefetch errata
MMU-500 erratum #841119 is tickled by a particular set of circumstances
interacting with the next-page prefetcher. Since said prefetcher is
quite dumb and actually detrimental to performance in some cases (by
causing unwanted TLB evictions for non-sequential access patterns), we
lose very little by turning it off, and what we gain is a guarantee that
the erratum is never hit.

As a bonus, the same workaround will also prevent erratum #826419 once
v7 short descriptor support is implemented.

CC: Catalin Marinas <catalin.marinas@arm.com>
CC: Will Deacon <will.deacon@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-05-03 18:23:02 +01:00
Robin Murphy e086d912d4 iommu/arm-smmu: Convert ThunderX workaround to new method
With a framework for implementation-specific funtionality in place, the
currently-FDT-dependent ThunderX workaround gets to be the first user.

Acked-by: Tirumalesh Chalamarla <tchalamarla@caviumnetworks.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-05-03 18:23:02 +01:00
Robin Murphy 67b65a3fb8 iommu/arm-smmu: Differentiate specific implementations
As the inevitable reality of implementation-specific errata workarounds
begin to accrue alongside our integration quirk handling, it's about
time the driver had a decent way of keeping track. Extend the per-SMMU
data so we can identify specific implementations in an efficient and
firmware-agnostic manner.

Acked-by: Tirumalesh Chalamarla <tchalamarla@caviumnetworks.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-05-03 18:23:01 +01:00
Tirumalesh Chalamarla 1bd37a6835 iommu/arm-smmu: Workaround for ThunderX erratum #27704
Due to erratum #27704, the CN88xx SMMUv2 implementation supports only
shared ASID and VMID numberspaces.

This patch ensures that ASID and VMIDs are unique across all SMMU
instances on affected Cavium systems.

Signed-off-by: Tirumalesh Chalamarla <tchalamarla@caviumnetworks.com>
Signed-off-by: Akula Geethasowjanya <Geethasowjanya.Akula@caviumnetworks.com>
[will: commit message, comments and formatting]
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-05-03 18:23:01 +01:00
Tirumalesh Chalamarla 4e3e9b6997 iommu/arm-smmu: Add support for 16 bit VMID
This patch adds support for 16-bit VMIDs on implementations of SMMUv2
that support it.

Signed-off-by: Tirumalesh Chalamarla <tchalamarla@caviumnetworks.com>
[will: commit messsage and comments]
Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-05-03 18:23:00 +01:00
Joerg Roedel fd6c50ee3a iommu/amd: Move get_device_id() and friends to beginning of file
They will be needed there later.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-21 18:24:02 +02:00
Joerg Roedel 9ee35e4c6f iommu/amd: Don't use IS_ERR_VALUE to check integer values
Use the better 'var < 0' check.

Fixes: 7aba6cb9ee ('iommu/amd: Make call-sites of get_device_id aware of its return value')
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-21 18:21:31 +02:00
Robin Murphy 9800699c64 iommu/arm-smmu: Don't allocate resources for bypass domains
Until we get fully plumbed into of_iommu_configure, our default
IOMMU_DOMAIN_DMA domains just bypass translation. Since we achieve that
by leaving the stream table entries set to bypass instead of pointing at
a translation context, the context bank we allocate for the domain is
completely wasted. Context banks are typically a rather limited
resource, so don't hog ones we don't need.

Reported-by: Eric Auger <eric.auger@linaro.org>
Tested-by: Eric Auger <eric.auger@linaro.org>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-21 16:47:32 +02:00
Will Deacon 5f634956cc iommu/arm-smmu: Fix stream-match conflict with IOMMU_DOMAIN_DMA
Commit cbf8277ef4 ("iommu/arm-smmu: Treat IOMMU_DOMAIN_DMA as bypass
for now") ignores requests to attach a device to the default domain
since, without IOMMU-basked DMA ops available everywhere, the default
domain will just lead to unexpected transaction faults being reported.

Unfortunately, the way this was implemented on SMMUv2 causes a
regression with VFIO PCI device passthrough under KVM on AMD Seattle.
On this system, the host controller device is associated with both a
pci_dev *and* a platform_device, and can therefore end up with duplicate
SMR entries, resulting in a stream-match conflict at runtime.

This patch amends the original fix so that attaching to IOMMU_DOMAIN_DMA
is rejected even before configuring the SMRs. This restores the old
behaviour for now, but we'll need to look at handing host controllers
specially when we come to supporting the default domain fully.

Reported-by: Eric Auger <eric.auger@linaro.org>
Tested-by: Eric Auger <eric.auger@linaro.org>
Tested-by: Yang Shi <yang.shi@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-21 16:47:32 +02:00
Omer Peleg 22e2f9fa63 iommu/vt-d: Use per-cpu IOVA caching
Commit 9257b4a2 ('iommu/iova: introduce per-cpu caching to iova allocation')
introduced per-CPU IOVA caches to massively improve scalability. Use them.

Signed-off-by: Omer Peleg <omer@cs.technion.ac.il>
[mad@cs.technion.ac.il: rebased, cleaned up and reworded the commit message]
Signed-off-by: Adam Morrison <mad@cs.technion.ac.il>
Reviewed-by: Shaohua Li <shli@fb.com>
Reviewed-by: Ben Serebrin <serebrin@google.com>
[dwmw2: split out VT-d part into a separate patch]
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2016-04-20 15:44:48 -04:00
Omer Peleg 9257b4a206 iommu/iova: introduce per-cpu caching to iova allocation
IOVA allocation has two problems that impede high-throughput I/O.
First, it can do a linear search over the allocated IOVA ranges.
Second, the rbtree spinlock that serializes IOVA allocations becomes
contended.

Address these problems by creating an API for caching allocated IOVA
ranges, so that the IOVA allocator isn't accessed frequently.  This
patch adds a per-CPU cache, from which CPUs can alloc/free IOVAs
without taking the rbtree spinlock.  The per-CPU caches are backed by
a global cache, to avoid invoking the (linear-time) IOVA allocator
without needing to make the per-CPU cache size excessive.  This design
is based on magazines, as described in "Magazines and Vmem: Extending
the Slab Allocator to Many CPUs and Arbitrary Resources" (currently
available at https://www.usenix.org/legacy/event/usenix01/bonwick.html)

Adding caching on top of the existing rbtree allocator maintains the
property that IOVAs are densely packed in the IO virtual address space,
which is important for keeping IOMMU page table usage low.

To keep the cache size reasonable, we bound the IOVA space a CPU can
cache by 32 MiB (we cache a bounded number of IOVA ranges, and only
ranges of size <= 128 KiB).  The shared global cache is bounded at
4 MiB of IOVA space.

Signed-off-by: Omer Peleg <omer@cs.technion.ac.il>
[mad@cs.technion.ac.il: rebased, cleaned up and reworded the commit message]
Signed-off-by: Adam Morrison <mad@cs.technion.ac.il>
Reviewed-by: Shaohua Li <shli@fb.com>
Reviewed-by: Ben Serebrin <serebrin@google.com>
[dwmw2: split out VT-d part into a separate patch]
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2016-04-20 15:42:24 -04:00
Omer Peleg 2aac630429 iommu/vt-d: change intel-iommu to use IOVA frame numbers
Make intel-iommu map/unmap/invalidate work with IOVA pfns instead of
pointers to "struct iova". This avoids using the iova struct from the IOVA
red-black tree and the resulting explicit find_iova() on unmap.

This patch will allow us to cache IOVAs in the next patch, in order to
avoid rbtree operations for the majority of map/unmap operations.

Note: In eliminating the find_iova() operation, we have also eliminated
the sanity check previously done in the unmap flow. Arguably, this was
overhead that is better avoided in production code, but it could be
brought back as a debug option for driver development.

Signed-off-by: Omer Peleg <omer@cs.technion.ac.il>
[mad@cs.technion.ac.il: rebased, fixed to not break iova api, and reworded
 the commit message]
Signed-off-by: Adam Morrison <mad@cs.technion.ac.il>
Reviewed-by: Shaohua Li <shli@fb.com>
Reviewed-by: Ben Serebrin <serebrin@google.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2016-04-20 15:07:22 -04:00
Omer Peleg 0824c5920b iommu/vt-d: avoid dev iotlb logic for domains with no dev iotlbs
This patch avoids taking the device_domain_lock in iommu_flush_dev_iotlb()
for domains with no dev iotlb devices.

Signed-off-by: Omer Peleg <omer@cs.technion.ac.il>
[gvdl@google.com: fixed locking issues]
Signed-off-by: Godfrey van der Linden <gvdl@google.com>
[mad@cs.technion.ac.il: rebased and reworded the commit message]
Signed-off-by: Adam Morrison <mad@cs.technion.ac.il>
Reviewed-by: Shaohua Li <shli@fb.com>
Reviewed-by: Ben Serebrin <serebrin@google.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2016-04-20 15:06:15 -04:00
Omer Peleg 769530e4ba iommu/vt-d: only unmap mapped entries
Current unmap implementation unmaps the entire area covered by the IOVA
range, which is a power-of-2 aligned region. The corresponding map,
however, only maps those pages originally mapped by the user. This
discrepancy can lead to unmapping of already unmapped entries, which is
unneeded work.

With this patch, only mapped pages are unmapped. This is also a baseline
for a map/unmap implementation based on IOVAs and not iova structures,
which will allow caching.

Signed-off-by: Omer Peleg <omer@cs.technion.ac.il>
[mad@cs.technion.ac.il: rebased and reworded the commit message]
Signed-off-by: Adam Morrison <mad@cs.technion.ac.il>
Reviewed-by: Shaohua Li <shli@fb.com>
Reviewed-by: Ben Serebrin <serebrin@google.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2016-04-20 15:06:01 -04:00
Omer Peleg f5c0c08b1e iommu/vt-d: correct flush_unmaps pfn usage
Change flush_unmaps() to correctly pass iommu_flush_iotlb_psi()
dma addresses.  (x86_64 mm and dma have the same size for pages
at the moment, but this usage improves consistency.)

Signed-off-by: Omer Peleg <omer@cs.technion.ac.il>
[mad@cs.technion.ac.il: rebased and reworded the commit message]
Signed-off-by: Adam Morrison <mad@cs.technion.ac.il>
Reviewed-by: Shaohua Li <shli@fb.com>
Reviewed-by: Ben Serebrin <serebrin@google.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2016-04-20 15:05:56 -04:00
Omer Peleg aa4732406e iommu/vt-d: per-cpu deferred invalidation queues
The IOMMU's IOTLB invalidation is a costly process.  When iommu mode
is not set to "strict", it is done asynchronously. Current code
amortizes the cost of invalidating IOTLB entries by batching all the
invalidations in the system and performing a single global invalidation
instead. The code queues pending invalidations in a global queue that
is accessed under the global "async_umap_flush_lock" spinlock, which
can result is significant spinlock contention.

This patch splits this deferred queue into multiple per-cpu deferred
queues, and thus gets rid of the "async_umap_flush_lock" and its
contention.  To keep existing deferred invalidation behavior, it still
invalidates the pending invalidations of all CPUs whenever a CPU
reaches its watermark or a timeout occurs.

Signed-off-by: Omer Peleg <omer@cs.technion.ac.il>
[mad@cs.technion.ac.il: rebased, cleaned up and reworded the commit message]
Signed-off-by: Adam Morrison <mad@cs.technion.ac.il>
Reviewed-by: Shaohua Li <shli@fb.com>
Reviewed-by: Ben Serebrin <serebrin@google.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2016-04-20 15:05:24 -04:00
Omer Peleg 314f1dc140 iommu/vt-d: refactoring of deferred flush entries
Currently, deferred flushes' info is striped between several lists in
the flush tables. Instead, move all information about a specific flush
to a single entry in this table.

This patch does not introduce any functional change.

Signed-off-by: Omer Peleg <omer@cs.technion.ac.il>
[mad@cs.technion.ac.il: rebased and reworded the commit message]
Signed-off-by: Adam Morrison <mad@cs.technion.ac.il>
Reviewed-by: Shaohua Li <shli@fb.com>
Reviewed-by: Ben Serebrin <serebrin@google.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2016-04-20 15:05:20 -04:00
Joerg Roedel cb6c27bb09 iommu/arm-smmu: Make use of phandle iterators in device-tree parsing
Remove the usage of of_parse_phandle_with_args() and replace
it by the phandle-iterator implementation so that we can
parse out all of the potentially present 128 stream-ids.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Rob Herring <robh@kernel.org>
2016-04-19 17:25:15 -05:00
Ingo Molnar 6666ea558b Linux 4.6-rc4
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQEcBAABAgAGBQJXFELfAAoJEHm+PkMAQRiGRYIH+wWsUva7TR9arN1ZrURvI17b
 KqyQH8Ov9zJBsIaq/rFXOr5KfNgx7BU9BL9h7QkBy693HXTWf+GTZ1czHM4N12C3
 0ZdHGrLwTHo2zdisiQaFORZSfhSVTUNGXGHXw13bUMgEqatPgkozXEnsvXXNdt1Z
 HtlcuJn3pcj+QIY7qDXZgTLTwgn248hi1AgNag+ntFcWiz21IYaMIi7/mCY9QUIi
 AY+Y3hqFQM7/8cVyThGS5wZPTg1YzdhsLJpoCk0TbS8FvMEnA+ylcTgc15C78bwu
 AxOwM3OCmH4gMsd7Dd/O+i9lE3K6PFrgzdDisYL3P7eHap+EdiLDvVzPDPPx0xg=
 =Q7r3
 -----END PGP SIGNATURE-----

Merge tag 'v4.6-rc4' into x86/asm, to pick up fixes

Signed-off-by: Ingo Molnar <mingo@kernel.org>
2016-04-19 10:38:52 +02:00
Dan Carpenter 2d8e1f039d iommu/amd: Signedness bug in acpihid_device_group()
"devid" needs to be signed for the error handling to work.

Fixes: b097d11a0f ('iommu/amd: Manage iommu_group for ACPI HID devices')
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-15 12:09:10 +02:00
Borislav Petkov 93984fbd4e x86/cpufeature: Replace cpu_has_apic with boot_cpu_has() usage
Signed-off-by: Borislav Petkov <bp@suse.de>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: iommu@lists.linux-foundation.org
Cc: linux-pm@vger.kernel.org
Cc: oprofile-list@lists.sf.net
Link: http://lkml.kernel.org/r/1459801503-15600-8-git-send-email-bp@alien8.de
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2016-04-13 11:37:41 +02:00
Jacek Lawrynowicz 338c3149a2 PCI: Add support for multiple DMA aliases
Solve IOMMU support issues with PCIe non-transparent bridges that use
Requester ID look-up tables (RID-LUT), e.g., the PEX8733.

The NTB connects devices in two independent PCI domains.  Devices separated
by the NTB are not able to discover each other.  A PCI packet being
forwared from one domain to another has to have its RID modified so it
appears on correct bus and completions are forwarded back to the original
domain through the NTB.  The RID is translated using a preprogrammed table
(LUT) and the PCI packet propagates upstream away from the NTB.  If the
destination system has IOMMU enabled, the packet will be discarded because
the new RID is unknown to the IOMMU.  Adding a DMA alias for the new RID
allows IOMMU to properly recognize the packet.

Each device behind the NTB has a unique RID assigned in the RID-LUT.  The
current DMA alias implementation supports only a single alias, so it's not
possible to support mutiple devices behind the NTB when IOMMU is enabled.

Enable all possible aliases on a given bus (256) that are stored in a
bitset.  Alias devfn is directly translated to a bit number.  The bitset is
not allocated for devices that have no need for DMA aliases.

More details can be found in the following article:
http://www.plxtech.com/files/pdf/technical/expresslane/RTC_Enabling%20MulitHostSystemDesigns.pdf

Signed-off-by: Jacek Lawrynowicz <jacek.lawrynowicz@intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Alex Williamson <alex.williamson@redhat.com>
Acked-by: David Woodhouse <David.Woodhouse@intel.com>
Acked-by: Joerg Roedel <jroedel@suse.de>
2016-04-11 14:34:32 -05:00
Joerg Roedel e315604834 iommu/amd: Fix checking of pci dma aliases
Commit 61289cb ('iommu/amd: Remove old alias handling code')
removed the old alias handling code from the AMD IOMMU
driver because this is now handled by the IOMMU core code.

But this also removed the handling of PCI aliases, which is
not handled by the core code. This caused issues with PCI
devices that have hidden PCIe-to-PCI bridges that rewrite
the request-id.

Fix this bug by re-introducing some of the removed functions
from commit 61289cbaf6 and add a alias field
'struct iommu_dev_data'. This field carrys the return value
of the get_alias() function and uses that instead of the
amd_iommu_alias_table[] array in the code.

Fixes: 61289cbaf6 ('iommu/amd: Remove old alias handling code')
Cc: stable@vger.kernel.org # v4.4+
Tested-by: Tomasz Golinski <tomaszg@math.uwb.edu.pl>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-11 16:07:51 +02:00
Robin Murphy e88ccab12a iommu/io-pgtable-arm-v7s: Support IOMMU_MMIO flag
Teach the short-descriptor format to create Device mappings when asked.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07 15:07:50 +02:00
Robin Murphy fb948251e4 iommu/io-pgtable-arm: Support IOMMU_MMIO flag
Teach the LPAE format to create Device mappings when asked.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Tested-by: Eric Auger <eric.auger@linaro.org>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07 15:07:50 +02:00
Dan Carpenter 0b74ecdfbe iommu/vt-d: Silence an uninitialized variable warning
My static checker complains that "dma_alias" is uninitialized unless we
are dealing with a pci device.  This is true but harmless.  Anyway, we
can flip the condition around to silence the warning.

Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07 14:51:47 +02:00
John Keeping fbedd9b990 iommu/rockchip: Fix "is stall active" check
Since commit cd6438c5f8 ("iommu/rockchip: Reconstruct to support multi
slaves") rk_iommu_is_stall_active() always returns false because the
bitwise AND operates on the boolean flag promoted to an integer and a
value that is either zero or BIT(2).

Explicitly convert the right-hand value to a boolean so that both sides
are guaranteed to be either zero or one.

rk_iommu_is_paging_enabled() does not suffer from the same problem since
RK_MMU_STATUS_PAGING_ENABLED is BIT(0), but let's apply the same change
for consistency and to make it clear that it's correct without needing
to lookup the value.

Fixes: cd6438c5f8 ("iommu/rockchip: Reconstruct to support multi slaves")
Signed-off-by: John Keeping <john@metanate.com>
Reviewed-by: Heiko Stuebner <heiko@sntech.de>
Tested-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07 14:50:18 +02:00
Andy Fleming a0d284d2b1 powerpc: Fix incorrect PPC32 PAMU dependency
The Freescale PAMU can be enabled on both 32 and 64-bit
Power chips. Commit 477ab7a19c restricted PAMU to PPC32.
PPC covers both.

Signed-off-by: Andy Fleming <afleming@gmail.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07 14:45:59 +02:00
Joerg Roedel eebb8034a5 iommu: Don't overwrite domain pointer when there is no default_domain
IOMMU drivers that do not support default domains, but make
use of the the group->domain pointer can get that pointer
overwritten with NULL on device add/remove.

Make sure this can't happen by only overwriting the domain
pointer when it is NULL.

Cc: stable@vger.kernel.org # v4.4+
Fixes: 1228236de5 ('iommu: Move default domain allocation to iommu_group_get_for_dev()')
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07 14:33:03 +02:00
Wan Zongshun 9a4d3bf56c iommu/amd: Set AMD iommu callbacks for amba bus
AMD Uart DMA belongs to ACPI HID type device, and its driver
is basing on AMBA Bus, need also IOMMU support.

This patch is just to set the AMD iommu callbacks for amba bus.

Signed-off-by: Wan Zongshun <Vincent.Wan@amd.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07 13:29:42 +02:00
Wan Zongshun b097d11a0f iommu/amd: Manage iommu_group for ACPI HID devices
This patch creates a new function for finding or creating an IOMMU
group for acpihid(ACPI Hardware ID) device.

The acpihid devices with the same devid will be put into same group and
there will have the same domain id and share the same page table.

Signed-off-by: Wan Zongshun <Vincent.Wan@amd.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07 13:29:42 +02:00
Wan Zongshun 2bf9a0a127 iommu/amd: Add iommu support for ACPI HID devices
Current IOMMU driver make assumption that the downstream devices are PCI.
With the newly added ACPI-HID IVHD device entry support, this is no
longer true. This patch is to add dev type check and to distinguish the
pci and acpihid device code path.

Signed-off-by: Wan Zongshun <Vincent.Wan@amd.com>
Signed-off-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07 13:29:42 +02:00
Wan Zongshun 7aba6cb9ee iommu/amd: Make call-sites of get_device_id aware of its return value
This patch is to make the call-sites of get_device_id aware of its
return value.

Signed-off-by: Wan Zongshun <Vincent.Wan@amd.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07 13:29:41 +02:00