Commit Graph

294 Commits

Author SHA1 Message Date
David Woodhouse 4ed6a540fa iommu/vt-d: Fix passthrough mode with translation-disabled devices
When we use 'intel_iommu=igfx_off' to disable translation for the
graphics, and when we discover that the BIOS has misconfigured the DMAR
setup for I/OAT, we use a special DUMMY_DEVICE_DOMAIN_INFO value in
dev->archdata.iommu to indicate that translation is disabled.

With passthrough mode, we were attempting to dereference that as a
normal pointer to a struct device_domain_info when setting up an
identity mapping for the affected device.

This fixes the problem by making device_to_iommu() explicitly check for
the special value and indicate that no IOMMU was found to handle the
devices in question.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
Cc: stable@vger.kernel.org (which means you can pick up 18436afdc now too)
2015-05-11 14:59:20 +01:00
Linus Torvalds 9f86262dcc Merge git://git.infradead.org/intel-iommu
Pull intel iommu updates from David Woodhouse:
 "This lays a little of the groundwork for upcoming Shared Virtual
  Memory support — fixing some bogus #defines for capability bits and
  adding the new ones, and starting to use the new wider page tables
  where we can, in anticipation of actually filling in the new fields
  therein.

  It also allows graphics devices to be assigned to VM guests again.
  This got broken in 3.17 by disallowing assignment of RMRR-afflicted
  devices.  Like USB, we do understand why there's an RMRR for graphics
  devices — and unlike USB, it's actually sane.  So we can make an
  exception for graphics devices, just as we do USB controllers.

  Finally, tone down the warning about the X2APIC_OPT_OUT bit, due to
  persistent requests.  X2APIC_OPT_OUT was added to the spec as a nasty
  hack to allow broken BIOSes to forbid us from using X2APIC when they
  do stupid and invasive things and would break if we did.

  Someone noticed that since Windows doesn't have full IOMMU support for
  DMA protection, setting the X2APIC_OPT_OUT bit made Windows avoid
  initialising the IOMMU on the graphics unit altogether.

  This means that it would be available for use in "driver mode", where
  the IOMMU registers are made available through a BAR of the graphics
  device and the graphics driver can do SVM all for itself.

  So they started setting the X2APIC_OPT_OUT bit on *all* platforms with
  SVM capabilities.  And even the platforms which *might*, if the
  planets had been aligned correctly, possibly have had SVM capability
  but which in practice actually don't"

* git://git.infradead.org/intel-iommu:
  iommu/vt-d: support extended root and context entries
  iommu/vt-d: Add new extended capabilities from v2.3 VT-d specification
  iommu/vt-d: Allow RMRR on graphics devices too
  iommu/vt-d: Print x2apic opt out info instead of printing a warning
  iommu/vt-d: kill bogus ecap_niotlb_iunits()
2015-04-26 17:47:46 -07:00
Linus Torvalds 79319a052c IOMMU Updates for Linux v4.1
Not much this time, but the changes include:
 
 	* Moving domain allocation into the iommu drivers to prepare for
 	  the introduction of default domains for devices
 
 	* Fixing the IO page-table code in the AMD IOMMU driver to
 	  correctly encode large page sizes
 
 	* Extension of the PCI support in the ARM-SMMU driver
 
 	* Various fixes and cleanups
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQIcBAABAgAGBQJVNFIPAAoJECvwRC2XARrj4v8QAMVsPJ+kmnLvqGDkO9v2i9z6
 sFX27h55HhK3Pgb5aEmEhvZd0Eec22KtuADr92LsRSjskgA4FgrzzSlo8w7+MbwM
 dtowij+5Bzx/jEeexM5gog0ZA9Brl725KSYBmwJIAroKAtl3YXsIA4TO7X/JtXJm
 0qWbCxLs9CX5uWyJawkeDl8UAaZYb8AHKv1UhJt8Z5yajM/qITMULi51g2Bgh8kx
 YaRHeZNj+mFQqb6IlNkmOhILN+dbTdxQREp+aJs1alGdkBGlJyfo6eK4weNOpA4x
 gc8EXUWZzj1GEPyWMpA/ZMzPzCbj9M6wTeXqRiTq31AMV10zcy545uYcLWks680M
 CYvWTmjeCvwsbuaj9cn+efa47foH2UoeXxBmXWOJDv4WxcjE1ejmlmSd8WYfwkh9
 hIkMzD8tW2iZf3ssnjCeQLa7f6ydL2P4cpnK2JH+N7hN9VOASAlciezroFxtCjU+
 18T7ozgUTbOXZZomBX7OcGQ8ElXMiHB/uaCyNO64yVzApsUnQfpHzcRI5OavOYn5
 dznjrzvNLCwHs3QFI4R7rsmIfPkOM0g5nY5drGwJ23+F+rVpLmpWVPR5hqT7a1HM
 tJVmzces6HzOu7P1Mo0IwvNbZEmNBGTHYjGtWs6e79MQxdriFT4I+DwvFOy7GUq/
 Is2b+HPwWhiWJHQXLTT2
 =FMxH
 -----END PGP SIGNATURE-----

Merge tag 'iommu-updates-v4.1' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu

Pull IOMMU updates from Joerg Roedel:
 "Not much this time, but the changes include:

   - moving domain allocation into the iommu drivers to prepare for the
     introduction of default domains for devices

   - fixing the IO page-table code in the AMD IOMMU driver to correctly
     encode large page sizes

   - extension of the PCI support in the ARM-SMMU driver

   - various fixes and cleanups"

* tag 'iommu-updates-v4.1' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (34 commits)
  iommu/amd: Correctly encode huge pages in iommu page tables
  iommu/amd: Optimize amd_iommu_iova_to_phys for new fetch_pte interface
  iommu/amd: Optimize alloc_new_range for new fetch_pte interface
  iommu/amd: Optimize iommu_unmap_page for new fetch_pte interface
  iommu/amd: Return the pte page-size in fetch_pte
  iommu/amd: Add support for contiguous dma allocator
  iommu/amd: Don't allocate with __GFP_ZERO in alloc_coherent
  iommu/amd: Ignore BUS_NOTIFY_UNBOUND_DRIVER event
  iommu/amd: Use BUS_NOTIFY_REMOVED_DEVICE
  iommu/tegra: smmu: Compute PFN mask at runtime
  iommu/tegra: gart: Set aperture at domain initialization time
  iommu/tegra: Setup aperture
  iommu: Remove domain_init and domain_free iommu_ops
  iommu/fsl: Make use of domain_alloc and domain_free
  iommu/rockchip: Make use of domain_alloc and domain_free
  iommu/ipmmu-vmsa: Make use of domain_alloc and domain_free
  iommu/shmobile: Make use of domain_alloc and domain_free
  iommu/msm: Make use of domain_alloc and domain_free
  iommu/tegra-gart: Make use of domain_alloc and domain_free
  iommu/tegra-smmu: Make use of domain_alloc and domain_free
  ...
2015-04-20 10:50:05 -07:00
Rafael J. Wysocki 9a9ca16e7a Merge branch 'device-properties'
* device-properties:
  device property: Introduce firmware node type for platform data
  device property: Make it possible to use secondary firmware nodes
  driver core: Implement device property accessors through fwnode ones
  driver core: property: Update fwnode_property_read_string_array()
  driver core: Add comments about returning array counts
  ACPI: Introduce has_acpi_companion()
  driver core / ACPI: Represent ACPI companions using fwnode_handle
2015-04-13 00:35:54 +02:00
Joerg Roedel 7f65ef01e1 Merge branches 'iommu/fixes', 'x86/vt-d', 'x86/amd', 'arm/smmu', 'arm/tegra' and 'core' into next
Conflicts:
	drivers/iommu/amd_iommu.c
	drivers/iommu/tegra-gart.c
	drivers/iommu/tegra-smmu.c
2015-04-02 13:33:19 +02:00
Joerg Roedel 00a77deb0f iommu/vt-d: Make use of domain_alloc and domain_free
Get rid of domain_init and domain_destroy and implement
domain_alloc/domain_free instead.

Reviewed-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-03-31 15:32:02 +02:00
David Woodhouse 03ecc32c52 iommu/vt-d: support extended root and context entries
Add a new function iommu_context_addr() which takes care of the
differences and returns a pointer to a context entry which may be
in either format. The formats are binary compatible for all the old
fields anyway; the new one is just larger and some of the reserved
bits in the original 128 are now meaningful.

So far, nothing actually uses the new fields in the extended context
entry. Modulo hardware bugs with interpreting the new-style tables,
this should basically be a no-op.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2015-03-25 15:46:13 +00:00
David Woodhouse 18436afdc1 iommu/vt-d: Allow RMRR on graphics devices too
Commit c875d2c1 ("iommu/vt-d: Exclude devices using RMRRs from IOMMU API
domains") prevents certain options for devices with RMRRs. This even
prevents those devices from getting a 1:1 mapping with 'iommu=pt',
because we don't have the code to handle *preserving* the RMRR regions
when moving the device between domains.

There's already an exclusion for USB devices, because we know the only
reason for RMRRs there is a misguided desire to keep legacy
keyboard/mouse emulation running in some theoretical OS which doesn't
have support for USB in its own right... but which *does* enable the
IOMMU.

Add an exclusion for graphics devices too, so that 'iommu=pt' works
there. We should be able to successfully assign graphics devices to
guests too, as long as the initial handling of stolen memory is
reconfigured appropriately. This has certainly worked in the past.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
Cc: stable@vger.kernel.org
2015-03-25 15:36:35 +00:00
Alex Williamson 509fca899d iommu/vt-d: Remove unused variable
Unused after commit 7168440690 ("iommu/vt-d: Detach domain *only*
from attached iommus").  Reported by 0-day builder.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-03-24 15:39:26 +01:00
Alex Williamson 7168440690 iommu/vt-d: Detach domain *only* from attached iommus
Device domains never span IOMMU hardware units, which allows the
domain ID space for each IOMMU to be an independent address space.
Therefore we can have multiple, independent domains, each with the
same domain->id, but attached to different hardware units.  This is
also why we need to do a heavy-weight search for VM domains since
they can span multiple IOMMUs hardware units and we don't require a
single global ID to use for all hardware units.

Therefore, if we call iommu_detach_domain() across all active IOMMU
hardware units for a non-VM domain, the result is that we clear domain
IDs that are not associated with our domain, allowing them to be
re-allocated and causing apparent coherency issues when the device
cannot access IOVAs for the intended domain.

This bug was introduced in commit fb170fb4c5 ("iommu/vt-d: Introduce
helper functions to make code symmetric for readability"), but is
significantly exacerbated by the more recent commit 62c22167dd
("iommu/vt-d: Fix dmar_domain leak in iommu_attach_device") which calls
domain_exit() more frequently to resolve a domain leak.

Fixes: fb170fb4c5 ("iommu/vt-d: Introduce helper functions to make code symmetric for readability")
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Cc: Jiang Liu <jiang.liu@linux.intel.com>
Cc: stable@vger.kernel.org # v3.17+
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-03-23 15:22:08 +01:00
Rafael J. Wysocki ca5b74d267 ACPI: Introduce has_acpi_companion()
Now that the ACPI companions of devices are represented by pointers
to struct fwnode_handle, it is not quite efficient to check whether
or not an ACPI companion of a device is present by evaluating the
ACPI_COMPANION() macro.

For this reason, introduce a special static inline routine for that,
has_acpi_companion(), and update the code to use it where applicable.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2015-03-16 23:49:08 +01:00
Quentin Lambert 2f119c7895 iommu/vt-d: Convert non-returned local variable to boolean when relevant
This patch was produced using Coccinelle. A simplified version of the
semantic patch is:

@r exists@
identifier f;
local idexpression u8 x;
identifier xname;
@@

f(...) {
...when any
(
  x@xname = 1;
|
  x@xname = 0;
)
...when any
}

@bad exists@
identifier r.f;
local idexpression u8 r.x
expression e1 != {0, 1}, e2;
@@

f(...) {
...when any
(
  x = e1;
|
  x + e2
)
...when any
}

@depends on !bad@
identifier r.f;
local idexpression u8 r.x;
identifier r.xname;
@@

f(...) {
...
++ bool xname;
- int xname;
<...
(
  x =
- 1
+ true
|
  x =
- -1
+ false
)
...>

}

Signed-off-by: Quentin Lambert <lambert.quentin@gmail.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-03-03 14:13:20 +01:00
Robin Murphy 0fb5fe874c iommu: Make IOVA domain page size explicit
Systems may contain heterogeneous IOMMUs supporting differing minimum
page sizes, which may also not be common with the CPU page size.
Thus it is practical to have an explicit notion of IOVA granularity
to simplify handling of mapping and allocation constraints.

As an initial step, move the IOVA page granularity from an implicit
compile-time constant to a per-domain property so we can make use
of it in IOVA domain context at runtime. To keep the abstraction tidy,
extend the little API of inline iova_* helpers to parallel some of the
equivalent PAGE_* macros.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-01-19 14:55:22 +01:00
Robin Murphy 1b72250076 iommu: Make IOVA domain low limit flexible
To share the IOVA allocator with other architectures, it needs to
accommodate more general aperture restrictions; move the lower limit
from a compile-time constant to a runtime domain property to allow
IOVA domains with different requirements to co-exist.

Also reword the slightly unclear description of alloc_iova since we're
touching it anyway.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-01-19 14:55:22 +01:00
Robin Murphy 85b4545629 iommu: Consolidate IOVA allocator code
In order to share the IOVA allocator with other architectures, break
the unnecssary dependency on the Intel IOMMU driver and move the
remaining IOVA internals to iova.c

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-01-19 14:55:22 +01:00
Joerg Roedel 6d1b9cc9ee iommu/vt-d: Remove dead code in device_notifier
This code only runs when action == BUS_NOTIFY_REMOVED_DEVICE,
so it can't be BUS_NOTIFY_DEL_DEVICE.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-01-05 12:23:38 +01:00
Joerg Roedel 62c22167dd iommu/vt-d: Fix dmar_domain leak in iommu_attach_device
Since commit 1196c2f a domain is only destroyed in the
notifier path if it is hot-unplugged. This caused a
domain leakage in iommu_attach_device when a driver was
unbound from the device and bound to VFIO. In this case the
device is attached to a new domain and unlinked from the old
domain. At this point nothing points to the old domain
anymore and its memory is leaked.
Fix this by explicitly freeing the old domain in
iommu_attach_domain.

Fixes: 1196c2f (iommu/vt-d: Fix dmar_domain leak in iommu_attach_device)
Cc: stable@vger.kernel.org # v3.18
Tested-by: Jerry Hoemann <jerry.hoemann@hp.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-01-05 12:23:38 +01:00
Joerg Roedel 76771c938e Merge branches 'arm/omap', 'arm/msm', 'arm/rockchip', 'arm/renesas', 'arm/smmu', 'x86/vt-d', 'x86/amd' and 'core' into next
Conflicts:
	drivers/iommu/arm-smmu.c
2014-12-02 13:07:13 +01:00
Jiang Liu cc4f14aa17 iommu/vt-d: Fix an off-by-one bug in __domain_mapping()
There's an off-by-one bug in function __domain_mapping(), which may
trigger the BUG_ON(nr_pages < lvl_pages) when
	(nr_pages + 1) & superpage_mask == 0

The issue was introduced by commit 9051aa0268 "intel-iommu: Combine
domain_pfn_mapping() and domain_sg_mapping()", which sets sg_res to
"nr_pages + 1" to avoid some of the 'sg_res==0' code paths.

It's safe to remove extra "+1" because sg_res is only used to calculate
page size now.

Reported-And-Tested-by: Sudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Cc: <stable@vger.kernel.org> # >= 3.0
Acked-By: David Woodhouse <David.Woodhouse@intel.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-12-02 13:03:09 +01:00
Jiang Liu ffebeb46dd iommu/vt-d: Enhance intel-iommu driver to support DMAR unit hotplug
Implement required callback functions for intel-iommu driver
to support DMAR unit hotplug.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Reviewed-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-11-18 11:18:36 +01:00
Jiang Liu 6b1972493a iommu/vt-d: Implement DMAR unit hotplug framework
On Intel platforms, an IO Hub (PCI/PCIe host bridge) may contain DMAR
units, so we need to support DMAR hotplug when supporting PCI host
bridge hotplug on Intel platforms.

According to Section 8.8 "Remapping Hardware Unit Hot Plug" in "Intel
Virtualization Technology for Directed IO Architecture Specification
Rev 2.2", ACPI BIOS should implement ACPI _DSM method under the ACPI
object for the PCI host bridge to support DMAR hotplug.

This patch introduces interfaces to parse ACPI _DSM method for
DMAR unit hotplug. It also implements state machines for DMAR unit
hot-addition and hot-removal.

The PCI host bridge hotplug driver should call dmar_hotplug_hotplug()
before scanning PCI devices connected for hot-addition and after
destroying all PCI devices for hot-removal.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Reviewed-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-11-18 11:18:35 +01:00
Jiang Liu 78d8e70461 iommu/vt-d: Dynamically allocate and free seq_id for DMAR units
Introduce functions to support dynamic IOMMU seq_id allocating and
releasing, which will be used to support DMAR hotplug.

Also rename IOMMU_UNITS_SUPPORTED as DMAR_UNITS_SUPPORTED.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Reviewed-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-11-18 11:18:35 +01:00
Jiang Liu c2a0b538d2 iommu/vt-d: Introduce helper function dmar_walk_resources()
Introduce helper function dmar_walk_resources to walk resource entries
in DMAR table and ACPI buffer object returned by ACPI _DSM method
for IOMMU hot-plug.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-11-18 11:18:35 +01:00
Li, Zhen-Hua 1a2262f90f x86/vt-d: Fix incorrect bit operations in setting values
The function context_set_address_root() and set_root_value are setting new
address in a wrong way, and this patch is trying to fix this problem.

According to Intel Vt-d specs(Feb 2011, Revision 1.3), Chapter 9.1 and 9.2,
field ctp in root entry is using bits 12:63, field asr in context entry is
using bits 12:63.

To set these fields, the following functions are used:
static inline void context_set_address_root(struct context_entry *context,
        unsigned long value);
and
static inline void set_root_value(struct root_entry *root, unsigned long value)

But they are using an invalid method to set these fields, in current code, only
a '|' operator is used to set it. This will not set the asr to the expected
value if it has an old value.

For example:
Before calling this function,
	context->lo = 0x3456789012111;
	value = 0x123456789abcef12;

After we call context_set_address_root(context, value), expected result is
	context->lo == 0x123456789abce111;

But the actual result is:
	context->lo == 0x1237577f9bbde111;

So we need to clear bits 12:63 before setting the new value, this will fix
this problem.

Signed-off-by: Li, Zhen-Hua <zhen-hual@hp.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-11-06 14:40:10 +01:00
Olav Haugan 315786ebbf iommu: Add iommu_map_sg() function
Mapping and unmapping are more often than not in the critical path.
map_sg allows IOMMU driver implementations to optimize the process
of mapping buffers into the IOMMU page tables.

Instead of mapping a buffer one page at a time and requiring potentially
expensive TLB operations for each page, this function allows the driver
to map all pages in one go and defer TLB maintenance until after all
pages have been mapped.

Additionally, the mapping operation would be faster in general since
clients does not have to keep calling map API over and over again for
each physically contiguous chunk of memory that needs to be mapped to a
virtually contiguous region.

Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-11-04 14:53:36 +01:00
Joerg Roedel 09b5269a1b Merge branches 'arm/exynos', 'arm/omap', 'arm/smmu', 'x86/vt-d', 'x86/amd' and 'core' into next
Conflicts:
	drivers/iommu/arm-smmu.c
2014-10-02 12:24:45 +02:00
Joerg Roedel 1196c2fb04 iommu/vt-d: Only remove domain when device is removed
This makes sure any RMRR mappings stay in place when the
driver is unbound from the device.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
Tested-by: Jerry Hoemann <jerry.hoemann@hp.com>
2014-10-02 11:18:58 +02:00
Joerg Roedel 5d587b8de5 iommu/vt-d: Convert to iommu_capable() API function
Cc: Jiang Liu <jiang.liu@linux.intel.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-09-25 15:47:37 +02:00
Joerg Roedel e7f9fa5498 iommu/vt-d: Defer domain removal if device is assigned to a driver
When the BUS_NOTIFY_DEL_DEVICE event is received the device
might still be attached to a driver. In this case the domain
can't be released as the mappings might still be in use.

Defer the domain removal in this case until we receivce the
BUS_NOTIFY_UNBOUND_DRIVER event.

Cc: Jiang Liu <jiang.liu@linux.intel.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: stable@vger.kernel.org   # v3.15, v3.16
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-08-18 13:37:56 +02:00
Alex Williamson c875d2c1b8 iommu/vt-d: Exclude devices using RMRRs from IOMMU API domains
The user of the IOMMU API domain expects to have full control of
the IOVA space for the domain.  RMRRs are fundamentally incompatible
with that idea.  We can neither map the RMRR into the IOMMU API
domain, nor can we guarantee that the device won't continue DMA with
the area described by the RMRR as part of the new domain.  Therefore
we must prevent such devices from being used by the IOMMU API.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: stable@vger.kernel.org
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-29 17:38:31 +02:00
Jiang Liu 161f693460 iommu/vt-d: Fix issue in computing domain's iommu_snooping flag
IOMMU units may dynamically attached to/detached from domains,
so we should scan all active IOMMU units when computing iommu_snooping
flag for a domain instead of only scanning IOMMU units associated
with the domain.

Also check snooping and superpage capabilities when hot-adding DMAR units.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-23 16:04:47 +02:00
Jiang Liu a156ef99e8 iommu/vt-d: Introduce helper function iova_size() to improve code readability
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-23 16:04:47 +02:00
Jiang Liu 162d1b10d4 iommu/vt-d: Introduce helper domain_pfn_within_range() to simplify code
Introduce helper function domain_pfn_within_range() to simplify code
and improve readability.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-23 16:04:47 +02:00
Jiang Liu d41a4adb1b iommu/vt-d: Simplify intel_unmap_sg() and kill duplicated code
Introduce intel_unmap() to reduce duplicated code in intel_unmap_sg()
and intel_unmap_page().

Also let dma_pte_free_pagetable() to call dma_pte_clear_range() directly,
so caller only needs to call dma_pte_free_pagetable().

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-23 16:04:47 +02:00
Jiang Liu 2a41ccee2f iommu/vt-d: Change iommu_enable/disable_translation to return void
Simplify error handling path by changing iommu_{enable|disable}_translation
to return void.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-23 16:04:47 +02:00
Jiang Liu 129ad28100 iommu/vt-d: Avoid freeing virtual machine domain in free_dmar_iommu()
Virtual machine domains are created by intel_iommu_domain_init() and
should be destroyed by intel_iommu_domain_destroy(). So avoid freeing
virtual machine domain data structure in free_dmar_iommu() when
doamin->iommu_count reaches zero, otherwise it may cause invalid
memory access because the IOMMU framework still holds references
to the domain structure.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-23 16:04:47 +02:00
Jiang Liu 2a46ddf77c iommu/vt-d: Fix possible invalid memory access caused by free_dmar_iommu()
Static identity and virtual machine domains may be cached in
iommu->domain_ids array after corresponding IOMMUs have been removed
from domain->iommu_bmp. So we should check domain->iommu_bmp before
decreasing domain->iommu_count in function free_dmar_iommu(), otherwise
it may cause free of inuse domain data structure.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-23 16:04:47 +02:00
Jiang Liu 44bde61428 iommu/vt-d: Allocate dynamic domain id for virtual domains only
Check the same domain id is allocated for si_domain on each IOMMU,
otherwise the IOTLB flush for si_domain will fail.

Now the rules to allocate and manage domain id are:
1) For normal and static identity domains, domain id is allocated
   when creating domain structure. And this id will be written into
   context entry.
2) For virtual machine domain, a virtual id is allocated when creating
   domain. And when binding virtual machine domain to an iommu, a real
   domain id is allocated on demand and this domain id will be written
   into context entry. So domain->id for virtual machine domain may be
   different from the domain id written into context entry(used by
   hardware).

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-23 16:04:47 +02:00
Jiang Liu fb170fb4c5 iommu/vt-d: Introduce helper functions to make code symmetric for readability
Introduce domain_attach_iommu()/domain_detach_iommu() and refine
iommu_attach_domain()/iommu_detach_domain() to make code symmetric
and improve readability.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-23 16:04:46 +02:00
Jiang Liu ab8dfe2515 iommu/vt-d: Introduce helper functions to improve code readability
Introduce domain_type_is_vm() and domain_type_is_vm_or_si() to improve
code readability.

Also kill useless macro DOMAIN_FLAG_P2P_MULTIPLE_DEVICES.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-23 16:04:46 +02:00
Jiang Liu 18fd779a41 iommu/vt-d: Use correct domain id to flush virtual machine domains
For virtual machine domains, domain->id is a virtual id, and the real
domain id written into context entry is dynamically allocated.
So use the real domain id instead of domain->id when flushing iotlbs
for virtual machine domains.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-23 16:04:46 +02:00
Jiang Liu c3b497c6bb iommu/vt-d: Match segment number when searching for dev_iotlb capable devices
For virtual machine and static identity domains, there may be devices
from different PCI segments associated with the same domain.
So function iommu_support_dev_iotlb() should also match PCI segment
number (iommu unit) when searching for dev_iotlb capable devices.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-23 16:04:46 +02:00
Joerg Roedel cbb24a25a8 Merge branch 'core' into x86/vt-d
Conflicts:
	drivers/iommu/intel-iommu.c
2014-07-23 16:04:37 +02:00
Thierry Reding b22f6434cf iommu: Constify struct iommu_ops
This structure is read-only data and should never be modified.

Signed-off-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-07 10:36:59 +02:00
Alex Williamson a5459cfece iommu/vt-d: Make use of IOMMU sysfs support
Register our DRHD IOMMUs, cross link devices, and provide a base set
of attributes for the IOMMU.  Note that IRQ remapping support parses
the DMAR table very early in boot, well before the iommu_class can
reasonably be setup, so our registration is split between
intel_iommu_init(), which occurs later, and alloc_iommu(), which
typically occurs much earlier, but may happen at any time later
with IOMMU hot-add support.

On a typical desktop system, this provides the following (pruned):

$ find /sys | grep dmar
/sys/devices/virtual/iommu/dmar0
/sys/devices/virtual/iommu/dmar0/devices
/sys/devices/virtual/iommu/dmar0/devices/0000:00:02.0
/sys/devices/virtual/iommu/dmar0/intel-iommu
/sys/devices/virtual/iommu/dmar0/intel-iommu/cap
/sys/devices/virtual/iommu/dmar0/intel-iommu/ecap
/sys/devices/virtual/iommu/dmar0/intel-iommu/address
/sys/devices/virtual/iommu/dmar0/intel-iommu/version
/sys/devices/virtual/iommu/dmar1
/sys/devices/virtual/iommu/dmar1/devices
/sys/devices/virtual/iommu/dmar1/devices/0000:00:00.0
/sys/devices/virtual/iommu/dmar1/devices/0000:00:01.0
/sys/devices/virtual/iommu/dmar1/devices/0000:00:16.0
/sys/devices/virtual/iommu/dmar1/devices/0000:00:1a.0
/sys/devices/virtual/iommu/dmar1/devices/0000:00:1b.0
/sys/devices/virtual/iommu/dmar1/devices/0000:00:1c.0
...
/sys/devices/virtual/iommu/dmar1/intel-iommu
/sys/devices/virtual/iommu/dmar1/intel-iommu/cap
/sys/devices/virtual/iommu/dmar1/intel-iommu/ecap
/sys/devices/virtual/iommu/dmar1/intel-iommu/address
/sys/devices/virtual/iommu/dmar1/intel-iommu/version
/sys/class/iommu/dmar0
/sys/class/iommu/dmar1

(devices also link back to the dmar units)

This makes address, version, capabilities, and extended capabilities
available, just like printed on boot.  I've tried not to duplicate
data that can be found in the DMAR table, with the exception of the
address, which provides an easy way to associate the sysfs device with
a DRHD entry in the DMAR.  It's tempting to add scopes and RMRR data
here, but the full DMAR table is already exposed under /sys/firmware/
and therefore already provides a way for userspace to learn such
details.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-04 12:35:59 +02:00
Alex Williamson 579305f75d iommu/vt-d: Update to use PCI DMA aliases
VT-d code currently makes use of pci_find_upstream_pcie_bridge() in
order to find the topology based alias of a device.  This function has
a few problems.  First, it doesn't check the entire alias path of the
device to the root bus, therefore if a PCIe device is masked upstream,
the wrong result is produced.  Also, it's known to get confused and
give up when it crosses a bridge from a conventional PCI bus to a PCIe
bus that lacks a PCIe capability.  The PCI-core provided DMA alias
support solves both of these problems and additionally adds support
for DMA function quirks allowing VT-d to work with devices like
Marvell and Ricoh with known broken requester IDs.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-04 12:35:58 +02:00
Alex Williamson e17f9ff413 iommu/vt-d: Use iommu_group_get_for_dev()
The IOMMU code now provides a common interface for finding or
creating an IOMMU group for a device on PCI buses.  Make use of it
and remove piles of code.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-04 12:35:58 +02:00
Yijing Wang aa4d066a2a iommu/vt-d: Suppress compiler warnings
suppress compiler warnings:
drivers/iommu/intel-iommu.c: In function ‘device_to_iommu’:
drivers/iommu/intel-iommu.c:673: warning: ‘segment’ may be used uninitialized in this function
drivers/iommu/intel-iommu.c: In function ‘get_domain_for_dev.clone.3’:
drivers/iommu/intel-iommu.c:2217: warning: ‘bridge_bus’ may be used uninitialized in this function
drivers/iommu/intel-iommu.c:2217: warning: ‘bridge_devfn’ may be used uninitialized in this function

Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-04 11:34:37 +02:00
Yijing Wang effad4b59f iommu/vt-d: Remove the useless dma_pte_addr
Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-04 11:34:20 +02:00
Joerg Roedel c3c75eb7fa iommu/vt-d: Don't use magic number in dma_pte_superpage
Use the already defined DMA_PTE_LARGE_PAGE for testing
instead of hardcoding the value again.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-04 11:34:18 +02:00
Yijing Wang 9b27e82d20 iommu/vt-d: Fix reference count in iommu_prepare_isa
Decrease the device reference count avoid memory leak.

Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-04 11:34:13 +02:00
Yijing Wang e16922af9d iommu/vt-d: Use inline function dma_pte_superpage instead of macros
Use inline function dma_pte_superpage() instead of macro for
better readability.

Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-04 11:34:07 +02:00
Yijing Wang 8f9d41b430 iommu/vt-d: Clear the redundant assignment for domain->nid
Alloc_domain() will initialize domain->nid to -1. So the
initialization for domain->nid in md_domain_init() is redundant,
clear it.

Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-04 11:34:00 +02:00
Yijing Wang 3a74ca0140 iommu/vt-d: Use list_for_each_safe() to simplify code
Use list_for_each_entry_safe() instead of list_entry()
to simplify code.

Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-04 11:16:20 +02:00
Jiang Liu 27e249501c iommu/vt-d: fix bug in handling multiple RMRRs for the same PCI device
Function dmar_iommu_notify_scope_dev() makes a wrong assumption that
there's one RMRR for each PCI device at most, which causes DMA failure
on some HP platforms. So enhance dmar_iommu_notify_scope_dev() to
handle multiple RMRRs for the same PCI device.

Fixbug: https://bugzilla.novell.com/show_bug.cgi?id=879482

Cc: <stable@vger.kernel.org> # 3.15
Reported-by: Tom Mingarelli <thomas.mingarelli@hp.com>
Tested-by: Linda Knippers <linda.knippers@hp.com>
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-06-20 14:18:04 +02:00
Akinobu Mita 3674643625 intel-iommu: integrate DMA CMA
This adds support for the DMA Contiguous Memory Allocator for
intel-iommu.  This change enables dma_alloc_coherent() to allocate big
contiguous memory.

It is achieved in the same way as nommu_dma_ops currently does, i.e.
trying to allocate memory by dma_alloc_from_contiguous() and
alloc_pages() is used as a fallback.

Signed-off-by: Akinobu Mita <akinobu.mita@gmail.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Don Dutile <ddutile@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2014-06-04 16:53:57 -07:00
David Woodhouse 9f05d3fb64 iommu/vt-d: Fix get_domain_for_dev() handling of upstream PCIe bridges
Commit 146922ec79 ("iommu/vt-d: Make get_domain_for_dev() take struct
device") introduced new variables bridge_bus and bridge_devfn to
identify the upstream PCIe to PCI bridge responsible for the given
target device. Leaving the original bus/devfn variables to identify
the target device itself, now that it is no longer assumed to be PCI
and we can no longer trivially find that information.

However, the patch failed to correctly use the new variables in all
cases; instead using the as-yet-uninitialised 'bus' and 'devfn'
variables.

Reported-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-04-14 22:01:30 -07:00
Jiang Liu adeb25905c iommu/vt-d: fix memory leakage caused by commit ea8ea46
Commit ea8ea46 "iommu/vt-d: Clean up and fix page table clear/free
behaviour" introduces possible leakage of DMA page tables due to:
        for (pte = page_address(pg); !first_pte_in_page(pte); pte++) {
                if (dma_pte_present(pte) && !dma_pte_superpage(pte))
                        freelist = dma_pte_list_pagetables(domain, level - 1,
                                                           pte, freelist);
        }

For the first pte in a page, first_pte_in_page(pte) will always be true,
thus dma_pte_list_pagetables() will never be called and leak DMA page
tables if level is bigger than 1.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-04-13 13:07:56 +01:00
Dan Carpenter 14d4056996 iommu/vt-d: returning free pointer in get_domain_for_dev()
If we hit this error condition then we want to return a NULL pointer and
not a freed variable.

Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-28 11:31:39 +00:00
David Woodhouse cf04eee8bf iommu/vt-d: Include ACPI devices in iommu=pt
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:08:10 +00:00
David Woodhouse 66077edc97 iommu/vt-d: Finally enable translation for non-PCI devices
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:08:08 +00:00
David Woodhouse 46333e375f iommu/vt-d: Remove to_pci_dev() in intel_map_page()
It might not be...

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:08:07 +00:00
David Woodhouse 7207d8f925 iommu/vt-d: Remove pdev from intel_iommu_attach_device()
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:08:05 +00:00
David Woodhouse ecb509ec2b iommu/vt-d: Remove pdev from iommu_no_mapping()
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:08:04 +00:00
David Woodhouse 5913c9bf0e iommu/vt-d: Make domain_add_dev_info() take struct device
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:08:03 +00:00
David Woodhouse bf9c9eda71 iommu/vt-d: Make domain_remove_one_dev_info() take struct device
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:08:01 +00:00
David Woodhouse 5040a918bd iommu/vt-d: Rename 'hwdev' variables to 'dev' now that that's the norm
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:08:00 +00:00
David Woodhouse 207e35920d iommu/vt-d: Remove some pointless to_pci_dev() calls
Mostly made redundant by using dev_name() instead of pci_name(), and one
instance of using *dev->dma_mask instead of pdev->dma_mask.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:07:58 +00:00
David Woodhouse d4b709f48e iommu/vt-d: Make get_valid_domain_for_dev() take struct device
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:07:57 +00:00
David Woodhouse 3bdb259116 iommu/vt-d: Make iommu_should_identity_map() take struct device
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:07:55 +00:00
David Woodhouse 0b9d975315 iommu/vt-d: Handle RMRRs for non-PCI devices
Should hopefully never happen (RMRRs are an abomination) but while we're
busy eliminating all the PCI assumptions, we might as well do it.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:07:54 +00:00
David Woodhouse 146922ec79 iommu/vt-d: Make get_domain_for_dev() take struct device
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:07:52 +00:00
David Woodhouse e1f167f3fd iommu/vt-d: Make domain_context_mapp{ed,ing}() take struct device
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:07:51 +00:00
David Woodhouse 156baca8d3 iommu/vt-d: Make device_to_iommu() cope with non-PCI devices
Pass the struct device to it, and also make it return the bus/devfn to use,
since that is also stored in the DMAR table.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:07:49 +00:00
David Woodhouse 9b226624bb iommu/vt-d: Make identity_mapping() take struct device not struct pci_dev
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:07:48 +00:00
David Woodhouse 41e80dca52 iommu/vt-d: Remove segment from struct device_domain_info()
It's accessible via info->iommu->segment so this is redundant.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:07:46 +00:00
David Woodhouse 7c7faa11ec iommu/vt-d: Remove device_to_iommu() call from domain_remove_dev_info()
This was problematic because it works by domain/bus/devfn and we want
to make device_to_iommu() use only a struct device * (for handling non-PCI
devices). Now that the iommu pointer is reliably stored in the
device_domain_info, we don't need to look it up.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:06:53 +00:00
David Woodhouse 8bbc441012 iommu/vt-d: Simplify iommu check in domain_remove_one_dev_info()
Now we store the iommu in the device_domain_info, we don't need to do a
lookup.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:06:51 +00:00
David Woodhouse 5a8f40e8c8 iommu/vt-d: Always store iommu in device_domain_info
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:06:44 +00:00
David Woodhouse e2f8c5f6d4 iommu/vt-d: Use domain_remove_one_dev_info() in domain_add_dev_info() error path
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:06:42 +00:00
David Woodhouse 0ac7266485 iommu/vt-d: use dmar_insert_dev_info() from dma_add_dev_info()
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:06:41 +00:00
David Woodhouse b718cd3d84 iommu/vt-d: Stop dmar_insert_dev_info() freeing domains on losing race
By moving this into get_domain_for_dev() we can make dmar_insert_dev_info()
suitable for use with "special" domains such as the si_domain, which
currently use domain_add_dev_info().

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:06:39 +00:00
David Woodhouse 64ae892bfe iommu/vt-d: Pass iommu to domain_context_mapping_one() and iommu_support_dev_iotlb()
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:06:37 +00:00
David Woodhouse 0bcb3e28c3 iommu/vt-d: Use struct device in device_domain_info, not struct pci_dev
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:06:36 +00:00
David Woodhouse 1525a29a7d iommu/vt-d: Make dmar_insert_dev_info() take struct device instead of struct pci_dev
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:06:34 +00:00
David Woodhouse 3d89194a94 iommu/vt-d: Make iommu_dummy() take struct device instead of struct pci_dev
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:06:33 +00:00
David Woodhouse 832bd85867 iommu/vt-d: Change scope lists to struct device, bus, devfn
It's not only for PCI devices any more, and the scope information for an
ACPI device provides the bus and devfn so that has to be stored here too.

It is the device pointer itself which needs to be protected with RCU,
so the __rcu annotation follows it into the definition of struct
dmar_dev_scope, since we're no longer just passing arrays of device
pointers around.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-24 14:05:08 +00:00
David Woodhouse d050196087 iommu/vt-d: Be less pessimistic about domain coherency where possible
In commit 2e12bc29 ("intel-iommu: Default to non-coherent for domains
unattached to iommus") we decided to err on the side of caution and
always assume that it's possible that a device will be attached which is
behind a non-coherent IOMMU.

In some cases, however, that just *cannot* happen. If there *are* no
IOMMUs in the system which are non-coherent, then we don't need to do
it. And flushing the dcache is a *significant* performance hit.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-19 17:25:48 +00:00
David Woodhouse 214e39aa36 iommu/vt-d: Honour intel_iommu=sp_off for non-VMM domains
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-19 17:22:13 +00:00
David Woodhouse ea8ea460c9 iommu/vt-d: Clean up and fix page table clear/free behaviour
There is a race condition between the existing clear/free code and the
hardware. The IOMMU is actually permitted to cache the intermediate
levels of the page tables, and doesn't need to walk the table from the
very top of the PGD each time. So the existing back-to-back calls to
dma_pte_clear_range() and dma_pte_free_pagetable() can lead to a
use-after-free where the IOMMU reads from a freed page table.

When freeing page tables we actually need to do the IOTLB flush, with
the 'invalidation hint' bit clear to indicate that it's not just a
leaf-node flush, after unlinking each page table page from the next level
up but before actually freeing it.

So in the rewritten domain_unmap() we just return a list of pages (using
pg->freelist to make a list of them), and then the caller is expected to
do the appropriate IOTLB flush (or tear down the domain completely,
whatever), before finally calling dma_free_pagelist() to free the pages.

As an added bonus, we no longer need to flush the CPU's data cache for
pages which are about to be *removed* from the page table hierarchy anyway,
in the non-cache-coherent case. This drastically improves the performance
of large unmaps.

As a side-effect of all these changes, this also fixes the fact that
intel_iommu_unmap() was neglecting to free the page tables for the range
in question after clearing them.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-19 17:21:41 +00:00
David Woodhouse 5cf0a76fa2 iommu/vt-d: Clean up size handling for intel_iommu_unmap()
We have this horrid API where iommu_unmap() can unmap more than it's asked
to, if the IOVA in question happens to be mapped with a large page.

Instead of propagating this nonsense to the point where we end up returning
the page order from dma_pte_clear_range(), let's just do it once and adjust
the 'size' parameter accordingly.

Augment pfn_to_dma_pte() to return the level at which the PTE was found,
which will also be useful later if we end up changing the API for
iommu_iova_to_phys() to behave the same way as is being discussed upstream.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2014-03-19 17:21:32 +00:00
Jiang Liu 75f05569d0 iommu/vt-d: Update IOMMU state when memory hotplug happens
If static identity domain is created, IOMMU driver needs to update
si_domain page table when memory hotplug event happens. Otherwise
PCI device DMA operations can't access the hot-added memory regions.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-03-04 17:51:06 +01:00
Jiang Liu 2e45528930 iommu/vt-d: Unify the way to process DMAR device scope array
Now we have a PCI bus notification based mechanism to update DMAR
device scope array, we could extend the mechanism to support boot
time initialization too, which will help to unify and simplify
the implementation.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-03-04 17:51:06 +01:00
Jiang Liu 59ce0515cd iommu/vt-d: Update DRHD/RMRR/ATSR device scope caches when PCI hotplug happens
Current Intel DMAR/IOMMU driver assumes that all PCI devices associated
with DMAR/RMRR/ATSR device scope arrays are created at boot time and
won't change at runtime, so it caches pointers of associated PCI device
object. That assumption may be wrong now due to:
1) introduction of PCI host bridge hotplug
2) PCI device hotplug through sysfs interfaces.

Wang Yijing has tried to solve this issue by caching <bus, dev, func>
tupple instead of the PCI device object pointer, but that's still
unreliable because PCI bus number may change in case of hotplug.
Please refer to http://lkml.org/lkml/2013/11/5/64
Message from Yingjing's mail:
after remove and rescan a pci device
[  611.857095] dmar: DRHD: handling fault status reg 2
[  611.857109] dmar: DMAR:[DMA Read] Request device [86:00.3] fault addr ffff7000
[  611.857109] DMAR:[fault reason 02] Present bit in context entry is clear
[  611.857524] dmar: DRHD: handling fault status reg 102
[  611.857534] dmar: DMAR:[DMA Read] Request device [86:00.3] fault addr ffff6000
[  611.857534] DMAR:[fault reason 02] Present bit in context entry is clear
[  611.857936] dmar: DRHD: handling fault status reg 202
[  611.857947] dmar: DMAR:[DMA Read] Request device [86:00.3] fault addr ffff5000
[  611.857947] DMAR:[fault reason 02] Present bit in context entry is clear
[  611.858351] dmar: DRHD: handling fault status reg 302
[  611.858362] dmar: DMAR:[DMA Read] Request device [86:00.3] fault addr ffff4000
[  611.858362] DMAR:[fault reason 02] Present bit in context entry is clear
[  611.860819] IPv6: ADDRCONF(NETDEV_UP): eth3: link is not ready
[  611.860983] dmar: DRHD: handling fault status reg 402
[  611.860995] dmar: INTR-REMAP: Request device [[86:00.3] fault index a4
[  611.860995] INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear

This patch introduces a new mechanism to update the DRHD/RMRR/ATSR device scope
caches by hooking PCI bus notification.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-03-04 17:51:06 +01:00
Jiang Liu 0e242612d9 iommu/vt-d: Use RCU to protect global resources in interrupt context
Global DMA and interrupt remapping resources may be accessed in
interrupt context, so use RCU instead of rwsem to protect them
in such cases.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-03-04 17:51:05 +01:00
Jiang Liu 3a5670e8ac iommu/vt-d: Introduce a rwsem to protect global data structures
Introduce a global rwsem dmar_global_lock, which will be used to
protect DMAR related global data structures from DMAR/PCI/memory
device hotplug operations in process context.

DMA and interrupt remapping related data structures are read most,
and only change when memory/PCI/DMAR hotplug event happens.
So a global rwsem solution is adopted for balance between simplicity
and performance.

For interrupt remapping driver, function intel_irq_remapping_supported(),
dmar_table_init(), intel_enable_irq_remapping(), disable_irq_remapping(),
reenable_irq_remapping() and enable_drhd_fault_handling() etc
are called during booting, suspending and resuming with interrupt
disabled, so no need to take the global lock.

For interrupt remapping entry allocation, the locking model is:
	down_read(&dmar_global_lock);
	/* Find corresponding iommu */
	iommu = map_hpet_to_ir(id);
	if (iommu)
		/*
		 * Allocate remapping entry and mark entry busy,
		 * the IOMMU won't be hot-removed until the
		 * allocated entry has been released.
		 */
		index = alloc_irte(iommu, irq, 1);
	up_read(&dmar_global_lock);

For DMA remmaping driver, we only uses the dmar_global_lock rwsem to
protect functions which are only called in process context. For any
function which may be called in interrupt context, we will use RCU
to protect them in following patches.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-03-04 17:51:05 +01:00
Jiang Liu b683b230a2 iommu/vt-d: Introduce macro for_each_dev_scope() to walk device scope entries
Introduce for_each_dev_scope()/for_each_active_dev_scope() to walk
{active} device scope entries. This will help following RCU lock
related patches.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-03-04 17:51:04 +01:00
Jiang Liu b5f82ddf22 iommu/vt-d: Fix error in detect ATS capability
Current Intel IOMMU driver only matches a PCIe root port with the first
DRHD unit with the samge segment number. It will report false result
if there are multiple DRHD units with the same segment number, thus fail
to detect ATS capability for some PCIe devices.

This patch refines function dmar_find_matched_atsr_unit() to search all
DRHD units with the same segment number.

An example DMAR table entries as below:
[1D0h 0464  2]                Subtable Type : 0002 <Root Port ATS Capability>
[1D2h 0466  2]                       Length : 0028
[1D4h 0468  1]                        Flags : 00
[1D5h 0469  1]                     Reserved : 00
[1D6h 0470  2]           PCI Segment Number : 0000

[1D8h 0472  1]      Device Scope Entry Type : 02
[1D9h 0473  1]                 Entry Length : 08
[1DAh 0474  2]                     Reserved : 0000
[1DCh 0476  1]               Enumeration ID : 00
[1DDh 0477  1]               PCI Bus Number : 00
[1DEh 0478  2]                     PCI Path : [02, 00]

[1E0h 0480  1]      Device Scope Entry Type : 02
[1E1h 0481  1]                 Entry Length : 08
[1E2h 0482  2]                     Reserved : 0000
[1E4h 0484  1]               Enumeration ID : 00
[1E5h 0485  1]               PCI Bus Number : 00
[1E6h 0486  2]                     PCI Path : [03, 00]

[1E8h 0488  1]      Device Scope Entry Type : 02
[1E9h 0489  1]                 Entry Length : 08
[1EAh 0490  2]                     Reserved : 0000
[1ECh 0492  1]               Enumeration ID : 00
[1EDh 0493  1]               PCI Bus Number : 00
[1EEh 0494  2]                     PCI Path : [03, 02]

[1F0h 0496  1]      Device Scope Entry Type : 02
[1F1h 0497  1]                 Entry Length : 08
[1F2h 0498  2]                     Reserved : 0000
[1F4h 0500  1]               Enumeration ID : 00
[1F5h 0501  1]               PCI Bus Number : 00
[1F6h 0502  2]                     PCI Path : [03, 03]

[1F8h 0504  2]                Subtable Type : 0002 <Root Port ATS Capability>
[1FAh 0506  2]                       Length : 0020
[1FCh 0508  1]                        Flags : 00
[1FDh 0509  1]                     Reserved : 00
[1FEh 0510  2]           PCI Segment Number : 0000

[200h 0512  1]      Device Scope Entry Type : 02
[201h 0513  1]                 Entry Length : 08
[202h 0514  2]                     Reserved : 0000
[204h 0516  1]               Enumeration ID : 00
[205h 0517  1]               PCI Bus Number : 40
[206h 0518  2]                     PCI Path : [02, 00]

[208h 0520  1]      Device Scope Entry Type : 02
[209h 0521  1]                 Entry Length : 08
[20Ah 0522  2]                     Reserved : 0000
[20Ch 0524  1]               Enumeration ID : 00
[20Dh 0525  1]               PCI Bus Number : 40
[20Eh 0526  2]                     PCI Path : [02, 02]

[210h 0528  1]      Device Scope Entry Type : 02
[211h 0529  1]                 Entry Length : 08
[212h 0530  2]                     Reserved : 0000
[214h 0532  1]               Enumeration ID : 00
[215h 0533  1]               PCI Bus Number : 40
[216h 0534  2]                     PCI Path : [03, 00]

[218h 0536  2]                Subtable Type : 0002 <Root Port ATS Capability>
[21Ah 0538  2]                       Length : 0020
[21Ch 0540  1]                        Flags : 00
[21Dh 0541  1]                     Reserved : 00
[21Eh 0542  2]           PCI Segment Number : 0000

[220h 0544  1]      Device Scope Entry Type : 02
[221h 0545  1]                 Entry Length : 08
[222h 0546  2]                     Reserved : 0000
[224h 0548  1]               Enumeration ID : 00
[225h 0549  1]               PCI Bus Number : 80
[226h 0550  2]                     PCI Path : [02, 00]

[228h 0552  1]      Device Scope Entry Type : 02
[229h 0553  1]                 Entry Length : 08
[22Ah 0554  2]                     Reserved : 0000
[22Ch 0556  1]               Enumeration ID : 00
[22Dh 0557  1]               PCI Bus Number : 80
[22Eh 0558  2]                     PCI Path : [02, 02]

[230h 0560  1]      Device Scope Entry Type : 02
[231h 0561  1]                 Entry Length : 08
[232h 0562  2]                     Reserved : 0000
[234h 0564  1]               Enumeration ID : 00
[235h 0565  1]               PCI Bus Number : 80
[236h 0566  2]                     PCI Path : [03, 00]

[238h 0568  2]                Subtable Type : 0002 <Root Port ATS Capability>
[23Ah 0570  2]                       Length : 0020
[23Ch 0572  1]                        Flags : 00
[23Dh 0573  1]                     Reserved : 00
[23Eh 0574  2]           PCI Segment Number : 0000

[240h 0576  1]      Device Scope Entry Type : 02
[241h 0577  1]                 Entry Length : 08
[242h 0578  2]                     Reserved : 0000
[244h 0580  1]               Enumeration ID : 00
[245h 0581  1]               PCI Bus Number : C0
[246h 0582  2]                     PCI Path : [02, 00]

[248h 0584  1]      Device Scope Entry Type : 02
[249h 0585  1]                 Entry Length : 08
[24Ah 0586  2]                     Reserved : 0000
[24Ch 0588  1]               Enumeration ID : 00
[24Dh 0589  1]               PCI Bus Number : C0
[24Eh 0590  2]                     PCI Path : [02, 02]

[250h 0592  1]      Device Scope Entry Type : 02
[251h 0593  1]                 Entry Length : 08
[252h 0594  2]                     Reserved : 0000
[254h 0596  1]               Enumeration ID : 00
[255h 0597  1]               PCI Bus Number : C0
[256h 0598  2]                     PCI Path : [03, 00]

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-03-04 17:51:04 +01:00
Jiang Liu a4eaa86c0c iommu/vt-d: Check for NULL pointer when freeing IOMMU data structure
Domain id 0 will be assigned to invalid translation without allocating
domain data structure if DMAR unit supports caching mode. So in function
free_dmar_iommu(), we should check whether the domain pointer is NULL,
otherwise it will cause system crash as below:
[    6.790519] BUG: unable to handle kernel NULL pointer dereference at 00000000000000c8
[    6.799520] IP: [<ffffffff810e2dc8>] __lock_acquire+0x11f8/0x1430
[    6.806493] PGD 0
[    6.817972] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[    6.823303] Modules linked in:
[    6.826862] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.14.0-rc1+ #126
[    6.834252] Hardware name: Intel Corporation BRICKLAND/BRICKLAND, BIOS BRIVTIN1.86B.0047.R00.1402050741 02/05/2014
[    6.845951] task: ffff880455a80000 ti: ffff880455a88000 task.ti: ffff880455a88000
[    6.854437] RIP: 0010:[<ffffffff810e2dc8>]  [<ffffffff810e2dc8>] __lock_acquire+0x11f8/0x1430
[    6.864154] RSP: 0000:ffff880455a89ce0  EFLAGS: 00010046
[    6.870179] RAX: 0000000000000046 RBX: 0000000000000002 RCX: 0000000000000000
[    6.878249] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 00000000000000c8
[    6.886318] RBP: ffff880455a89d40 R08: 0000000000000002 R09: 0000000000000001
[    6.894387] R10: 0000000000000000 R11: 0000000000000001 R12: ffff880455a80000
[    6.902458] R13: 0000000000000000 R14: 00000000000000c8 R15: 0000000000000000
[    6.910520] FS:  0000000000000000(0000) GS:ffff88045b800000(0000) knlGS:0000000000000000
[    6.919687] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    6.926198] CR2: 00000000000000c8 CR3: 0000000001e0e000 CR4: 00000000001407f0
[    6.934269] Stack:
[    6.936588]  ffffffffffffff10 ffffffff810f59db 0000000000000010 0000000000000246
[    6.945219]  ffff880455a89d10 0000000000000000 ffffffff82bcb980 0000000000000046
[    6.953850]  0000000000000000 0000000000000000 0000000000000002 0000000000000000
[    6.962482] Call Trace:
[    6.965300]  [<ffffffff810f59db>] ? vprintk_emit+0x4fb/0x5a0
[    6.971716]  [<ffffffff810e3185>] lock_acquire+0x185/0x200
[    6.977941]  [<ffffffff821fbbee>] ? init_dmars+0x839/0xa1d
[    6.984167]  [<ffffffff81870b06>] _raw_spin_lock_irqsave+0x56/0x90
[    6.991158]  [<ffffffff821fbbee>] ? init_dmars+0x839/0xa1d
[    6.997380]  [<ffffffff821fbbee>] init_dmars+0x839/0xa1d
[    7.003410]  [<ffffffff8147d575>] ? pci_get_dev_by_id+0x75/0xd0
[    7.010119]  [<ffffffff821fc146>] intel_iommu_init+0x2f0/0x502
[    7.016735]  [<ffffffff821a7947>] ? iommu_setup+0x27d/0x27d
[    7.023056]  [<ffffffff821a796f>] pci_iommu_init+0x28/0x52
[    7.029282]  [<ffffffff81002162>] do_one_initcall+0xf2/0x220
[    7.035702]  [<ffffffff810a4a29>] ? parse_args+0x2c9/0x450
[    7.041919]  [<ffffffff8219d1b1>] kernel_init_freeable+0x1c9/0x25b
[    7.048919]  [<ffffffff8219c8d2>] ? do_early_param+0x8a/0x8a
[    7.055336]  [<ffffffff8184d3f0>] ? rest_init+0x150/0x150
[    7.061461]  [<ffffffff8184d3fe>] kernel_init+0xe/0x100
[    7.067393]  [<ffffffff8187b5fc>] ret_from_fork+0x7c/0xb0
[    7.073518]  [<ffffffff8184d3f0>] ? rest_init+0x150/0x150
[    7.079642] Code: 01 76 18 89 05 46 04 36 01 41 be 01 00 00 00 e9 2f 02 00 00 0f 1f 80 00 00 00 00 41 be 01 00 00 00 e9 1d 02 00 00 0f 1f 44 00 00 <49> 81 3e c0 31 34 82 b8 01 00 00 00 0f 44 d8 41 83 ff 01 0f 87
[    7.104944] RIP  [<ffffffff810e2dc8>] __lock_acquire+0x11f8/0x1430
[    7.112008]  RSP <ffff880455a89ce0>
[    7.115988] CR2: 00000000000000c8
[    7.119784] ---[ end trace 13d756f0f462c538 ]---
[    7.125034] note: swapper/0[1] exited with preempt_count 1
[    7.131285] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009
[    7.131285]

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-03-04 17:51:03 +01:00
Jiang Liu 9ebd682e5a iommu/vt-d: Fix incorrect iommu_count for si_domain
The iommu_count field in si_domain(static identity domain) is
initialized to zero and never increases. It will underflow
when tearing down iommu unit in function free_dmar_iommu()
and leak memory. So refine code to correctly manage
si_domain->iommu_count.

Warning message caused by si_domain memory leak:
[   14.609681] IOMMU: Setting RMRR:
[   14.613496] Ignoring identity map for HW passthrough device 0000:00:1a.0 [0xbdcfd000 - 0xbdd1dfff]
[   14.623809] Ignoring identity map for HW passthrough device 0000:00:1d.0 [0xbdcfd000 - 0xbdd1dfff]
[   14.634162] IOMMU: Prepare 0-16MiB unity mapping for LPC
[   14.640329] Ignoring identity map for HW passthrough device 0000:00:1f.0 [0x0 - 0xffffff]
[   14.673360] IOMMU: dmar init failed
[   14.678157] kmem_cache_destroy iommu_devinfo: Slab cache still has objects
[   14.686076] CPU: 12 PID: 1 Comm: swapper/0 Not tainted 3.13.0-rc1-gerry+ #59
[   14.694176] Hardware name: Intel Corporation LH Pass ........../SVRBD-ROW_T, BIOS SE5C600.86B.99.99.x059.091020121352 09/10/2012
[   14.707412]  0000000000000000 ffff88042dd33db0 ffffffff8156223d ffff880c2cc37c00
[   14.716407]  ffff88042dd33dc8 ffffffff811790b1 ffff880c2d3533b8 ffff88042dd33e00
[   14.725468]  ffffffff81dc7a6a ffffffff81b1e8e0 ffffffff81f84058 ffffffff81d8a711
[   14.734464] Call Trace:
[   14.737453]  [<ffffffff8156223d>] dump_stack+0x4d/0x66
[   14.743430]  [<ffffffff811790b1>] kmem_cache_destroy+0xf1/0x100
[   14.750279]  [<ffffffff81dc7a6a>] intel_iommu_init+0x122/0x56a
[   14.757035]  [<ffffffff81d8a711>] ? iommu_setup+0x27d/0x27d
[   14.763491]  [<ffffffff81d8a739>] pci_iommu_init+0x28/0x52
[   14.769846]  [<ffffffff81000342>] do_one_initcall+0x122/0x180
[   14.776506]  [<ffffffff81077738>] ? parse_args+0x1e8/0x320
[   14.782866]  [<ffffffff81d850e8>] kernel_init_freeable+0x1e1/0x26c
[   14.789994]  [<ffffffff81d84833>] ? do_early_param+0x88/0x88
[   14.796556]  [<ffffffff8154ffc0>] ? rest_init+0xd0/0xd0
[   14.802626]  [<ffffffff8154ffce>] kernel_init+0xe/0x130
[   14.808698]  [<ffffffff815756ac>] ret_from_fork+0x7c/0xb0
[   14.814963]  [<ffffffff8154ffc0>] ? rest_init+0xd0/0xd0
[   14.821640] kmem_cache_destroy iommu_domain: Slab cache still has objects
[   14.829456] CPU: 12 PID: 1 Comm: swapper/0 Not tainted 3.13.0-rc1-gerry+ #59
[   14.837562] Hardware name: Intel Corporation LH Pass ........../SVRBD-ROW_T, BIOS SE5C600.86B.99.99.x059.091020121352 09/10/2012
[   14.850803]  0000000000000000 ffff88042dd33db0 ffffffff8156223d ffff88102c1ee3c0
[   14.861222]  ffff88042dd33dc8 ffffffff811790b1 ffff880c2d3533b8 ffff88042dd33e00
[   14.870284]  ffffffff81dc7a76 ffffffff81b1e8e0 ffffffff81f84058 ffffffff81d8a711
[   14.879271] Call Trace:
[   14.882227]  [<ffffffff8156223d>] dump_stack+0x4d/0x66
[   14.888197]  [<ffffffff811790b1>] kmem_cache_destroy+0xf1/0x100
[   14.895034]  [<ffffffff81dc7a76>] intel_iommu_init+0x12e/0x56a
[   14.901781]  [<ffffffff81d8a711>] ? iommu_setup+0x27d/0x27d
[   14.908238]  [<ffffffff81d8a739>] pci_iommu_init+0x28/0x52
[   14.914594]  [<ffffffff81000342>] do_one_initcall+0x122/0x180
[   14.921244]  [<ffffffff81077738>] ? parse_args+0x1e8/0x320
[   14.927598]  [<ffffffff81d850e8>] kernel_init_freeable+0x1e1/0x26c
[   14.934738]  [<ffffffff81d84833>] ? do_early_param+0x88/0x88
[   14.941309]  [<ffffffff8154ffc0>] ? rest_init+0xd0/0xd0
[   14.947380]  [<ffffffff8154ffce>] kernel_init+0xe/0x130
[   14.953430]  [<ffffffff815756ac>] ret_from_fork+0x7c/0xb0
[   14.959689]  [<ffffffff8154ffc0>] ? rest_init+0xd0/0xd0
[   14.966299] kmem_cache_destroy iommu_iova: Slab cache still has objects
[   14.973923] CPU: 12 PID: 1 Comm: swapper/0 Not tainted 3.13.0-rc1-gerry+ #59
[   14.982020] Hardware name: Intel Corporation LH Pass ........../SVRBD-ROW_T, BIOS SE5C600.86B.99.99.x059.091020121352 09/10/2012
[   14.995263]  0000000000000000 ffff88042dd33db0 ffffffff8156223d ffff88042cb5c980
[   15.004265]  ffff88042dd33dc8 ffffffff811790b1 ffff880c2d3533b8 ffff88042dd33e00
[   15.013322]  ffffffff81dc7a82 ffffffff81b1e8e0 ffffffff81f84058 ffffffff81d8a711
[   15.022318] Call Trace:
[   15.025238]  [<ffffffff8156223d>] dump_stack+0x4d/0x66
[   15.031202]  [<ffffffff811790b1>] kmem_cache_destroy+0xf1/0x100
[   15.038038]  [<ffffffff81dc7a82>] intel_iommu_init+0x13a/0x56a
[   15.044786]  [<ffffffff81d8a711>] ? iommu_setup+0x27d/0x27d
[   15.051242]  [<ffffffff81d8a739>] pci_iommu_init+0x28/0x52
[   15.057601]  [<ffffffff81000342>] do_one_initcall+0x122/0x180
[   15.064254]  [<ffffffff81077738>] ? parse_args+0x1e8/0x320
[   15.070608]  [<ffffffff81d850e8>] kernel_init_freeable+0x1e1/0x26c
[   15.077747]  [<ffffffff81d84833>] ? do_early_param+0x88/0x88
[   15.084300]  [<ffffffff8154ffc0>] ? rest_init+0xd0/0xd0
[   15.090362]  [<ffffffff8154ffce>] kernel_init+0xe/0x130
[   15.096431]  [<ffffffff815756ac>] ret_from_fork+0x7c/0xb0
[   15.102693]  [<ffffffff8154ffc0>] ? rest_init+0xd0/0xd0
[   15.189273] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-03-04 17:51:02 +01:00
Jiang Liu 92d03cc8d0 iommu/vt-d: Reduce duplicated code to handle virtual machine domains
Reduce duplicated code to handle virtual machine domains, there's no
functionality changes. It also improves code readability.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-03-04 17:51:01 +01:00
Jiang Liu e85bb5d4d1 iommu/vt-d: Free resources if failed to create domain for PCIe endpoint
Enhance function get_domain_for_dev() to release allocated resources
if failed to create domain for PCIe endpoint, otherwise the allocated
resources will get lost.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-03-04 17:51:01 +01:00
Jiang Liu 745f2586e7 iommu/vt-d: Simplify function get_domain_for_dev()
Function get_domain_for_dev() is a little complex, simplify it
by factoring out dmar_search_domain_by_dev_info() and
dmar_insert_dev_info().

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-03-04 17:51:01 +01:00
Jiang Liu b94e4117f8 iommu/vt-d: Move private structures and variables into intel-iommu.c
Move private structures and variables into intel-iommu.c, which will
help to simplify locking policy for hotplug. Also delete redundant
declarations.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-03-04 17:51:00 +01:00
Jiang Liu 7e7dfab71a iommu/vt-d: Avoid caching stale domain_device_info when hot-removing PCI device
Function device_notifier() in intel-iommu.c only remove domain_device_info
data structure associated with a PCI device when handling PCI device
driver unbinding events. If a PCI device has never been bound to a PCI
device driver, there won't be BUS_NOTIFY_UNBOUND_DRIVER event when
hot-removing the PCI device. So associated domain_device_info data
structure may get lost.

On the other hand, if iommu_pass_through is enabled, function
iommu_prepare_static_indentify_mapping() will create domain_device_info
data structure for each PCIe to PCIe bridge and PCIe endpoint,
no matter whether there are drivers associated with those PCIe devices
or not. So those domain_device_info data structures will get lost when
hot-removing the assocated PCIe devices if they have never bound to
any PCI device driver.

To be even worse, it's not only an memory leak issue, but also an
caching of stale information bug because the memory are kept in
device_domain_list and domain->devices lists.

Fix the bug by trying to remove domain_device_info data structure when
handling BUS_NOTIFY_DEL_DEVICE event.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-03-04 17:51:00 +01:00
Jiang Liu 816997d03b iommu/vt-d: Avoid caching stale domain_device_info and fix memory leak
Function device_notifier() in intel-iommu.c fails to remove
device_domain_info data structures for PCI devices if they are
associated with si_domain because iommu_no_mapping() returns true
for those PCI devices. This will cause memory leak and caching of
stale information in domain->devices list.

So fix the issue by not calling iommu_no_mapping() and skipping check
of iommu_pass_through.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-03-04 17:50:59 +01:00
Jiang Liu 989d51fc99 iommu/vt-d: Avoid double free of g_iommus on error recovery path
Array 'g_iommus' may be freed twice on error recovery path in function
init_dmars() and free_dmar_iommu(), thus cause random system crash as
below.

[    6.774301] IOMMU: dmar init failed
[    6.778310] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[    6.785615] software IO TLB [mem 0x76bcf000-0x7abcf000] (64MB) mapped at [ffff880076bcf000-ffff88007abcefff]
[    6.796887] general protection fault: 0000 [#1] SMP DEBUG_PAGEALLOC
[    6.804173] Modules linked in:
[    6.807731] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.14.0-rc1+ #108
[    6.815122] Hardware name: Intel Corporation BRICKLAND/BRICKLAND, BIOS BRIVTIN1.86B.0047.R00.1402050741 02/05/2014
[    6.836000] task: ffff880455a80000 ti: ffff880455a88000 task.ti: ffff880455a88000
[    6.844487] RIP: 0010:[<ffffffff8143eea6>]  [<ffffffff8143eea6>] memcpy+0x6/0x110
[    6.853039] RSP: 0000:ffff880455a89cc8  EFLAGS: 00010293
[    6.859064] RAX: ffff006568636163 RBX: ffff00656863616a RCX: 0000000000000005
[    6.867134] RDX: 0000000000000005 RSI: ffffffff81cdc439 RDI: ffff006568636163
[    6.875205] RBP: ffff880455a89d30 R08: 000000000001bc3b R09: 0000000000000000
[    6.883275] R10: 0000000000000000 R11: ffffffff81cdc43e R12: ffff880455a89da8
[    6.891338] R13: ffff006568636163 R14: 0000000000000005 R15: ffffffff81cdc439
[    6.899408] FS:  0000000000000000(0000) GS:ffff88045b800000(0000) knlGS:0000000000000000
[    6.908575] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    6.915088] CR2: ffff88047e1ff000 CR3: 0000000001e0e000 CR4: 00000000001407f0
[    6.923160] Stack:
[    6.925487]  ffffffff8143c904 ffff88045b407e00 ffff006568636163 ffff006568636163
[    6.934113]  ffffffff8120a1a9 ffffffff81cdc43e 0000000000000007 0000000000000000
[    6.942747]  ffff880455a89da8 ffff006568636163 0000000000000007 ffffffff81cdc439
[    6.951382] Call Trace:
[    6.954197]  [<ffffffff8143c904>] ? vsnprintf+0x124/0x6f0
[    6.960323]  [<ffffffff8120a1a9>] ? __kmalloc_track_caller+0x169/0x360
[    6.967716]  [<ffffffff81440e1b>] kvasprintf+0x6b/0x80
[    6.973552]  [<ffffffff81432bf1>] kobject_set_name_vargs+0x21/0x70
[    6.980552]  [<ffffffff8143393d>] kobject_init_and_add+0x4d/0x90
[    6.987364]  [<ffffffff812067c9>] ? __kmalloc+0x169/0x370
[    6.993492]  [<ffffffff8102dbbc>] ? cache_add_dev+0x17c/0x4f0
[    7.000005]  [<ffffffff8102ddfa>] cache_add_dev+0x3ba/0x4f0
[    7.006327]  [<ffffffff821a87ca>] ? i8237A_init_ops+0x14/0x14
[    7.012842]  [<ffffffff821a87f8>] cache_sysfs_init+0x2e/0x61
[    7.019260]  [<ffffffff81002162>] do_one_initcall+0xf2/0x220
[    7.025679]  [<ffffffff810a4a29>] ? parse_args+0x2c9/0x450
[    7.031903]  [<ffffffff8219d1b1>] kernel_init_freeable+0x1c9/0x25b
[    7.038904]  [<ffffffff8219c8d2>] ? do_early_param+0x8a/0x8a
[    7.045322]  [<ffffffff8184d5e0>] ? rest_init+0x150/0x150
[    7.051447]  [<ffffffff8184d5ee>] kernel_init+0xe/0x100
[    7.057380]  [<ffffffff8187b87c>] ret_from_fork+0x7c/0xb0
[    7.063503]  [<ffffffff8184d5e0>] ? rest_init+0x150/0x150
[    7.069628] Code: 89 e5 53 48 89 fb 75 16 80 7f 3c 00 75 05 e8 d2 f9 ff ff 48 8b 43 58 48 2b 43 50 88 43 4e 5b 5d c3 90 90 90 90 48 89 f8 48 89 d1 <f3> a4 c3 03 83 e2 07 f3 48 a5 89 d1 f3 a4 c3 20 4c 8b 06 4c 8b
[    7.094960] RIP  [<ffffffff8143eea6>] memcpy+0x6/0x110
[    7.100856]  RSP <ffff880455a89cc8>
[    7.104864] ---[ end trace b5d3fdc6c6c28083 ]---
[    7.110142] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[    7.110142]
[    7.120540] Kernel Offset: 0x0 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffff9fffffff)

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-03-04 17:50:59 +01:00
Linus Torvalds b3a4bcaa5a IOMMU Updates for Linux v3.14
A few patches have been queued up for this merge window:
 
 	* Improvements for the ARM-SMMU driver
 	  (IOMMU_EXEC support, IOMMU group support)
 	* Updates and fixes for the shmobile IOMMU driver
 	* Various fixes to generic IOMMU code and the
 	  Intel IOMMU driver
 	* Some cleanups in IOMMU drivers (dev_is_pci() usage)
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1.4.11 (GNU/Linux)
 
 iQIcBAABAgAGBQJS6XrCAAoJECvwRC2XARrjgy4P/itemtg2U+603Ldje8WcPo0E
 OCO/0VVSmCTYKUJDZY0hiVwqmhe5gFL3Hm/gGwkS0+UJenFXMmi+aVaPp4pCpgH+
 dL2HD3dIEvi14bisrdxG/8MdR6mIx0qzKtnZLkKSR4LXwucyLHvC/DaCoOytb7Yk
 7s+eEuo0hj0jAkiqSG/zLEtKElTEnoAAkLOjMy46orecJ5q4HusPZekLtWZs2ETe
 x3NS63Unb9g1iSQJWIA7HnQlxWIr2+iynoamHHJRiVFzqRF0W0sGvQY3auG0DSCn
 70WRNE1rKfEkfXMJxosRQ4394YUQdAkt8MBENNcJcC6E1n5PBi0cEZXH6mCnEIlG
 jXzIKUY9fz68ZboaqIxXv4Hb+JLlPXCvPBvQzIQiKRgVxd8nncEjn5I9MHdf+je5
 BmJlzJLJvP4cFvW8Hc8k2Oq101b1kEcSCLARWWvE9/bk9xIUyrqBkR4XjC0vb6qq
 1HbKVdZ7KFKCkBHy9xMpr7CUjKiDiiLeUmqlhyjcK9spicuNIZQnC11HemL6/USP
 oR6Ext9RGhvz+ch656+5+L6f6FURVP8/ywKiJ3RjmvXV5/fCYo3WMitOB2qzlWCy
 SYXAczAOMOdOo+1Dxbghrr+7HzUWPqgfPmntZEPGMZhfuZ6xXr+7pGLjAhHb4vcR
 SZxqkDo1cprqrR9KFAWC
 =YKLk
 -----END PGP SIGNATURE-----

Merge tag 'iommu-updates-v3.14' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu

Pull IOMMU Updates from Joerg Roedel:
 "A few patches have been queued up for this merge window:

   - improvements for the ARM-SMMU driver (IOMMU_EXEC support, IOMMU
     group support)
   - updates and fixes for the shmobile IOMMU driver
   - various fixes to generic IOMMU code and the Intel IOMMU driver
   - some cleanups in IOMMU drivers (dev_is_pci() usage)"

* tag 'iommu-updates-v3.14' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (36 commits)
  iommu/vt-d: Fix signedness bug in alloc_irte()
  iommu/vt-d: free all resources if failed to initialize DMARs
  iommu/vt-d, trivial: clean sparse warnings
  iommu/vt-d: fix wrong return value of dmar_table_init()
  iommu/vt-d: release invalidation queue when destroying IOMMU unit
  iommu/vt-d: fix access after free issue in function free_dmar_iommu()
  iommu/vt-d: keep shared resources when failed to initialize iommu devices
  iommu/vt-d: fix invalid memory access when freeing DMAR irq
  iommu/vt-d, trivial: simplify code with existing macros
  iommu/vt-d, trivial: use defined macro instead of hardcoding
  iommu/vt-d: mark internal functions as static
  iommu/vt-d, trivial: clean up unused code
  iommu/vt-d, trivial: check suitable flag in function detect_intel_iommu()
  iommu/vt-d, trivial: print correct domain id of static identity domain
  iommu/vt-d, trivial: refine support of 64bit guest address
  iommu/vt-d: fix resource leakage on error recovery path in iommu_init_domains()
  iommu/vt-d: fix a race window in allocating domain ID for virtual machines
  iommu/vt-d: fix PCI device reference leakage on error recovery path
  drm/msm: Fix link error with !MSM_IOMMU
  iommu/vt-d: use dedicated bitmap to track remapping entry allocation status
  ...
2014-01-29 20:00:13 -08:00
Alex Williamson 08336fd218 intel-iommu: fix off-by-one in pagetable freeing
dma_pte_free_level() has an off-by-one error when checking whether a pte
is completely covered by a range.  Take for example the case of
attempting to free pfn 0x0 - 0x1ff, ie.  512 entries covering the first
2M superpage.

The level_size() is 0x200 and we test:

  static void dma_pte_free_level(...
	...

	if (!(0 > 0 || 0x1ff < 0 + 0x200)) {
		...
	}

Clearly the 2nd test is true, which means we fail to take the branch to
clear and free the pagetable entry.  As a result, we're leaking
pagetables and failing to install new pages over the range.

This was found with a PCI device assigned to a QEMU guest using vfio-pci
without a VGA device present.  The first 1M of guest address space is
mapped with various combinations of 4K pages, but eventually the range
is entirely freed and replaced with a 2M contiguous mapping.
intel-iommu errors out with something like:

  ERROR: DMA PTE for vPFN 0x0 already set (to 5c2b8003 not 849c00083)

In this case 5c2b8003 is the pointer to the previous leaf page that was
neither freed nor cleared and 849c00083 is the superpage entry that
we're trying to replace it with.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2014-01-21 16:19:41 -08:00
Jiang Liu 9bdc531ec6 iommu/vt-d: free all resources if failed to initialize DMARs
Enhance intel_iommu_init() to free all resources if failed to
initialize DMAR hardware.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-01-09 12:44:30 +01:00
Jiang Liu b707cb027e iommu/vt-d, trivial: clean sparse warnings
Clean up most sparse warnings in Intel DMA and interrupt remapping
drivers.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-01-09 12:44:16 +01:00
Jiang Liu 5ced12af69 iommu/vt-d: fix access after free issue in function free_dmar_iommu()
Function free_dmar_iommu() may access domain->iommu_lock by
	spin_unlock_irqrestore(&domain->iommu_lock, flags);
after freeing corresponding domain structure.

Sample stack dump:
[    8.912818] =========================
[    8.917072] [ BUG: held lock freed! ]
[    8.921335] 3.13.0-rc1-gerry+ #12 Not tainted
[    8.926375] -------------------------
[    8.930629] swapper/0/1 is freeing memory ffff880c23b56040-ffff880c23b5613f, with a lock still held there!
[    8.941675]  (&(&domain->iommu_lock)->rlock){......}, at: [<ffffffff81dc775c>] init_dmars+0x72c/0x95b
[    8.952582] 1 lock held by swapper/0/1:
[    8.957031]  #0:  (&(&domain->iommu_lock)->rlock){......}, at: [<ffffffff81dc775c>] init_dmars+0x72c/0x95b
[    8.968487]
[    8.968487] stack backtrace:
[    8.973602] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.13.0-rc1-gerry+ #12
[    8.981556] Hardware name: Intel Corporation LH Pass ........../SVRBD-ROW_T, BIOS SE5C600.86B.99.99.x059.091020121352 09/10/2012
[    8.994742]  ffff880c23b56040 ffff88042dd33c98 ffffffff815617fd ffff88042dd38b28
[    9.003566]  ffff88042dd33cd0 ffffffff810a977a ffff880c23b56040 0000000000000086
[    9.012403]  ffff88102c4923c0 ffff88042ddb4800 ffffffff81b1e8c0 ffff88042dd33d28
[    9.021240] Call Trace:
[    9.024138]  [<ffffffff815617fd>] dump_stack+0x4d/0x66
[    9.030057]  [<ffffffff810a977a>] debug_check_no_locks_freed+0x15a/0x160
[    9.037723]  [<ffffffff811aa1c2>] kmem_cache_free+0x62/0x5b0
[    9.044225]  [<ffffffff81465e27>] domain_exit+0x197/0x1c0
[    9.050418]  [<ffffffff81dc7788>] init_dmars+0x758/0x95b
[    9.056527]  [<ffffffff81dc7dfa>] intel_iommu_init+0x351/0x438
[    9.063207]  [<ffffffff81d8a711>] ? iommu_setup+0x27d/0x27d
[    9.069601]  [<ffffffff81d8a739>] pci_iommu_init+0x28/0x52
[    9.075910]  [<ffffffff81000342>] do_one_initcall+0x122/0x180
[    9.082509]  [<ffffffff81077738>] ? parse_args+0x1e8/0x320
[    9.088815]  [<ffffffff81d850e8>] kernel_init_freeable+0x1e1/0x26c
[    9.095895]  [<ffffffff81d84833>] ? do_early_param+0x88/0x88
[    9.102396]  [<ffffffff8154f580>] ? rest_init+0xd0/0xd0
[    9.108410]  [<ffffffff8154f58e>] kernel_init+0xe/0x130
[    9.114423]  [<ffffffff81574a2c>] ret_from_fork+0x7c/0xb0
[    9.120612]  [<ffffffff8154f580>] ? rest_init+0xd0/0xd0

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-01-09 12:43:42 +01:00
Jiang Liu a868e6b7b6 iommu/vt-d: keep shared resources when failed to initialize iommu devices
Data structure drhd->iommu is shared between DMA remapping driver and
interrupt remapping driver, so DMA remapping driver shouldn't release
drhd->iommu when it failed to initialize IOMMU devices. Otherwise it
may cause invalid memory access to the interrupt remapping driver.

Sample stack dump:
[   13.315090] BUG: unable to handle kernel paging request at ffffc9000605a088
[   13.323221] IP: [<ffffffff81461bac>] qi_submit_sync+0x15c/0x400
[   13.330107] PGD 82f81e067 PUD c2f81e067 PMD 82e846067 PTE 0
[   13.336818] Oops: 0002 [#1] SMP
[   13.340757] Modules linked in:
[   13.344422] CPU: 0 PID: 4 Comm: kworker/0:0 Not tainted 3.13.0-rc1-gerry+ #7
[   13.352474] Hardware name: Intel Corporation LH Pass ........../SVRBD-ROW_T,                                               BIOS SE5C600.86B.99.99.x059.091020121352 09/10/2012
[   13.365659] Workqueue: events work_for_cpu_fn
[   13.370774] task: ffff88042ddf00d0 ti: ffff88042ddee000 task.ti: ffff88042dde                                              e000
[   13.379389] RIP: 0010:[<ffffffff81461bac>]  [<ffffffff81461bac>] qi_submit_sy                                              nc+0x15c/0x400
[   13.389055] RSP: 0000:ffff88042ddef940  EFLAGS: 00010002
[   13.395151] RAX: 00000000000005e0 RBX: 0000000000000082 RCX: 0000000200000025
[   13.403308] RDX: ffffc9000605a000 RSI: 0000000000000010 RDI: ffff88042ddb8610
[   13.411446] RBP: ffff88042ddef9a0 R08: 00000000000005d0 R09: 0000000000000001
[   13.419599] R10: 0000000000000000 R11: 000000000000005d R12: 000000000000005c
[   13.427742] R13: ffff88102d84d300 R14: 0000000000000174 R15: ffff88042ddb4800
[   13.435877] FS:  0000000000000000(0000) GS:ffff88043de00000(0000) knlGS:00000                                              00000000000
[   13.445168] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   13.451749] CR2: ffffc9000605a088 CR3: 0000000001a0b000 CR4: 00000000000407f0
[   13.459895] Stack:
[   13.462297]  ffff88042ddb85d0 000000000000005d ffff88042ddef9b0 0000000000000                                              5d0
[   13.471147]  00000000000005c0 ffff88042ddb8000 000000000000005c 0000000000000                                              015
[   13.480001]  ffff88042ddb4800 0000000000000282 ffff88042ddefa40 ffff88042ddef                                              ac0
[   13.488855] Call Trace:
[   13.491771]  [<ffffffff8146848d>] modify_irte+0x9d/0xd0
[   13.497778]  [<ffffffff8146886d>] intel_setup_ioapic_entry+0x10d/0x290
[   13.505250]  [<ffffffff810a92a6>] ? trace_hardirqs_on_caller+0x16/0x1e0
[   13.512824]  [<ffffffff810346b0>] ? default_init_apic_ldr+0x60/0x60
[   13.519998]  [<ffffffff81468be0>] setup_ioapic_remapped_entry+0x20/0x30
[   13.527566]  [<ffffffff8103683a>] io_apic_setup_irq_pin+0x12a/0x2c0
[   13.534742]  [<ffffffff8136673b>] ? acpi_pci_irq_find_prt_entry+0x2b9/0x2d8
[   13.544102]  [<ffffffff81037fd5>] io_apic_setup_irq_pin_once+0x85/0xa0
[   13.551568]  [<ffffffff8103816f>] ? mp_find_ioapic_pin+0x8f/0xf0
[   13.558434]  [<ffffffff81038044>] io_apic_set_pci_routing+0x34/0x70
[   13.565621]  [<ffffffff8102f4cf>] mp_register_gsi+0xaf/0x1c0
[   13.572111]  [<ffffffff8102f5ee>] acpi_register_gsi_ioapic+0xe/0x10
[   13.579286]  [<ffffffff8102f33f>] acpi_register_gsi+0xf/0x20
[   13.585779]  [<ffffffff81366b86>] acpi_pci_irq_enable+0x171/0x1e3
[   13.592764]  [<ffffffff8146d771>] pcibios_enable_device+0x31/0x40
[   13.599744]  [<ffffffff81320e9b>] do_pci_enable_device+0x3b/0x60
[   13.606633]  [<ffffffff81322248>] pci_enable_device_flags+0xc8/0x120
[   13.613887]  [<ffffffff813222f3>] pci_enable_device+0x13/0x20
[   13.620484]  [<ffffffff8132fa7e>] pcie_port_device_register+0x1e/0x510
[   13.627947]  [<ffffffff810a92a6>] ? trace_hardirqs_on_caller+0x16/0x1e0
[   13.635510]  [<ffffffff810a947d>] ? trace_hardirqs_on+0xd/0x10
[   13.642189]  [<ffffffff813302b8>] pcie_portdrv_probe+0x58/0xc0
[   13.648877]  [<ffffffff81323ba5>] local_pci_probe+0x45/0xa0
[   13.655266]  [<ffffffff8106bc44>] work_for_cpu_fn+0x14/0x20
[   13.661656]  [<ffffffff8106fa79>] process_one_work+0x369/0x710
[   13.668334]  [<ffffffff8106fa02>] ? process_one_work+0x2f2/0x710
[   13.675215]  [<ffffffff81071d56>] ? worker_thread+0x46/0x690
[   13.681714]  [<ffffffff81072194>] worker_thread+0x484/0x690
[   13.688109]  [<ffffffff81071d10>] ? cancel_delayed_work_sync+0x20/0x20
[   13.695576]  [<ffffffff81079c60>] kthread+0xf0/0x110
[   13.701300]  [<ffffffff8108e7bf>] ? local_clock+0x3f/0x50
[   13.707492]  [<ffffffff81079b70>] ? kthread_create_on_node+0x250/0x250
[   13.714959]  [<ffffffff81574d2c>] ret_from_fork+0x7c/0xb0
[   13.721152]  [<ffffffff81079b70>] ? kthread_create_on_node+0x250/0x250

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-01-09 12:43:40 +01:00
Jiang Liu b5f36d9e61 iommu/vt-d: fix invalid memory access when freeing DMAR irq
In function free_dmar_iommu(), it sets IRQ handler data to NULL
before calling free_irq(), which will cause invalid memory access
because free_irq() will access IRQ handler data when calling
function dmar_msi_mask(). So only set IRQ handler data to NULL
after calling free_irq().

Sample stack dump:
[   13.094010] BUG: unable to handle kernel NULL pointer dereference at 0000000000000048
[   13.103215] IP: [<ffffffff810a97cd>] __lock_acquire+0x4d/0x12a0
[   13.110104] PGD 0
[   13.112614] Oops: 0000 [#1] SMP
[   13.116585] Modules linked in:
[   13.120260] CPU: 60 PID: 1 Comm: swapper/0 Tainted: G        W    3.13.0-rc1-gerry+ #9
[   13.129367] Hardware name: Intel Corporation LH Pass ........../SVRBD-ROW_T, BIOS SE5C600.86B.99.99.x059.091020121352 09/10/2012
[   13.142555] task: ffff88042dd38010 ti: ffff88042dd32000 task.ti: ffff88042dd32000
[   13.151179] RIP: 0010:[<ffffffff810a97cd>]  [<ffffffff810a97cd>] __lock_acquire+0x4d/0x12a0
[   13.160867] RSP: 0000:ffff88042dd33b78  EFLAGS: 00010046
[   13.166969] RAX: 0000000000000046 RBX: 0000000000000002 RCX: 0000000000000000
[   13.175122] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000048
[   13.183274] RBP: ffff88042dd33bd8 R08: 0000000000000002 R09: 0000000000000001
[   13.191417] R10: 0000000000000000 R11: 0000000000000001 R12: ffff88042dd38010
[   13.199571] R13: 0000000000000000 R14: 0000000000000048 R15: 0000000000000000
[   13.207725] FS:  0000000000000000(0000) GS:ffff88103f200000(0000) knlGS:0000000000000000
[   13.217014] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   13.223596] CR2: 0000000000000048 CR3: 0000000001a0b000 CR4: 00000000000407e0
[   13.231747] Stack:
[   13.234160]  0000000000000004 0000000000000046 ffff88042dd33b98 ffffffff810a567d
[   13.243059]  ffff88042dd33c08 ffffffff810bb14c ffffffff828995a0 0000000000000046
[   13.251969]  0000000000000000 0000000000000000 0000000000000002 0000000000000000
[   13.260862] Call Trace:
[   13.263775]  [<ffffffff810a567d>] ? trace_hardirqs_off+0xd/0x10
[   13.270571]  [<ffffffff810bb14c>] ? vprintk_emit+0x23c/0x570
[   13.277058]  [<ffffffff810ab1e3>] lock_acquire+0x93/0x120
[   13.283269]  [<ffffffff814623f7>] ? dmar_msi_mask+0x47/0x70
[   13.289677]  [<ffffffff8156b449>] _raw_spin_lock_irqsave+0x49/0x90
[   13.296748]  [<ffffffff814623f7>] ? dmar_msi_mask+0x47/0x70
[   13.303153]  [<ffffffff814623f7>] dmar_msi_mask+0x47/0x70
[   13.309354]  [<ffffffff810c0d93>] irq_shutdown+0x53/0x60
[   13.315467]  [<ffffffff810bdd9d>] __free_irq+0x26d/0x280
[   13.321580]  [<ffffffff810be920>] free_irq+0xf0/0x180
[   13.327395]  [<ffffffff81466591>] free_dmar_iommu+0x271/0x2b0
[   13.333996]  [<ffffffff810a947d>] ? trace_hardirqs_on+0xd/0x10
[   13.340696]  [<ffffffff81461a17>] free_iommu+0x17/0x50
[   13.346597]  [<ffffffff81dc75a5>] init_dmars+0x691/0x77a
[   13.352711]  [<ffffffff81dc7afd>] intel_iommu_init+0x351/0x438
[   13.359400]  [<ffffffff81d8a711>] ? iommu_setup+0x27d/0x27d
[   13.365806]  [<ffffffff81d8a739>] pci_iommu_init+0x28/0x52
[   13.372114]  [<ffffffff81000342>] do_one_initcall+0x122/0x180
[   13.378707]  [<ffffffff81077738>] ? parse_args+0x1e8/0x320
[   13.385016]  [<ffffffff81d850e8>] kernel_init_freeable+0x1e1/0x26c
[   13.392100]  [<ffffffff81d84833>] ? do_early_param+0x88/0x88
[   13.398596]  [<ffffffff8154f8b0>] ? rest_init+0xd0/0xd0
[   13.404614]  [<ffffffff8154f8be>] kernel_init+0xe/0x130
[   13.410626]  [<ffffffff81574d6c>] ret_from_fork+0x7c/0xb0
[   13.416829]  [<ffffffff8154f8b0>] ? rest_init+0xd0/0xd0
[   13.422842] Code: ec 99 00 85 c0 8b 05 53 05 a5 00 41 0f 45 d8 85 c0 0f 84 ff 00 00 00 8b 05 99 f9 7e 01 49 89 fe 41 89 f7 85 c0 0f 84 03 01 00 00 <49> 8b 06 be 01 00 00 00 48 3d c0 0e 01 82 0f 44 de 41 83 ff 01
[   13.450191] RIP  [<ffffffff810a97cd>] __lock_acquire+0x4d/0x12a0
[   13.458598]  RSP <ffff88042dd33b78>
[   13.462671] CR2: 0000000000000048
[   13.466551] ---[ end trace c5bd26a37c81d760 ]---

Reviewed-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-01-09 12:43:38 +01:00
Jiang Liu 7c9197791a iommu/vt-d, trivial: simplify code with existing macros
Simplify vt-d related code with existing macros and introduce a new
macro for_each_active_drhd_unit() to enumerate all active DRHD unit.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-01-09 12:43:37 +01:00
Jiang Liu b8a2d2881e iommu/vt-d, trivial: clean up unused code
Remove dead code from VT-d related files.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>

Conflicts:

	drivers/iommu/dmar.c
2014-01-09 12:43:31 +01:00
Jiang Liu 9544c003e8 iommu/vt-d, trivial: print correct domain id of static identity domain
Field si_domain->id is set by iommu_attach_domain(), so we should only
print domain id for static identity domain after calling
iommu_attach_domain(si_domain, iommu), otherwise it's always zero.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-01-09 12:43:28 +01:00
Jiang Liu 5c645b35b7 iommu/vt-d, trivial: refine support of 64bit guest address
In Intel IOMMU driver, it calculate page table level from adjusted guest
address width as 'level = (agaw - 30) / 9', which assumes (agaw -30)
could be divided by 9. On the other hand, 64bit is a valid agaw and
(64 - 30) can't be divided by 9, so it needs special handling.

This patch enhances Intel IOMMU driver to correctly handle 64bit agaw.
It's mainly for code readability because there's no hardware supporting
64bit agaw yet.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-01-09 12:43:27 +01:00
Jiang Liu 852bdb04f8 iommu/vt-d: fix resource leakage on error recovery path in iommu_init_domains()
Release allocated resources on error recovery path in function
iommu_init_domains().

Also improve printk messages in iommu_init_domains().

Acked-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-01-09 12:43:25 +01:00
Jiang Liu 18d99165d3 iommu/vt-d: fix a race window in allocating domain ID for virtual machines
Function intel_iommu_domain_init() may be concurrently called by upper
layer without serialization, so use atomic_t to protect domain id
allocation.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-01-09 12:43:24 +01:00
Yijing Wang dbad086433 iommu/vt-d: Use dev_is_pci() to check whether it is pci device
Use PCI standard marco dev_is_pci() instead of directly compare
pci_bus_type to check whether it is pci device.

Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2014-01-07 15:21:45 +01:00
Yijing Wang bca2b916f3 iommu/vt-d: Use list_for_each_entry_safe() for dmar_domain->devices traversal
Replace list_for_each_safe() + list_entry() with the simpler
list_for_each_entry_safe().

Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2013-11-01 14:18:48 +01:00
Julian Stecklina f9423606ad iommu/vt-d: Fixed interaction of VFIO_IOMMU_MAP_DMA with IOMMU address limits
The BUG_ON in drivers/iommu/intel-iommu.c:785 can be triggered from userspace via
VFIO by calling the VFIO_IOMMU_MAP_DMA ioctl on a vfio device with any address
beyond the addressing capabilities of the IOMMU. The problem is that the ioctl code
calls iommu_iova_to_phys before it calls iommu_map. iommu_map handles the case that
it gets addresses beyond the addressing capabilities of its IOMMU.
intel_iommu_iova_to_phys does not.

This patch fixes iommu_iova_to_phys to return NULL for addresses beyond what the
IOMMU can handle. This in turn causes the ioctl call to fail in iommu_map and
(correctly) return EFAULT to the user with a helpful warning message in the kernel
log.

Signed-off-by: Julian Stecklina <jsteckli@os.inf.tu-dresden.de>
Acked-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2013-11-01 12:46:25 +01:00
Alex Williamson 3269ee0bd6 intel-iommu: Fix leaks in pagetable freeing
At best the current code only seems to free the leaf pagetables and
the root.  If you're unlucky enough to have a large gap (like any
QEMU guest with more than 3G of memory), only the first chunk of leaf
pagetables are freed (plus the root).  This is a massive memory leak.
This patch re-writes the pagetable freeing function to use a
recursive algorithm and manages to not only free all the pagetables,
but does it without any apparent performance loss versus the current
broken version.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Cc: stable@vger.kernel.org
Reviewed-by: Marcelo Tosatti <mtosatti@redhat.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2013-08-14 22:21:04 +02:00
Alex Williamson c14d26905d iommu/{vt-d,amd}: Remove multifunction assumption around grouping
If a device is multifunction and does not have ACS enabled then we
assume that the entire package lacks ACS and use function 0 as the
base of the group.  The PCIe spec however states that components are
permitted to implement ACS on some, none, or all of their applicable
functions.  It's therefore conceivable that function 0 may be fully
independent and support ACS while other functions do not.  Instead
use the lowest function of the slot that does not have ACS enabled
as the base of the group.  This may be the current device, which is
intentional.  So long as we use a consistent algorithm, all the
non-ACS functions will be grouped together and ACS functions will
get separate groups.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2013-06-20 17:21:09 +02:00
Joerg Roedel 0c4513be3d Merge branches 'iommu/fixes', 'x86/vt-d', 'x86/amd', 'ppc/pamu', 'core' and 'arm/tegra' into next 2013-05-02 12:10:19 +02:00
Varun Sethi 61e015ac5b iommu: Move swap_pci_ref function to drivers/iommu/pci.h.
The swap_pci_ref function is used by the IOMMU API code for
swapping pci device pointers, while determining the iommu
group for the device.
Currently this function was being implemented for different
IOMMU drivers.  This patch moves the function to a new file,
drivers/iommu/pci.h so that the implementation can be
shared across various IOMMU drivers.

Signed-off-by: Varun Sethi <Varun.Sethi@freescale.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2013-04-23 14:55:00 +02:00
Takao Indoh 3a93c841c2 iommu/vt-d: Disable translation if already enabled
This patch disables translation(dma-remapping) before its initialization
if it is already enabled.

This is needed for kexec/kdump boot. If dma-remapping is enabled in the
first kernel, it need to be disabled before initializing its page table
during second kernel boot. Wei Hu also reported that this is needed
when second kernel boots with intel_iommu=off.

Basically iommu->gcmd is used to know whether translation is enabled or
disabled, but it is always zero at boot time even when translation is
enabled since iommu->gcmd is initialized without considering such a
case. Therefor this patch synchronizes iommu->gcmd value with global
command register when iommu structure is allocated.

Signed-off-by: Takao Indoh <indou.takao@jp.fujitsu.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2013-04-23 14:47:08 +02:00
Varun Sethi bb5547acfc iommu/fsl: Make iova dma_addr_t in the iommu_iova_to_phys API.
This is required in case of PAMU, as it can support a window size of up
to 64G (even on 32bit).

Signed-off-by: Varun Sethi <Varun.Sethi@freescale.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2013-04-02 18:20:53 +02:00
Linus Torvalds fffddfd6c8 Merge branch 'drm-next' of git://people.freedesktop.org/~airlied/linux
Pull drm merge from Dave Airlie:
 "Highlights:

   - TI LCD controller KMS driver

   - TI OMAP KMS driver merged from staging

   - drop gma500 stub driver

   - the fbcon locking fixes

   - the vgacon dirty like zebra fix.

   - open firmware videomode and hdmi common code helpers

   - major locking rework for kms object handling - pageflip/cursor
     won't block on polling anymore!

   - fbcon helper and prime helper cleanups

   - i915: all over the map, haswell power well enhancements, valleyview
     macro horrors cleaned up, killing lots of legacy GTT code,

   - radeon: CS ioctl unification, deprecated UMS support, gpu reset
     rework, VM fixes

   - nouveau: reworked thermal code, external dp/tmds encoder support
     (anx9805), fences sleep instead of polling,

   - exynos: all over the driver fixes."

Lovely conflict in radeon/evergreen_cs.c between commit de0babd60d
("drm/radeon: enforce use of radeon_get_ib_value when reading user cmd")
and the new changes that modified that evergreen_dma_cs_parse()
function.

* 'drm-next' of git://people.freedesktop.org/~airlied/linux: (508 commits)
  drm/tilcdc: only build on arm
  drm/i915: Revert hdmi HDP pin checks
  drm/tegra: Add list of framebuffers to debugfs
  drm/tegra: Fix color expansion
  drm/tegra: Split DC_CMD_STATE_CONTROL register write
  drm/tegra: Implement page-flipping support
  drm/tegra: Implement VBLANK support
  drm/tegra: Implement .mode_set_base()
  drm/tegra: Add plane support
  drm/tegra: Remove bogus tegra_framebuffer structure
  drm: Add consistency check for page-flipping
  drm/radeon: Use generic HDMI infoframe helpers
  drm/tegra: Use generic HDMI infoframe helpers
  drm: Add EDID helper documentation
  drm: Add HDMI infoframe helpers
  video: Add generic HDMI infoframe helpers
  drm: Add some missing forward declarations
  drm: Move mode tables to drm_edid.c
  drm: Remove duplicate drm_mode_cea_vic()
  gma500: Fix n, m1 and m2 clock limits for sdvo and lvds
  ...
2013-02-25 16:46:44 -08:00
Linus Torvalds 5800700f66 Merge branch 'x86-apic-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull x86/apic changes from Ingo Molnar:
 "Main changes:

   - Multiple MSI support added to the APIC, PCI and AHCI code - acked
     by all relevant maintainers, by Alexander Gordeev.

     The advantage is that multiple AHCI ports can have multiple MSI
     irqs assigned, and can thus spread to multiple CPUs.

     [ Drivers can make use of this new facility via the
       pci_enable_msi_block_auto() method ]

   - x86 IOAPIC code from interrupt remapping cleanups from Joerg
     Roedel:

     These patches move all interrupt remapping specific checks out of
     the x86 core code and replaces the respective call-sites with
     function pointers.  As a result the interrupt remapping code is
     better abstraced from x86 core interrupt handling code.

   - Various smaller improvements, fixes and cleanups."

* 'x86-apic-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (26 commits)
  x86/intel/irq_remapping: Clean up x2apic opt-out security warning mess
  x86, kvm: Fix intialization warnings in kvm.c
  x86, irq: Move irq_remapped out of x86 core code
  x86, io_apic: Introduce eoi_ioapic_pin call-back
  x86, msi: Introduce x86_msi.compose_msi_msg call-back
  x86, irq: Introduce setup_remapped_irq()
  x86, irq: Move irq_remapped() check into free_remapped_irq
  x86, io-apic: Remove !irq_remapped() check from __target_IO_APIC_irq()
  x86, io-apic: Move CONFIG_IRQ_REMAP code out of x86 core
  x86, irq: Add data structure to keep AMD specific irq remapping information
  x86, irq: Move irq_remapping_enabled declaration to iommu code
  x86, io_apic: Remove irq_remapping_enabled check in setup_timer_IRQ0_pin
  x86, io_apic: Move irq_remapping_enabled checks out of check_timer()
  x86, io_apic: Convert setup_ioapic_entry to function pointer
  x86, io_apic: Introduce set_affinity function pointer
  x86, msi: Use IRQ remapping specific setup_msi_irqs routine
  x86, hpet: Introduce x86_msi_ops.setup_hpet_msi
  x86, io_apic: Introduce x86_io_apic_ops.print_entries for debugging
  x86, io_apic: Introduce x86_io_apic_ops.disable()
  x86, apic: Mask IO-APIC and PIC unconditionally on LAPIC resume
  ...
2013-02-19 19:07:27 -08:00
Daniel Vetter 210561ffd7 intel/iommu: force writebuffer-flush quirk on Gen 4 Chipsets
We already have the quirk entry for the mobile platform, but also
reports on some desktop versions. So be paranoid and set it
everywhere.

References: http://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg33138.html
Cc: stable@vger.kernel.org
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: "Sankaran, Rajesh" <rajesh.sankaran@intel.com>
Reported-and-tested-by: Mihai Moldovan <ionic@ionic.de>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
2013-02-20 00:21:50 +01:00
Joerg Roedel 078e1ee26a x86, irq: Move irq_remapping_enabled declaration to iommu code
Remove the last left-over from this flag from x86 code.

Signed-off-by: Joerg Roedel <joro@8bytes.org>
Acked-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
2013-01-28 12:17:26 +01:00
Daniel Vetter 9452618e74 iommu/intel: disable DMAR for g4x integrated gfx
DMAR support on g4x/gm45 integrated gpus seems to be totally busted.
So don't bother, but instead disable it by default to allow distros to
unconditionally enable DMAR support.

v2: Actually wire up the right quirk entry, spotted by Adam Jackson.

Note that according to intel marketing materials only g45 and gm45
support DMAR/VT-d. So we have reports for all relevant gen4 pci ids by
now. Still, keep all the other gen4 ids in the quirk table in case the
marketing stuff confused me again, which would not be the first time.

Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=51921
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=538163
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=538163
Cc: Adam Jackson <ajax@redhat.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: stable@vger.kernel.org
Acked-By: David Woodhouse <David.Woodhouse@intel.com>
Tested-by: stathis <stathis@npcglib.org>
Tested-by: Mihai Moldovan <ionic@ionic.de>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
2013-01-23 01:00:41 +01:00
Greg Kroah-Hartman d34d6517ea Drivers: iommu: remove __dev* attributes.
CONFIG_HOTPLUG is going away as an option.  As a result, the __dev*
markings need to be removed.

This change removes the use of __devinit, __devexit_p, __devinitdata,
and __devexit from these drivers.

Based on patches originally written by Bill Pemberton, but redone by me
in order to handle some of the coding style issues better, by hand.

Cc: Bill Pemberton <wfp5p@virginia.edu>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Ohad Ben-Cohen <ohad@wizery.com>
Cc: Tony Lindgren <tony@atomide.com>
Cc: Omar Ramirez Luna <omar.luna@linaro.org>
Cc: Mauro Carvalho Chehab <mchehab@redhat.com>
Cc: Hiroshi Doyu <hdoyu@nvidia.com>
Cc: Stephen Warren <swarren@wwwdotorg.org>
Cc: Bharat Nihalani <bnihalani@nvidia.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-01-03 15:57:14 -08:00
Linus Torvalds 787314c35f IOMMU Updates for Linux v3.8
A few new features this merge-window. The most important one is
 probably, that dma-debug now warns if a dma-handle is not checked with
 dma_mapping_error by the device driver. This requires minor changes to
 some architectures which make use of dma-debug. Most of these changes
 have the respective Acks by the Arch-Maintainers.
 Besides that there are updates to the AMD IOMMU driver for refactor the
 IOMMU-Groups support and to make sure it does not trigger a hardware
 erratum.
 The OMAP changes (for which I pulled in a branch from Tony Lindgren's
 tree) have a conflict in linux-next with the arm-soc tree. The conflict
 is in the file arch/arm/mach-omap2/clock44xx_data.c which is deleted in
 the arm-soc tree. It is safe to delete the file too so solve the
 conflict. Similar changes are done in the arm-soc tree in the common
 clock framework migration. A missing hunk from the patch in the IOMMU
 tree will be submitted as a seperate patch when the merge-window is
 closed.
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1.4.11 (GNU/Linux)
 
 iQIcBAABAgAGBQJQzbQQAAoJECvwRC2XARrjXCIP/2RxBzbVOiaPOorl+ZWbsZ41
 lzWiXsCHJkh4BK4/qGsVeKhiNd9LcbQUlhywnBbhWxym3spzmjGtvU2Hcg8QiO/M
 R83r9S4e8Z6DnF9Gcats1Ns9BufgpyhLXg3XoXPxtyHOgRS59fvYi6xXOxyX30Dy
 uhbj+WL6UD0zvOMNztEnM1p6UhX+XlpvzKDTR5+G5xKdVPkcgeiaKSwqz739caTn
 QE2NpqIh+8Mwuu1nIapk8h07xhUYU5eGMXa38u1LvDwSHsrsCMLC+lXIjtInn7Gw
 Bv+XcCHgtOaoPQwwk/xd2HVwJQxO9HNb5YX51EIjwP0C5S/3yW9Ji1RgqFb6Ewqq
 jIkF6ckwUheLWsBGkw5UknI/f7RX3MDiTWkziYLIniYKKewm+ymGfgIqPt2TzLIO
 tMZZiIssKvy7wOXQ5JjpYJg5Xmrau6opNwdEguC8pWkJT7qsn+3SeLjMt0Lh9IoY
 +37DOgOLb3O3/vnZJ3i0KMRZBfVeaRj5HaGmlxFCYUZCNQymIPTih9Jtqm+WuVcu
 YaGQCTtynsQ0JVh8YEekLzSfgd3OODP68fyCg1CQNixEgvUi2hd/toX2/Z1wkkSA
 JC9bZarcoPkSWqaTAA2HvmaaxvRR+0UbhFPopFTQarVV0MVLZWBxoyuKy/nMrmMd
 UgTzrDYy74UKdrSTwIXg
 =pPHZ
 -----END PGP SIGNATURE-----

Merge tag 'iommu-updates-v3.8' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu

Pull IOMMU updates from Joerg Roedel:
 "A few new features this merge-window.  The most important one is
  probably, that dma-debug now warns if a dma-handle is not checked with
  dma_mapping_error by the device driver.  This requires minor changes
  to some architectures which make use of dma-debug.  Most of these
  changes have the respective Acks by the Arch-Maintainers.

  Besides that there are updates to the AMD IOMMU driver for refactor
  the IOMMU-Groups support and to make sure it does not trigger a
  hardware erratum.

  The OMAP changes (for which I pulled in a branch from Tony Lindgren's
  tree) have a conflict in linux-next with the arm-soc tree.  The
  conflict is in the file arch/arm/mach-omap2/clock44xx_data.c which is
  deleted in the arm-soc tree.  It is safe to delete the file too so
  solve the conflict.  Similar changes are done in the arm-soc tree in
  the common clock framework migration.  A missing hunk from the patch
  in the IOMMU tree will be submitted as a seperate patch when the
  merge-window is closed."

* tag 'iommu-updates-v3.8' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (29 commits)
  ARM: dma-mapping: support debug_dma_mapping_error
  ARM: OMAP4: hwmod data: ipu and dsp to use parent clocks instead of leaf clocks
  iommu/omap: Adapt to runtime pm
  iommu/omap: Migrate to hwmod framework
  iommu/omap: Keep mmu enabled when requested
  iommu/omap: Remove redundant clock handling on ISR
  iommu/amd: Remove obsolete comment
  iommu/amd: Don't use 512GB pages
  iommu/tegra: smmu: Move bus_set_iommu after probe for multi arch
  iommu/tegra: gart: Move bus_set_iommu after probe for multi arch
  iommu/tegra: smmu: Remove unnecessary PTC/TLB flush all
  tile: dma_debug: add debug_dma_mapping_error support
  sh: dma_debug: add debug_dma_mapping_error support
  powerpc: dma_debug: add debug_dma_mapping_error support
  mips: dma_debug: add debug_dma_mapping_error support
  microblaze: dma-mapping: support debug_dma_mapping_error
  ia64: dma_debug: add debug_dma_mapping_error support
  c6x: dma_debug: add debug_dma_mapping_error support
  ARM64: dma_debug: add debug_dma_mapping_error support
  intel-iommu: Prevent devices with RMRRs from being placed into SI Domain
  ...
2012-12-20 10:07:25 -08:00
Woodhouse, David 6491d4d028 intel-iommu: Free old page tables before creating superpage
The dma_pte_free_pagetable() function will only free a page table page
if it is asked to free the *entire* 2MiB range that it covers. So if a
page table page was used for one or more small mappings, it's likely to
end up still present in the page tables... but with no valid PTEs.

This was fine when we'd only be repopulating it with 4KiB PTEs anyway
but the same virtual address range can end up being reused for a
*large-page* mapping. And in that case were were trying to insert the
large page into the second-level page table, and getting a complaint
from the sanity check in __domain_mapping() because there was already a
corresponding entry. This was *relatively* harmless; it led to a memory
leak of the old page table page, but no other ill-effects.

Fix it by calling dma_pte_clear_range (hopefully redundant) and
dma_pte_free_pagetable() before setting up the new large page.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
Tested-by: Ravi Murty <Ravi.Murty@intel.com>
Tested-by: Sudeep Dutt <sudeep.dutt@intel.com>
Cc: stable@kernel.org [3.0+]
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-12-20 10:06:04 -08:00
Joerg Roedel 9c6ecf6a3a Merge branches 'iommu/fixes', 'dma-debug', 'x86/amd', 'x86/vt-d', 'arm/tegra' and 'arm/omap' into next 2012-12-16 12:24:09 +01:00
Tom Mingarelli ea2447f700 intel-iommu: Prevent devices with RMRRs from being placed into SI Domain
This patch is to prevent non-USB devices that have RMRRs associated with them from
being placed into the SI Domain during init. This fixes the issue where the RMRR info
for devices being placed in and out of the SI Domain gets lost.

Signed-off-by: Thomas Mingarelli <thomas.mingarelli@hp.com>
Tested-by: Shuah Khan <shuah.khan@hp.com>
Reviewed-by: Donald Dutile <ddutile@redhat.com>
Reviewed-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2012-11-21 16:55:32 +01:00
Alex Williamson 3da4af0aff intel-iommu: Fix lookup in add device
We can't assume this device exists, fall back to the bridge itself.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Tested-by: Matthew Thode <prometheanfire@gentoo.org>
Cc: stable@vger.kernel.org
Signed-off-by: Joerg Roedel <joro@8bytes.org>
2012-11-17 13:27:15 +01:00
Linus Torvalds c0703c12ef IOMMU Updates for Linux v3.7-rc1
This time the IOMMU updates contain a bunch of fixes and cleanups to
 various IOMMU drivers and the DMA debug code. New features are the
 code for IRQ remapping support with the AMD IOMMU (preperation for that
 was already merged in the last release) and a debugfs interface to
 export some statistics in the NVidia Tegra IOMMU driver.
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1.4.11 (GNU/Linux)
 
 iQIbBAABAgAGBQJQbvi2AAoJECvwRC2XARrjIOcP92esm3auz49N2+rq5TAto6hX
 4MbpUFgkpKQRdFUSgYOsuzjh+I46ZpWEVajCxM+9oJz4I3PalDH0hvN7VkcZTjvR
 bENRaLOZNR39rtIPrNVlne2RX7DXjKxJF5+K7jD6uMGJdItPfXk1Vo0VrLw30+XD
 E3pWNFCS+qJkWl7d0UGUJ6CzS1BZuHZdAx+XdHTa9DIV88jANiUefIV35cAVuqrL
 xVDDDMs7ITIYn6vTpEj7/zXCHNcFS7Ki4FQxAQ4DFvpPdrvLtPMeXnVSBMg0LPpe
 Gb2r8tjywo83yjZMoDxca6w20SJPkQHoWwlansGlg5j0KwqPhvD1M/OXwzbq6avd
 fqL69kZTXttLY+qtnlM5YfAqySXevGmGusgmqEXT7GvbAH6dSqMDdPU2W6lt4zBd
 80LyAdcwP5X5hT91KiOfGGpFY4np97dUC7QFbwAdgDO7t6R+c2OfSte6azzWkYm4
 2eYCsIKy+5U6XLSNabp7QNAxnic3mt6/AVMydZsVLyjeQZ8ybCwslQjPxSNwQrxc
 bx8z8KFY37UuFUs77dYFIChM8BJP6erKL0W1BOt0Ve2PWgeOaEl3y5n3mwBe+GXa
 JU9SPqy0juNZIYJWU1ZE/90ULo6ZpB3wM7QpMgX1pJ7KTd1gYDjSXhAEJi5Eje8r
 GNHlYwjxsQZQcvR19g8=
 =Q5kj
 -----END PGP SIGNATURE-----

Merge tag 'iommu-updates-v3.7-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu

Pull IOMMU updates from Joerg Roedel:
 "This time the IOMMU updates contain a bunch of fixes and cleanups to
  various IOMMU drivers and the DMA debug code.  New features are the
  code for IRQ remapping support with the AMD IOMMU (preperation for
  that was already merged in the last release) and a debugfs interface
  to export some statistics in the NVidia Tegra IOMMU driver."

* tag 'iommu-updates-v3.7-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (27 commits)
  iommu/amd: Remove obsolete comment line
  dma-debug: Remove local BUS_NOTIFY_UNBOUND_DRIVER define
  iommu/amd: Fix possible use after free in get_irq_table()
  iommu/amd: Report irq remapping through IOMMU-API
  iommu/amd: Print message to system log when irq remapping is enabled
  iommu/irq: Use amd_iommu_irq_ops if supported
  iommu/amd: Make sure irq remapping still works on dma init failure
  iommu/amd: Add initialization routines for AMD interrupt remapping
  iommu/amd: Add call-back routine for HPET MSI
  iommu/amd: Implement MSI routines for interrupt remapping
  iommu/amd: Add IOAPIC remapping routines
  iommu/amd: Add routines to manage irq remapping tables
  iommu/amd: Add IRTE invalidation routine
  iommu/amd: Make sure IOMMU is not considered to translate itself
  iommu/amd: Split device table initialization into irq and dma part
  iommu/amd: Check if IOAPIC information is correct
  iommu/amd: Allocate data structures to keep track of irq remapping tables
  iommu/amd: Add slab-cache for irq remapping tables
  iommu/amd: Keep track of HPET and IOAPIC device ids
  iommu/amd: Fix features reporting
  ...
2012-10-08 06:33:44 +09:00
Alex Williamson 2e12bc29fc intel-iommu: Default to non-coherent for domains unattached to iommus
domain_update_iommu_coherency() currently defaults to setting domains
as coherent when the domain is not attached to any iommus.  This
allows for a window in domain_context_mapping_one() where such a
domain can update context entries non-coherently, and only after
update the domain capability to clear iommu_coherency.

This can be seen using KVM device assignment on VT-d systems that
do not support coherency in the ecap register.  When a device is
added to a guest, a domain is created (iommu_coherency = 0), the
device is attached, and ranges are mapped.  If we then hot unplug
the device, the coherency is updated and set to the default (1)
since no iommus are attached to the domain.  A subsequent attach
of a device makes use of the same dmar domain (now marked coherent)
updates context entries with coherency enabled, and only disables
coherency as the last step in the process.

To fix this, switch domain_update_iommu_coherency() to use the
safer, non-coherent default for domains not attached to iommus.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Tested-by: Donald Dutile <ddutile@redhat.com>
Acked-by: Donald Dutile <ddutile@redhat.com>
Acked-by: Chris Wright <chrisw@sous-sol.org>
Cc: stable@vger.kernel.org
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2012-09-18 16:17:31 +02:00
Bjorn Helgaas 78890b5989 Merge commit 'v3.6-rc5' into next
* commit 'v3.6-rc5': (1098 commits)
  Linux 3.6-rc5
  HID: tpkbd: work even if the new Lenovo Keyboard driver is not configured
  Remove user-triggerable BUG from mpol_to_str
  xen/pciback: Fix proper FLR steps.
  uml: fix compile error in deliver_alarm()
  dj: memory scribble in logi_dj
  Fix order of arguments to compat_put_time[spec|val]
  xen: Use correct masking in xen_swiotlb_alloc_coherent.
  xen: fix logical error in tlb flushing
  xen/p2m: Fix one-off error in checking the P2M tree directory.
  powerpc: Don't use __put_user() in patch_instruction
  powerpc: Make sure IPI handlers see data written by IPI senders
  powerpc: Restore correct DSCR in context switch
  powerpc: Fix DSCR inheritance in copy_thread()
  powerpc: Keep thread.dscr and thread.dscr_inherit in sync
  powerpc: Update DSCR on all CPUs when writing sysfs dscr_default
  powerpc/powernv: Always go into nap mode when CPU is offline
  powerpc: Give hypervisor decrementer interrupts their own handler
  powerpc/vphn: Fix arch_update_cpu_topology() return value
  ARM: gemini: fix the gemini build
  ...

Conflicts:
	drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c
	drivers/rapidio/devices/tsi721.c
2012-09-13 08:41:01 -06:00
Yijing Wang 62f87c0e31 PCI: Introduce pci_pcie_type(dev) to replace pci_dev->pcie_type
Introduce an inline function pci_pcie_type(dev) to extract PCIe
device type from pci_dev->pcie_flags_reg field, and prepare for
removing pci_dev->pcie_type.

Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Jiang Liu <jiang.liu@huawei.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2012-08-23 09:40:57 -06:00
Alex Williamson a4ff1fc2cc iommu/intel: Fix ACS path checking
SR-IOV can create buses without a bridge.  There may be other cases
where this happens as well.  In these cases skip to the parent bus
and continue testing devices there.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Tested-by: David Ahern <dsahern@gmail.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2012-08-06 18:09:51 +02:00
Julia Lawall d2900bd63f iommu/intel: add missing free_domain_mem
Add missing free_domain_mem on failure path after alloc_domain.

A simplified version of the semantic match that finds this
problem is as follows: (http://coccinelle.lip6.fr/)

// <smpl>
@km exists@
local idexpression e;
expression e1,e2,e3;
type T,T1;
identifier f;
@@

* e = alloc_domain(...)
... when any
    when != e = e1
    when != e1 = (T)e
    when != e1(...,(T)e,...)
    when != &e->f
if(...) { ... when != e2(...,(T1)e,...)
                 when != e3 = e
                 when forall
(
             return <+...e...+>;
|
*             return ...;
) }
// </smpl>

Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2012-08-03 16:04:15 +02:00
Linus Torvalds 97027da6ad IOMMU Updates for Linux v3.6-rc1
The most important part of these updates is the IOMMU groups code
 enhancement written by Alex Williamson. It abstracts the problem that a
 given hardware IOMMU can't isolate any given device from any other
 device (e.g. 32 bit PCI devices can't usually be isolated). Devices that
 can't be isolated are grouped together. This code is required for the
 upcoming VFIO framework.
 
 Another IOMMU-API change written by be is the introduction of domain
 attributes. This makes it easier to handle GART-like IOMMUs with the
 IOMMU-API because now the start-address and the size of the domain
 address space can be queried.
 
 Besides that there are a few cleanups and fixes for the NVidia Tegra
 IOMMU drivers and the reworked init-code for the AMD IOMMU. The later is
 from my patch-set to support interrupt remapping. The rest of this
 patch-set requires x86 changes which are not mergabe yet. So full
 support for interrupt remapping with AMD IOMMUs will come in a future
 merge window.
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1.4.11 (GNU/Linux)
 
 iQIcBAABAgAGBQJQDV/MAAoJECvwRC2XARrjSDcP+gJbtSHDMyZ71zyfQfAZcxJt
 rTqLbdZRtIjrjgtKSEDp8u5Bo5TK9dAYoZVuJMOZewFzwI/fSfbRsWp1PU0I88Fr
 ZzM+/o1N9MLvf1e3kRVOzNzUfku+jTQgUBD4txsbtQzc/IeGHe9qP1Bqzs/xg4Pk
 SjWu7pLNYxaER10z76nRodNn6zGjsc7GFdOW8cJu2HOAHhisIAR291jSQgd6Rz9r
 zWqSTsXIEzYt2CtU3G2/tFJ554Mp8v5F80gHo+0Ldw8aNxlD6nGtbqGNt+KI8qTv
 MUL8KJ0TNms9CZdti1CSlSNp51VgJi2GaWKCaDAkYuuER2IbC/8Yp/p2DIIA0GNp
 HpziIs+dauZPWfZHc6oU7lJAClGAG4MUx7CysVIOzl7ML/Bf4mjYv0faGf5YQfyE
 weOR+OPPIWDUwgjzHKMAboA4ijkE/v+EKjOaN/S9rEqFEMKC99fwGkf9wUcpZTne
 8lzdI2JrgYNDWMVNYlomeLD4lBAbxb/QsnRUa33igjr0MclvMDkp5HaO631Z1+Zx
 be2z8Rl1CtMwS4qeaOXoeaoNWHU26+oJRZNtCGi/Fw4aKqYXP1dnE/m0GtqEP9Yi
 +CU2rKbZn3j0+ZcQjCQop8FREPrZ2/Uaji70b6G7WZ2ApcqBxzBffpbMKOmd6T1D
 HIzGh0fpdYNDuwn6Txit
 =MbAC
 -----END PGP SIGNATURE-----

Merge tag 'iommu-updates-v3.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu

Pull IOMMU updates from Joerg Roedel:
 "The most important part of these updates is the IOMMU groups code
  enhancement written by Alex Williamson.  It abstracts the problem that
  a given hardware IOMMU can't isolate any given device from any other
  device (e.g.  32 bit PCI devices can't usually be isolated).  Devices
  that can't be isolated are grouped together.  This code is required
  for the upcoming VFIO framework.

  Another IOMMU-API change written by me is the introduction of domain
  attributes.  This makes it easier to handle GART-like IOMMUs with the
  IOMMU-API because now the start-address and the size of the domain
  address space can be queried.

  Besides that there are a few cleanups and fixes for the NVidia Tegra
  IOMMU drivers and the reworked init-code for the AMD IOMMU.  The
  latter is from my patch-set to support interrupt remapping.  The rest
  of this patch-set requires x86 changes which are not mergabe yet.  So
  full support for interrupt remapping with AMD IOMMUs will come in a
  future merge window."

* tag 'iommu-updates-v3.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (33 commits)
  iommu/amd: Fix hotplug with iommu=pt
  iommu/amd: Add missing spin_lock initialization
  iommu/amd: Convert iommu initialization to state machine
  iommu/amd: Introduce amd_iommu_init_dma routine
  iommu/amd: Move unmap_flush message to amd_iommu_init_dma_ops()
  iommu/amd: Split enable_iommus() routine
  iommu/amd: Introduce early_amd_iommu_init routine
  iommu/amd: Move informational prinks out of iommu_enable
  iommu/amd: Split out PCI related parts of IOMMU initialization
  iommu/amd: Use acpi_get_table instead of acpi_table_parse
  iommu/amd: Fix sparse warnings
  iommu/tegra: Don't call alloc_pdir with as->lock
  iommu/tegra: smmu: Fix unsleepable memory allocation at alloc_pdir()
  iommu/tegra: smmu: Remove unnecessary sanity check at alloc_pdir()
  iommu/exynos: Implement DOMAIN_ATTR_GEOMETRY attribute
  iommu/tegra: Implement DOMAIN_ATTR_GEOMETRY attribute
  iommu/msm: Implement DOMAIN_ATTR_GEOMETRY attribute
  iommu/omap: Implement DOMAIN_ATTR_GEOMETRY attribute
  iommu/vt-d: Implement DOMAIN_ATTR_GEOMETRY attribute
  iommu/amd: Implement DOMAIN_ATTR_GEOMETRY attribute
  ...
2012-07-24 16:24:11 -07:00
Joerg Roedel 395e51f18d Merge branches 'iommu/fixes', 'x86/amd', 'groups', 'arm/tegra' and 'api/domain-attr' into next
Conflicts:
	drivers/iommu/iommu.c
	include/linux/iommu.h
2012-07-23 12:17:00 +02:00
Joerg Roedel 8a0e715b73 iommu/vt-d: Implement DOMAIN_ATTR_GEOMETRY attribute
Implement the attribute for the Intel IOMMU driver.

Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2012-07-11 12:15:45 +02:00
Alex Williamson 783f157bc5 intel-iommu: Make use of DMA quirks and ACS checks in IOMMU groups
Work around broken devices and adhere to ACS support when determining
IOMMU grouping.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2012-06-25 13:48:29 +02:00