Commit Graph

115 Commits

Author SHA1 Message Date
Robin Murphy 2d471b20c5 iommu: Streamline registration interface
Rather than have separate opaque setter functions that are easy to
overlook and lead to repetitive boilerplate in drivers, let's pass the
relevant initialisation parameters directly to iommu_device_register().

Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Link: https://lore.kernel.org/r/ab001b87c533b6f4db71eb90db6f888953986c36.1617285386.git.robin.murphy@arm.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2021-04-16 17:20:45 +02:00
Dafna Hirschfeld b34ea31fe0 iommu/mediatek: Always enable the clk on resume
In mtk_iommu_runtime_resume always enable the clk, even
if m4u_dom is null. Otherwise the 'suspend' cb might
disable the clk which is already disabled causing the warning:

[    1.586104] infra_m4u already disabled
[    1.586133] WARNING: CPU: 0 PID: 121 at drivers/clk/clk.c:952 clk_core_disable+0xb0/0xb8
[    1.594391] mtk-iommu 10205000.iommu: bound 18001000.larb (ops mtk_smi_larb_component_ops)
[    1.598108] Modules linked in:
[    1.598114] CPU: 0 PID: 121 Comm: kworker/0:2 Not tainted 5.12.0-rc5 #69
[    1.609246] mtk-iommu 10205000.iommu: bound 14027000.larb (ops mtk_smi_larb_component_ops)
[    1.617487] Hardware name: Google Elm (DT)
[    1.617491] Workqueue: pm pm_runtime_work
[    1.620545] mtk-iommu 10205000.iommu: bound 19001000.larb (ops mtk_smi_larb_component_ops)

[    1.627229] pstate: 60000085 (nZCv daIf -PAN -UAO -TCO BTYPE=--)
[    1.659297] pc : clk_core_disable+0xb0/0xb8
[    1.663475] lr : clk_core_disable+0xb0/0xb8
[    1.667652] sp : ffff800011b9bbe0
[    1.670959] x29: ffff800011b9bbe0 x28: 0000000000000000
[    1.676267] x27: ffff800011448000 x26: ffff8000100cfd98
[    1.681574] x25: ffff800011b9bd48 x24: 0000000000000000
[    1.686882] x23: 0000000000000000 x22: ffff8000106fad90
[    1.692189] x21: 000000000000000a x20: ffff0000c0048500
[    1.697496] x19: ffff0000c0048500 x18: ffffffffffffffff
[    1.702804] x17: 0000000000000000 x16: 0000000000000000
[    1.708112] x15: ffff800011460300 x14: fffffffffffe0000
[    1.713420] x13: ffff8000114602d8 x12: 0720072007200720
[    1.718727] x11: 0720072007200720 x10: 0720072007200720
[    1.724035] x9 : ffff800011b9bbe0 x8 : ffff800011b9bbe0
[    1.729342] x7 : 0000000000000009 x6 : ffff8000114b8328
[    1.734649] x5 : 0000000000000000 x4 : 0000000000000000
[    1.739956] x3 : 00000000ffffffff x2 : ffff800011460298
[    1.745263] x1 : 1af1d7de276f4500 x0 : 0000000000000000
[    1.750572] Call trace:
[    1.753010]  clk_core_disable+0xb0/0xb8
[    1.756840]  clk_core_disable_lock+0x24/0x40
[    1.761105]  clk_disable+0x20/0x30
[    1.764501]  mtk_iommu_runtime_suspend+0x88/0xa8
[    1.769114]  pm_generic_runtime_suspend+0x2c/0x48
[    1.773815]  __rpm_callback+0xe0/0x178
[    1.777559]  rpm_callback+0x24/0x88
[    1.781041]  rpm_suspend+0xdc/0x470
[    1.784523]  rpm_idle+0x12c/0x170
[    1.787831]  pm_runtime_work+0xa8/0xc0
[    1.791573]  process_one_work+0x1e8/0x360
[    1.795580]  worker_thread+0x44/0x478
[    1.799237]  kthread+0x150/0x158
[    1.802460]  ret_from_fork+0x10/0x30
[    1.806034] ---[ end trace 82402920ef64573b ]---
[    1.810728] ------------[ cut here ]------------

In addition, we now don't need to enable the clock from the
function mtk_iommu_hw_init since it is already enabled by the resume.

Fixes: c0b57581b7 ("iommu/mediatek: Add power-domain operation")
Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
Reviewed-by: Yong Wu <yong.wu@mediatek.com>
Link: https://lore.kernel.org/r/20210416105449.4744-1-dafna.hirschfeld@collabora.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2021-04-16 17:14:51 +02:00
Yong Wu 18d8c74ec5 iommu/mediatek: Allow building as module
The IOMMU in many SoC depends on the MM clocks and power-domain which
are device_initcall normally, thus the subsys_init here is not helpful.
This patch switches it to module_platform_driver which also allow the
driver built as module.

Correspondingly switch the config to tristate, and update the
iommu_ops's owner.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Link: https://lore.kernel.org/r/20210326032337.24578-2-yong.wu@mediatek.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2021-04-07 10:33:58 +02:00
Dan Carpenter a92a90ac62 iommu/mediatek: Fix error code in probe()
This error path is supposed to return -EINVAL.  It used to return
directly but we added some clean up and accidentally removed the
error code.  Also I fixed a typo in the error message.

Fixes: c0b57581b7 ("iommu/mediatek: Add power-domain operation")
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Reviewed-by: Yong Wu <yong.wu@mediatek.com>
Link: https://lore.kernel.org/r/YB0+GU5akSdu29Vu@mwanda
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2021-02-08 09:11:06 +01:00
Colin Ian King 7a5661739d iommu/mediatek: Fix unsigned domid comparison with less than zero
Currently the check for domid < 0 is always false because domid
is unsigned. Fix this by casting domid to an int before making
the comparison.

Signed-off-by: Colin Ian King <colin.king@canonical.com>
Acked-by: Will Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/20210204150001.102672-1-colin.king@canonical.com
Addresses-Coverity: ("Unsigned comparison against 0")
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2021-02-05 09:49:47 +01:00
Yong Wu 9e3489e06f iommu/mediatek: Add mt8192 support
Add mt8192 iommu support.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-33-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01 11:31:19 +00:00
Yong Wu 23357572be iommu/mediatek: Remove unnecessary check in attach_device
This priv_data is set in the of_xlate. if of_xlate failed, it should
not enter attach_device. remove the unnecessary check.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-32-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01 11:31:19 +00:00
Yong Wu 8d2c749e52 iommu/mediatek: Support master use iova over 32bit
After extending v7s, our pagetable already support iova reach
16GB(34bit). the master got the iova via dma_alloc_attrs may reach
34bits, but its HW register still is 32bit. then how to set the
bit32/bit33 iova? this depend on a SMI larb setting(bank_sel).

we separate whole 16GB iova to four banks:
bank: 0: 0~4G; 1: 4~8G; 2: 8-12G; 3: 12-16G;
The bank number is (iova >> 32).

We will preassign which bank the larbs belong to. currently we don't
have a interface for master to adjust its bank number.

Each a bank is a iova_region which is a independent iommu-domain.
the iova range for each iommu-domain can't cross 4G.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Acked-by: Krzysztof Kozlowski <krzk@kernel.org> #for memory part
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-31-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01 11:31:19 +00:00
Yong Wu ab1d5281a6 iommu/mediatek: Add iova reserved function
For multiple iommu_domains, we need to reserve some iova regions. Take a
example, If the default iova region is 0 ~ 4G, but the 0x4000_0000 ~
0x43ff_ffff is only for the special CCU0 domain. Thus we should exclude
this region for the default iova region.

Signed-off-by: Anan sun <anan.sun@mediatek.com>
Signed-off-by: Chao Hao <chao.hao@mediatek.com>
Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-30-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01 11:31:19 +00:00
Yong Wu c3045f3924 iommu/mediatek: Support for multi domains
Some HW IP(ex: CCU) require the special iova range. That means the iova
got from dma_alloc_attrs for that devices must locate in his special range.
In this patch, we prepare a iommu group(domain) for each a iova range
requirement.

Meanwhile we still use one pagetable which support 16GB iova.

After this patch, If the iova range of a master is over 4G, the master
should:
a) Declare its special dma-ranges in its dtsi node. For example, If we
   preassign the iova 4G-8G for vcodec, then the vcodec dtsi node should
   add this:
   /*
    * iova start at 0x1_0000_0000, pa still start at 0x4000_0000
    * size is 0x1_0000_0000.
    */
   dma-ranges = <0x1 0x0 0x0 0x40000000 0x1 0x0>;  /* 4G ~ 8G */
 Note: we don't have a actual bus concept here. the master doesn't have its
 special parent node, thus this dma-ranges can only be put in the master's
 node.

b) Update the dma_mask:
  dma_set_mask_and_coherent(dev, DMA_BIT_MASK(33));

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-29-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01 11:31:19 +00:00
Yong Wu 803cf9e5a6 iommu/mediatek: Add get_domain_id from dev->dma_range_map
Add a new interface _get_domain_id from dev->dma_range_map,
The iommu consumer device will use dma-ranges in dtsi node to indicate
its dma address region requirement. In this iommu driver, we will get
the requirement and decide which iova domain it should locate.

In the lastest SoC, there will be several iova-regions(domains), we will
compare and calculate which domain is right. If the start/end of device
requirement equal some region. it is best fit of course. If it is inside
some region, it is also ok. the iova requirement of a device should not
be inside two or more regions.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-28-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01 11:31:19 +00:00
Yong Wu 585e58f498 iommu/mediatek: Add iova_region structure
Add a new structure for the iova_region. Each a region will be a
independent iommu domain.

For the previous SoC, there is single iova region(0~4G). For the SoC
that need support multi-domains, there will be several regions.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-27-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01 11:31:19 +00:00
Yong Wu b7875eb945 iommu/mediatek: Move geometry.aperture updating into domain_finalise
Move the domain geometry.aperture updating into domain_finalise.
This is a preparing patch for updating the domain region. We know the
detailed iova region in the attach_device.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-26-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01 11:31:19 +00:00
Yong Wu 4f956c97d2 iommu/mediatek: Move domain_finalise into attach_device
Currently domain_alloc only has a parameter(type), We have no chance to
input some special data. This patch moves the domain_finalise into
attach_device which has the device information, then could update
the domain's geometry.aperture ranges for each a device.

Strictly, I should use the data from mtk_iommu_get_m4u_data as the
parameter of mtk_iommu_domain_finalise in this patch. but dom->data
only is used in tlb ops in which the data is get from the m4u_list, thus
it is ok here.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-25-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01 11:31:18 +00:00
Yong Wu 08500c43d4 iommu/mediatek: Adjust the structure
Add "struct mtk_iommu_data *" in the "struct mtk_iommu_domain",
reduce the call mtk_iommu_get_m4u_data().
No functional change.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-24-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01 11:31:18 +00:00
Yong Wu ef0f0986b6 iommu/mediatek: Support report iova 34bit translation fault in ISR
If the iova is over 32bit, the fault status register bit is a little
different.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-23-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01 11:31:18 +00:00
Yong Wu bfed873114 iommu/mediatek: Support up to 34bit iova in tlb flush
If the iova is 34bit, the iova[32][33] is the bit0/1 in the tlb flush
register. Add a new macro for this.

In the macro, since (iova + size - 1) may be end with 0xfff, then the
bit0/1 always is 1, thus add a mask.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-22-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01 11:31:18 +00:00
Yong Wu c0b57581b7 iommu/mediatek: Add power-domain operation
In the previous SoC, the M4U HW is in the EMI power domain which is
always on. the latest M4U is in the display power domain which may be
turned on/off, thus we have to add pm_runtime interface for it.

When the engine work, the engine always enable the power and clocks for
smi-larb/smi-common, then the M4U's power will always be powered on
automatically via the device link with smi-common.

Note: we don't enable the M4U power in iommu_map/unmap for tlb flush.
If its power already is on, of course it is ok. if the power is off,
the main tlb will be reset while M4U power on, thus the tlb flush while
m4u power off is unnecessary, just skip it.
Therefore, we increase the ref_count for pm when pm status is ACTIVE,
otherwise, skip it. Meanwhile, the tlb_flush_range is called so often,
thus, update pm ref_count while the SoC has power-domain to avoid touch the
dev->power.lock. and the tlb_flush_all only is called when boot, so no
need check if the SoC has power-domain to keep code clean.

There will be one case that pm runctime status is not expected when tlb
flush. After boot, the display may call dma_alloc_attrs before it call
pm_runtime_get(disp-dev), then the m4u's pm status is not active inside
the dma_alloc_attrs. Since it only happens after boot, the tlb is clean
at that time, I also think this is ok.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-21-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01 11:31:18 +00:00
Yong Wu 34665c7929 iommu/mediatek: Add pm runtime callback
In pm runtime case, all the registers backup/restore and bclk are
controlled in the pm_runtime callback, Rename the original
suspend/resume to the runtime_suspend/resume.

Use pm_runtime_force_suspend/resume as the normal suspend/resume.
iommu should suspend after iommu consumer devices, thus use _LATE_.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-20-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01 11:31:18 +00:00
Yong Wu baf94e6ebf iommu/mediatek: Add device link for smi-common and m4u
In the lastest SoC, M4U has its special power domain. thus, If the engine
begin to work, it should help enable the power for M4U firstly.
Currently if the engine work, it always enable the power/clocks for
smi-larbs/smi-common. This patch adds device_link for smi-common and M4U.
then, if smi-common power is enabled, the M4U power also is powered on
automatically.

Normally M4U connect with several smi-larbs and their smi-common always
are the same, In this patch it get smi-common dev from the last smi-larb
device, then add the device_link.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-19-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01 11:31:18 +00:00
Yong Wu 986d9ec5f1 iommu/mediatek: Add error handle for mtk_iommu_probe
In the original code, we lack the error handle. This patch adds them.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-18-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01 11:31:17 +00:00
Yong Wu 7f37a91dc8 iommu/mediatek: Move hw_init into attach_device
In attach device, it will update the pagetable base address register.
Move the hw_init function also here. Then it only need call
pm_runtime_get/put one time here if m4u has power domain.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-17-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01 11:31:17 +00:00
Yong Wu 9bdfe4c175 iommu/mediatek: Update oas for v7s
This patch only updates oas in different SoCs.

If the SoC supports 4GB-mode and current dram size is 4GB, the oas is 33.
otherwise, it's still 32. In the lastest SoC, the oas is 35bits.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-16-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01 11:31:17 +00:00
Yong Wu 2f317da433 iommu/mediatek: Add a flag for iova 34bits case
Add a HW flag for if the HW support 34bit IOVA. the previous SoC
still use 32bit. normally the lvl1 pgtable size is 16KB when ias == 32.
if ias == 34, lvl1 pgtable size is 16KB * 4. The purpose of this patch
is to save 16KB*3 continuous memory for the previous SoC.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-15-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01 11:31:17 +00:00
Yong Wu 40596d2f2b iommu/io-pgtable-arm-v7s: Extend PA34 for MediaTek
MediaTek extend the bit5 in lvl1 and lvl2 descriptor as PA34.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Acked-by: Will Deacon <will@kernel.org>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-11-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01 11:31:17 +00:00
Yong Wu 66a2891554 iommu/mediatek: Use the common mtk-memory-port.h
Use the common memory header(larb-port) in the source code.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Acked-by: Krzysztof Kozlowski <krzk@kernel.org>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Link: https://lore.kernel.org/r/20210111111914.22211-9-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-02-01 11:31:16 +00:00
Yong Wu 0954d61a59 iommu/mediatek: Remove the tlb-ops for v7s
Until now, we have already used the tlb operations from iommu framework,
then the tlb operations for v7s can be removed.

Correspondingly, Switch the paramenter "cookie" to the internal structure.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Acked-by: Will Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/20210107122909.16317-8-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-01-27 12:32:27 +00:00
Yong Wu f21ae3b100 iommu/mediatek: Gather iova in iommu_unmap to achieve tlb sync once
In current iommu_unmap, this code is:

	iommu_iotlb_gather_init(&iotlb_gather);
	ret = __iommu_unmap(domain, iova, size, &iotlb_gather);
	iommu_iotlb_sync(domain, &iotlb_gather);

We could gather the whole iova range in __iommu_unmap, and then do tlb
synchronization in the iommu_iotlb_sync.

This patch implement this, Gather the range in mtk_iommu_unmap.
then iommu_iotlb_sync call tlb synchronization for the gathered iova range.
we don't call iommu_iotlb_gather_add_page since our tlb synchronization
could be regardless of granule size.

In this way, gather->start is impossible ULONG_MAX, remove the checking.

This patch aims to do tlb synchronization *once* in the iommu_unmap.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Acked-by: Will Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/20210107122909.16317-7-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-01-27 12:32:27 +00:00
Yong Wu 862c3715de iommu: Switch gather->end to the inclusive end
Currently gather->end is "unsigned long" which may be overflow in
arch32 in the corner case: 0xfff00000 + 0x100000(iova + size).
Although it doesn't affect the size(end - start), it affects the checking
"gather->end < end"

This patch changes this "end" to the real end address
(end = start + size - 1). Correspondingly, update the length to
"end - start + 1".

Fixes: a7d20dc19d ("iommu: Introduce struct iommu_iotlb_gather for batching TLB flushes")
Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Acked-by: Will Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/20210107122909.16317-5-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-01-27 12:32:27 +00:00
Yong Wu 20143451ef iommu/mediatek: Add iotlb_sync_map to sync whole the iova range
Remove IO_PGTABLE_QUIRK_TLBI_ON_MAP to avoid tlb sync for each a small
chunk memory, Use the new iotlb_sync_map to tlb_sync once for whole the
iova range of iommu_map.

Signed-off-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Acked-by: Will Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/20210107122909.16317-4-yong.wu@mediatek.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-01-27 12:32:26 +00:00
Robin Murphy fefe8527a1 iommu/io-pgtable: Remove tlb_flush_leaf
The only user of tlb_flush_leaf is a particularly hairy corner of the
Arm short-descriptor code, which wants a synchronous invalidation to
minimise the races inherent in trying to split a large page mapping.
This is already far enough into "here be dragons" territory that no
sensible caller should ever hit it, and thus it really doesn't need
optimising. Although using tlb_flush_walk there may technically be
more heavyweight than needed, it does the job and saves everyone else
having to carry around useless baggage.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Steven Price <steven.price@arm.com>
Link: https://lore.kernel.org/r/9844ab0c5cb3da8b2f89c6c2da16941910702b41.1606324115.git.robin.murphy@arm.com
Signed-off-by: Will Deacon <will@kernel.org>
2020-12-08 15:23:37 +00:00
Fabien Parent 3c213562fb iommu/mediatek: Add support for MT8167
Add support for the IOMMU on MT8167

Signed-off-by: Fabien Parent <fparent@baylibre.com>
Reviewed-by: Yong Wu <yong.wu@mediatek.com>
Link: https://lore.kernel.org/r/20200907101649.1573134-3-fparent@baylibre.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-09-18 10:29:12 +02:00
Fabien Parent d1b5ef0054 iommu/mediatek: Add flag for legacy ivrp paddr
Add a new flag in order to select which IVRP_PADDR format is used
by an SoC.

Signed-off-by: Fabien Parent <fparent@baylibre.com>
Reviewed-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Matthias Brugger <matthias.bgg@gmail.com>
Link: https://lore.kernel.org/r/20200907101649.1573134-2-fparent@baylibre.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-09-18 10:29:07 +02:00
Miles Chen c2c59456e1 iommu/mediatek: Check 4GB mode by reading infracfg
In previous discussion [1] and [2], we found that it is risky to
use max_pfn or totalram_pages to tell if 4GB mode is enabled.

Check 4GB mode by reading infracfg register, remove the usage
of the un-exported symbol max_pfn.

This is a step towards building mtk_iommu as a kernel module.

[1] https://lore.kernel.org/lkml/20200603161132.2441-1-miles.chen@mediatek.com/
[2] https://lore.kernel.org/lkml/20200604080120.2628-1-miles.chen@mediatek.com/
[3] https://lore.kernel.org/lkml/20200715205120.GA778876@bogus/

Cc: Mike Rapoport <rppt@linux.ibm.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: Yong Wu <yong.wu@mediatek.com>
Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Rob Herring <robh@kernel.org>
Cc: Matthias Brugger <matthias.bgg@gmail.com>
Cc: Joerg Roedel <joro@8bytes.org>
Reviewed-by: Matthias Brugger <matthias.bgg@gmail.com>
Signed-off-by: Miles Chen <miles.chen@mediatek.com>
Link: https://lore.kernel.org/r/20200904104038.4979-1-miles.chen@mediatek.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-09-04 13:20:36 +02:00
Krzysztof Kozlowski f53dd97874 iommu/mediatek: Drop of_match_ptr to fix -Wunused-const-variable
The of_device_id is included unconditionally by of.h header and used
in the driver as well.  Remove of_match_ptr to fix W=1 compile test
warning with !CONFIG_OF:

    drivers/iommu/mtk_iommu.c:833:34: warning: 'mtk_iommu_of_ids' defined but not used [-Wunused-const-variable=]
      833 | static const struct of_device_id mtk_iommu_of_ids[] = {

Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Krzysztof Kozlowski <krzk@kernel.org>
Link: https://lore.kernel.org/r/20200727181842.8441-1-krzk@kernel.org
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-09-04 10:40:49 +02:00
Joerg Roedel 56fbacc9bf Merge branches 'arm/renesas', 'arm/qcom', 'arm/mediatek', 'arm/omap', 'arm/exynos', 'arm/smmu', 'ppc/pamu', 'x86/vt-d', 'x86/amd' and 'core' into next 2020-07-29 14:42:00 +02:00
Baolin Wang f34ce7a701 iommu: Add gfp parameter to io_pgtable_ops->map()
Now the ARM page tables are always allocated by GFP_ATOMIC parameter,
but the iommu_ops->map() function has been added a gfp_t parameter by
commit 781ca2de89 ("iommu: Add gfp parameter to iommu_ops::map"),
thus io_pgtable_ops->map() should use the gfp parameter passed from
iommu_ops->map() to allocate page pages, which can avoid wasting the
memory allocators atomic pools for some non-atomic contexts.

Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Acked-by: Will Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/3093df4cb95497aaf713fca623ce4ecebb197c2e.1591930156.git.baolin.wang@linux.alibaba.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-07-24 14:29:47 +02:00
Chao Hao 068c86e92f iommu/mediatek: Add mt6779 basic support
1. Start from mt6779, INVLDT_SEL move to offset=0x2c, so we add
   REG_MMU_INV_SEL_GEN2 definition and mt6779 uses it.
2. Add mt6779_data to support mm_iommu HW init.

Signed-off-by: Chao Hao <chao.hao@mediatek.com>
Reviewed-by: Matthias Brugger <matthias.bgg@gmail.com>
Cc: Yong Wu <yong.wu@mediatek.com>
Link: https://lore.kernel.org/r/20200703044127.27438-11-chao.hao@mediatek.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-07-10 16:13:11 +02:00
Chao Hao 864444130e iommu/mediatek: Modify MMU_CTRL register setting
The MMU_CTRL register of MT8173 is different from other SoCs.
The in_order_wr_en is bit[9] which is zero by default.
Other SoCs have the vitcim_tlb_en feature mapped to bit[12].
This bit is set to one by default. We need to preserve the bit
when setting F_MMU_TF_PROT_TO_PROGRAM_ADDR as otherwise the
bit will be cleared and IOMMU performance will drop.

Signed-off-by: Chao Hao <chao.hao@mediatek.com>
Reviewed-by: Matthias Brugger <matthias.bgg@gmail.com>
Cc: Matthias Brugger <matthias.bgg@gmail.com>
Cc: Yong Wu <yong.wu@mediatek.com>
Link: https://lore.kernel.org/r/20200703044127.27438-10-chao.hao@mediatek.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-07-10 16:13:11 +02:00
Chao Hao 829316b3bc iommu/mediatek: Extend protect pa alignment value
Starting with mt6779, iommu needs to extend to 256 bytes from 128
bytes which can send the max number of data for memory protection
pa alignment. So we can use a separate patch to modify it.

Signed-off-by: Chao Hao <chao.hao@mediatek.com>
Reviewed-by: Matthias Brugger <matthias.bgg@gmail.com>
Link: https://lore.kernel.org/r/20200703044127.27438-9-chao.hao@mediatek.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-07-10 16:13:11 +02:00
Chao Hao 35c1b48d27 iommu/mediatek: Add REG_MMU_WR_LEN_CTRL register definition
Some platforms(ex: mt6779) need to improve performance by setting
REG_MMU_WR_LEN_CTRL register. And we can use WR_THROT_EN macro to control
whether we need to set the register. If the register uses default value,
iommu will send command to EMI without restriction, when the number of
commands become more and more, it will drop the EMI performance. So when
more than ten_commands(default value) don't be handled for EMI, iommu will
stop send command to EMI for keeping EMI's performace by enabling write
throttling mechanism(bit[5][21]=0) in MMU_WR_LEN_CTRL register.

Signed-off-by: Chao Hao <chao.hao@mediatek.com>
Reviewed-by: Matthias Brugger <matthias.bgg@gmail.com>
Cc: Matthias Brugger <matthias.bgg@gmail.com>
Link: https://lore.kernel.org/r/20200703044127.27438-8-chao.hao@mediatek.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-07-10 16:13:11 +02:00
Chao Hao 37276e00da iommu/mediatek: Add sub_comm id in translation fault
The max larb number that a iommu HW support is 8(larb0~larb7 in the below
diagram).
If the larb's number is over 8, we use a sub_common for merging
several larbs into one larb. At this case, we will extend larb_id:
bit[11:9] means common-id;
bit[8:7] means subcommon-id;
>From these two variables, we could get the real larb number when
translation fault happen.
The diagram is as below:
		 EMI
		  |
		IOMMU
		  |
           -----------------
	   |               |
	common1   	common0
	   |		   |
	   -----------------
		  |
             smi common
		  |
  ------------------------------------
  |       |       |       |     |    |
 3'd0    3'd1    3'd2    3'd3  ...  3'd7   <-common_id(max is 8)
  |       |       |       |     |    |
Larb0   Larb1     |     Larb3  ... Larb7
		  |
	    smi sub common
		  |
     --------------------------
     |        |       |       |
    2'd0     2'd1    2'd2    2'd3   <-sub_common_id(max is 4)
     |        |       |       |
   Larb8    Larb9   Larb10  Larb11

In this patch we extend larb_remap[] to larb_remap[8][4] for this.
larb_remap[x][y]: x means common-id above, y means subcommon_id above.

We can also distinguish if the M4U HW has sub_common by HAS_SUB_COMM
macro.

Signed-off-by: Chao Hao <chao.hao@mediatek.com>
Reviewed-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Matthias Brugger <matthias.bgg@gmail.com>
Cc: Matthias Brugger <matthias.bgg@gmail.com>
Link: https://lore.kernel.org/r/20200703044127.27438-7-chao.hao@mediatek.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-07-10 16:13:11 +02:00
Chao Hao b053bc7183 iommu/mediatek: Move inv_sel_reg into the plat_data
For mt6779, MMU_INV_SEL register's offset is changed from
0x38 to 0x2c, so we can put inv_sel_reg in the plat_data to
use it.
In addition, we renamed it to REG_MMU_INV_SEL_GEN1 and use it
before mt6779.

Signed-off-by: Chao Hao <chao.hao@mediatek.com>
Reviewed-by: Matthias Brugger <matthias.bgg@gmail.com>
Cc: Yong Wu <yong.wu@mediatek.com>
Link: https://lore.kernel.org/r/20200703044127.27438-6-chao.hao@mediatek.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-07-10 16:13:11 +02:00
Chao Hao 4bb2bf4c6a iommu/mediatek: Setting MISC_CTRL register
Add F_MMU_IN_ORDER_WR_EN_MASK and F_MMU_STANDARD_AXI_MODE_EN_MASK
definitions in MISC_CTRL register.
F_MMU_STANDARD_AXI_MODE_EN_MASK:
If we set F_MMU_STANDARD_AXI_MODE_EN_MASK (bit[3][19] = 0, not follow
standard AXI protocol), the iommu will priorize sending of urgent read
command over a normal read command. This improves the performance.
F_MMU_IN_ORDER_WR_EN_MASK:
If we set F_MMU_IN_ORDER_WR_EN_MASK (bit[1][17] = 0, out-of-order write),
the iommu will re-order write commands and send the write commands with
higher priority. Otherwise the sending of write commands will be done in
order. The feature is controlled by OUT_ORDER_WR_EN platform data flag.

Suggested-by: Yong Wu <yong.wu@mediatek.com>
Signed-off-by: Chao Hao <chao.hao@mediatek.com>
Reviewed-by: Matthias Brugger <matthias.bgg@gmail.com>
Cc: Matthias Brugger <matthias.bgg@gmail.com>
Link: https://lore.kernel.org/r/20200703044127.27438-5-chao.hao@mediatek.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-07-10 16:13:10 +02:00
Chao Hao 6b71779622 iommu/mediatek: Use a u32 flags to describe different HW features
Given the fact that we are adding more and more plat_data bool values,
it would make sense to use a u32 flags register and add the appropriate
macro definitions to set and check for a flag present.
No functional change.

Suggested-by: Matthias Brugger <matthias.bgg@gmail.com>
Signed-off-by: Chao Hao <chao.hao@mediatek.com>
Reviewed-by: Matthias Brugger <matthias.bgg@gmail.com>
Cc: Yong Wu <yong.wu@mediatek.com>
Link: https://lore.kernel.org/r/20200703044127.27438-4-chao.hao@mediatek.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-07-10 16:13:10 +02:00
Chao Hao 75eed35087 iommu/mediatek: Rename the register STANDARD_AXI_MODE(0x48) to MISC_CTRL
For iommu offset=0x48 register, only the previous mt8173/mt8183 use the
name STANDARD_AXI_MODE, all the latest SoC extend the register more
feature by different bits, for example: axi_mode, in_order_en, coherent_en
and so on. So rename REG_MMU_MISC_CTRL may be more proper.

This patch only rename the register name, no functional change.

Signed-off-by: Chao Hao <chao.hao@mediatek.com>
Reviewed-by: Yong Wu <yong.wu@mediatek.com>
Reviewed-by: Matthias Brugger <matthias.bgg@gmail.com>
Link: https://lore.kernel.org/r/20200703044127.27438-3-chao.hao@mediatek.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-07-10 16:13:10 +02:00
Joerg Roedel 80e4592a77 iommu/mediatek: Convert to probe/release_device() call-backs
Convert the Mediatek IOMMU driver to use the probe_device() and
release_device() call-backs of iommu_ops, so that the iommu core code
does the group and sysfs setup.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
Link: https://lore.kernel.org/r/20200429133712.31431-23-joro@8bytes.org
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-05-05 14:36:13 +02:00
Joerg Roedel 3524b5592c iommu/mediatek: Use accessor functions for iommu private data
Make use of dev_iommu_priv_set/get() functions.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
Reviewed-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
Link: https://lore.kernel.org/r/20200326150841.10083-14-joro@8bytes.org
2020-03-27 11:14:52 +01:00
Robin Murphy d1e5f26f14 iommu/io-pgtable-arm: Rationalise TTBRn handling
TTBR1 values have so far been redundant since no users implement any
support for split address spaces. Crucially, though, one of the main
reasons for wanting to do so is to be able to manage each half entirely
independently, e.g. context-switching one set of mappings without
disturbing the other. Thus it seems unlikely that tying two tables
together in a single io_pgtable_cfg would ever be particularly desirable
or useful.

Streamline the configs to just a single conceptual TTBR value
representing the allocated table. This paves the way for future users to
support split address spaces by simply allocating a table and dealing
with the detailed TTBRn logistics themselves.

Tested-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
[will: Drop change to ttbr value]
Signed-off-by: Will Deacon <will@kernel.org>
2020-01-10 15:39:23 +00:00
Joerg Roedel 9b3a713fee Merge branches 'iommu/fixes', 'arm/qcom', 'arm/renesas', 'arm/rockchip', 'arm/mediatek', 'arm/tegra', 'arm/smmu', 'x86/amd', 'x86/vt-d', 'virtio' and 'core' into next 2019-11-12 17:11:25 +01:00