The GIC interface numbering does not necessarily follow the logical
CPU numbering, especially for complex topologies such as multi-cluster
systems.
Fortunately we can easily probe the GIC to create a mapping as the
Interrupt Processor Targets Registers for the first 32 interrupts are
read-only, and each field returns a value that always corresponds to
the processor reading the register.
Initially all mappings target all CPUs in case an IPI is required to
boot secondary CPUs. It is refined as those CPUs discover what their
actual mapping is.
Signed-off-by: Nicolas Pitre <nico@linaro.org>
Acked-by: Will Deacon <will.deacon@arm.com>
Pull more ARM updates from Russell King.
This got a fair number of conflicts with the <asm/system.h> split, but
also with some other sparse-irq and header file include cleanups. They
all looked pretty trivial, though.
* 'for-linus' of git://git.linaro.org/people/rmk/linux-arm: (59 commits)
ARM: fix Kconfig warning for HAVE_BPF_JIT
ARM: 7361/1: provide XIP_VIRT_ADDR for no-MMU builds
ARM: 7349/1: integrator: convert to sparse irqs
ARM: 7259/3: net: JIT compiler for packet filters
ARM: 7334/1: add jump label support
ARM: 7333/2: jump label: detect %c support for ARM
ARM: 7338/1: add support for early console output via semihosting
ARM: use set_current_blocked() and block_sigmask()
ARM: exec: remove redundant set_fs(USER_DS)
ARM: 7332/1: extract out code patch function from kprobes
ARM: 7331/1: extract out insn generation code from ftrace
ARM: 7330/1: ftrace: use canonical Thumb-2 wide instruction format
ARM: 7351/1: ftrace: remove useless memory checks
ARM: 7316/1: kexec: EOI active and mask all interrupts in kexec crash path
ARM: Versatile Express: add NO_IOPORT
ARM: get rid of asm/irq.h in asm/prom.h
ARM: 7319/1: Print debug info for SIGBUS in user faults
ARM: 7318/1: gic: refactor irq_start assignment
ARM: 7317/1: irq: avoid NULL check in for_each_irq_desc loop
ARM: 7315/1: perf: add support for the Cortex-A7 PMU
...
The irq_start and hwirq_base assignment code is fairly hairy and ended
up being difficult to read following a conflict resolution for 3.2.
This patch rearranges the code slightly to make it easier to read.
Cc: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
This patch removes the simplistic implementation of irq_domains and enables
the powerpc infrastructure for all irq_domain users. The powerpc
infrastructure includes support for complex mappings between Linux and
hardware irq numbers, and can manage allocation of irq_descs.
This patch also converts the few users of irq_domain_add()/irq_domain_del()
to call irq_domain_add_legacy() instead.
v3: Fix bug that set up too many irqs in translation range.
v2: Fix removal of irq_alloc_descs() call in gic driver
Signed-off-by: Grant Likely <grant.likely@secretlab.ca>
Cc: Rob Herring <rob.herring@calxeda.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Milton Miller <miltonm@bga.com>
Tested-by: Olof Johansson <olof@lixom.net>
Part of the series to unify the irq remapping mechanisms in the
kernel. A follow up patch will copy the powerpc implementation into
kernel/irq/irqdomain.c, which will be a lot easier if the structures
are identical.
Where they differ, I've chose to use the powerpc names since there is
a lot more code using those names.
Signed-off-by: Grant Likely <grant.likely@secretlab.ca>
Cc: Rob Herring <rob.herring@calxeda.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Milton Miller <miltonm@bga.com>
Tested-by: Olof Johansson <olof@lixom.net>
It turns out that the logical CPU mapping is useful even when !CONFIG_SMP
for manipulation of devices like interrupt and power controllers when
running a UP kernel on a CPU other than 0. This can happen when kexecing
a UP image from an SMP kernel.
In the future, multi-cluster systems running AMP configurations will
require something similar for mapping cluster IDs, so it makes sense to
decouple this logic in preparation for this support.
Acked-by: Yang Bai <hamo.by@gmail.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Reported-by: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Commit 4294f8baa ("ARM: gic: add irq_domain support") defines irq_start
as irq_start = (irq_start & ~31) + 16; On a platform with a GIC and a
CPU without PPIs, this results in irq_start being off by 16.
This patch fixes gic_init so that we only carve out a PPI space when
PPIs exist for the GIC being initialised.
Cc: Rob Herring <rob.herring@calxeda.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
When multiple GICs exist on a platform (RealView PB1176/11MP),
we must make sure the PM notifier block is only registered
once, otherwise we end up corrupting the PM notifier list.
The fix is to only register the notifier when initializing
the first GIC, as the power management functions seem
to iterate over all the registered GICs.
Tested on PB11MP and PB1176.
Reported-by: Will Deacon <will.deacon@arm.com>
Tested-by: Will Deacon <will.deacon@arm.com>
Cc: Colin Cross <ccross@android.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Now that MULTI_IRQ_HANDLER is selected by all the in-tree
GIC users, make it mandatory and remove the unused macros.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Provide the GIC code with a low level handler that can be used
by platforms using CONFIG_MULTI_IRQ_HANDLER.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
The GIC support code is heavily using the fact that hardware
implementations are exposing banked registers. Unfortunately, it
looks like at least one GIC implementation (EXYNOS) offers both
the distributor and the CPU interfaces at different addresses,
depending on the CPU.
This problem is solved by allowing the distributor and CPU interface
addresses to be per-cpu variables for the platforms that require it.
The EXYNOS code is updated not to mess with the GIC internals while
handling interrupts, and struct gic_chip_data is back to being private.
The DT binding for the gic is updated to allow an optional "cpu-offset"
value, which is used to compute the various base addresses.
Finally, a new config option (GIC_NON_BANKED) is used to control this
feature, so the overhead is only present on kernels compiled with
support for EXYNOS.
Tested on Origen (EXYNOS4) and Panda (OMAP4).
Cc: Kukjin Kim <kgene.kim@samsung.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Thomas Abraham <thomas.abraham@linaro.org>
Acked-by: Rob Herring <rob.herring@calxeda.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
* 'next/dt' of git://git.linaro.org/people/arnd/arm-soc:
ARM: gic: use module.h instead of export.h
ARM: gic: fix irq_alloc_descs handling for sparse irq
ARM: gic: add OF based initialization
ARM: gic: add irq_domain support
irq: support domains with non-zero hwirq base
of/irq: introduce of_irq_init
ARM: at91: add at91sam9g20 and Calao USB A9G20 DT support
ARM: at91: dt: at91sam9g45 family and board device tree files
arm/mx5: add device tree support for imx51 babbage
arm/mx5: add device tree support for imx53 boards
ARM: msm: Add devicetree support for msm8660-surf
msm_serial: Add devicetree support
msm_serial: Use relative resources for iomem
Fix up conflicts in arch/arm/mach-at91/{at91sam9260.c,at91sam9g45.c}
The module.h cleanup series is not merged at this point, so use the
older header file for now, to make it build either way.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Commit "ARM: gic: add irq_domain support" (b49b6ff) breaks SPARSE_IRQ
on platforms with GIC. When SPARSE_IRQ is enabled, all NR_IRQS or
mach_desc->nr_irqs will be allocated by arch_probe_nr_irqs(). This caused
irq_alloc_descs to allocate irq_descs after the pre-allocated space.
Make irq_alloc_descs search for an exact irq range and assume it has
been pre-allocated on failure. For DT probing dynamic allocation is used.
DT enabled platforms should set their nr_irqs to NR_IRQ_LEGACY and have all
irq_chips allocate their irq_descs with irq_alloc_descs if SPARSE_IRQ is
enabled.
gic_init irq_start param is changed to be signed with negative meaning do
dynamic Linux irq assigment.
Signed-off-by: Rob Herring <rob.herring@calxeda.com>
This adds ARM gic interrupt controller initialization using device tree
data.
The initialization function is intended to be called by of_irq_init
function like this:
const static struct of_device_id irq_match[] = {
{ .compatible = "arm,cortex-a9-gic", .data = gic_of_init, },
{}
};
static void __init init_irqs(void)
{
of_irq_init(irq_match);
}
Signed-off-by: Rob Herring <rob.herring@calxeda.com>
Reviewed-by: Jamie Iles <jamie@jamieiles.com>
Tested-by: Thomas Abraham <thomas.abraham@linaro.org>
Acked-by: Grant Likely <grant.likely@secretlab.ca>
Convert the gic interrupt controller to use irq domains in preparation
for device-tree binding and MULTI_IRQ. This allows for translation between
GIC interrupt IDs and Linux irq numbers.
The meaning of irq_offset has changed. It now is just the number of skipped
GIC interrupt IDs for the controller. It will be 16 for primary GIC and 32
for secondary GICs.
Signed-off-by: Rob Herring <rob.herring@calxeda.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Jamie Iles <jamie@jamieiles.com>
Tested-by: Thomas Abraham <thomas.abraham@linaro.org>
Acked-by: Grant Likely <grant.likely@secretlab.ca>
* 'core-locking-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (27 commits)
rtmutex: Add missing rcu_read_unlock() in debug_rt_mutex_print_deadlock()
lockdep: Comment all warnings
lib: atomic64: Change the type of local lock to raw_spinlock_t
locking, lib/atomic64: Annotate atomic64_lock::lock as raw
locking, x86, iommu: Annotate qi->q_lock as raw
locking, x86, iommu: Annotate irq_2_ir_lock as raw
locking, x86, iommu: Annotate iommu->register_lock as raw
locking, dma, ipu: Annotate bank_lock as raw
locking, ARM: Annotate low level hw locks as raw
locking, drivers/dca: Annotate dca_lock as raw
locking, powerpc: Annotate uic->lock as raw
locking, x86: mce: Annotate cmci_discover_lock as raw
locking, ACPI: Annotate c3_lock as raw
locking, oprofile: Annotate oprofilefs lock as raw
locking, video: Annotate vga console lock as raw
locking, latencytop: Annotate latency_lock as raw
locking, timer_stats: Annotate table_lock as raw
locking, rwsem: Annotate inner lock as raw
locking, semaphores: Annotate inner lock as raw
locking, sched: Annotate thread_group_cputimer as raw
...
Fix up conflicts in kernel/posix-cpu-timers.c manually: making
cputimer->cputime a raw lock conflicted with the ABBA fix in commit
bcd5cff721 ("cputimer: Cure lock inversion").
This patch remove the hardcoded link between local timers and PPIs,
and convert the PPI users (TWD, MCT and MSM timers) to the new
*_percpu_irq interface. Also some collateral cleanup
(local_timer_ack() is gone, and the interrupt handler is strictly
private to each driver).
PPIs are now useable for more than just the local timers.
Additional testing by David Brown (msm8250 and msm8660) and
Shawn Guo (imx6q).
Cc: David Brown <davidb@codeaurora.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Acked-by: David Brown <davidb@codeaurora.org>
Tested-by: David Brown <davidb@codeaurora.org>
Tested-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
PPI handling is a bit of an odd beast. It uses its own low level
handling code and is hardwired to the local timers (hence lacking
a registration interface).
Instead, switch the low handling to the normal SPI handling code.
PPIs are handled by the handle_percpu_devid_irq flow.
This also allows the removal of some duplicated code.
Cc: Kukjin Kim <kgene.kim@samsung.com>
Cc: David Brown <davidb@codeaurora.org>
Cc: Bryan Huntsman <bryanh@codeaurora.org>
Cc: Tony Lindgren <tony@atomide.com>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: Magnus Damm <magnus.damm@gmail.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Acked-by: David Brown <davidb@codeaurora.org>
Tested-by: David Brown <davidb@codeaurora.org>
Tested-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
The GIC driver must convert logical CPU numbers passed in from Linux
into physical CPU numbers that are understood by the hardware.
This patch uses the new cpu_logical_map macro for performing the
conversion inside the GIC driver.
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Tegra can benefit from the IRQCHIP_MASK_ON_SUSPEND flag, allow it
to be passed to the gic irq chip.
Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Reviewed-by: Kevin Hilman <khilman@ti.com>
Tested-and-Acked-by: Shawn Guo <shawn.guo@linaro.org>
Tested-by: Vishwanath BS <vishwanath.bs@ti.com>
When the cpu is powered down in a low power mode, the gic cpu
interface may be reset, and when the cpu cluster is powered
down, the gic distributor may also be reset.
This patch uses CPU_PM_ENTER and CPU_PM_EXIT notifiers to save
and restore the gic cpu interface registers, and the
CPU_CLUSTER_PM_ENTER and CPU_CLUSTER_PM_EXIT notifiers to save
and restore the gic distributor registers.
Original-author: Gary King <gking@nvidia.com>
Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Tested-and-Acked-by: Shawn Guo <shawn.guo@linaro.org>
Tested-by: Vishwanath BS <vishwanath.bs@ti.com>
Annotate the low level hardware locks which must not be preempted.
In mainline this change documents the low level nature of
the lock - otherwise there's no functional difference. Lockdep
and Sparse checking will work as usual.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
The irq_set_affinity() method can be called with masks which include
offline CPUs. This allows offline CPUs to have interrupts routed to
them by writing to /proc/irq/*/smp_affinity after hotplug has taken
a CPU offline. Fix this by ensuring that we select a target CPU
present in both the required affinity and the online CPU mask.
Ensure that we return IRQ_SET_MASK_OK (which happens to be 0) on
success to ensure generic code copies the new mask into the irq_data
structure.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
irqdesc's node member is supposed to mark the numa node number for the
interrupt. Our use of it is non-standard. Remove this, replacing the
functionality with a test of the affinity mask.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Since Samsung EXYNOS4210 cannot support register banking in GIC,
so needs to update CPU interface base address.
The 'gic_chip_data' is used for it, this patch moves gic_chip_data
structure declaraton to arch/arm/include/asm/hardware/gic.h to use
it.
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Changhwan Youn <chaos.youn@samsung.com>
Signed-off-by: Kukjin Kim <kgene.kim@samsung.com>
The GIC register accesses today make use of readl()/writel()
which prove to be very expensive when used along with mandatory
barriers. This mandatory barriers also introduces an un-necessary
and expensive l2x0_sync() operation. On Cortex-A9 MP cores, GIC
IO accesses from CPU are direct and doesn't go through L2X0 write
buffer.
A DSB before writel_relaxed() in gic_raise_softirq() is added to be
compliant with the Barrier Litmus document - the mailbox scenario.
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Currently, the gic uses handle_level_irq for handling SPIs (Shared
Peripheral Interrupts), requiring active interrupts to be masked at
the distributor level during IRQ handling.
On a virtualised system, only the CPU interfaces are virtualised in
hardware. Accesses to the distributor must be trapped by the
hypervisor, adding latency to the critical interrupt path in Linux.
This patch modifies the GIC code to use handle_fasteoi_irq for handling
interrupts, which only requires us to signal EOI to the CPU interface
when handling is complete. Cascaded IRQ handling is also updated to use
the chained IRQ enter/exit functions to honour the flow control of the
parent chip.
Note that commit 846afbd1 ("GIC: Dont disable INT in ack callback")
broke cascading interrupts by forgetting to add IRQ masking. This is
no longer an issue because the unmask call is now unnecessary.
Tested on Versatile Express and Realview EB (1176 w/ cascaded GICs).
Tested-and-reviewed-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Tested-and-acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Reorder
irq_set_chip()
irq_set_chip_data()
irq_set_handler()
to
irq_set_chip()
irq_set_handler()
irq_set_chip_data()
so the next patch can combine irq_set_chip() and irq_set_handler() to
irq_set_chip_and_handler().
Automated conversion with coccinelle.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Few architectures combine the GIC with an external interrupt
controller. On such systems it may be necessary to update both
the GIC registers and the external controller's registers to control
IRQ behavior.
This can be addressed in couple of possible methods.
1. Export common GIC routines along with 'struct irq_chip gic_chip'
and allow architectures to have custom function by override.
2. Provide architecture specific function pointer hooks
within GIC library and leave platforms to add the necessary
code as part of these hooks.
First one might be non-intrusive but have few shortcomings like arch
needs to have there own custom gic library. Locks used should be
common since it caters to same IRQs etc. Maintenance point of view
also it leads to multiple file fixes.
The second probably is cleaner and portable. It ensures that all the
common GIC infrastructure is not touched and also provides archs to
address their specific issue.
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Acked-by: Colin Cross <ccross@android.com>
Tested-by: Colin Cross <ccross@android.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
This cleans up after the conversion to irq_data. Rename the function
to match the method, and remove the now useless lookup of the irq
descriptor which is never used. Move the bitmask calculation out of
the irq_controller_lock region.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Avoid adding nasty genirq-specific code to local timers to enable PPI
interrupts. Instead, provide a gic function to do this.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
This avoids writing unnecessarily to gic_data[] from other CPUs,
making this a mostly read-only variable.
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Tested-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Every architecture using the GIC has a gic_cpu_base_addr pointer for
GIC 0 for their entry assembly code to use to decode the cause of the
current interrupt. Move this into the common GIC code.
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Tested-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
We don't need to re-pass the base address for the CPU interfaces to the
GIC for secondary CPUs, as it will never be different from the boot CPU
- and even if it was, we'd overwrite the boot CPU's base address.
Get rid of this argument, and rename to gic_secondary_init().
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Tested-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Provide gic_init() which initializes the GIC distributor and current
CPU's GIC interface for the boot (or single) CPU.
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Tested-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
gic_set_cpu will directly use irq_desc[]. If CONFIG_SPARSE_IRQ is
enabled, there is no irq_desc[]. So we need use irq_to_desc(irq) to
get the descriptor for irq.
Signed-off-by: Chao Xie <chao.xie@marvell.com>
Acked-by: Kyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
This change limits number of GIC-originating interrupts to the
platform maximum (defined by NR_IRQS) while still initialising
all distributor registers.
Signed-off-by: Pawel Moll <pawel.moll@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Software generated interrupts (SGI) are used for IPIs by the kernel.
While previous revisions of the GIC hardware were specified not to
implement enable bits for SGIs, more recent hardware is now permitted
to implement these bits in a per-CPU banked register.
The priority registers for the PPI and SGIs are also per-CPU banked
registers, so ensure that these are also appropriately initialized.
Reported-by: Scott Valentine <svalentine@concentris-systems.com>
Acked-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Masking in the ack callback fails to work with handle_percpu_irq and handle_edge_irq.
The interrupt stays disabled after the first invocation since percpu and edge irq do
not unmask an interrupt after handling it. For handle_level_irq masking in the ack
is redundant because ack is always called after mask in the mask_ack function.
Masking in the ack function is required only when __do_IRQ was used instead of flow
handlers, but using __do_IRQ has been deprecated.
Remove the masking of interrupt from the ack callback.
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Signed-off-by: Jeff Ohlstein <johlstei@codeaurora.org>
Signed-off-by: Daniel Walker <dwalker@codeaurora.org>
Implement set_type() to allow configuration of the trigger type.
Cc: Abhijeet Dharmapurikar <adharmap@quicinc.com>
Acked-by: Linus Walleij <linus.walleij@stericsson.com>
Signed-off-by: Rabin Vincent <rabin.vincent@stericsson.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Nicolas Pitre