Merge branch 'for-linus' of master.kernel.org:/home/rmk/linux-2.6-arm

* 'for-linus' of master.kernel.org:/home/rmk/linux-2.6-arm: (237 commits)
  ARM: 7004/1: fix traps.h compile warnings
  ARM: 6998/2: kernel: use proper memory barriers for bitops
  ARM: 6997/1: ep93xx: increase NR_BANKS to 16 for support of 128MB RAM
  ARM: Fix build errors caused by adding generic macros
  ARM: CPU hotplug: ensure we migrate all IRQs off a downed CPU
  ARM: CPU hotplug: pass in proper affinity mask on IRQ migration
  ARM: GIC: avoid routing interrupts to offline CPUs
  ARM: CPU hotplug: fix abuse of irqdesc->node
  ARM: 6981/2: mmci: adjust calculation of f_min
  ARM: 7000/1: LPAE: Use long long printk format for displaying the pud
  ARM: 6999/1: head, zImage: Always Enter the kernel in ARM state
  ARM: btc: avoid invalidating the branch target cache on kernel TLB maintanence
  ARM: ARM_DMA_ZONE_SIZE is no more
  ARM: mach-shark: move ARM_DMA_ZONE_SIZE to mdesc->dma_zone_size
  ARM: mach-sa1100: move ARM_DMA_ZONE_SIZE to mdesc->dma_zone_size
  ARM: mach-realview: move from ARM_DMA_ZONE_SIZE to mdesc->dma_zone_size
  ARM: mach-pxa: move from ARM_DMA_ZONE_SIZE to mdesc->dma_zone_size
  ARM: mach-ixp4xx: move from ARM_DMA_ZONE_SIZE to mdesc->dma_zone_size
  ARM: mach-h720x: move from ARM_DMA_ZONE_SIZE to mdesc->dma_zone_size
  ARM: mach-davinci: move from ARM_DMA_ZONE_SIZE to mdesc->dma_zone_size
  ...
This commit is contained in:
Linus Torvalds 2011-07-24 10:20:54 -07:00
commit b6844e8f64
211 changed files with 7206 additions and 5761 deletions

View File

@ -164,3 +164,8 @@ In either case, the following conditions must be met:
- The boot loader is expected to call the kernel image by jumping
directly to the first instruction of the kernel image.
On CPUs supporting the ARM instruction set, the entry must be
made in ARM state, even for a Thumb-2 kernel.
On CPUs supporting only the Thumb instruction set such as
Cortex-M class CPUs, the entry must be made in Thumb state.

View File

@ -0,0 +1,42 @@
ROM-able zImage boot from eSD
-----------------------------
An ROM-able zImage compiled with ZBOOT_ROM_SDHI may be written to eSD and
SuperH Mobile ARM will to boot directly from the SDHI hardware block.
This is achieved by the mask ROM loading the first portion of the image into
MERAM and then jumping to it. This portion contains loader code which
copies the entire image to SDRAM and jumps to it. From there the zImage
boot code proceeds as normal, uncompressing the image into its final
location and then jumping to it.
This code has been tested on an mackerel board using the developer 1A eSD
boot mode which is configured using the following jumper settings.
8 7 6 5 4 3 2 1
x|x|x|x| |x|x|
S4 -+-+-+-+-+-+-+-
| | | |x| | |x on
The eSD card needs to be present in SDHI slot 1 (CN7).
As such S1 and S33 also need to be configured as per
the notes in arch/arm/mach-shmobile/board-mackerel.c.
A partial zImage must be written to physical partition #1 (boot)
of the eSD at sector 0 in vrl4 format. A utility vrl4 is supplied to
accomplish this.
e.g.
vrl4 < zImage | dd of=/dev/sdX bs=512 count=17
A full copy of _the same_ zImage should be written to physical partition #1
(boot) of the eSD at sector 0. This should _not_ be in vrl4 format.
vrl4 < zImage | dd of=/dev/sdX bs=512
Note: The commands above assume that the physical partition has been
switched. No such facility currently exists in the Linux Kernel.
Physical partitions are described in the eSD specification. At the time of
writing they are not the same as partitions that are typically configured
using fdisk and visible through /proc/partitions

View File

@ -0,0 +1,267 @@
Kernel-provided User Helpers
============================
These are segment of kernel provided user code reachable from user space
at a fixed address in kernel memory. This is used to provide user space
with some operations which require kernel help because of unimplemented
native feature and/or instructions in many ARM CPUs. The idea is for this
code to be executed directly in user mode for best efficiency but which is
too intimate with the kernel counter part to be left to user libraries.
In fact this code might even differ from one CPU to another depending on
the available instruction set, or whether it is a SMP systems. In other
words, the kernel reserves the right to change this code as needed without
warning. Only the entry points and their results as documented here are
guaranteed to be stable.
This is different from (but doesn't preclude) a full blown VDSO
implementation, however a VDSO would prevent some assembly tricks with
constants that allows for efficient branching to those code segments. And
since those code segments only use a few cycles before returning to user
code, the overhead of a VDSO indirect far call would add a measurable
overhead to such minimalistic operations.
User space is expected to bypass those helpers and implement those things
inline (either in the code emitted directly by the compiler, or part of
the implementation of a library call) when optimizing for a recent enough
processor that has the necessary native support, but only if resulting
binaries are already to be incompatible with earlier ARM processors due to
useage of similar native instructions for other things. In other words
don't make binaries unable to run on earlier processors just for the sake
of not using these kernel helpers if your compiled code is not going to
use new instructions for other purpose.
New helpers may be added over time, so an older kernel may be missing some
helpers present in a newer kernel. For this reason, programs must check
the value of __kuser_helper_version (see below) before assuming that it is
safe to call any particular helper. This check should ideally be
performed only once at process startup time, and execution aborted early
if the required helpers are not provided by the kernel version that
process is running on.
kuser_helper_version
--------------------
Location: 0xffff0ffc
Reference declaration:
extern int32_t __kuser_helper_version;
Definition:
This field contains the number of helpers being implemented by the
running kernel. User space may read this to determine the availability
of a particular helper.
Usage example:
#define __kuser_helper_version (*(int32_t *)0xffff0ffc)
void check_kuser_version(void)
{
if (__kuser_helper_version < 2) {
fprintf(stderr, "can't do atomic operations, kernel too old\n");
abort();
}
}
Notes:
User space may assume that the value of this field never changes
during the lifetime of any single process. This means that this
field can be read once during the initialisation of a library or
startup phase of a program.
kuser_get_tls
-------------
Location: 0xffff0fe0
Reference prototype:
void * __kuser_get_tls(void);
Input:
lr = return address
Output:
r0 = TLS value
Clobbered registers:
none
Definition:
Get the TLS value as previously set via the __ARM_NR_set_tls syscall.
Usage example:
typedef void * (__kuser_get_tls_t)(void);
#define __kuser_get_tls (*(__kuser_get_tls_t *)0xffff0fe0)
void foo()
{
void *tls = __kuser_get_tls();
printf("TLS = %p\n", tls);
}
Notes:
- Valid only if __kuser_helper_version >= 1 (from kernel version 2.6.12).
kuser_cmpxchg
-------------
Location: 0xffff0fc0
Reference prototype:
int __kuser_cmpxchg(int32_t oldval, int32_t newval, volatile int32_t *ptr);
Input:
r0 = oldval
r1 = newval
r2 = ptr
lr = return address
Output:
r0 = success code (zero or non-zero)
C flag = set if r0 == 0, clear if r0 != 0
Clobbered registers:
r3, ip, flags
Definition:
Atomically store newval in *ptr only if *ptr is equal to oldval.
Return zero if *ptr was changed or non-zero if no exchange happened.
The C flag is also set if *ptr was changed to allow for assembly
optimization in the calling code.
Usage example:
typedef int (__kuser_cmpxchg_t)(int oldval, int newval, volatile int *ptr);
#define __kuser_cmpxchg (*(__kuser_cmpxchg_t *)0xffff0fc0)
int atomic_add(volatile int *ptr, int val)
{
int old, new;
do {
old = *ptr;
new = old + val;
} while(__kuser_cmpxchg(old, new, ptr));
return new;
}
Notes:
- This routine already includes memory barriers as needed.
- Valid only if __kuser_helper_version >= 2 (from kernel version 2.6.12).
kuser_memory_barrier
--------------------
Location: 0xffff0fa0
Reference prototype:
void __kuser_memory_barrier(void);
Input:
lr = return address
Output:
none
Clobbered registers:
none
Definition:
Apply any needed memory barrier to preserve consistency with data modified
manually and __kuser_cmpxchg usage.
Usage example:
typedef void (__kuser_dmb_t)(void);
#define __kuser_dmb (*(__kuser_dmb_t *)0xffff0fa0)
Notes:
- Valid only if __kuser_helper_version >= 3 (from kernel version 2.6.15).
kuser_cmpxchg64
---------------
Location: 0xffff0f60
Reference prototype:
int __kuser_cmpxchg64(const int64_t *oldval,
const int64_t *newval,
volatile int64_t *ptr);
Input:
r0 = pointer to oldval
r1 = pointer to newval
r2 = pointer to target value
lr = return address
Output:
r0 = success code (zero or non-zero)
C flag = set if r0 == 0, clear if r0 != 0
Clobbered registers:
r3, lr, flags
Definition:
Atomically store the 64-bit value pointed by *newval in *ptr only if *ptr
is equal to the 64-bit value pointed by *oldval. Return zero if *ptr was
changed or non-zero if no exchange happened.
The C flag is also set if *ptr was changed to allow for assembly
optimization in the calling code.
Usage example:
typedef int (__kuser_cmpxchg64_t)(const int64_t *oldval,
const int64_t *newval,
volatile int64_t *ptr);
#define __kuser_cmpxchg64 (*(__kuser_cmpxchg64_t *)0xffff0f60)
int64_t atomic_add64(volatile int64_t *ptr, int64_t val)
{
int64_t old, new;
do {
old = *ptr;
new = old + val;
} while(__kuser_cmpxchg64(&old, &new, ptr));
return new;
}
Notes:
- This routine already includes memory barriers as needed.
- Due to the length of this sequence, this spans 2 conventional kuser
"slots", therefore 0xffff0f80 is not used as a valid entry point.
- Valid only if __kuser_helper_version >= 5 (from kernel version 3.1).

View File

@ -0,0 +1,21 @@
* ARM Performance Monitor Units
ARM cores often have a PMU for counting cpu and cache events like cache misses
and hits. The interface to the PMU is part of the ARM ARM. The ARM PMU
representation in the device tree should be done as under:-
Required properties:
- compatible : should be one of
"arm,cortex-a9-pmu"
"arm,cortex-a8-pmu"
"arm,arm1176-pmu"
"arm,arm1136-pmu"
- interrupts : 1 combined interrupt or 1 per core.
Example:
pmu {
compatible = "arm,cortex-a9-pmu";
interrupts = <100 101>;
};

View File

@ -10,7 +10,7 @@ config ARM
select GENERIC_ATOMIC64 if (CPU_V6 || !CPU_32v6K || !AEABI)
select HAVE_OPROFILE if (HAVE_PERF_EVENTS)
select HAVE_ARCH_KGDB
select HAVE_KPROBES if (!XIP_KERNEL && !THUMB2_KERNEL)
select HAVE_KPROBES if !XIP_KERNEL
select HAVE_KRETPROBES if (HAVE_KPROBES)
select HAVE_FUNCTION_TRACER if (!XIP_KERNEL)
select HAVE_FTRACE_MCOUNT_RECORD if (!XIP_KERNEL)
@ -37,6 +37,9 @@ config ARM
Europe. There is an ARM Linux project with a web page at
<http://www.arm.linux.org.uk/>.
config ARM_HAS_SG_CHAIN
bool
config HAVE_PWM
bool
@ -1347,7 +1350,6 @@ config SMP_ON_UP
config HAVE_ARM_SCU
bool
depends on SMP
help
This option enables support for the ARM system coherency unit
@ -1716,17 +1718,34 @@ config ZBOOT_ROM
Say Y here if you intend to execute your compressed kernel image
(zImage) directly from ROM or flash. If unsure, say N.
choice
prompt "Include SD/MMC loader in zImage (EXPERIMENTAL)"
depends on ZBOOT_ROM && ARCH_SH7372 && EXPERIMENTAL
default ZBOOT_ROM_NONE
help
Include experimental SD/MMC loading code in the ROM-able zImage.
With this enabled it is possible to write the the ROM-able zImage
kernel image to an MMC or SD card and boot the kernel straight
from the reset vector. At reset the processor Mask ROM will load
the first part of the the ROM-able zImage which in turn loads the
rest the kernel image to RAM.
config ZBOOT_ROM_NONE
bool "No SD/MMC loader in zImage (EXPERIMENTAL)"
help
Do not load image from SD or MMC
config ZBOOT_ROM_MMCIF
bool "Include MMCIF loader in zImage (EXPERIMENTAL)"
depends on ZBOOT_ROM && ARCH_SH7372 && EXPERIMENTAL
help
Say Y here to include experimental MMCIF loading code in the
ROM-able zImage. With this enabled it is possible to write the
the ROM-able zImage kernel image to an MMC card and boot the
kernel straight from the reset vector. At reset the processor
Mask ROM will load the first part of the the ROM-able zImage
which in turn loads the rest the kernel image to RAM using the
MMCIF hardware block.
Load image from MMCIF hardware block.
config ZBOOT_ROM_SH_MOBILE_SDHI
bool "Include SuperH Mobile SDHI loader in zImage (EXPERIMENTAL)"
help
Load image from SDHI hardware block
endchoice
config CMDLINE
string "Default kernel command string"

View File

@ -6,13 +6,19 @@
OBJS =
# Ensure that mmcif loader code appears early in the image
# Ensure that MMCIF loader code appears early in the image
# to minimise that number of bocks that have to be read in
# order to load it.
ifeq ($(CONFIG_ZBOOT_ROM_MMCIF),y)
ifeq ($(CONFIG_ARCH_SH7372),y)
OBJS += mmcif-sh7372.o
endif
# Ensure that SDHI loader code appears early in the image
# to minimise that number of bocks that have to be read in
# order to load it.
ifeq ($(CONFIG_ZBOOT_ROM_SH_MOBILE_SDHI),y)
OBJS += sdhi-shmobile.o
OBJS += sdhi-sh7372.o
endif
AFLAGS_head.o += -DTEXT_OFFSET=$(TEXT_OFFSET)

View File

@ -25,14 +25,14 @@
/* load board-specific initialization code */
#include <mach/zboot.h>
#ifdef CONFIG_ZBOOT_ROM_MMCIF
/* Load image from MMC */
adr sp, __tmp_stack + 128
#if defined(CONFIG_ZBOOT_ROM_MMCIF) || defined(CONFIG_ZBOOT_ROM_SH_MOBILE_SDHI)
/* Load image from MMC/SD */
adr sp, __tmp_stack + 256
ldr r0, __image_start
ldr r1, __image_end
subs r1, r1, r0
ldr r0, __load_base
bl mmcif_loader
bl mmc_loader
/* Jump to loaded code */
ldr r0, __loaded
@ -51,9 +51,9 @@ __loaded:
.long __continue
.align
__tmp_stack:
.space 128
.space 256
__continue:
#endif /* CONFIG_ZBOOT_ROM_MMCIF */
#endif /* CONFIG_ZBOOT_ROM_MMC || CONFIG_ZBOOT_ROM_SH_MOBILE_SDHI */
b 1f
__atags:@ tag #1

View File

@ -353,7 +353,8 @@ not_relocated: mov r0, #0
mov r0, #0 @ must be zero
mov r1, r7 @ restore architecture number
mov r2, r8 @ restore atags pointer
mov pc, r4 @ call kernel
ARM( mov pc, r4 ) @ call kernel
THUMB( bx r4 ) @ entry point is always ARM
.align 2
.type LC0, #object

View File

@ -40,7 +40,7 @@
* to an MMC card
* # dd if=vrl4.out of=/dev/sdx bs=512 seek=1
*/
asmlinkage void mmcif_loader(unsigned char *buf, unsigned long len)
asmlinkage void mmc_loader(unsigned char *buf, unsigned long len)
{
mmc_init_progress();
mmc_update_progress(MMC_PROGRESS_ENTER);

View File

@ -0,0 +1,95 @@
/*
* SuperH Mobile SDHI
*
* Copyright (C) 2010 Magnus Damm
* Copyright (C) 2010 Kuninori Morimoto
* Copyright (C) 2010 Simon Horman
*
* This file is subject to the terms and conditions of the GNU General Public
* License. See the file "COPYING" in the main directory of this archive
* for more details.
*
* Parts inspired by u-boot
*/
#include <linux/io.h>
#include <mach/mmc.h>
#include <linux/mmc/boot.h>
#include <linux/mmc/tmio.h>
#include "sdhi-shmobile.h"
#define PORT179CR 0xe60520b3
#define PORT180CR 0xe60520b4
#define PORT181CR 0xe60520b5
#define PORT182CR 0xe60520b6
#define PORT183CR 0xe60520b7
#define PORT184CR 0xe60520b8
#define SMSTPCR3 0xe615013c
#define CR_INPUT_ENABLE 0x10
#define CR_FUNCTION1 0x01
#define SDHI1_BASE (void __iomem *)0xe6860000
#define SDHI_BASE SDHI1_BASE
/* SuperH Mobile SDHI loader
*
* loads the zImage from an SD card starting from block 0
* on physical partition 1
*
* The image must be start with a vrl4 header and
* the zImage must start at offset 512 of the image. That is,
* at block 1 (=byte 512) of physical partition 1
*
* Use the following line to write the vrl4 formated zImage
* to an SD card
* # dd if=vrl4.out of=/dev/sdx bs=512
*/
asmlinkage void mmc_loader(unsigned short *buf, unsigned long len)
{
int high_capacity;
mmc_init_progress();
mmc_update_progress(MMC_PROGRESS_ENTER);
/* Initialise SDHI1 */
/* PORT184CR: GPIO_FN_SDHICMD1 Control */
__raw_writeb(CR_FUNCTION1, PORT184CR);
/* PORT179CR: GPIO_FN_SDHICLK1 Control */
__raw_writeb(CR_INPUT_ENABLE|CR_FUNCTION1, PORT179CR);
/* PORT181CR: GPIO_FN_SDHID1_3 Control */
__raw_writeb(CR_FUNCTION1, PORT183CR);
/* PORT182CR: GPIO_FN_SDHID1_2 Control */
__raw_writeb(CR_FUNCTION1, PORT182CR);
/* PORT183CR: GPIO_FN_SDHID1_1 Control */
__raw_writeb(CR_FUNCTION1, PORT181CR);
/* PORT180CR: GPIO_FN_SDHID1_0 Control */
__raw_writeb(CR_FUNCTION1, PORT180CR);
/* Enable clock to SDHI1 hardware block */
__raw_writel(__raw_readl(SMSTPCR3) & ~(1 << 13), SMSTPCR3);
/* setup SDHI hardware */
mmc_update_progress(MMC_PROGRESS_INIT);
high_capacity = sdhi_boot_init(SDHI_BASE);
if (high_capacity < 0)
goto err;
mmc_update_progress(MMC_PROGRESS_LOAD);
/* load kernel */
if (sdhi_boot_do_read(SDHI_BASE, high_capacity,
0, /* Kernel is at block 1 */
(len + TMIO_BBS - 1) / TMIO_BBS, buf))
goto err;
/* Disable clock to SDHI1 hardware block */
__raw_writel(__raw_readl(SMSTPCR3) & (1 << 13), SMSTPCR3);
mmc_update_progress(MMC_PROGRESS_DONE);
return;
err:
for(;;);
}

View File

@ -0,0 +1,449 @@
/*
* SuperH Mobile SDHI
*
* Copyright (C) 2010 Magnus Damm
* Copyright (C) 2010 Kuninori Morimoto
* Copyright (C) 2010 Simon Horman
*
* This file is subject to the terms and conditions of the GNU General Public
* License. See the file "COPYING" in the main directory of this archive
* for more details.
*
* Parts inspired by u-boot
*/
#include <linux/io.h>
#include <linux/mmc/host.h>
#include <linux/mmc/core.h>
#include <linux/mmc/mmc.h>
#include <linux/mmc/sd.h>
#include <linux/mmc/tmio.h>
#include <mach/sdhi.h>
#define OCR_FASTBOOT (1<<29)
#define OCR_HCS (1<<30)
#define OCR_BUSY (1<<31)
#define RESP_CMD12 0x00000030
static inline u16 sd_ctrl_read16(void __iomem *base, int addr)
{
return __raw_readw(base + addr);
}
static inline u32 sd_ctrl_read32(void __iomem *base, int addr)
{
return __raw_readw(base + addr) |
__raw_readw(base + addr + 2) << 16;
}
static inline void sd_ctrl_write16(void __iomem *base, int addr, u16 val)
{
__raw_writew(val, base + addr);
}
static inline void sd_ctrl_write32(void __iomem *base, int addr, u32 val)
{
__raw_writew(val, base + addr);
__raw_writew(val >> 16, base + addr + 2);
}
#define ALL_ERROR (TMIO_STAT_CMD_IDX_ERR | TMIO_STAT_CRCFAIL | \
TMIO_STAT_STOPBIT_ERR | TMIO_STAT_DATATIMEOUT | \
TMIO_STAT_RXOVERFLOW | TMIO_STAT_TXUNDERRUN | \
TMIO_STAT_CMDTIMEOUT | TMIO_STAT_ILL_ACCESS | \
TMIO_STAT_ILL_FUNC)
static int sdhi_intr(void __iomem *base)
{
unsigned long state = sd_ctrl_read32(base, CTL_STATUS);
if (state & ALL_ERROR) {
sd_ctrl_write32(base, CTL_STATUS, ~ALL_ERROR);
sd_ctrl_write32(base, CTL_IRQ_MASK,
ALL_ERROR |
sd_ctrl_read32(base, CTL_IRQ_MASK));
return -EINVAL;
}
if (state & TMIO_STAT_CMDRESPEND) {
sd_ctrl_write32(base, CTL_STATUS, ~TMIO_STAT_CMDRESPEND);
sd_ctrl_write32(base, CTL_IRQ_MASK,
TMIO_STAT_CMDRESPEND |
sd_ctrl_read32(base, CTL_IRQ_MASK));
return 0;
}
if (state & TMIO_STAT_RXRDY) {
sd_ctrl_write32(base, CTL_STATUS, ~TMIO_STAT_RXRDY);
sd_ctrl_write32(base, CTL_IRQ_MASK,
TMIO_STAT_RXRDY | TMIO_STAT_TXUNDERRUN |
sd_ctrl_read32(base, CTL_IRQ_MASK));
return 0;
}
if (state & TMIO_STAT_DATAEND) {
sd_ctrl_write32(base, CTL_STATUS, ~TMIO_STAT_DATAEND);
sd_ctrl_write32(base, CTL_IRQ_MASK,
TMIO_STAT_DATAEND |
sd_ctrl_read32(base, CTL_IRQ_MASK));
return 0;
}
return -EAGAIN;
}
static int sdhi_boot_wait_resp_end(void __iomem *base)
{
int err = -EAGAIN, timeout = 10000000;
while (timeout--) {
err = sdhi_intr(base);
if (err != -EAGAIN)
break;
udelay(1);
}
return err;
}
/* SDHI_CLK_CTRL */
#define CLK_MMC_ENABLE (1 << 8)
#define CLK_MMC_INIT (1 << 6) /* clk / 256 */
static void sdhi_boot_mmc_clk_stop(void __iomem *base)
{
sd_ctrl_write16(base, CTL_CLK_AND_WAIT_CTL, 0x0000);
msleep(10);
sd_ctrl_write16(base, CTL_SD_CARD_CLK_CTL, ~CLK_MMC_ENABLE &
sd_ctrl_read16(base, CTL_SD_CARD_CLK_CTL));
msleep(10);
}
static void sdhi_boot_mmc_clk_start(void __iomem *base)
{
sd_ctrl_write16(base, CTL_SD_CARD_CLK_CTL, CLK_MMC_ENABLE |
sd_ctrl_read16(base, CTL_SD_CARD_CLK_CTL));
msleep(10);
sd_ctrl_write16(base, CTL_CLK_AND_WAIT_CTL, CLK_MMC_ENABLE);
msleep(10);
}
static void sdhi_boot_reset(void __iomem *base)
{
sd_ctrl_write16(base, CTL_RESET_SD, 0x0000);
msleep(10);
sd_ctrl_write16(base, CTL_RESET_SD, 0x0001);
msleep(10);
}
/* Set MMC clock / power.
* Note: This controller uses a simple divider scheme therefore it cannot
* run a MMC card at full speed (20MHz). The max clock is 24MHz on SD, but as
* MMC wont run that fast, it has to be clocked at 12MHz which is the next
* slowest setting.
*/
static int sdhi_boot_mmc_set_ios(void __iomem *base, struct mmc_ios *ios)
{
if (sd_ctrl_read32(base, CTL_STATUS) & TMIO_STAT_CMD_BUSY)
return -EBUSY;
if (ios->clock)
sd_ctrl_write16(base, CTL_SD_CARD_CLK_CTL,
ios->clock | CLK_MMC_ENABLE);
/* Power sequence - OFF -> ON -> UP */
switch (ios->power_mode) {
case MMC_POWER_OFF: /* power down SD bus */
sdhi_boot_mmc_clk_stop(base);
break;
case MMC_POWER_ON: /* power up SD bus */
break;
case MMC_POWER_UP: /* start bus clock */
sdhi_boot_mmc_clk_start(base);
break;
}
switch (ios->bus_width) {
case MMC_BUS_WIDTH_1:
sd_ctrl_write16(base, CTL_SD_MEM_CARD_OPT, 0x80e0);
break;
case MMC_BUS_WIDTH_4:
sd_ctrl_write16(base, CTL_SD_MEM_CARD_OPT, 0x00e0);
break;
}
/* Let things settle. delay taken from winCE driver */
udelay(140);
return 0;
}
/* These are the bitmasks the tmio chip requires to implement the MMC response
* types. Note that R1 and R6 are the same in this scheme. */
#define RESP_NONE 0x0300
#define RESP_R1 0x0400
#define RESP_R1B 0x0500
#define RESP_R2 0x0600
#define RESP_R3 0x0700
#define DATA_PRESENT 0x0800
#define TRANSFER_READ 0x1000
static int sdhi_boot_request(void __iomem *base, struct mmc_command *cmd)
{
int err, c = cmd->opcode;
switch (mmc_resp_type(cmd)) {
case MMC_RSP_NONE: c |= RESP_NONE; break;
case MMC_RSP_R1: c |= RESP_R1; break;
case MMC_RSP_R1B: c |= RESP_R1B; break;
case MMC_RSP_R2: c |= RESP_R2; break;
case MMC_RSP_R3: c |= RESP_R3; break;
default:
return -EINVAL;
}
/* No interrupts so this may not be cleared */
sd_ctrl_write32(base, CTL_STATUS, ~TMIO_STAT_CMDRESPEND);
sd_ctrl_write32(base, CTL_IRQ_MASK, TMIO_STAT_CMDRESPEND |
sd_ctrl_read32(base, CTL_IRQ_MASK));
sd_ctrl_write32(base, CTL_ARG_REG, cmd->arg);
sd_ctrl_write16(base, CTL_SD_CMD, c);
sd_ctrl_write32(base, CTL_IRQ_MASK,
~(TMIO_STAT_CMDRESPEND | ALL_ERROR) &
sd_ctrl_read32(base, CTL_IRQ_MASK));
err = sdhi_boot_wait_resp_end(base);
if (err)
return err;
cmd->resp[0] = sd_ctrl_read32(base, CTL_RESPONSE);
return 0;
}
static int sdhi_boot_do_read_single(void __iomem *base, int high_capacity,
unsigned long block, unsigned short *buf)
{
int err, i;
/* CMD17 - Read */
{
struct mmc_command cmd;
cmd.opcode = MMC_READ_SINGLE_BLOCK | \
TRANSFER_READ | DATA_PRESENT;
if (high_capacity)
cmd.arg = block;
else
cmd.arg = block * TMIO_BBS;
cmd.flags = MMC_RSP_R1;
err = sdhi_boot_request(base, &cmd);
if (err)
return err;
}
sd_ctrl_write32(base, CTL_IRQ_MASK,
~(TMIO_STAT_DATAEND | TMIO_STAT_RXRDY |
TMIO_STAT_TXUNDERRUN) &
sd_ctrl_read32(base, CTL_IRQ_MASK));
err = sdhi_boot_wait_resp_end(base);
if (err)
return err;
sd_ctrl_write16(base, CTL_SD_XFER_LEN, TMIO_BBS);
for (i = 0; i < TMIO_BBS / sizeof(*buf); i++)
*buf++ = sd_ctrl_read16(base, RESP_CMD12);
err = sdhi_boot_wait_resp_end(base);
if (err)
return err;
return 0;
}
int sdhi_boot_do_read(void __iomem *base, int high_capacity,
unsigned long offset, unsigned short count,
unsigned short *buf)
{
unsigned long i;
int err = 0;
for (i = 0; i < count; i++) {
err = sdhi_boot_do_read_single(base, high_capacity, offset + i,
buf + (i * TMIO_BBS /
sizeof(*buf)));
if (err)
return err;
}
return 0;
}
#define VOLTAGES (MMC_VDD_32_33 | MMC_VDD_33_34)
int sdhi_boot_init(void __iomem *base)
{
bool sd_v2 = false, sd_v1_0 = false;
unsigned short cid;
int err, high_capacity = 0;
sdhi_boot_mmc_clk_stop(base);
sdhi_boot_reset(base);
/* mmc0: clock 400000Hz busmode 1 powermode 2 cs 0 Vdd 21 width 0 timing 0 */
{
struct mmc_ios ios;
ios.power_mode = MMC_POWER_ON;
ios.bus_width = MMC_BUS_WIDTH_1;
ios.clock = CLK_MMC_INIT;
err = sdhi_boot_mmc_set_ios(base, &ios);
if (err)
return err;
}
/* CMD0 */
{
struct mmc_command cmd;
msleep(1);
cmd.opcode = MMC_GO_IDLE_STATE;
cmd.arg = 0;
cmd.flags = MMC_RSP_NONE;
err = sdhi_boot_request(base, &cmd);
if (err)
return err;
msleep(2);
}
/* CMD8 - Test for SD version 2 */
{
struct mmc_command cmd;
cmd.opcode = SD_SEND_IF_COND;
cmd.arg = (VOLTAGES != 0) << 8 | 0xaa;
cmd.flags = MMC_RSP_R1;
err = sdhi_boot_request(base, &cmd); /* Ignore error */
if ((cmd.resp[0] & 0xff) == 0xaa)
sd_v2 = true;
}
/* CMD55 - Get OCR (SD) */
{
int timeout = 1000;
struct mmc_command cmd;
cmd.arg = 0;
do {
cmd.opcode = MMC_APP_CMD;
cmd.flags = MMC_RSP_R1;
cmd.arg = 0;
err = sdhi_boot_request(base, &cmd);
if (err)
break;
cmd.opcode = SD_APP_OP_COND;
cmd.flags = MMC_RSP_R3;
cmd.arg = (VOLTAGES & 0xff8000);
if (sd_v2)
cmd.arg |= OCR_HCS;
cmd.arg |= OCR_FASTBOOT;
err = sdhi_boot_request(base, &cmd);
if (err)
break;
msleep(1);
} while((!(cmd.resp[0] & OCR_BUSY)) && --timeout);
if (!err && timeout) {
if (!sd_v2)
sd_v1_0 = true;
high_capacity = (cmd.resp[0] & OCR_HCS) == OCR_HCS;
}
}
/* CMD1 - Get OCR (MMC) */
if (!sd_v2 && !sd_v1_0) {
int timeout = 1000;
struct mmc_command cmd;
do {
cmd.opcode = MMC_SEND_OP_COND;
cmd.arg = VOLTAGES | OCR_HCS;
cmd.flags = MMC_RSP_R3;
err = sdhi_boot_request(base, &cmd);
if (err)
return err;
msleep(1);
} while((!(cmd.resp[0] & OCR_BUSY)) && --timeout);
if (!timeout)
return -EAGAIN;
high_capacity = (cmd.resp[0] & OCR_HCS) == OCR_HCS;
}
/* CMD2 - Get CID */
{
struct mmc_command cmd;
cmd.opcode = MMC_ALL_SEND_CID;
cmd.arg = 0;
cmd.flags = MMC_RSP_R2;
err = sdhi_boot_request(base, &cmd);
if (err)
return err;
}
/* CMD3
* MMC: Set the relative address
* SD: Get the relative address
* Also puts the card into the standby state
*/
{
struct mmc_command cmd;
cmd.opcode = MMC_SET_RELATIVE_ADDR;
cmd.arg = 0;
cmd.flags = MMC_RSP_R1;
err = sdhi_boot_request(base, &cmd);
if (err)
return err;
cid = cmd.resp[0] >> 16;
}
/* CMD9 - Get CSD */
{
struct mmc_command cmd;
cmd.opcode = MMC_SEND_CSD;
cmd.arg = cid << 16;
cmd.flags = MMC_RSP_R2;
err = sdhi_boot_request(base, &cmd);
if (err)
return err;
}
/* CMD7 - Select the card */
{
struct mmc_command cmd;
cmd.opcode = MMC_SELECT_CARD;
//cmd.arg = rca << 16;
cmd.arg = cid << 16;
//cmd.flags = MMC_RSP_R1B;
cmd.flags = MMC_RSP_R1;
err = sdhi_boot_request(base, &cmd);
if (err)
return err;
}
/* CMD16 - Set the block size */
{
struct mmc_command cmd;
cmd.opcode = MMC_SET_BLOCKLEN;
cmd.arg = TMIO_BBS;
cmd.flags = MMC_RSP_R1;
err = sdhi_boot_request(base, &cmd);
if (err)
return err;
}
return high_capacity;
}

View File

@ -0,0 +1,11 @@
#ifndef SDHI_MOBILE_H
#define SDHI_MOBILE_H
#include <linux/compiler.h>
int sdhi_boot_do_read(void __iomem *base, int high_capacity,
unsigned long offset, unsigned short count,
unsigned short *buf);
int sdhi_boot_init(void __iomem *base);
#endif

View File

@ -33,20 +33,24 @@ SECTIONS
*(.text.*)
*(.fixup)
*(.gnu.warning)
*(.glue_7t)
*(.glue_7)
}
.rodata : {
*(.rodata)
*(.rodata.*)
*(.glue_7)
*(.glue_7t)
}
.piggydata : {
*(.piggydata)
. = ALIGN(4);
}
. = ALIGN(4);
_etext = .;
.got.plt : { *(.got.plt) }
_got_start = .;
.got : { *(.got) }
_got_end = .;
.got.plt : { *(.got.plt) }
_edata = .;
. = BSS_START;

View File

@ -79,6 +79,8 @@ struct dmabounce_device_info {
struct dmabounce_pool large;
rwlock_t lock;
int (*needs_bounce)(struct device *, dma_addr_t, size_t);
};
#ifdef STATS
@ -210,114 +212,91 @@ static struct safe_buffer *find_safe_buffer_dev(struct device *dev,
if (!dev || !dev->archdata.dmabounce)
return NULL;
if (dma_mapping_error(dev, dma_addr)) {
if (dev)
dev_err(dev, "Trying to %s invalid mapping\n", where);
else
pr_err("unknown device: Trying to %s invalid mapping\n", where);
dev_err(dev, "Trying to %s invalid mapping\n", where);
return NULL;
}
return find_safe_buffer(dev->archdata.dmabounce, dma_addr);
}
static int needs_bounce(struct device *dev, dma_addr_t dma_addr, size_t size)
{
if (!dev || !dev->archdata.dmabounce)
return 0;
if (dev->dma_mask) {
unsigned long limit, mask = *dev->dma_mask;
limit = (mask + 1) & ~mask;
if (limit && size > limit) {
dev_err(dev, "DMA mapping too big (requested %#x "
"mask %#Lx)\n", size, *dev->dma_mask);
return -E2BIG;
}
/* Figure out if we need to bounce from the DMA mask. */
if ((dma_addr | (dma_addr + size - 1)) & ~mask)
return 1;
}
return !!dev->archdata.dmabounce->needs_bounce(dev, dma_addr, size);
}
static inline dma_addr_t map_single(struct device *dev, void *ptr, size_t size,
enum dma_data_direction dir)
{
struct dmabounce_device_info *device_info = dev->archdata.dmabounce;
dma_addr_t dma_addr;
int needs_bounce = 0;
struct safe_buffer *buf;
if (device_info)
DO_STATS ( device_info->map_op_count++ );
dma_addr = virt_to_dma(dev, ptr);
if (dev->dma_mask) {
unsigned long mask = *dev->dma_mask;
unsigned long limit;
limit = (mask + 1) & ~mask;
if (limit && size > limit) {
dev_err(dev, "DMA mapping too big (requested %#x "
"mask %#Lx)\n", size, *dev->dma_mask);
return ~0;
}
/*
* Figure out if we need to bounce from the DMA mask.
*/
needs_bounce = (dma_addr | (dma_addr + size - 1)) & ~mask;
buf = alloc_safe_buffer(device_info, ptr, size, dir);
if (buf == NULL) {
dev_err(dev, "%s: unable to map unsafe buffer %p!\n",
__func__, ptr);
return ~0;
}
if (device_info && (needs_bounce || dma_needs_bounce(dev, dma_addr, size))) {
struct safe_buffer *buf;
dev_dbg(dev, "%s: unsafe buffer %p (dma=%#x) mapped to %p (dma=%#x)\n",
__func__, buf->ptr, virt_to_dma(dev, buf->ptr),
buf->safe, buf->safe_dma_addr);
buf = alloc_safe_buffer(device_info, ptr, size, dir);
if (buf == 0) {
dev_err(dev, "%s: unable to map unsafe buffer %p!\n",
__func__, ptr);
return ~0;
}
dev_dbg(dev,
"%s: unsafe buffer %p (dma=%#x) mapped to %p (dma=%#x)\n",
__func__, buf->ptr, virt_to_dma(dev, buf->ptr),
buf->safe, buf->safe_dma_addr);
if ((dir == DMA_TO_DEVICE) ||
(dir == DMA_BIDIRECTIONAL)) {
dev_dbg(dev, "%s: copy unsafe %p to safe %p, size %d\n",
__func__, ptr, buf->safe, size);
memcpy(buf->safe, ptr, size);
}
ptr = buf->safe;
dma_addr = buf->safe_dma_addr;
} else {
/*
* We don't need to sync the DMA buffer since
* it was allocated via the coherent allocators.
*/
__dma_single_cpu_to_dev(ptr, size, dir);
if (dir == DMA_TO_DEVICE || dir == DMA_BIDIRECTIONAL) {
dev_dbg(dev, "%s: copy unsafe %p to safe %p, size %d\n",
__func__, ptr, buf->safe, size);
memcpy(buf->safe, ptr, size);
}
return dma_addr;
return buf->safe_dma_addr;
}
static inline void unmap_single(struct device *dev, dma_addr_t dma_addr,
static inline void unmap_single(struct device *dev, struct safe_buffer *buf,
size_t size, enum dma_data_direction dir)
{
struct safe_buffer *buf = find_safe_buffer_dev(dev, dma_addr, "unmap");
BUG_ON(buf->size != size);
BUG_ON(buf->direction != dir);
if (buf) {
BUG_ON(buf->size != size);
BUG_ON(buf->direction != dir);
dev_dbg(dev, "%s: unsafe buffer %p (dma=%#x) mapped to %p (dma=%#x)\n",
__func__, buf->ptr, virt_to_dma(dev, buf->ptr),
buf->safe, buf->safe_dma_addr);
dev_dbg(dev,
"%s: unsafe buffer %p (dma=%#x) mapped to %p (dma=%#x)\n",
__func__, buf->ptr, virt_to_dma(dev, buf->ptr),
buf->safe, buf->safe_dma_addr);
DO_STATS(dev->archdata.dmabounce->bounce_count++);
DO_STATS(dev->archdata.dmabounce->bounce_count++);
if (dir == DMA_FROM_DEVICE || dir == DMA_BIDIRECTIONAL) {
void *ptr = buf->ptr;
if (dir == DMA_FROM_DEVICE || dir == DMA_BIDIRECTIONAL) {
void *ptr = buf->ptr;
dev_dbg(dev, "%s: copy back safe %p to unsafe %p size %d\n",
__func__, buf->safe, ptr, size);
memcpy(ptr, buf->safe, size);
dev_dbg(dev,
"%s: copy back safe %p to unsafe %p size %d\n",
__func__, buf->safe, ptr, size);
memcpy(ptr, buf->safe, size);
/*
* Since we may have written to a page cache page,
* we need to ensure that the data will be coherent
* with user mappings.
*/
__cpuc_flush_dcache_area(ptr, size);
}
free_safe_buffer(dev->archdata.dmabounce, buf);
} else {
__dma_single_dev_to_cpu(dma_to_virt(dev, dma_addr), size, dir);
/*
* Since we may have written to a page cache page,
* we need to ensure that the data will be coherent
* with user mappings.
*/
__cpuc_flush_dcache_area(ptr, size);
}
free_safe_buffer(dev->archdata.dmabounce, buf);
}
/* ************************************************** */
@ -328,45 +307,28 @@ static inline void unmap_single(struct device *dev, dma_addr_t dma_addr,
* substitute the safe buffer for the unsafe one.
* (basically move the buffer from an unsafe area to a safe one)
*/
dma_addr_t __dma_map_single(struct device *dev, void *ptr, size_t size,
enum dma_data_direction dir)
{
dev_dbg(dev, "%s(ptr=%p,size=%d,dir=%x)\n",
__func__, ptr, size, dir);
BUG_ON(!valid_dma_direction(dir));
return map_single(dev, ptr, size, dir);
}
EXPORT_SYMBOL(__dma_map_single);
/*
* see if a mapped address was really a "safe" buffer and if so, copy
* the data from the safe buffer back to the unsafe buffer and free up
* the safe buffer. (basically return things back to the way they
* should be)
*/
void __dma_unmap_single(struct device *dev, dma_addr_t dma_addr, size_t size,
enum dma_data_direction dir)
{
dev_dbg(dev, "%s(ptr=%p,size=%d,dir=%x)\n",
__func__, (void *) dma_addr, size, dir);
unmap_single(dev, dma_addr, size, dir);
}
EXPORT_SYMBOL(__dma_unmap_single);
dma_addr_t __dma_map_page(struct device *dev, struct page *page,
unsigned long offset, size_t size, enum dma_data_direction dir)
{
dma_addr_t dma_addr;
int ret;
dev_dbg(dev, "%s(page=%p,off=%#lx,size=%zx,dir=%x)\n",
__func__, page, offset, size, dir);
BUG_ON(!valid_dma_direction(dir));
dma_addr = pfn_to_dma(dev, page_to_pfn(page)) + offset;
ret = needs_bounce(dev, dma_addr, size);
if (ret < 0)
return ~0;
if (ret == 0) {
__dma_page_cpu_to_dev(page, offset, size, dir);
return dma_addr;
}
if (PageHighMem(page)) {
dev_err(dev, "DMA buffer bouncing of HIGHMEM pages "
"is not supported\n");
dev_err(dev, "DMA buffer bouncing of HIGHMEM pages is not supported\n");
return ~0;
}
@ -383,10 +345,19 @@ EXPORT_SYMBOL(__dma_map_page);
void __dma_unmap_page(struct device *dev, dma_addr_t dma_addr, size_t size,
enum dma_data_direction dir)
{
dev_dbg(dev, "%s(ptr=%p,size=%d,dir=%x)\n",
__func__, (void *) dma_addr, size, dir);
struct safe_buffer *buf;
unmap_single(dev, dma_addr, size, dir);
dev_dbg(dev, "%s(dma=%#x,size=%d,dir=%x)\n",
__func__, dma_addr, size, dir);
buf = find_safe_buffer_dev(dev, dma_addr, __func__);
if (!buf) {
__dma_page_dev_to_cpu(pfn_to_page(dma_to_pfn(dev, dma_addr)),
dma_addr & ~PAGE_MASK, size, dir);
return;
}
unmap_single(dev, buf, size, dir);
}
EXPORT_SYMBOL(__dma_unmap_page);
@ -461,7 +432,8 @@ static int dmabounce_init_pool(struct dmabounce_pool *pool, struct device *dev,
}
int dmabounce_register_dev(struct device *dev, unsigned long small_buffer_size,
unsigned long large_buffer_size)
unsigned long large_buffer_size,
int (*needs_bounce_fn)(struct device *, dma_addr_t, size_t))
{
struct dmabounce_device_info *device_info;
int ret;
@ -497,6 +469,7 @@ int dmabounce_register_dev(struct device *dev, unsigned long small_buffer_size,
device_info->dev = dev;
INIT_LIST_HEAD(&device_info->safe_buffers);
rwlock_init(&device_info->lock);
device_info->needs_bounce = needs_bounce_fn;
#ifdef STATS
device_info->total_allocs = 0;

View File

@ -179,22 +179,21 @@ static int gic_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
{
void __iomem *reg = gic_dist_base(d) + GIC_DIST_TARGET + (gic_irq(d) & ~3);
unsigned int shift = (d->irq % 4) * 8;
unsigned int cpu = cpumask_first(mask_val);
unsigned int cpu = cpumask_any_and(mask_val, cpu_online_mask);
u32 val, mask, bit;
if (cpu >= 8)
if (cpu >= 8 || cpu >= nr_cpu_ids)
return -EINVAL;
mask = 0xff << shift;
bit = 1 << (cpu + shift);
spin_lock(&irq_controller_lock);
d->node = cpu;
val = readl_relaxed(reg) & ~mask;
writel_relaxed(val | bit, reg);
spin_unlock(&irq_controller_lock);
return 0;
return IRQ_SET_MASK_OK;
}
#endif

View File

@ -243,6 +243,12 @@ static struct resource it8152_mem = {
* ITE8152 chip can address up to 64MByte, so all the devices
* connected to ITE8152 (PCI and USB) should have limited DMA window
*/
static int it8152_needs_bounce(struct device *dev, dma_addr_t dma_addr, size_t size)
{
dev_dbg(dev, "%s: dma_addr %08x, size %08x\n",
__func__, dma_addr, size);
return (dma_addr + size - PHYS_OFFSET) >= SZ_64M;
}
/*
* Setup DMA mask to 64MB on devices connected to ITE8152. Ignore all
@ -254,7 +260,7 @@ static int it8152_pci_platform_notify(struct device *dev)
if (dev->dma_mask)
*dev->dma_mask = (SZ_64M - 1) | PHYS_OFFSET;
dev->coherent_dma_mask = (SZ_64M - 1) | PHYS_OFFSET;
dmabounce_register_dev(dev, 2048, 4096);
dmabounce_register_dev(dev, 2048, 4096, it8152_needs_bounce);
}
return 0;
}
@ -267,14 +273,6 @@ static int it8152_pci_platform_notify_remove(struct device *dev)
return 0;
}
int dma_needs_bounce(struct device *dev, dma_addr_t dma_addr, size_t size)
{
dev_dbg(dev, "%s: dma_addr %08x, size %08x\n",
__func__, dma_addr, size);
return (dev->bus == &pci_bus_type) &&
((dma_addr + size - PHYS_OFFSET) >= SZ_64M);
}
int dma_set_coherent_mask(struct device *dev, u64 mask)
{
if (mask >= PHYS_OFFSET + SZ_64M - 1)

View File

@ -579,7 +579,36 @@ sa1111_configure_smc(struct sa1111 *sachip, int sdram, unsigned int drac,
sachip->dev->coherent_dma_mask &= sa1111_dma_mask[drac >> 2];
}
#endif
#ifdef CONFIG_DMABOUNCE
/*
* According to the "Intel StrongARM SA-1111 Microprocessor Companion
* Chip Specification Update" (June 2000), erratum #7, there is a
* significant bug in the SA1111 SDRAM shared memory controller. If
* an access to a region of memory above 1MB relative to the bank base,
* it is important that address bit 10 _NOT_ be asserted. Depending
* on the configuration of the RAM, bit 10 may correspond to one
* of several different (processor-relative) address bits.
*
* This routine only identifies whether or not a given DMA address
* is susceptible to the bug.
*
* This should only get called for sa1111_device types due to the
* way we configure our device dma_masks.
*/
static int sa1111_needs_bounce(struct device *dev, dma_addr_t addr, size_t size)
{
/*
* Section 4.6 of the "Intel StrongARM SA-1111 Development Module
* User's Guide" mentions that jumpers R51 and R52 control the
* target of SA-1111 DMA (either SDRAM bank 0 on Assabet, or
* SDRAM bank 1 on Neponset). The default configuration selects
* Assabet, so any address in bank 1 is necessarily invalid.
*/
return (machine_is_assabet() || machine_is_pfs168()) &&
(addr >= 0xc8000000 || (addr + size) >= 0xc8000000);
}
#endif
static void sa1111_dev_release(struct device *_dev)
@ -644,7 +673,8 @@ sa1111_init_one_child(struct sa1111 *sachip, struct resource *parent,
dev->dev.dma_mask = &dev->dma_mask;
if (dev->dma_mask != 0xffffffffUL) {
ret = dmabounce_register_dev(&dev->dev, 1024, 4096);
ret = dmabounce_register_dev(&dev->dev, 1024, 4096,
sa1111_needs_bounce);
if (ret) {
dev_err(&dev->dev, "SA1111: Failed to register"
" with dmabounce\n");
@ -818,34 +848,6 @@ static void __sa1111_remove(struct sa1111 *sachip)
kfree(sachip);
}
/*
* According to the "Intel StrongARM SA-1111 Microprocessor Companion
* Chip Specification Update" (June 2000), erratum #7, there is a
* significant bug in the SA1111 SDRAM shared memory controller. If
* an access to a region of memory above 1MB relative to the bank base,
* it is important that address bit 10 _NOT_ be asserted. Depending
* on the configuration of the RAM, bit 10 may correspond to one
* of several different (processor-relative) address bits.
*
* This routine only identifies whether or not a given DMA address
* is susceptible to the bug.
*
* This should only get called for sa1111_device types due to the
* way we configure our device dma_masks.
*/
int dma_needs_bounce(struct device *dev, dma_addr_t addr, size_t size)
{
/*
* Section 4.6 of the "Intel StrongARM SA-1111 Development Module
* User's Guide" mentions that jumpers R51 and R52 control the
* target of SA-1111 DMA (either SDRAM bank 0 on Assabet, or
* SDRAM bank 1 on Neponset). The default configuration selects
* Assabet, so any address in bank 1 is necessarily invalid.
*/
return ((machine_is_assabet() || machine_is_pfs168()) &&
(addr >= 0xc8000000 || (addr + size) >= 0xc8000000));
}
struct sa1111_save_data {
unsigned int skcr;
unsigned int skpcr;

View File

@ -293,4 +293,13 @@
.macro ldrusr, reg, ptr, inc, cond=al, rept=1, abort=9001f
usracc ldr, \reg, \ptr, \inc, \cond, \rept, \abort
.endm
/* Utility macro for declaring string literals */
.macro string name:req, string
.type \name , #object
\name:
.asciz "\string"
.size \name , . - \name
.endm
#endif /* __ASM_ASSEMBLER_H__ */

View File

@ -26,8 +26,8 @@
#include <linux/compiler.h>
#include <asm/system.h>
#define smp_mb__before_clear_bit() mb()
#define smp_mb__after_clear_bit() mb()
#define smp_mb__before_clear_bit() smp_mb()
#define smp_mb__after_clear_bit() smp_mb()
/*
* These functions are the basis of our bit ops.

View File

@ -115,39 +115,8 @@ static inline void __dma_page_dev_to_cpu(struct page *page, unsigned long off,
___dma_page_dev_to_cpu(page, off, size, dir);
}
/*
* Return whether the given device DMA address mask can be supported
* properly. For example, if your device can only drive the low 24-bits
* during bus mastering, then you would pass 0x00ffffff as the mask
* to this function.
*
* FIXME: This should really be a platform specific issue - we should
* return false if GFP_DMA allocations may not satisfy the supplied 'mask'.
*/
static inline int dma_supported(struct device *dev, u64 mask)
{
if (mask < ISA_DMA_THRESHOLD)
return 0;
return 1;
}
static inline int dma_set_mask(struct device *dev, u64 dma_mask)
{
#ifdef CONFIG_DMABOUNCE
if (dev->archdata.dmabounce) {
if (dma_mask >= ISA_DMA_THRESHOLD)
return 0;
else
return -EIO;
}
#endif
if (!dev->dma_mask || !dma_supported(dev, dma_mask))
return -EIO;
*dev->dma_mask = dma_mask;
return 0;
}
extern int dma_supported(struct device *, u64);
extern int dma_set_mask(struct device *, u64);
/*
* DMA errors are defined by all-bits-set in the DMA address.
@ -256,14 +225,14 @@ int dma_mmap_writecombine(struct device *, struct vm_area_struct *,
* @dev: valid struct device pointer
* @small_buf_size: size of buffers to use with small buffer pool
* @large_buf_size: size of buffers to use with large buffer pool (can be 0)
* @needs_bounce_fn: called to determine whether buffer needs bouncing
*
* This function should be called by low-level platform code to register
* a device as requireing DMA buffer bouncing. The function will allocate
* appropriate DMA pools for the device.
*
*/
extern int dmabounce_register_dev(struct device *, unsigned long,
unsigned long);
unsigned long, int (*)(struct device *, dma_addr_t, size_t));
/**
* dmabounce_unregister_dev
@ -277,31 +246,9 @@ extern int dmabounce_register_dev(struct device *, unsigned long,
*/
extern void dmabounce_unregister_dev(struct device *);
/**
* dma_needs_bounce
*
* @dev: valid struct device pointer
* @dma_handle: dma_handle of unbounced buffer
* @size: size of region being mapped
*
* Platforms that utilize the dmabounce mechanism must implement
* this function.
*
* The dmabounce routines call this function whenever a dma-mapping
* is requested to determine whether a given buffer needs to be bounced
* or not. The function must return 0 if the buffer is OK for
* DMA access and 1 if the buffer needs to be bounced.
*
*/
extern int dma_needs_bounce(struct device*, dma_addr_t, size_t);
/*
* The DMA API, implemented by dmabounce.c. See below for descriptions.
*/
extern dma_addr_t __dma_map_single(struct device *, void *, size_t,
enum dma_data_direction);
extern void __dma_unmap_single(struct device *, dma_addr_t, size_t,
enum dma_data_direction);
extern dma_addr_t __dma_map_page(struct device *, struct page *,
unsigned long, size_t, enum dma_data_direction);
extern void __dma_unmap_page(struct device *, dma_addr_t, size_t,
@ -328,13 +275,6 @@ static inline int dmabounce_sync_for_device(struct device *d, dma_addr_t addr,
}
static inline dma_addr_t __dma_map_single(struct device *dev, void *cpu_addr,
size_t size, enum dma_data_direction dir)
{
__dma_single_cpu_to_dev(cpu_addr, size, dir);
return virt_to_dma(dev, cpu_addr);
}
static inline dma_addr_t __dma_map_page(struct device *dev, struct page *page,
unsigned long offset, size_t size, enum dma_data_direction dir)
{
@ -342,12 +282,6 @@ static inline dma_addr_t __dma_map_page(struct device *dev, struct page *page,
return pfn_to_dma(dev, page_to_pfn(page)) + offset;
}
static inline void __dma_unmap_single(struct device *dev, dma_addr_t handle,
size_t size, enum dma_data_direction dir)
{
__dma_single_dev_to_cpu(dma_to_virt(dev, handle), size, dir);
}
static inline void __dma_unmap_page(struct device *dev, dma_addr_t handle,
size_t size, enum dma_data_direction dir)
{
@ -373,14 +307,18 @@ static inline void __dma_unmap_page(struct device *dev, dma_addr_t handle,
static inline dma_addr_t dma_map_single(struct device *dev, void *cpu_addr,
size_t size, enum dma_data_direction dir)
{
unsigned long offset;
struct page *page;
dma_addr_t addr;
BUG_ON(!virt_addr_valid(cpu_addr));
BUG_ON(!virt_addr_valid(cpu_addr + size - 1));
BUG_ON(!valid_dma_direction(dir));
addr = __dma_map_single(dev, cpu_addr, size, dir);
debug_dma_map_page(dev, virt_to_page(cpu_addr),
(unsigned long)cpu_addr & ~PAGE_MASK, size,
dir, addr, true);
page = virt_to_page(cpu_addr);
offset = (unsigned long)cpu_addr & ~PAGE_MASK;
addr = __dma_map_page(dev, page, offset, size, dir);
debug_dma_map_page(dev, page, offset, size, dir, addr, true);
return addr;
}
@ -430,7 +368,7 @@ static inline void dma_unmap_single(struct device *dev, dma_addr_t handle,
size_t size, enum dma_data_direction dir)
{
debug_dma_unmap_page(dev, handle, size, dir, true);
__dma_unmap_single(dev, handle, size, dir);
__dma_unmap_page(dev, handle, size, dir);
}
/**

View File

@ -1,15 +1,16 @@
#ifndef __ASM_ARM_DMA_H
#define __ASM_ARM_DMA_H
#include <asm/memory.h>
/*
* This is the maximum virtual address which can be DMA'd from.
*/
#ifndef ARM_DMA_ZONE_SIZE
#define MAX_DMA_ADDRESS 0xffffffff
#ifndef CONFIG_ZONE_DMA
#define MAX_DMA_ADDRESS 0xffffffffUL
#else
#define MAX_DMA_ADDRESS (PAGE_OFFSET + ARM_DMA_ZONE_SIZE)
#define MAX_DMA_ADDRESS ({ \
extern unsigned long arm_dma_zone_size; \
arm_dma_zone_size ? \
(PAGE_OFFSET + arm_dma_zone_size) : 0xffffffffUL; })
#endif
#ifdef CONFIG_ISA_DMA_API

View File

@ -4,8 +4,8 @@
* Interrupt handling. Preserves r7, r8, r9
*/
.macro arch_irq_handler_default
get_irqnr_preamble r5, lr
1: get_irqnr_and_base r0, r6, r5, lr
get_irqnr_preamble r6, lr
1: get_irqnr_and_base r0, r2, r6, lr
movne r1, sp
@
@ routine called with r0 = irq number, r1 = struct pt_regs *
@ -17,17 +17,17 @@
/*
* XXX
*
* this macro assumes that irqstat (r6) and base (r5) are
* this macro assumes that irqstat (r2) and base (r6) are
* preserved from get_irqnr_and_base above
*/
ALT_SMP(test_for_ipi r0, r6, r5, lr)
ALT_SMP(test_for_ipi r0, r2, r6, lr)
ALT_UP_B(9997f)
movne r1, sp
adrne lr, BSYM(1b)
bne do_IPI
#ifdef CONFIG_LOCAL_TIMERS
test_for_ltirq r0, r6, r5, lr
test_for_ltirq r0, r2, r6, lr
movne r0, sp
adrne lr, BSYM(1b)
bne do_local_timer
@ -40,7 +40,7 @@
.align 5
.global \symbol_name
\symbol_name:
mov r4, lr
mov r8, lr
arch_irq_handler_default
mov pc, r4
mov pc, r8
.endm

View File

@ -4,22 +4,26 @@
/*
* HWCAP flags - for elf_hwcap (in kernel) and AT_HWCAP
*/
#define HWCAP_SWP 1
#define HWCAP_HALF 2
#define HWCAP_THUMB 4
#define HWCAP_26BIT 8 /* Play it safe */
#define HWCAP_FAST_MULT 16
#define HWCAP_FPA 32
#define HWCAP_VFP 64
#define HWCAP_EDSP 128
#define HWCAP_JAVA 256
#define HWCAP_IWMMXT 512
#define HWCAP_CRUNCH 1024
#define HWCAP_THUMBEE 2048
#define HWCAP_NEON 4096
#define HWCAP_VFPv3 8192
#define HWCAP_VFPv3D16 16384
#define HWCAP_TLS 32768
#define HWCAP_SWP (1 << 0)
#define HWCAP_HALF (1 << 1)
#define HWCAP_THUMB (1 << 2)
#define HWCAP_26BIT (1 << 3) /* Play it safe */
#define HWCAP_FAST_MULT (1 << 4)
#define HWCAP_FPA (1 << 5)
#define HWCAP_VFP (1 << 6)
#define HWCAP_EDSP (1 << 7)
#define HWCAP_JAVA (1 << 8)
#define HWCAP_IWMMXT (1 << 9)
#define HWCAP_CRUNCH (1 << 10)
#define HWCAP_THUMBEE (1 << 11)
#define HWCAP_NEON (1 << 12)
#define HWCAP_VFPv3 (1 << 13)
#define HWCAP_VFPv3D16 (1 << 14)
#define HWCAP_TLS (1 << 15)
#define HWCAP_VFPv4 (1 << 16)
#define HWCAP_IDIVA (1 << 17)
#define HWCAP_IDIVT (1 << 18)
#define HWCAP_IDIV (HWCAP_IDIVA | HWCAP_IDIVT)
#if defined(__KERNEL__) && !defined(__ASSEMBLY__)
/*

View File

@ -24,12 +24,6 @@
#define MAX_INSN_SIZE 2
#define MAX_STACK_SIZE 64 /* 32 would probably be OK */
/*
* This undefined instruction must be unique and
* reserved solely for kprobes' use.
*/
#define KPROBE_BREAKPOINT_INSTRUCTION 0xe7f001f8
#define regs_return_value(regs) ((regs)->ARM_r0)
#define flush_insn_slot(p) do { } while (0)
#define kretprobe_blacklist_size 0
@ -38,14 +32,17 @@ typedef u32 kprobe_opcode_t;
struct kprobe;
typedef void (kprobe_insn_handler_t)(struct kprobe *, struct pt_regs *);
typedef unsigned long (kprobe_check_cc)(unsigned long);
typedef void (kprobe_insn_singlestep_t)(struct kprobe *, struct pt_regs *);
typedef void (kprobe_insn_fn_t)(void);
/* Architecture specific copy of original instruction. */
struct arch_specific_insn {
kprobe_opcode_t *insn;
kprobe_insn_handler_t *insn_handler;
kprobe_check_cc *insn_check_cc;
kprobe_opcode_t *insn;
kprobe_insn_handler_t *insn_handler;
kprobe_check_cc *insn_check_cc;
kprobe_insn_singlestep_t *insn_singlestep;
kprobe_insn_fn_t *insn_fn;
};
struct prev_kprobe {
@ -62,20 +59,9 @@ struct kprobe_ctlblk {
};
void arch_remove_kprobe(struct kprobe *);
void kretprobe_trampoline(void);
int kprobe_fault_handler(struct pt_regs *regs, unsigned int fsr);
int kprobe_exceptions_notify(struct notifier_block *self,
unsigned long val, void *data);
enum kprobe_insn {
INSN_REJECTED,
INSN_GOOD,
INSN_GOOD_NO_SLOT
};
enum kprobe_insn arm_kprobe_decode_insn(kprobe_opcode_t,
struct arch_specific_insn *);
void __init arm_kprobe_decode_init(void);
#endif /* _ARM_KPROBES_H */

View File

@ -23,6 +23,10 @@ struct machine_desc {
unsigned int nr_irqs; /* number of IRQs */
#ifdef CONFIG_ZONE_DMA
unsigned long dma_zone_size; /* size of DMA-able area */
#endif
unsigned int video_start; /* start of video RAM */
unsigned int video_end; /* end of video RAM */

View File

@ -203,18 +203,6 @@ static inline unsigned long __phys_to_virt(unsigned long x)
#define PHYS_OFFSET PLAT_PHYS_OFFSET
#endif
/*
* The DMA mask corresponding to the maximum bus address allocatable
* using GFP_DMA. The default here places no restriction on DMA
* allocations. This must be the smallest DMA mask in the system,
* so a successful GFP_DMA allocation will always satisfy this.
*/
#ifndef ARM_DMA_ZONE_SIZE
#define ISA_DMA_THRESHOLD (0xffffffffULL)
#else
#define ISA_DMA_THRESHOLD (PHYS_OFFSET + ARM_DMA_ZONE_SIZE - 1)
#endif
/*
* PFNs are used to describe any physical page; this means
* PFN 0 == physical address 0.

View File

@ -24,6 +24,8 @@ enum arm_perf_pmu_ids {
ARM_PERF_PMU_ID_V6MP,
ARM_PERF_PMU_ID_CA8,
ARM_PERF_PMU_ID_CA9,
ARM_PERF_PMU_ID_CA5,
ARM_PERF_PMU_ID_CA15,
ARM_NUM_PMU_IDS,
};

View File

@ -52,7 +52,7 @@ reserve_pmu(enum arm_pmu_type device);
* a cookie.
*/
extern int
release_pmu(struct platform_device *pdev);
release_pmu(enum arm_pmu_type type);
/**
* init_pmu() - Initialise the PMU.

View File

@ -82,13 +82,13 @@ extern void cpu_do_switch_mm(unsigned long pgd_phys, struct mm_struct *mm);
extern void cpu_set_pte_ext(pte_t *ptep, pte_t pte, unsigned int ext);
extern void cpu_reset(unsigned long addr) __attribute__((noreturn));
#else
#define cpu_proc_init() processor._proc_init()
#define cpu_proc_fin() processor._proc_fin()
#define cpu_reset(addr) processor.reset(addr)
#define cpu_do_idle() processor._do_idle()
#define cpu_dcache_clean_area(addr,sz) processor.dcache_clean_area(addr,sz)
#define cpu_set_pte_ext(ptep,pte,ext) processor.set_pte_ext(ptep,pte,ext)
#define cpu_do_switch_mm(pgd,mm) processor.switch_mm(pgd,mm)
#define cpu_proc_init processor._proc_init
#define cpu_proc_fin processor._proc_fin
#define cpu_reset processor.reset
#define cpu_do_idle processor._do_idle
#define cpu_dcache_clean_area processor.dcache_clean_area
#define cpu_set_pte_ext processor.set_pte_ext
#define cpu_do_switch_mm processor.switch_mm
#endif
extern void cpu_resume(void);

View File

@ -69,8 +69,9 @@
#define PSR_c 0x000000ff /* Control */
/*
* ARMv7 groups of APSR bits
* ARMv7 groups of PSR bits
*/
#define APSR_MASK 0xf80f0000 /* N, Z, C, V, Q and GE flags */
#define PSR_ISET_MASK 0x01000010 /* ISA state (J, T) mask */
#define PSR_IT_MASK 0x0600fc00 /* If-Then execution state mask */
#define PSR_ENDIAN_MASK 0x00000200 /* Endianness state mask */
@ -199,6 +200,14 @@ extern unsigned long profile_pc(struct pt_regs *regs);
#define predicate(x) ((x) & 0xf0000000)
#define PREDICATE_ALWAYS 0xe0000000
/*
* True if instr is a 32-bit thumb instruction. This works if instr
* is the first or only half-word of a thumb instruction. It also works
* when instr holds all 32-bits of a wide thumb instruction if stored
* in the form (first_half<<16)|(second_half)
*/
#define is_wide_instruction(instr) ((unsigned)(instr) >= 0xe800)
/*
* kprobe-based event tracer support
*/

View File

@ -1,6 +1,10 @@
#ifndef _ASMARM_SCATTERLIST_H
#define _ASMARM_SCATTERLIST_H
#ifdef CONFIG_ARM_HAS_SG_CHAIN
#define ARCH_HAS_SG_CHAIN
#endif
#include <asm/memory.h>
#include <asm/types.h>
#include <asm-generic/scatterlist.h>

View File

@ -187,12 +187,16 @@ struct tagtable {
#define __tag __used __attribute__((__section__(".taglist.init")))
#define __tagtable(tag, fn) \
static struct tagtable __tagtable_##fn __tag = { tag, fn }
static const struct tagtable __tagtable_##fn __tag = { tag, fn }
/*
* Memory map description
*/
#define NR_BANKS 8
#ifdef CONFIG_ARCH_EP93XX
# define NR_BANKS 16
#else
# define NR_BANKS 8
#endif
struct membank {
phys_addr_t start;

View File

@ -0,0 +1,22 @@
#ifndef __ASM_ARM_SUSPEND_H
#define __ASM_ARM_SUSPEND_H
#include <asm/memory.h>
#include <asm/tlbflush.h>
extern void cpu_resume(void);
/*
* Hide the first two arguments to __cpu_suspend - these are an implementation
* detail which platform code shouldn't have to know about.
*/
static inline int cpu_suspend(unsigned long arg, int (*fn)(unsigned long))
{
extern int __cpu_suspend(int, long, unsigned long,
int (*)(unsigned long));
int ret = __cpu_suspend(0, PHYS_OFFSET - PAGE_OFFSET, arg, fn);
flush_tlb_all();
return ret;
}
#endif

View File

@ -27,5 +27,7 @@
void *tcm_alloc(size_t len);
void tcm_free(void *addr, size_t len);
bool tcm_dtcm_present(void);
bool tcm_itcm_present(void);
#endif

View File

@ -34,16 +34,12 @@
#define TLB_V6_D_ASID (1 << 17)
#define TLB_V6_I_ASID (1 << 18)
#define TLB_BTB (1 << 28)
/* Unified Inner Shareable TLB operations (ARMv7 MP extensions) */
#define TLB_V7_UIS_PAGE (1 << 19)
#define TLB_V7_UIS_FULL (1 << 20)
#define TLB_V7_UIS_ASID (1 << 21)
/* Inner Shareable BTB operation (ARMv7 MP extensions) */
#define TLB_V7_IS_BTB (1 << 22)
#define TLB_BARRIER (1 << 28)
#define TLB_L2CLEAN_FR (1 << 29) /* Feroceon */
#define TLB_DCLEAN (1 << 30)
#define TLB_WB (1 << 31)
@ -58,7 +54,7 @@
* v4wb - ARMv4 with write buffer without I TLB flush entry instruction
* v4wbi - ARMv4 with write buffer with I TLB flush entry instruction
* fr - Feroceon (v4wbi with non-outer-cacheable page table walks)
* fa - Faraday (v4 with write buffer with UTLB and branch target buffer (BTB))
* fa - Faraday (v4 with write buffer with UTLB)
* v6wbi - ARMv6 with write buffer with I TLB flush entry instruction
* v7wbi - identical to v6wbi
*/
@ -99,7 +95,7 @@
# define v4_always_flags (-1UL)
#endif
#define fa_tlb_flags (TLB_WB | TLB_BTB | TLB_DCLEAN | \
#define fa_tlb_flags (TLB_WB | TLB_DCLEAN | TLB_BARRIER | \
TLB_V4_U_FULL | TLB_V4_U_PAGE)
#ifdef CONFIG_CPU_TLB_FA
@ -166,7 +162,7 @@
# define v4wb_always_flags (-1UL)
#endif
#define v6wbi_tlb_flags (TLB_WB | TLB_DCLEAN | TLB_BTB | \
#define v6wbi_tlb_flags (TLB_WB | TLB_DCLEAN | TLB_BARRIER | \
TLB_V6_I_FULL | TLB_V6_D_FULL | \
TLB_V6_I_PAGE | TLB_V6_D_PAGE | \
TLB_V6_I_ASID | TLB_V6_D_ASID)
@ -184,9 +180,9 @@
# define v6wbi_always_flags (-1UL)
#endif
#define v7wbi_tlb_flags_smp (TLB_WB | TLB_DCLEAN | TLB_V7_IS_BTB | \
#define v7wbi_tlb_flags_smp (TLB_WB | TLB_DCLEAN | TLB_BARRIER | \
TLB_V7_UIS_FULL | TLB_V7_UIS_PAGE | TLB_V7_UIS_ASID)
#define v7wbi_tlb_flags_up (TLB_WB | TLB_DCLEAN | TLB_BTB | \
#define v7wbi_tlb_flags_up (TLB_WB | TLB_DCLEAN | TLB_BARRIER | \
TLB_V6_U_FULL | TLB_V6_U_PAGE | TLB_V6_U_ASID)
#ifdef CONFIG_CPU_TLB_V7
@ -341,15 +337,7 @@ static inline void local_flush_tlb_all(void)
if (tlb_flag(TLB_V7_UIS_FULL))
asm("mcr p15, 0, %0, c8, c3, 0" : : "r" (zero) : "cc");
if (tlb_flag(TLB_BTB)) {
/* flush the branch target cache */
asm("mcr p15, 0, %0, c7, c5, 6" : : "r" (zero) : "cc");
dsb();
isb();
}
if (tlb_flag(TLB_V7_IS_BTB)) {
/* flush the branch target cache */
asm("mcr p15, 0, %0, c7, c1, 6" : : "r" (zero) : "cc");
if (tlb_flag(TLB_BARRIER)) {
dsb();
isb();
}
@ -389,17 +377,8 @@ static inline void local_flush_tlb_mm(struct mm_struct *mm)
asm("mcr p15, 0, %0, c8, c3, 2" : : "r" (asid) : "cc");
#endif
if (tlb_flag(TLB_BTB)) {
/* flush the branch target cache */
asm("mcr p15, 0, %0, c7, c5, 6" : : "r" (zero) : "cc");
if (tlb_flag(TLB_BARRIER))
dsb();
}
if (tlb_flag(TLB_V7_IS_BTB)) {
/* flush the branch target cache */
asm("mcr p15, 0, %0, c7, c1, 6" : : "r" (zero) : "cc");
dsb();
isb();
}
}
static inline void
@ -439,17 +418,8 @@ local_flush_tlb_page(struct vm_area_struct *vma, unsigned long uaddr)
asm("mcr p15, 0, %0, c8, c3, 1" : : "r" (uaddr) : "cc");
#endif
if (tlb_flag(TLB_BTB)) {
/* flush the branch target cache */
asm("mcr p15, 0, %0, c7, c5, 6" : : "r" (zero) : "cc");
if (tlb_flag(TLB_BARRIER))
dsb();
}
if (tlb_flag(TLB_V7_IS_BTB)) {
/* flush the branch target cache */
asm("mcr p15, 0, %0, c7, c1, 6" : : "r" (zero) : "cc");
dsb();
isb();
}
}
static inline void local_flush_tlb_kernel_page(unsigned long kaddr)
@ -482,15 +452,7 @@ static inline void local_flush_tlb_kernel_page(unsigned long kaddr)
if (tlb_flag(TLB_V7_UIS_PAGE))
asm("mcr p15, 0, %0, c8, c3, 1" : : "r" (kaddr) : "cc");
if (tlb_flag(TLB_BTB)) {
/* flush the branch target cache */
asm("mcr p15, 0, %0, c7, c5, 6" : : "r" (zero) : "cc");
dsb();
isb();
}
if (tlb_flag(TLB_V7_IS_BTB)) {
/* flush the branch target cache */
asm("mcr p15, 0, %0, c7, c1, 6" : : "r" (zero) : "cc");
if (tlb_flag(TLB_BARRIER)) {
dsb();
isb();
}

View File

@ -3,6 +3,9 @@
#include <linux/list.h>
struct pt_regs;
struct task_struct;
struct undef_hook {
struct list_head node;
u32 instr_mask;

View File

@ -37,7 +37,12 @@ obj-$(CONFIG_HAVE_ARM_TWD) += smp_twd.o
obj-$(CONFIG_DYNAMIC_FTRACE) += ftrace.o
obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
obj-$(CONFIG_KEXEC) += machine_kexec.o relocate_kernel.o
obj-$(CONFIG_KPROBES) += kprobes.o kprobes-decode.o
obj-$(CONFIG_KPROBES) += kprobes.o kprobes-common.o
ifdef CONFIG_THUMB2_KERNEL
obj-$(CONFIG_KPROBES) += kprobes-thumb.o
else
obj-$(CONFIG_KPROBES) += kprobes-arm.o
endif
obj-$(CONFIG_ATAGS_PROC) += atags.o
obj-$(CONFIG_OABI_COMPAT) += sys_oabi-compat.o
obj-$(CONFIG_ARM_THUMBEE) += thumbee.o

View File

@ -59,6 +59,9 @@ int main(void)
DEFINE(TI_TP_VALUE, offsetof(struct thread_info, tp_value));
DEFINE(TI_FPSTATE, offsetof(struct thread_info, fpstate));
DEFINE(TI_VFPSTATE, offsetof(struct thread_info, vfpstate));
#ifdef CONFIG_SMP
DEFINE(VFP_CPU, offsetof(union vfp_state, hard.cpu));
#endif
#ifdef CONFIG_ARM_THUMBEE
DEFINE(TI_THUMBEE_STATE, offsetof(struct thread_info, thumbee_state));
#endif

View File

@ -29,21 +29,53 @@
#include <asm/entry-macro-multi.S>
/*
* Interrupt handling. Preserves r7, r8, r9
* Interrupt handling.
*/
.macro irq_handler
#ifdef CONFIG_MULTI_IRQ_HANDLER
ldr r5, =handle_arch_irq
ldr r1, =handle_arch_irq
mov r0, sp
ldr r5, [r5]
ldr r1, [r1]
adr lr, BSYM(9997f)
teq r5, #0
movne pc, r5
teq r1, #0
movne pc, r1
#endif
arch_irq_handler_default
9997:
.endm
.macro pabt_helper
@ PABORT handler takes pt_regs in r2, fault address in r4 and psr in r5
#ifdef MULTI_PABORT
ldr ip, .LCprocfns
mov lr, pc
ldr pc, [ip, #PROCESSOR_PABT_FUNC]
#else
bl CPU_PABORT_HANDLER
#endif
.endm
.macro dabt_helper
@
@ Call the processor-specific abort handler:
@
@ r2 - pt_regs
@ r4 - aborted context pc
@ r5 - aborted context psr
@
@ The abort handler must return the aborted address in r0, and
@ the fault status register in r1. r9 must be preserved.
@
#ifdef MULTI_DABORT
ldr ip, .LCprocfns
mov lr, pc
ldr pc, [ip, #PROCESSOR_DABT_FUNC]
#else
bl CPU_DABORT_HANDLER
#endif
.endm
#ifdef CONFIG_KPROBES
.section .kprobes.text,"ax",%progbits
#else
@ -126,106 +158,74 @@ ENDPROC(__und_invalid)
SPFIX( subeq sp, sp, #4 )
stmia sp, {r1 - r12}
ldmia r0, {r1 - r3}
add r5, sp, #S_SP - 4 @ here for interlock avoidance
mov r4, #-1 @ "" "" "" ""
add r0, sp, #(S_FRAME_SIZE + \stack_hole - 4)
SPFIX( addeq r0, r0, #4 )
str r1, [sp, #-4]! @ save the "real" r0 copied
ldmia r0, {r3 - r5}
add r7, sp, #S_SP - 4 @ here for interlock avoidance
mov r6, #-1 @ "" "" "" ""
add r2, sp, #(S_FRAME_SIZE + \stack_hole - 4)
SPFIX( addeq r2, r2, #4 )
str r3, [sp, #-4]! @ save the "real" r0 copied
@ from the exception stack
mov r1, lr
mov r3, lr
@
@ We are now ready to fill in the remaining blanks on the stack:
@
@ r0 - sp_svc
@ r1 - lr_svc
@ r2 - lr_<exception>, already fixed up for correct return/restart
@ r3 - spsr_<exception>
@ r4 - orig_r0 (see pt_regs definition in ptrace.h)
@ r2 - sp_svc
@ r3 - lr_svc
@ r4 - lr_<exception>, already fixed up for correct return/restart
@ r5 - spsr_<exception>
@ r6 - orig_r0 (see pt_regs definition in ptrace.h)
@
stmia r5, {r0 - r4}
stmia r7, {r2 - r6}
#ifdef CONFIG_TRACE_IRQFLAGS
bl trace_hardirqs_off
#endif
.endm
.align 5
__dabt_svc:
svc_entry
@
@ get ready to re-enable interrupts if appropriate
@
mrs r9, cpsr
tst r3, #PSR_I_BIT
biceq r9, r9, #PSR_I_BIT
@
@ Call the processor-specific abort handler:
@
@ r2 - aborted context pc
@ r3 - aborted context cpsr
@
@ The abort handler must return the aborted address in r0, and
@ the fault status register in r1. r9 must be preserved.
@
#ifdef MULTI_DABORT
ldr r4, .LCprocfns
mov lr, pc
ldr pc, [r4, #PROCESSOR_DABT_FUNC]
#else
bl CPU_DABORT_HANDLER
#endif
@
@ set desired IRQ state, then call main handler
@
debug_entry r1
msr cpsr_c, r9
mov r2, sp
bl do_DataAbort
dabt_helper
@
@ IRQs off again before pulling preserved data off the stack
@
disable_irq_notrace
@
@ restore SPSR and restart the instruction
@
ldr r2, [sp, #S_PSR]
svc_exit r2 @ return from exception
#ifdef CONFIG_TRACE_IRQFLAGS
tst r5, #PSR_I_BIT
bleq trace_hardirqs_on
tst r5, #PSR_I_BIT
blne trace_hardirqs_off
#endif
svc_exit r5 @ return from exception
UNWIND(.fnend )
ENDPROC(__dabt_svc)
.align 5
__irq_svc:
svc_entry
irq_handler
#ifdef CONFIG_TRACE_IRQFLAGS
bl trace_hardirqs_off
#endif
#ifdef CONFIG_PREEMPT
get_thread_info tsk
ldr r8, [tsk, #TI_PREEMPT] @ get preempt count
add r7, r8, #1 @ increment it
str r7, [tsk, #TI_PREEMPT]
#endif
irq_handler
#ifdef CONFIG_PREEMPT
str r8, [tsk, #TI_PREEMPT] @ restore preempt count
ldr r0, [tsk, #TI_FLAGS] @ get flags
teq r8, #0 @ if preempt count != 0
movne r0, #0 @ force flags to 0
tst r0, #_TIF_NEED_RESCHED
blne svc_preempt
#endif
ldr r4, [sp, #S_PSR] @ irqs are already disabled
#ifdef CONFIG_TRACE_IRQFLAGS
tst r4, #PSR_I_BIT
bleq trace_hardirqs_on
@ The parent context IRQs must have been enabled to get here in
@ the first place, so there's no point checking the PSR I bit.
bl trace_hardirqs_on
#endif
svc_exit r4 @ return from exception
svc_exit r5 @ return from exception
UNWIND(.fnend )
ENDPROC(__irq_svc)
@ -251,7 +251,6 @@ __und_svc:
#else
svc_entry
#endif
@
@ call emulation code, which returns using r9 if it has emulated
@ the instruction, or the more conventional lr if we are to treat
@ -260,15 +259,16 @@ __und_svc:
@ r0 - instruction
@
#ifndef CONFIG_THUMB2_KERNEL
ldr r0, [r2, #-4]
ldr r0, [r4, #-4]
#else
ldrh r0, [r2, #-2] @ Thumb instruction at LR - 2
ldrh r0, [r4, #-2] @ Thumb instruction at LR - 2
and r9, r0, #0xf800
cmp r9, #0xe800 @ 32-bit instruction if xx >= 0
ldrhhs r9, [r2] @ bottom 16 bits
ldrhhs r9, [r4] @ bottom 16 bits
orrhs r0, r9, r0, lsl #16
#endif
adr r9, BSYM(1f)
mov r2, r4
bl call_fpe
mov r0, sp @ struct pt_regs *regs
@ -282,45 +282,35 @@ __und_svc:
@
@ restore SPSR and restart the instruction
@
ldr r2, [sp, #S_PSR] @ Get SVC cpsr
svc_exit r2 @ return from exception
ldr r5, [sp, #S_PSR] @ Get SVC cpsr
#ifdef CONFIG_TRACE_IRQFLAGS
tst r5, #PSR_I_BIT
bleq trace_hardirqs_on
tst r5, #PSR_I_BIT
blne trace_hardirqs_off
#endif
svc_exit r5 @ return from exception
UNWIND(.fnend )
ENDPROC(__und_svc)
.align 5
__pabt_svc:
svc_entry
@
@ re-enable interrupts if appropriate
@
mrs r9, cpsr
tst r3, #PSR_I_BIT
biceq r9, r9, #PSR_I_BIT
mov r0, r2 @ pass address of aborted instruction.
#ifdef MULTI_PABORT
ldr r4, .LCprocfns
mov lr, pc
ldr pc, [r4, #PROCESSOR_PABT_FUNC]
#else
bl CPU_PABORT_HANDLER
#endif
debug_entry r1
msr cpsr_c, r9 @ Maybe enable interrupts
mov r2, sp @ regs
bl do_PrefetchAbort @ call abort handler
pabt_helper
@
@ IRQs off again before pulling preserved data off the stack
@
disable_irq_notrace
@
@ restore SPSR and restart the instruction
@
ldr r2, [sp, #S_PSR]
svc_exit r2 @ return from exception
#ifdef CONFIG_TRACE_IRQFLAGS
tst r5, #PSR_I_BIT
bleq trace_hardirqs_on
tst r5, #PSR_I_BIT
blne trace_hardirqs_off
#endif
svc_exit r5 @ return from exception
UNWIND(.fnend )
ENDPROC(__pabt_svc)
@ -351,23 +341,23 @@ ENDPROC(__pabt_svc)
ARM( stmib sp, {r1 - r12} )
THUMB( stmia sp, {r0 - r12} )
ldmia r0, {r1 - r3}
ldmia r0, {r3 - r5}
add r0, sp, #S_PC @ here for interlock avoidance
mov r4, #-1 @ "" "" "" ""
mov r6, #-1 @ "" "" "" ""
str r1, [sp] @ save the "real" r0 copied
str r3, [sp] @ save the "real" r0 copied
@ from the exception stack
@
@ We are now ready to fill in the remaining blanks on the stack:
@
@ r2 - lr_<exception>, already fixed up for correct return/restart
@ r3 - spsr_<exception>
@ r4 - orig_r0 (see pt_regs definition in ptrace.h)
@ r4 - lr_<exception>, already fixed up for correct return/restart
@ r5 - spsr_<exception>
@ r6 - orig_r0 (see pt_regs definition in ptrace.h)
@
@ Also, separately save sp_usr and lr_usr
@
stmia r0, {r2 - r4}
stmia r0, {r4 - r6}
ARM( stmdb r0, {sp, lr}^ )
THUMB( store_user_sp_lr r0, r1, S_SP - S_PC )
@ -380,10 +370,14 @@ ENDPROC(__pabt_svc)
@ Clear FP to mark the first stack frame
@
zero_fp
#ifdef CONFIG_IRQSOFF_TRACER
bl trace_hardirqs_off
#endif
.endm
.macro kuser_cmpxchg_check
#if __LINUX_ARM_ARCH__ < 6 && !defined(CONFIG_NEEDS_SYSCALL_FOR_CMPXCHG)
#if !defined(CONFIG_CPU_32v6K) && !defined(CONFIG_NEEDS_SYSCALL_FOR_CMPXCHG)
#ifndef CONFIG_MMU
#warning "NPTL on non MMU needs fixing"
#else
@ -391,8 +385,8 @@ ENDPROC(__pabt_svc)
@ if it was interrupted in a critical region. Here we
@ perform a quick test inline since it should be false
@ 99.9999% of the time. The rest is done out of line.
cmp r2, #TASK_SIZE
blhs kuser_cmpxchg_fixup
cmp r4, #TASK_SIZE
blhs kuser_cmpxchg64_fixup
#endif
#endif
.endm
@ -401,32 +395,9 @@ ENDPROC(__pabt_svc)
__dabt_usr:
usr_entry
kuser_cmpxchg_check
@
@ Call the processor-specific abort handler:
@
@ r2 - aborted context pc
@ r3 - aborted context cpsr
@
@ The abort handler must return the aborted address in r0, and
@ the fault status register in r1.
@
#ifdef MULTI_DABORT
ldr r4, .LCprocfns
mov lr, pc
ldr pc, [r4, #PROCESSOR_DABT_FUNC]
#else
bl CPU_DABORT_HANDLER
#endif
@
@ IRQs on, then call the main handler
@
debug_entry r1
enable_irq
mov r2, sp
adr lr, BSYM(ret_from_exception)
b do_DataAbort
dabt_helper
b ret_from_exception
UNWIND(.fnend )
ENDPROC(__dabt_usr)
@ -434,28 +405,8 @@ ENDPROC(__dabt_usr)
__irq_usr:
usr_entry
kuser_cmpxchg_check
#ifdef CONFIG_IRQSOFF_TRACER
bl trace_hardirqs_off
#endif
get_thread_info tsk
#ifdef CONFIG_PREEMPT
ldr r8, [tsk, #TI_PREEMPT] @ get preempt count
add r7, r8, #1 @ increment it
str r7, [tsk, #TI_PREEMPT]
#endif
irq_handler
#ifdef CONFIG_PREEMPT
ldr r0, [tsk, #TI_PREEMPT]
str r8, [tsk, #TI_PREEMPT]
teq r0, r7
ARM( strne r0, [r0, -r0] )
THUMB( movne r0, #0 )
THUMB( strne r0, [r0] )
#endif
get_thread_info tsk
mov why, #0
b ret_to_user_from_irq
UNWIND(.fnend )
@ -467,6 +418,9 @@ ENDPROC(__irq_usr)
__und_usr:
usr_entry
mov r2, r4
mov r3, r5
@
@ fall through to the emulation code, which returns using r9 if
@ it has emulated the instruction, or the more conventional lr
@ -682,19 +636,8 @@ ENDPROC(__und_usr_unknown)
.align 5
__pabt_usr:
usr_entry
mov r0, r2 @ pass address of aborted instruction.
#ifdef MULTI_PABORT
ldr r4, .LCprocfns
mov lr, pc
ldr pc, [r4, #PROCESSOR_PABT_FUNC]
#else
bl CPU_PABORT_HANDLER
#endif
debug_entry r1
enable_irq @ Enable interrupts
mov r2, sp @ regs
bl do_PrefetchAbort @ call abort handler
pabt_helper
UNWIND(.fnend )
/* fall through */
/*
@ -758,31 +701,12 @@ ENDPROC(__switch_to)
/*
* User helpers.
*
* These are segment of kernel provided user code reachable from user space
* at a fixed address in kernel memory. This is used to provide user space
* with some operations which require kernel help because of unimplemented
* native feature and/or instructions in many ARM CPUs. The idea is for
* this code to be executed directly in user mode for best efficiency but
* which is too intimate with the kernel counter part to be left to user
* libraries. In fact this code might even differ from one CPU to another
* depending on the available instruction set and restrictions like on
* SMP systems. In other words, the kernel reserves the right to change
* this code as needed without warning. Only the entry points and their
* results are guaranteed to be stable.
*
* Each segment is 32-byte aligned and will be moved to the top of the high
* vector page. New segments (if ever needed) must be added in front of
* existing ones. This mechanism should be used only for things that are
* really small and justified, and not be abused freely.
*
* User space is expected to implement those things inline when optimizing
* for a processor that has the necessary native support, but only if such
* resulting binaries are already to be incompatible with earlier ARM
* processors due to the use of unsupported instructions other than what
* is provided here. In other words don't make binaries unable to run on
* earlier processors just for the sake of not using these kernel helpers
* if your compiled code is not going to use the new instructions for other
* purpose.
* See Documentation/arm/kernel_user_helpers.txt for formal definitions.
*/
THUMB( .arm )
@ -799,97 +723,104 @@ ENDPROC(__switch_to)
__kuser_helper_start:
/*
* Reference prototype:
*
* void __kernel_memory_barrier(void)
*
* Input:
*
* lr = return address
*
* Output:
*
* none
*
* Clobbered:
*
* none
*
* Definition and user space usage example:
*
* typedef void (__kernel_dmb_t)(void);
* #define __kernel_dmb (*(__kernel_dmb_t *)0xffff0fa0)
*
* Apply any needed memory barrier to preserve consistency with data modified
* manually and __kuser_cmpxchg usage.
*
* This could be used as follows:
*
* #define __kernel_dmb() \
* asm volatile ( "mov r0, #0xffff0fff; mov lr, pc; sub pc, r0, #95" \
* : : : "r0", "lr","cc" )
* Due to the length of some sequences, __kuser_cmpxchg64 spans 2 regular
* kuser "slots", therefore 0xffff0f80 is not used as a valid entry point.
*/
__kuser_cmpxchg64: @ 0xffff0f60
#if defined(CONFIG_NEEDS_SYSCALL_FOR_CMPXCHG)
/*
* Poor you. No fast solution possible...
* The kernel itself must perform the operation.
* A special ghost syscall is used for that (see traps.c).
*/
stmfd sp!, {r7, lr}
ldr r7, 1f @ it's 20 bits
swi __ARM_NR_cmpxchg64
ldmfd sp!, {r7, pc}
1: .word __ARM_NR_cmpxchg64
#elif defined(CONFIG_CPU_32v6K)
stmfd sp!, {r4, r5, r6, r7}
ldrd r4, r5, [r0] @ load old val
ldrd r6, r7, [r1] @ load new val
smp_dmb arm
1: ldrexd r0, r1, [r2] @ load current val
eors r3, r0, r4 @ compare with oldval (1)
eoreqs r3, r1, r5 @ compare with oldval (2)
strexdeq r3, r6, r7, [r2] @ store newval if eq
teqeq r3, #1 @ success?
beq 1b @ if no then retry
smp_dmb arm
rsbs r0, r3, #0 @ set returned val and C flag
ldmfd sp!, {r4, r5, r6, r7}
bx lr
#elif !defined(CONFIG_SMP)
#ifdef CONFIG_MMU
/*
* The only thing that can break atomicity in this cmpxchg64
* implementation is either an IRQ or a data abort exception
* causing another process/thread to be scheduled in the middle of
* the critical sequence. The same strategy as for cmpxchg is used.
*/
stmfd sp!, {r4, r5, r6, lr}
ldmia r0, {r4, r5} @ load old val
ldmia r1, {r6, lr} @ load new val
1: ldmia r2, {r0, r1} @ load current val
eors r3, r0, r4 @ compare with oldval (1)
eoreqs r3, r1, r5 @ compare with oldval (2)
2: stmeqia r2, {r6, lr} @ store newval if eq
rsbs r0, r3, #0 @ set return val and C flag
ldmfd sp!, {r4, r5, r6, pc}
.text
kuser_cmpxchg64_fixup:
@ Called from kuser_cmpxchg_fixup.
@ r4 = address of interrupted insn (must be preserved).
@ sp = saved regs. r7 and r8 are clobbered.
@ 1b = first critical insn, 2b = last critical insn.
@ If r4 >= 1b and r4 <= 2b then saved pc_usr is set to 1b.
mov r7, #0xffff0fff
sub r7, r7, #(0xffff0fff - (0xffff0f60 + (1b - __kuser_cmpxchg64)))
subs r8, r4, r7
rsbcss r8, r8, #(2b - 1b)
strcs r7, [sp, #S_PC]
#if __LINUX_ARM_ARCH__ < 6
bcc kuser_cmpxchg32_fixup
#endif
mov pc, lr
.previous
#else
#warning "NPTL on non MMU needs fixing"
mov r0, #-1
adds r0, r0, #0
usr_ret lr
#endif
#else
#error "incoherent kernel configuration"
#endif
/* pad to next slot */
.rept (16 - (. - __kuser_cmpxchg64)/4)
.word 0
.endr
.align 5
__kuser_memory_barrier: @ 0xffff0fa0
smp_dmb arm
usr_ret lr
.align 5
/*
* Reference prototype:
*
* int __kernel_cmpxchg(int oldval, int newval, int *ptr)
*
* Input:
*
* r0 = oldval
* r1 = newval
* r2 = ptr
* lr = return address
*
* Output:
*
* r0 = returned value (zero or non-zero)
* C flag = set if r0 == 0, clear if r0 != 0
*
* Clobbered:
*
* r3, ip, flags
*
* Definition and user space usage example:
*
* typedef int (__kernel_cmpxchg_t)(int oldval, int newval, int *ptr);
* #define __kernel_cmpxchg (*(__kernel_cmpxchg_t *)0xffff0fc0)
*
* Atomically store newval in *ptr if *ptr is equal to oldval for user space.
* Return zero if *ptr was changed or non-zero if no exchange happened.
* The C flag is also set if *ptr was changed to allow for assembly
* optimization in the calling code.
*
* Notes:
*
* - This routine already includes memory barriers as needed.
*
* For example, a user space atomic_add implementation could look like this:
*
* #define atomic_add(ptr, val) \
* ({ register unsigned int *__ptr asm("r2") = (ptr); \
* register unsigned int __result asm("r1"); \
* asm volatile ( \
* "1: @ atomic_add\n\t" \
* "ldr r0, [r2]\n\t" \
* "mov r3, #0xffff0fff\n\t" \
* "add lr, pc, #4\n\t" \
* "add r1, r0, %2\n\t" \
* "add pc, r3, #(0xffff0fc0 - 0xffff0fff)\n\t" \
* "bcc 1b" \
* : "=&r" (__result) \
* : "r" (__ptr), "rIL" (val) \
* : "r0","r3","ip","lr","cc","memory" ); \
* __result; })
*/
__kuser_cmpxchg: @ 0xffff0fc0
#if defined(CONFIG_NEEDS_SYSCALL_FOR_CMPXCHG)
@ -925,15 +856,15 @@ __kuser_cmpxchg: @ 0xffff0fc0
usr_ret lr
.text
kuser_cmpxchg_fixup:
kuser_cmpxchg32_fixup:
@ Called from kuser_cmpxchg_check macro.
@ r2 = address of interrupted insn (must be preserved).
@ r4 = address of interrupted insn (must be preserved).
@ sp = saved regs. r7 and r8 are clobbered.
@ 1b = first critical insn, 2b = last critical insn.
@ If r2 >= 1b and r2 <= 2b then saved pc_usr is set to 1b.
@ If r4 >= 1b and r4 <= 2b then saved pc_usr is set to 1b.
mov r7, #0xffff0fff
sub r7, r7, #(0xffff0fff - (0xffff0fc0 + (1b - __kuser_cmpxchg)))
subs r8, r2, r7
subs r8, r4, r7
rsbcss r8, r8, #(2b - 1b)
strcs r7, [sp, #S_PC]
mov pc, lr
@ -963,39 +894,6 @@ kuser_cmpxchg_fixup:
.align 5
/*
* Reference prototype:
*
* int __kernel_get_tls(void)
*
* Input:
*
* lr = return address
*
* Output:
*
* r0 = TLS value
*
* Clobbered:
*
* none
*
* Definition and user space usage example:
*
* typedef int (__kernel_get_tls_t)(void);
* #define __kernel_get_tls (*(__kernel_get_tls_t *)0xffff0fe0)
*
* Get the TLS value as previously set via the __ARM_NR_set_tls syscall.
*
* This could be used as follows:
*
* #define __kernel_get_tls() \
* ({ register unsigned int __val asm("r0"); \
* asm( "mov r0, #0xffff0fff; mov lr, pc; sub pc, r0, #31" \
* : "=r" (__val) : : "lr","cc" ); \
* __val; })
*/
__kuser_get_tls: @ 0xffff0fe0
ldr r0, [pc, #(16 - 8)] @ read TLS, set in kuser_get_tls_init
usr_ret lr
@ -1004,19 +902,6 @@ __kuser_get_tls: @ 0xffff0fe0
.word 0 @ 0xffff0ff0 software TLS value, then
.endr @ pad up to __kuser_helper_version
/*
* Reference declaration:
*
* extern unsigned int __kernel_helper_version;
*
* Definition and user space usage example:
*
* #define __kernel_helper_version (*(unsigned int *)0xffff0ffc)
*
* User space may read this to determine the curent number of helpers
* available.
*/
__kuser_helper_version: @ 0xffff0ffc
.word ((__kuser_helper_end - __kuser_helper_start) >> 5)

View File

@ -121,15 +121,13 @@
.endm
#else /* CONFIG_THUMB2_KERNEL */
.macro svc_exit, rpsr
ldr lr, [sp, #S_SP] @ top of the stack
ldrd r0, r1, [sp, #S_LR] @ calling lr and pc
clrex @ clear the exclusive monitor
ldr r0, [sp, #S_SP] @ top of the stack
ldr r1, [sp, #S_PC] @ return address
tst r0, #4 @ orig stack 8-byte aligned?
stmdb r0, {r1, \rpsr} @ rfe context
stmdb lr!, {r0, r1, \rpsr} @ calling lr and rfe context
ldmia sp, {r0 - r12}
ldr lr, [sp, #S_LR]
addeq sp, sp, #S_FRAME_SIZE - 8 @ aligned
addne sp, sp, #S_FRAME_SIZE - 4 @ not aligned
mov sp, lr
ldr lr, [sp], #4
rfeia sp!
.endm
@ -165,25 +163,6 @@
.endm
#endif /* !CONFIG_THUMB2_KERNEL */
@
@ Debug exceptions are taken as prefetch or data aborts.
@ We must disable preemption during the handler so that
@ we can access the debug registers safely.
@
.macro debug_entry, fsr
#if defined(CONFIG_HAVE_HW_BREAKPOINT) && defined(CONFIG_PREEMPT)
ldr r4, =0x40f @ mask out fsr.fs
and r5, r4, \fsr
cmp r5, #2 @ debug exception
bne 1f
get_thread_info r10
ldr r6, [r10, #TI_PREEMPT] @ get preempt count
add r11, r6, #1 @ increment it
str r11, [r10, #TI_PREEMPT]
1:
#endif
.endm
/*
* These are the registers used in the syscall handler, and allow us to
* have in theory up to 7 arguments to a function - r0 to r6.

View File

@ -32,8 +32,16 @@
* numbers for r1.
*
*/
.arm
__HEAD
ENTRY(stext)
THUMB( adr r9, BSYM(1f) ) @ Kernel is always entered in ARM.
THUMB( bx r9 ) @ If this is a Thumb-2 kernel,
THUMB( .thumb ) @ switch to Thumb now.
THUMB(1: )
setmode PSR_F_BIT | PSR_I_BIT | SVC_MODE, r9 @ ensure svc mode
@ and irqs disabled
#ifndef CONFIG_CPU_CP15

View File

@ -71,8 +71,16 @@
* crap here - that's what the boot loader (or in extreme, well justified
* circumstances, zImage) is for.
*/
.arm
__HEAD
ENTRY(stext)
THUMB( adr r9, BSYM(1f) ) @ Kernel is always entered in ARM.
THUMB( bx r9 ) @ If this is a Thumb-2 kernel,
THUMB( .thumb ) @ switch to Thumb now.
THUMB(1: )
setmode PSR_F_BIT | PSR_I_BIT | SVC_MODE, r9 @ ensure svc mode
@ and irqs disabled
mrc p15, 0, r9, c0, c0 @ get processor id

View File

@ -796,7 +796,7 @@ unlock:
/*
* Called from either the Data Abort Handler [watchpoint] or the
* Prefetch Abort Handler [breakpoint] with preemption disabled.
* Prefetch Abort Handler [breakpoint] with interrupts disabled.
*/
static int hw_breakpoint_pending(unsigned long addr, unsigned int fsr,
struct pt_regs *regs)
@ -804,8 +804,10 @@ static int hw_breakpoint_pending(unsigned long addr, unsigned int fsr,
int ret = 0;
u32 dscr;
/* We must be called with preemption disabled. */
WARN_ON(preemptible());
preempt_disable();
if (interrupts_enabled(regs))
local_irq_enable();
/* We only handle watchpoints and hardware breakpoints. */
ARM_DBG_READ(c1, 0, dscr);
@ -824,10 +826,6 @@ static int hw_breakpoint_pending(unsigned long addr, unsigned int fsr,
ret = 1; /* Unhandled fault. */
}
/*
* Re-enable preemption after it was disabled in the
* low-level exception handling code.
*/
preempt_enable();
return ret;

View File

@ -131,54 +131,63 @@ int __init arch_probe_nr_irqs(void)
#ifdef CONFIG_HOTPLUG_CPU
static bool migrate_one_irq(struct irq_data *d)
static bool migrate_one_irq(struct irq_desc *desc)
{
unsigned int cpu = cpumask_any_and(d->affinity, cpu_online_mask);
struct irq_data *d = irq_desc_get_irq_data(desc);
const struct cpumask *affinity = d->affinity;
struct irq_chip *c;
bool ret = false;
if (cpu >= nr_cpu_ids) {
cpu = cpumask_any(cpu_online_mask);
/*
* If this is a per-CPU interrupt, or the affinity does not
* include this CPU, then we have nothing to do.
*/
if (irqd_is_per_cpu(d) || !cpumask_test_cpu(smp_processor_id(), affinity))
return false;
if (cpumask_any_and(affinity, cpu_online_mask) >= nr_cpu_ids) {
affinity = cpu_online_mask;
ret = true;
}
pr_debug("IRQ%u: moving from cpu%u to cpu%u\n", d->irq, d->node, cpu);
d->chip->irq_set_affinity(d, cpumask_of(cpu), true);
c = irq_data_get_irq_chip(d);
if (c->irq_set_affinity)
c->irq_set_affinity(d, affinity, true);
else
pr_debug("IRQ%u: unable to set affinity\n", d->irq);
return ret;
}
/*
* The CPU has been marked offline. Migrate IRQs off this CPU. If
* the affinity settings do not allow other CPUs, force them onto any
* The current CPU has been marked offline. Migrate IRQs off this CPU.
* If the affinity settings do not allow other CPUs, force them onto any
* available CPU.
*
* Note: we must iterate over all IRQs, whether they have an attached
* action structure or not, as we need to get chained interrupts too.
*/
void migrate_irqs(void)
{
unsigned int i, cpu = smp_processor_id();
unsigned int i;
struct irq_desc *desc;
unsigned long flags;
local_irq_save(flags);
for_each_irq_desc(i, desc) {
struct irq_data *d = &desc->irq_data;
bool affinity_broken = false;
if (!desc)
continue;
raw_spin_lock(&desc->lock);
do {
if (desc->action == NULL)
break;
if (d->node != cpu)
break;
affinity_broken = migrate_one_irq(d);
} while (0);
affinity_broken = migrate_one_irq(desc);
raw_spin_unlock(&desc->lock);
if (affinity_broken && printk_ratelimit())
pr_warning("IRQ%u no longer affine to CPU%u\n", i, cpu);
pr_warning("IRQ%u no longer affine to CPU%u\n", i,
smp_processor_id());
}
local_irq_restore(flags);

View File

@ -0,0 +1,999 @@
/*
* arch/arm/kernel/kprobes-decode.c
*
* Copyright (C) 2006, 2007 Motorola Inc.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* General Public License for more details.
*/
/*
* We do not have hardware single-stepping on ARM, This
* effort is further complicated by the ARM not having a
* "next PC" register. Instructions that change the PC
* can't be safely single-stepped in a MP environment, so
* we have a lot of work to do:
*
* In the prepare phase:
* *) If it is an instruction that does anything
* with the CPU mode, we reject it for a kprobe.
* (This is out of laziness rather than need. The
* instructions could be simulated.)
*
* *) Otherwise, decode the instruction rewriting its
* registers to take fixed, ordered registers and
* setting a handler for it to run the instruction.
*
* In the execution phase by an instruction's handler:
*
* *) If the PC is written to by the instruction, the
* instruction must be fully simulated in software.
*
* *) Otherwise, a modified form of the instruction is
* directly executed. Its handler calls the
* instruction in insn[0]. In insn[1] is a
* "mov pc, lr" to return.
*
* Before calling, load up the reordered registers
* from the original instruction's registers. If one
* of the original input registers is the PC, compute
* and adjust the appropriate input register.
*
* After call completes, copy the output registers to
* the original instruction's original registers.
*
* We don't use a real breakpoint instruction since that
* would have us in the kernel go from SVC mode to SVC
* mode losing the link register. Instead we use an
* undefined instruction. To simplify processing, the
* undefined instruction used for kprobes must be reserved
* exclusively for kprobes use.
*
* TODO: ifdef out some instruction decoding based on architecture.
*/
#include <linux/kernel.h>
#include <linux/kprobes.h>
#include "kprobes.h"
#define sign_extend(x, signbit) ((x) | (0 - ((x) & (1 << (signbit)))))
#define branch_displacement(insn) sign_extend(((insn) & 0xffffff) << 2, 25)
#if __LINUX_ARM_ARCH__ >= 6
#define BLX(reg) "blx "reg" \n\t"
#else
#define BLX(reg) "mov lr, pc \n\t" \
"mov pc, "reg" \n\t"
#endif
/*
* To avoid the complications of mimicing single-stepping on a
* processor without a Next-PC or a single-step mode, and to
* avoid having to deal with the side-effects of boosting, we
* simulate or emulate (almost) all ARM instructions.
*
* "Simulation" is where the instruction's behavior is duplicated in
* C code. "Emulation" is where the original instruction is rewritten
* and executed, often by altering its registers.
*
* By having all behavior of the kprobe'd instruction completed before
* returning from the kprobe_handler(), all locks (scheduler and
* interrupt) can safely be released. There is no need for secondary
* breakpoints, no race with MP or preemptable kernels, nor having to
* clean up resources counts at a later time impacting overall system
* performance. By rewriting the instruction, only the minimum registers
* need to be loaded and saved back optimizing performance.
*
* Calling the insnslot_*_rwflags version of a function doesn't hurt
* anything even when the CPSR flags aren't updated by the
* instruction. It's just a little slower in return for saving
* a little space by not having a duplicate function that doesn't
* update the flags. (The same optimization can be said for
* instructions that do or don't perform register writeback)
* Also, instructions can either read the flags, only write the
* flags, or read and write the flags. To save combinations
* rather than for sheer performance, flag functions just assume
* read and write of flags.
*/
static void __kprobes simulate_bbl(struct kprobe *p, struct pt_regs *regs)
{
kprobe_opcode_t insn = p->opcode;
long iaddr = (long)p->addr;
int disp = branch_displacement(insn);
if (insn & (1 << 24))
regs->ARM_lr = iaddr + 4;
regs->ARM_pc = iaddr + 8 + disp;
}
static void __kprobes simulate_blx1(struct kprobe *p, struct pt_regs *regs)
{
kprobe_opcode_t insn = p->opcode;
long iaddr = (long)p->addr;
int disp = branch_displacement(insn);
regs->ARM_lr = iaddr + 4;
regs->ARM_pc = iaddr + 8 + disp + ((insn >> 23) & 0x2);
regs->ARM_cpsr |= PSR_T_BIT;
}
static void __kprobes simulate_blx2bx(struct kprobe *p, struct pt_regs *regs)
{
kprobe_opcode_t insn = p->opcode;
int rm = insn & 0xf;
long rmv = regs->uregs[rm];
if (insn & (1 << 5))
regs->ARM_lr = (long)p->addr + 4;
regs->ARM_pc = rmv & ~0x1;
regs->ARM_cpsr &= ~PSR_T_BIT;
if (rmv & 0x1)
regs->ARM_cpsr |= PSR_T_BIT;
}
static void __kprobes simulate_mrs(struct kprobe *p, struct pt_regs *regs)
{
kprobe_opcode_t insn = p->opcode;
int rd = (insn >> 12) & 0xf;
unsigned long mask = 0xf8ff03df; /* Mask out execution state */
regs->uregs[rd] = regs->ARM_cpsr & mask;
}
static void __kprobes simulate_mov_ipsp(struct kprobe *p, struct pt_regs *regs)
{
regs->uregs[12] = regs->uregs[13];
}
static void __kprobes
emulate_ldrdstrd(struct kprobe *p, struct pt_regs *regs)
{
kprobe_opcode_t insn = p->opcode;
unsigned long pc = (unsigned long)p->addr + 8;
int rt = (insn >> 12) & 0xf;
int rn = (insn >> 16) & 0xf;
int rm = insn & 0xf;
register unsigned long rtv asm("r0") = regs->uregs[rt];
register unsigned long rt2v asm("r1") = regs->uregs[rt+1];
register unsigned long rnv asm("r2") = (rn == 15) ? pc
: regs->uregs[rn];
register unsigned long rmv asm("r3") = regs->uregs[rm];
__asm__ __volatile__ (
BLX("%[fn]")
: "=r" (rtv), "=r" (rt2v), "=r" (rnv)
: "0" (rtv), "1" (rt2v), "2" (rnv), "r" (rmv),
[fn] "r" (p->ainsn.insn_fn)
: "lr", "memory", "cc"
);
regs->uregs[rt] = rtv;
regs->uregs[rt+1] = rt2v;
if (is_writeback(insn))
regs->uregs[rn] = rnv;
}
static void __kprobes
emulate_ldr(struct kprobe *p, struct pt_regs *regs)
{
kprobe_opcode_t insn = p->opcode;
unsigned long pc = (unsigned long)p->addr + 8;
int rt = (insn >> 12) & 0xf;
int rn = (insn >> 16) & 0xf;
int rm = insn & 0xf;
register unsigned long rtv asm("r0");
register unsigned long rnv asm("r2") = (rn == 15) ? pc
: regs->uregs[rn];
register unsigned long rmv asm("r3") = regs->uregs[rm];
__asm__ __volatile__ (
BLX("%[fn]")
: "=r" (rtv), "=r" (rnv)
: "1" (rnv), "r" (rmv), [fn] "r" (p->ainsn.insn_fn)
: "lr", "memory", "cc"
);
if (rt == 15)
load_write_pc(rtv, regs);
else
regs->uregs[rt] = rtv;
if (is_writeback(insn))
regs->uregs[rn] = rnv;
}
static void __kprobes
emulate_str(struct kprobe *p, struct pt_regs *regs)
{
kprobe_opcode_t insn = p->opcode;
unsigned long rtpc = (unsigned long)p->addr + str_pc_offset;
unsigned long rnpc = (unsigned long)p->addr + 8;
int rt = (insn >> 12) & 0xf;
int rn = (insn >> 16) & 0xf;
int rm = insn & 0xf;
register unsigned long rtv asm("r0") = (rt == 15) ? rtpc
: regs->uregs[rt];
register unsigned long rnv asm("r2") = (rn == 15) ? rnpc
: regs->uregs[rn];
register unsigned long rmv asm("r3") = regs->uregs[rm];
__asm__ __volatile__ (
BLX("%[fn]")
: "=r" (rnv)
: "r" (rtv), "0" (rnv), "r" (rmv), [fn] "r" (p->ainsn.insn_fn)
: "lr", "memory", "cc"
);
if (is_writeback(insn))
regs->uregs[rn] = rnv;
}
static void __kprobes
emulate_rd12rn16rm0rs8_rwflags(struct kprobe *p, struct pt_regs *regs)
{
kprobe_opcode_t insn = p->opcode;
unsigned long pc = (unsigned long)p->addr + 8;
int rd = (insn >> 12) & 0xf;
int rn = (insn >> 16) & 0xf;
int rm = insn & 0xf;
int rs = (insn >> 8) & 0xf;
register unsigned long rdv asm("r0") = regs->uregs[rd];
register unsigned long rnv asm("r2") = (rn == 15) ? pc
: regs->uregs[rn];
register unsigned long rmv asm("r3") = (rm == 15) ? pc
: regs->uregs[rm];
register unsigned long rsv asm("r1") = regs->uregs[rs];
unsigned long cpsr = regs->ARM_cpsr;
__asm__ __volatile__ (
"msr cpsr_fs, %[cpsr] \n\t"
BLX("%[fn]")
"mrs %[cpsr], cpsr \n\t"
: "=r" (rdv), [cpsr] "=r" (cpsr)
: "0" (rdv), "r" (rnv), "r" (rmv), "r" (rsv),
"1" (cpsr), [fn] "r" (p->ainsn.insn_fn)
: "lr", "memory", "cc"
);
if (rd == 15)
alu_write_pc(rdv, regs);
else
regs->uregs[rd] = rdv;
regs->ARM_cpsr = (regs->ARM_cpsr & ~APSR_MASK) | (cpsr & APSR_MASK);
}
static void __kprobes
emulate_rd12rn16rm0_rwflags_nopc(struct kprobe *p, struct pt_regs *regs)
{
kprobe_opcode_t insn = p->opcode;
int rd = (insn >> 12) & 0xf;
int rn = (insn >> 16) & 0xf;
int rm = insn & 0xf;
register unsigned long rdv asm("r0") = regs->uregs[rd];
register unsigned long rnv asm("r2") = regs->uregs[rn];
register unsigned long rmv asm("r3") = regs->uregs[rm];
unsigned long cpsr = regs->ARM_cpsr;
__asm__ __volatile__ (
"msr cpsr_fs, %[cpsr] \n\t"
BLX("%[fn]")
"mrs %[cpsr], cpsr \n\t"
: "=r" (rdv), [cpsr] "=r" (cpsr)
: "0" (rdv), "r" (rnv), "r" (rmv),
"1" (cpsr), [fn] "r" (p->ainsn.insn_fn)
: "lr", "memory", "cc"
);
regs->uregs[rd] = rdv;
regs->ARM_cpsr = (regs->ARM_cpsr & ~APSR_MASK) | (cpsr & APSR_MASK);
}
static void __kprobes
emulate_rd16rn12rm0rs8_rwflags_nopc(struct kprobe *p, struct pt_regs *regs)
{
kprobe_opcode_t insn = p->opcode;
int rd = (insn >> 16) & 0xf;
int rn = (insn >> 12) & 0xf;
int rm = insn & 0xf;
int rs = (insn >> 8) & 0xf;
register unsigned long rdv asm("r2") = regs->uregs[rd];
register unsigned long rnv asm("r0") = regs->uregs[rn];
register unsigned long rmv asm("r3") = regs->uregs[rm];
register unsigned long rsv asm("r1") = regs->uregs[rs];
unsigned long cpsr = regs->ARM_cpsr;
__asm__ __volatile__ (
"msr cpsr_fs, %[cpsr] \n\t"
BLX("%[fn]")
"mrs %[cpsr], cpsr \n\t"
: "=r" (rdv), [cpsr] "=r" (cpsr)
: "0" (rdv), "r" (rnv), "r" (rmv), "r" (rsv),
"1" (cpsr), [fn] "r" (p->ainsn.insn_fn)
: "lr", "memory", "cc"
);
regs->uregs[rd] = rdv;
regs->ARM_cpsr = (regs->ARM_cpsr & ~APSR_MASK) | (cpsr & APSR_MASK);
}
static void __kprobes
emulate_rd12rm0_noflags_nopc(struct kprobe *p, struct pt_regs *regs)
{
kprobe_opcode_t insn = p->opcode;
int rd = (insn >> 12) & 0xf;
int rm = insn & 0xf;
register unsigned long rdv asm("r0") = regs->uregs[rd];
register unsigned long rmv asm("r3") = regs->uregs[rm];
__asm__ __volatile__ (
BLX("%[fn]")
: "=r" (rdv)
: "0" (rdv), "r" (rmv), [fn] "r" (p->ainsn.insn_fn)
: "lr", "memory", "cc"
);
regs->uregs[rd] = rdv;
}
static void __kprobes
emulate_rdlo12rdhi16rn0rm8_rwflags_nopc(struct kprobe *p, struct pt_regs *regs)
{
kprobe_opcode_t insn = p->opcode;
int rdlo = (insn >> 12) & 0xf;
int rdhi = (insn >> 16) & 0xf;
int rn = insn & 0xf;
int rm = (insn >> 8) & 0xf;
register unsigned long rdlov asm("r0") = regs->uregs[rdlo];
register unsigned long rdhiv asm("r2") = regs->uregs[rdhi];
register unsigned long rnv asm("r3") = regs->uregs[rn];
register unsigned long rmv asm("r1") = regs->uregs[rm];
unsigned long cpsr = regs->ARM_cpsr;
__asm__ __volatile__ (
"msr cpsr_fs, %[cpsr] \n\t"
BLX("%[fn]")
"mrs %[cpsr], cpsr \n\t"
: "=r" (rdlov), "=r" (rdhiv), [cpsr] "=r" (cpsr)
: "0" (rdlov), "1" (rdhiv), "r" (rnv), "r" (rmv),
"2" (cpsr), [fn] "r" (p->ainsn.insn_fn)
: "lr", "memory", "cc"
);
regs->uregs[rdlo] = rdlov;
regs->uregs[rdhi] = rdhiv;
regs->ARM_cpsr = (regs->ARM_cpsr & ~APSR_MASK) | (cpsr & APSR_MASK);
}
/*
* For the instruction masking and comparisons in all the "space_*"
* functions below, Do _not_ rearrange the order of tests unless
* you're very, very sure of what you are doing. For the sake of
* efficiency, the masks for some tests sometimes assume other test
* have been done prior to them so the number of patterns to test
* for an instruction set can be as broad as possible to reduce the
* number of tests needed.
*/
static const union decode_item arm_1111_table[] = {
/* Unconditional instructions */
/* memory hint 1111 0100 x001 xxxx xxxx xxxx xxxx xxxx */
/* PLDI (immediate) 1111 0100 x101 xxxx xxxx xxxx xxxx xxxx */
/* PLDW (immediate) 1111 0101 x001 xxxx xxxx xxxx xxxx xxxx */
/* PLD (immediate) 1111 0101 x101 xxxx xxxx xxxx xxxx xxxx */
DECODE_SIMULATE (0xfe300000, 0xf4100000, kprobe_simulate_nop),
/* memory hint 1111 0110 x001 xxxx xxxx xxxx xxx0 xxxx */
/* PLDI (register) 1111 0110 x101 xxxx xxxx xxxx xxx0 xxxx */
/* PLDW (register) 1111 0111 x001 xxxx xxxx xxxx xxx0 xxxx */
/* PLD (register) 1111 0111 x101 xxxx xxxx xxxx xxx0 xxxx */
DECODE_SIMULATE (0xfe300010, 0xf6100000, kprobe_simulate_nop),
/* BLX (immediate) 1111 101x xxxx xxxx xxxx xxxx xxxx xxxx */
DECODE_SIMULATE (0xfe000000, 0xfa000000, simulate_blx1),
/* CPS 1111 0001 0000 xxx0 xxxx xxxx xx0x xxxx */
/* SETEND 1111 0001 0000 0001 xxxx xxxx 0000 xxxx */
/* SRS 1111 100x x1x0 xxxx xxxx xxxx xxxx xxxx */
/* RFE 1111 100x x0x1 xxxx xxxx xxxx xxxx xxxx */
/* Coprocessor instructions... */
/* MCRR2 1111 1100 0100 xxxx xxxx xxxx xxxx xxxx */
/* MRRC2 1111 1100 0101 xxxx xxxx xxxx xxxx xxxx */
/* LDC2 1111 110x xxx1 xxxx xxxx xxxx xxxx xxxx */
/* STC2 1111 110x xxx0 xxxx xxxx xxxx xxxx xxxx */
/* CDP2 1111 1110 xxxx xxxx xxxx xxxx xxx0 xxxx */
/* MCR2 1111 1110 xxx0 xxxx xxxx xxxx xxx1 xxxx */
/* MRC2 1111 1110 xxx1 xxxx xxxx xxxx xxx1 xxxx */
/* Other unallocated instructions... */
DECODE_END
};
static const union decode_item arm_cccc_0001_0xx0____0xxx_table[] = {
/* Miscellaneous instructions */
/* MRS cpsr cccc 0001 0000 xxxx xxxx xxxx 0000 xxxx */
DECODE_SIMULATEX(0x0ff000f0, 0x01000000, simulate_mrs,
REGS(0, NOPC, 0, 0, 0)),
/* BX cccc 0001 0010 xxxx xxxx xxxx 0001 xxxx */
DECODE_SIMULATE (0x0ff000f0, 0x01200010, simulate_blx2bx),
/* BLX (register) cccc 0001 0010 xxxx xxxx xxxx 0011 xxxx */
DECODE_SIMULATEX(0x0ff000f0, 0x01200030, simulate_blx2bx,
REGS(0, 0, 0, 0, NOPC)),
/* CLZ cccc 0001 0110 xxxx xxxx xxxx 0001 xxxx */
DECODE_EMULATEX (0x0ff000f0, 0x01600010, emulate_rd12rm0_noflags_nopc,
REGS(0, NOPC, 0, 0, NOPC)),
/* QADD cccc 0001 0000 xxxx xxxx xxxx 0101 xxxx */
/* QSUB cccc 0001 0010 xxxx xxxx xxxx 0101 xxxx */
/* QDADD cccc 0001 0100 xxxx xxxx xxxx 0101 xxxx */
/* QDSUB cccc 0001 0110 xxxx xxxx xxxx 0101 xxxx */
DECODE_EMULATEX (0x0f9000f0, 0x01000050, emulate_rd12rn16rm0_rwflags_nopc,
REGS(NOPC, NOPC, 0, 0, NOPC)),
/* BXJ cccc 0001 0010 xxxx xxxx xxxx 0010 xxxx */
/* MSR cccc 0001 0x10 xxxx xxxx xxxx 0000 xxxx */
/* MRS spsr cccc 0001 0100 xxxx xxxx xxxx 0000 xxxx */
/* BKPT 1110 0001 0010 xxxx xxxx xxxx 0111 xxxx */
/* SMC cccc 0001 0110 xxxx xxxx xxxx 0111 xxxx */
/* And unallocated instructions... */
DECODE_END
};
static const union decode_item arm_cccc_0001_0xx0____1xx0_table[] = {
/* Halfword multiply and multiply-accumulate */
/* SMLALxy cccc 0001 0100 xxxx xxxx xxxx 1xx0 xxxx */
DECODE_EMULATEX (0x0ff00090, 0x01400080, emulate_rdlo12rdhi16rn0rm8_rwflags_nopc,
REGS(NOPC, NOPC, NOPC, 0, NOPC)),
/* SMULWy cccc 0001 0010 xxxx xxxx xxxx 1x10 xxxx */
DECODE_OR (0x0ff000b0, 0x012000a0),
/* SMULxy cccc 0001 0110 xxxx xxxx xxxx 1xx0 xxxx */
DECODE_EMULATEX (0x0ff00090, 0x01600080, emulate_rd16rn12rm0rs8_rwflags_nopc,
REGS(NOPC, 0, NOPC, 0, NOPC)),
/* SMLAxy cccc 0001 0000 xxxx xxxx xxxx 1xx0 xxxx */
DECODE_OR (0x0ff00090, 0x01000080),
/* SMLAWy cccc 0001 0010 xxxx xxxx xxxx 1x00 xxxx */
DECODE_EMULATEX (0x0ff000b0, 0x01200080, emulate_rd16rn12rm0rs8_rwflags_nopc,
REGS(NOPC, NOPC, NOPC, 0, NOPC)),
DECODE_END
};
static const union decode_item arm_cccc_0000_____1001_table[] = {
/* Multiply and multiply-accumulate */
/* MUL cccc 0000 0000 xxxx xxxx xxxx 1001 xxxx */
/* MULS cccc 0000 0001 xxxx xxxx xxxx 1001 xxxx */
DECODE_EMULATEX (0x0fe000f0, 0x00000090, emulate_rd16rn12rm0rs8_rwflags_nopc,
REGS(NOPC, 0, NOPC, 0, NOPC)),
/* MLA cccc 0000 0010 xxxx xxxx xxxx 1001 xxxx */
/* MLAS cccc 0000 0011 xxxx xxxx xxxx 1001 xxxx */
DECODE_OR (0x0fe000f0, 0x00200090),
/* MLS cccc 0000 0110 xxxx xxxx xxxx 1001 xxxx */
DECODE_EMULATEX (0x0ff000f0, 0x00600090, emulate_rd16rn12rm0rs8_rwflags_nopc,
REGS(NOPC, NOPC, NOPC, 0, NOPC)),
/* UMAAL cccc 0000 0100 xxxx xxxx xxxx 1001 xxxx */
DECODE_OR (0x0ff000f0, 0x00400090),
/* UMULL cccc 0000 1000 xxxx xxxx xxxx 1001 xxxx */
/* UMULLS cccc 0000 1001 xxxx xxxx xxxx 1001 xxxx */
/* UMLAL cccc 0000 1010 xxxx xxxx xxxx 1001 xxxx */
/* UMLALS cccc 0000 1011 xxxx xxxx xxxx 1001 xxxx */
/* SMULL cccc 0000 1100 xxxx xxxx xxxx 1001 xxxx */
/* SMULLS cccc 0000 1101 xxxx xxxx xxxx 1001 xxxx */
/* SMLAL cccc 0000 1110 xxxx xxxx xxxx 1001 xxxx */
/* SMLALS cccc 0000 1111 xxxx xxxx xxxx 1001 xxxx */
DECODE_EMULATEX (0x0f8000f0, 0x00800090, emulate_rdlo12rdhi16rn0rm8_rwflags_nopc,
REGS(NOPC, NOPC, NOPC, 0, NOPC)),
DECODE_END
};
static const union decode_item arm_cccc_0001_____1001_table[] = {
/* Synchronization primitives */
/* SMP/SWPB cccc 0001 0x00 xxxx xxxx xxxx 1001 xxxx */
DECODE_EMULATEX (0x0fb000f0, 0x01000090, emulate_rd12rn16rm0_rwflags_nopc,
REGS(NOPC, NOPC, 0, 0, NOPC)),
/* LDREX/STREX{,D,B,H} cccc 0001 1xxx xxxx xxxx xxxx 1001 xxxx */
/* And unallocated instructions... */
DECODE_END
};
static const union decode_item arm_cccc_000x_____1xx1_table[] = {
/* Extra load/store instructions */
/* STRHT cccc 0000 xx10 xxxx xxxx xxxx 1011 xxxx */
/* ??? cccc 0000 xx10 xxxx xxxx xxxx 11x1 xxxx */
/* LDRHT cccc 0000 xx11 xxxx xxxx xxxx 1011 xxxx */
/* LDRSBT cccc 0000 xx11 xxxx xxxx xxxx 1101 xxxx */
/* LDRSHT cccc 0000 xx11 xxxx xxxx xxxx 1111 xxxx */
DECODE_REJECT (0x0f200090, 0x00200090),
/* LDRD/STRD lr,pc,{... cccc 000x x0x0 xxxx 111x xxxx 1101 xxxx */
DECODE_REJECT (0x0e10e0d0, 0x0000e0d0),
/* LDRD (register) cccc 000x x0x0 xxxx xxxx xxxx 1101 xxxx */
/* STRD (register) cccc 000x x0x0 xxxx xxxx xxxx 1111 xxxx */
DECODE_EMULATEX (0x0e5000d0, 0x000000d0, emulate_ldrdstrd,
REGS(NOPCWB, NOPCX, 0, 0, NOPC)),
/* LDRD (immediate) cccc 000x x1x0 xxxx xxxx xxxx 1101 xxxx */
/* STRD (immediate) cccc 000x x1x0 xxxx xxxx xxxx 1111 xxxx */
DECODE_EMULATEX (0x0e5000d0, 0x004000d0, emulate_ldrdstrd,
REGS(NOPCWB, NOPCX, 0, 0, 0)),
/* STRH (register) cccc 000x x0x0 xxxx xxxx xxxx 1011 xxxx */
DECODE_EMULATEX (0x0e5000f0, 0x000000b0, emulate_str,
REGS(NOPCWB, NOPC, 0, 0, NOPC)),
/* LDRH (register) cccc 000x x0x1 xxxx xxxx xxxx 1011 xxxx */
/* LDRSB (register) cccc 000x x0x1 xxxx xxxx xxxx 1101 xxxx */
/* LDRSH (register) cccc 000x x0x1 xxxx xxxx xxxx 1111 xxxx */
DECODE_EMULATEX (0x0e500090, 0x00100090, emulate_ldr,
REGS(NOPCWB, NOPC, 0, 0, NOPC)),
/* STRH (immediate) cccc 000x x1x0 xxxx xxxx xxxx 1011 xxxx */
DECODE_EMULATEX (0x0e5000f0, 0x004000b0, emulate_str,
REGS(NOPCWB, NOPC, 0, 0, 0)),
/* LDRH (immediate) cccc 000x x1x1 xxxx xxxx xxxx 1011 xxxx */
/* LDRSB (immediate) cccc 000x x1x1 xxxx xxxx xxxx 1101 xxxx */
/* LDRSH (immediate) cccc 000x x1x1 xxxx xxxx xxxx 1111 xxxx */
DECODE_EMULATEX (0x0e500090, 0x00500090, emulate_ldr,
REGS(NOPCWB, NOPC, 0, 0, 0)),
DECODE_END
};
static const union decode_item arm_cccc_000x_table[] = {
/* Data-processing (register) */
/* <op>S PC, ... cccc 000x xxx1 xxxx 1111 xxxx xxxx xxxx */
DECODE_REJECT (0x0e10f000, 0x0010f000),
/* MOV IP, SP 1110 0001 1010 0000 1100 0000 0000 1101 */
DECODE_SIMULATE (0xffffffff, 0xe1a0c00d, simulate_mov_ipsp),
/* TST (register) cccc 0001 0001 xxxx xxxx xxxx xxx0 xxxx */
/* TEQ (register) cccc 0001 0011 xxxx xxxx xxxx xxx0 xxxx */
/* CMP (register) cccc 0001 0101 xxxx xxxx xxxx xxx0 xxxx */
/* CMN (register) cccc 0001 0111 xxxx xxxx xxxx xxx0 xxxx */
DECODE_EMULATEX (0x0f900010, 0x01100000, emulate_rd12rn16rm0rs8_rwflags,
REGS(ANY, 0, 0, 0, ANY)),
/* MOV (register) cccc 0001 101x xxxx xxxx xxxx xxx0 xxxx */
/* MVN (register) cccc 0001 111x xxxx xxxx xxxx xxx0 xxxx */
DECODE_EMULATEX (0x0fa00010, 0x01a00000, emulate_rd12rn16rm0rs8_rwflags,
REGS(0, ANY, 0, 0, ANY)),
/* AND (register) cccc 0000 000x xxxx xxxx xxxx xxx0 xxxx */
/* EOR (register) cccc 0000 001x xxxx xxxx xxxx xxx0 xxxx */
/* SUB (register) cccc 0000 010x xxxx xxxx xxxx xxx0 xxxx */
/* RSB (register) cccc 0000 011x xxxx xxxx xxxx xxx0 xxxx */
/* ADD (register) cccc 0000 100x xxxx xxxx xxxx xxx0 xxxx */
/* ADC (register) cccc 0000 101x xxxx xxxx xxxx xxx0 xxxx */
/* SBC (register) cccc 0000 110x xxxx xxxx xxxx xxx0 xxxx */
/* RSC (register) cccc 0000 111x xxxx xxxx xxxx xxx0 xxxx */
/* ORR (register) cccc 0001 100x xxxx xxxx xxxx xxx0 xxxx */
/* BIC (register) cccc 0001 110x xxxx xxxx xxxx xxx0 xxxx */
DECODE_EMULATEX (0x0e000010, 0x00000000, emulate_rd12rn16rm0rs8_rwflags,
REGS(ANY, ANY, 0, 0, ANY)),
/* TST (reg-shift reg) cccc 0001 0001 xxxx xxxx xxxx 0xx1 xxxx */
/* TEQ (reg-shift reg) cccc 0001 0011 xxxx xxxx xxxx 0xx1 xxxx */
/* CMP (reg-shift reg) cccc 0001 0101 xxxx xxxx xxxx 0xx1 xxxx */
/* CMN (reg-shift reg) cccc 0001 0111 xxxx xxxx xxxx 0xx1 xxxx */
DECODE_EMULATEX (0x0f900090, 0x01100010, emulate_rd12rn16rm0rs8_rwflags,
REGS(ANY, 0, NOPC, 0, ANY)),
/* MOV (reg-shift reg) cccc 0001 101x xxxx xxxx xxxx 0xx1 xxxx */
/* MVN (reg-shift reg) cccc 0001 111x xxxx xxxx xxxx 0xx1 xxxx */
DECODE_EMULATEX (0x0fa00090, 0x01a00010, emulate_rd12rn16rm0rs8_rwflags,
REGS(0, ANY, NOPC, 0, ANY)),
/* AND (reg-shift reg) cccc 0000 000x xxxx xxxx xxxx 0xx1 xxxx */
/* EOR (reg-shift reg) cccc 0000 001x xxxx xxxx xxxx 0xx1 xxxx */
/* SUB (reg-shift reg) cccc 0000 010x xxxx xxxx xxxx 0xx1 xxxx */
/* RSB (reg-shift reg) cccc 0000 011x xxxx xxxx xxxx 0xx1 xxxx */
/* ADD (reg-shift reg) cccc 0000 100x xxxx xxxx xxxx 0xx1 xxxx */
/* ADC (reg-shift reg) cccc 0000 101x xxxx xxxx xxxx 0xx1 xxxx */
/* SBC (reg-shift reg) cccc 0000 110x xxxx xxxx xxxx 0xx1 xxxx */
/* RSC (reg-shift reg) cccc 0000 111x xxxx xxxx xxxx 0xx1 xxxx */
/* ORR (reg-shift reg) cccc 0001 100x xxxx xxxx xxxx 0xx1 xxxx */
/* BIC (reg-shift reg) cccc 0001 110x xxxx xxxx xxxx 0xx1 xxxx */
DECODE_EMULATEX (0x0e000090, 0x00000010, emulate_rd12rn16rm0rs8_rwflags,
REGS(ANY, ANY, NOPC, 0, ANY)),
DECODE_END
};
static const union decode_item arm_cccc_001x_table[] = {
/* Data-processing (immediate) */
/* MOVW cccc 0011 0000 xxxx xxxx xxxx xxxx xxxx */
/* MOVT cccc 0011 0100 xxxx xxxx xxxx xxxx xxxx */
DECODE_EMULATEX (0x0fb00000, 0x03000000, emulate_rd12rm0_noflags_nopc,
REGS(0, NOPC, 0, 0, 0)),
/* YIELD cccc 0011 0010 0000 xxxx xxxx 0000 0001 */
DECODE_OR (0x0fff00ff, 0x03200001),
/* SEV cccc 0011 0010 0000 xxxx xxxx 0000 0100 */
DECODE_EMULATE (0x0fff00ff, 0x03200004, kprobe_emulate_none),
/* NOP cccc 0011 0010 0000 xxxx xxxx 0000 0000 */
/* WFE cccc 0011 0010 0000 xxxx xxxx 0000 0010 */
/* WFI cccc 0011 0010 0000 xxxx xxxx 0000 0011 */
DECODE_SIMULATE (0x0fff00fc, 0x03200000, kprobe_simulate_nop),
/* DBG cccc 0011 0010 0000 xxxx xxxx ffff xxxx */
/* unallocated hints cccc 0011 0010 0000 xxxx xxxx xxxx xxxx */
/* MSR (immediate) cccc 0011 0x10 xxxx xxxx xxxx xxxx xxxx */
DECODE_REJECT (0x0fb00000, 0x03200000),
/* <op>S PC, ... cccc 001x xxx1 xxxx 1111 xxxx xxxx xxxx */
DECODE_REJECT (0x0e10f000, 0x0210f000),
/* TST (immediate) cccc 0011 0001 xxxx xxxx xxxx xxxx xxxx */
/* TEQ (immediate) cccc 0011 0011 xxxx xxxx xxxx xxxx xxxx */
/* CMP (immediate) cccc 0011 0101 xxxx xxxx xxxx xxxx xxxx */
/* CMN (immediate) cccc 0011 0111 xxxx xxxx xxxx xxxx xxxx */
DECODE_EMULATEX (0x0f900000, 0x03100000, emulate_rd12rn16rm0rs8_rwflags,
REGS(ANY, 0, 0, 0, 0)),
/* MOV (immediate) cccc 0011 101x xxxx xxxx xxxx xxxx xxxx */
/* MVN (immediate) cccc 0011 111x xxxx xxxx xxxx xxxx xxxx */
DECODE_EMULATEX (0x0fa00000, 0x03a00000, emulate_rd12rn16rm0rs8_rwflags,
REGS(0, ANY, 0, 0, 0)),
/* AND (immediate) cccc 0010 000x xxxx xxxx xxxx xxxx xxxx */
/* EOR (immediate) cccc 0010 001x xxxx xxxx xxxx xxxx xxxx */
/* SUB (immediate) cccc 0010 010x xxxx xxxx xxxx xxxx xxxx */
/* RSB (immediate) cccc 0010 011x xxxx xxxx xxxx xxxx xxxx */
/* ADD (immediate) cccc 0010 100x xxxx xxxx xxxx xxxx xxxx */
/* ADC (immediate) cccc 0010 101x xxxx xxxx xxxx xxxx xxxx */
/* SBC (immediate) cccc 0010 110x xxxx xxxx xxxx xxxx xxxx */
/* RSC (immediate) cccc 0010 111x xxxx xxxx xxxx xxxx xxxx */
/* ORR (immediate) cccc 0011 100x xxxx xxxx xxxx xxxx xxxx */
/* BIC (immediate) cccc 0011 110x xxxx xxxx xxxx xxxx xxxx */
DECODE_EMULATEX (0x0e000000, 0x02000000, emulate_rd12rn16rm0rs8_rwflags,
REGS(ANY, ANY, 0, 0, 0)),
DECODE_END
};
static const union decode_item arm_cccc_0110_____xxx1_table[] = {
/* Media instructions */
/* SEL cccc 0110 1000 xxxx xxxx xxxx 1011 xxxx */
DECODE_EMULATEX (0x0ff000f0, 0x068000b0, emulate_rd12rn16rm0_rwflags_nopc,
REGS(NOPC, NOPC, 0, 0, NOPC)),
/* SSAT cccc 0110 101x xxxx xxxx xxxx xx01 xxxx */
/* USAT cccc 0110 111x xxxx xxxx xxxx xx01 xxxx */
DECODE_OR(0x0fa00030, 0x06a00010),
/* SSAT16 cccc 0110 1010 xxxx xxxx xxxx 0011 xxxx */
/* USAT16 cccc 0110 1110 xxxx xxxx xxxx 0011 xxxx */
DECODE_EMULATEX (0x0fb000f0, 0x06a00030, emulate_rd12rn16rm0_rwflags_nopc,
REGS(0, NOPC, 0, 0, NOPC)),
/* REV cccc 0110 1011 xxxx xxxx xxxx 0011 xxxx */
/* REV16 cccc 0110 1011 xxxx xxxx xxxx 1011 xxxx */
/* RBIT cccc 0110 1111 xxxx xxxx xxxx 0011 xxxx */
/* REVSH cccc 0110 1111 xxxx xxxx xxxx 1011 xxxx */
DECODE_EMULATEX (0x0fb00070, 0x06b00030, emulate_rd12rm0_noflags_nopc,
REGS(0, NOPC, 0, 0, NOPC)),
/* ??? cccc 0110 0x00 xxxx xxxx xxxx xxx1 xxxx */
DECODE_REJECT (0x0fb00010, 0x06000010),
/* ??? cccc 0110 0xxx xxxx xxxx xxxx 1011 xxxx */
DECODE_REJECT (0x0f8000f0, 0x060000b0),
/* ??? cccc 0110 0xxx xxxx xxxx xxxx 1101 xxxx */
DECODE_REJECT (0x0f8000f0, 0x060000d0),
/* SADD16 cccc 0110 0001 xxxx xxxx xxxx 0001 xxxx */
/* SADDSUBX cccc 0110 0001 xxxx xxxx xxxx 0011 xxxx */
/* SSUBADDX cccc 0110 0001 xxxx xxxx xxxx 0101 xxxx */
/* SSUB16 cccc 0110 0001 xxxx xxxx xxxx 0111 xxxx */
/* SADD8 cccc 0110 0001 xxxx xxxx xxxx 1001 xxxx */
/* SSUB8 cccc 0110 0001 xxxx xxxx xxxx 1111 xxxx */
/* QADD16 cccc 0110 0010 xxxx xxxx xxxx 0001 xxxx */
/* QADDSUBX cccc 0110 0010 xxxx xxxx xxxx 0011 xxxx */
/* QSUBADDX cccc 0110 0010 xxxx xxxx xxxx 0101 xxxx */
/* QSUB16 cccc 0110 0010 xxxx xxxx xxxx 0111 xxxx */
/* QADD8 cccc 0110 0010 xxxx xxxx xxxx 1001 xxxx */
/* QSUB8 cccc 0110 0010 xxxx xxxx xxxx 1111 xxxx */
/* SHADD16 cccc 0110 0011 xxxx xxxx xxxx 0001 xxxx */
/* SHADDSUBX cccc 0110 0011 xxxx xxxx xxxx 0011 xxxx */
/* SHSUBADDX cccc 0110 0011 xxxx xxxx xxxx 0101 xxxx */
/* SHSUB16 cccc 0110 0011 xxxx xxxx xxxx 0111 xxxx */
/* SHADD8 cccc 0110 0011 xxxx xxxx xxxx 1001 xxxx */
/* SHSUB8 cccc 0110 0011 xxxx xxxx xxxx 1111 xxxx */
/* UADD16 cccc 0110 0101 xxxx xxxx xxxx 0001 xxxx */
/* UADDSUBX cccc 0110 0101 xxxx xxxx xxxx 0011 xxxx */
/* USUBADDX cccc 0110 0101 xxxx xxxx xxxx 0101 xxxx */
/* USUB16 cccc 0110 0101 xxxx xxxx xxxx 0111 xxxx */
/* UADD8 cccc 0110 0101 xxxx xxxx xxxx 1001 xxxx */
/* USUB8 cccc 0110 0101 xxxx xxxx xxxx 1111 xxxx */
/* UQADD16 cccc 0110 0110 xxxx xxxx xxxx 0001 xxxx */
/* UQADDSUBX cccc 0110 0110 xxxx xxxx xxxx 0011 xxxx */
/* UQSUBADDX cccc 0110 0110 xxxx xxxx xxxx 0101 xxxx */
/* UQSUB16 cccc 0110 0110 xxxx xxxx xxxx 0111 xxxx */
/* UQADD8 cccc 0110 0110 xxxx xxxx xxxx 1001 xxxx */
/* UQSUB8 cccc 0110 0110 xxxx xxxx xxxx 1111 xxxx */
/* UHADD16 cccc 0110 0111 xxxx xxxx xxxx 0001 xxxx */
/* UHADDSUBX cccc 0110 0111 xxxx xxxx xxxx 0011 xxxx */
/* UHSUBADDX cccc 0110 0111 xxxx xxxx xxxx 0101 xxxx */
/* UHSUB16 cccc 0110 0111 xxxx xxxx xxxx 0111 xxxx */
/* UHADD8 cccc 0110 0111 xxxx xxxx xxxx 1001 xxxx */
/* UHSUB8 cccc 0110 0111 xxxx xxxx xxxx 1111 xxxx */
DECODE_EMULATEX (0x0f800010, 0x06000010, emulate_rd12rn16rm0_rwflags_nopc,
REGS(NOPC, NOPC, 0, 0, NOPC)),
/* PKHBT cccc 0110 1000 xxxx xxxx xxxx x001 xxxx */
/* PKHTB cccc 0110 1000 xxxx xxxx xxxx x101 xxxx */
DECODE_EMULATEX (0x0ff00030, 0x06800010, emulate_rd12rn16rm0_rwflags_nopc,
REGS(NOPC, NOPC, 0, 0, NOPC)),
/* ??? cccc 0110 1001 xxxx xxxx xxxx 0111 xxxx */
/* ??? cccc 0110 1101 xxxx xxxx xxxx 0111 xxxx */
DECODE_REJECT (0x0fb000f0, 0x06900070),
/* SXTB16 cccc 0110 1000 1111 xxxx xxxx 0111 xxxx */
/* SXTB cccc 0110 1010 1111 xxxx xxxx 0111 xxxx */
/* SXTH cccc 0110 1011 1111 xxxx xxxx 0111 xxxx */
/* UXTB16 cccc 0110 1100 1111 xxxx xxxx 0111 xxxx */
/* UXTB cccc 0110 1110 1111 xxxx xxxx 0111 xxxx */
/* UXTH cccc 0110 1111 1111 xxxx xxxx 0111 xxxx */
DECODE_EMULATEX (0x0f8f00f0, 0x068f0070, emulate_rd12rm0_noflags_nopc,
REGS(0, NOPC, 0, 0, NOPC)),
/* SXTAB16 cccc 0110 1000 xxxx xxxx xxxx 0111 xxxx */
/* SXTAB cccc 0110 1010 xxxx xxxx xxxx 0111 xxxx */
/* SXTAH cccc 0110 1011 xxxx xxxx xxxx 0111 xxxx */
/* UXTAB16 cccc 0110 1100 xxxx xxxx xxxx 0111 xxxx */
/* UXTAB cccc 0110 1110 xxxx xxxx xxxx 0111 xxxx */
/* UXTAH cccc 0110 1111 xxxx xxxx xxxx 0111 xxxx */
DECODE_EMULATEX (0x0f8000f0, 0x06800070, emulate_rd12rn16rm0_rwflags_nopc,
REGS(NOPCX, NOPC, 0, 0, NOPC)),
DECODE_END
};
static const union decode_item arm_cccc_0111_____xxx1_table[] = {
/* Media instructions */
/* UNDEFINED cccc 0111 1111 xxxx xxxx xxxx 1111 xxxx */
DECODE_REJECT (0x0ff000f0, 0x07f000f0),
/* SMLALD cccc 0111 0100 xxxx xxxx xxxx 00x1 xxxx */
/* SMLSLD cccc 0111 0100 xxxx xxxx xxxx 01x1 xxxx */
DECODE_EMULATEX (0x0ff00090, 0x07400010, emulate_rdlo12rdhi16rn0rm8_rwflags_nopc,
REGS(NOPC, NOPC, NOPC, 0, NOPC)),
/* SMUAD cccc 0111 0000 xxxx 1111 xxxx 00x1 xxxx */
/* SMUSD cccc 0111 0000 xxxx 1111 xxxx 01x1 xxxx */
DECODE_OR (0x0ff0f090, 0x0700f010),
/* SMMUL cccc 0111 0101 xxxx 1111 xxxx 00x1 xxxx */
DECODE_OR (0x0ff0f0d0, 0x0750f010),
/* USAD8 cccc 0111 1000 xxxx 1111 xxxx 0001 xxxx */
DECODE_EMULATEX (0x0ff0f0f0, 0x0780f010, emulate_rd16rn12rm0rs8_rwflags_nopc,
REGS(NOPC, 0, NOPC, 0, NOPC)),
/* SMLAD cccc 0111 0000 xxxx xxxx xxxx 00x1 xxxx */
/* SMLSD cccc 0111 0000 xxxx xxxx xxxx 01x1 xxxx */
DECODE_OR (0x0ff00090, 0x07000010),
/* SMMLA cccc 0111 0101 xxxx xxxx xxxx 00x1 xxxx */
DECODE_OR (0x0ff000d0, 0x07500010),
/* USADA8 cccc 0111 1000 xxxx xxxx xxxx 0001 xxxx */
DECODE_EMULATEX (0x0ff000f0, 0x07800010, emulate_rd16rn12rm0rs8_rwflags_nopc,
REGS(NOPC, NOPCX, NOPC, 0, NOPC)),
/* SMMLS cccc 0111 0101 xxxx xxxx xxxx 11x1 xxxx */
DECODE_EMULATEX (0x0ff000d0, 0x075000d0, emulate_rd16rn12rm0rs8_rwflags_nopc,
REGS(NOPC, NOPC, NOPC, 0, NOPC)),
/* SBFX cccc 0111 101x xxxx xxxx xxxx x101 xxxx */
/* UBFX cccc 0111 111x xxxx xxxx xxxx x101 xxxx */
DECODE_EMULATEX (0x0fa00070, 0x07a00050, emulate_rd12rm0_noflags_nopc,
REGS(0, NOPC, 0, 0, NOPC)),
/* BFC cccc 0111 110x xxxx xxxx xxxx x001 1111 */
DECODE_EMULATEX (0x0fe0007f, 0x07c0001f, emulate_rd12rm0_noflags_nopc,
REGS(0, NOPC, 0, 0, 0)),
/* BFI cccc 0111 110x xxxx xxxx xxxx x001 xxxx */
DECODE_EMULATEX (0x0fe00070, 0x07c00010, emulate_rd12rm0_noflags_nopc,
REGS(0, NOPC, 0, 0, NOPCX)),
DECODE_END
};
static const union decode_item arm_cccc_01xx_table[] = {
/* Load/store word and unsigned byte */
/* LDRB/STRB pc,[...] cccc 01xx x0xx xxxx xxxx xxxx xxxx xxxx */
DECODE_REJECT (0x0c40f000, 0x0440f000),
/* STRT cccc 01x0 x010 xxxx xxxx xxxx xxxx xxxx */
/* LDRT cccc 01x0 x011 xxxx xxxx xxxx xxxx xxxx */
/* STRBT cccc 01x0 x110 xxxx xxxx xxxx xxxx xxxx */
/* LDRBT cccc 01x0 x111 xxxx xxxx xxxx xxxx xxxx */
DECODE_REJECT (0x0d200000, 0x04200000),
/* STR (immediate) cccc 010x x0x0 xxxx xxxx xxxx xxxx xxxx */
/* STRB (immediate) cccc 010x x1x0 xxxx xxxx xxxx xxxx xxxx */
DECODE_EMULATEX (0x0e100000, 0x04000000, emulate_str,
REGS(NOPCWB, ANY, 0, 0, 0)),
/* LDR (immediate) cccc 010x x0x1 xxxx xxxx xxxx xxxx xxxx */
/* LDRB (immediate) cccc 010x x1x1 xxxx xxxx xxxx xxxx xxxx */
DECODE_EMULATEX (0x0e100000, 0x04100000, emulate_ldr,
REGS(NOPCWB, ANY, 0, 0, 0)),
/* STR (register) cccc 011x x0x0 xxxx xxxx xxxx xxxx xxxx */
/* STRB (register) cccc 011x x1x0 xxxx xxxx xxxx xxxx xxxx */
DECODE_EMULATEX (0x0e100000, 0x06000000, emulate_str,
REGS(NOPCWB, ANY, 0, 0, NOPC)),
/* LDR (register) cccc 011x x0x1 xxxx xxxx xxxx xxxx xxxx */
/* LDRB (register) cccc 011x x1x1 xxxx xxxx xxxx xxxx xxxx */
DECODE_EMULATEX (0x0e100000, 0x06100000, emulate_ldr,
REGS(NOPCWB, ANY, 0, 0, NOPC)),
DECODE_END
};
static const union decode_item arm_cccc_100x_table[] = {
/* Block data transfer instructions */
/* LDM cccc 100x x0x1 xxxx xxxx xxxx xxxx xxxx */
/* STM cccc 100x x0x0 xxxx xxxx xxxx xxxx xxxx */
DECODE_CUSTOM (0x0e400000, 0x08000000, kprobe_decode_ldmstm),
/* STM (user registers) cccc 100x x1x0 xxxx xxxx xxxx xxxx xxxx */
/* LDM (user registers) cccc 100x x1x1 xxxx 0xxx xxxx xxxx xxxx */
/* LDM (exception ret) cccc 100x x1x1 xxxx 1xxx xxxx xxxx xxxx */
DECODE_END
};
const union decode_item kprobe_decode_arm_table[] = {
/*
* Unconditional instructions
* 1111 xxxx xxxx xxxx xxxx xxxx xxxx xxxx
*/
DECODE_TABLE (0xf0000000, 0xf0000000, arm_1111_table),
/*
* Miscellaneous instructions
* cccc 0001 0xx0 xxxx xxxx xxxx 0xxx xxxx
*/
DECODE_TABLE (0x0f900080, 0x01000000, arm_cccc_0001_0xx0____0xxx_table),
/*
* Halfword multiply and multiply-accumulate
* cccc 0001 0xx0 xxxx xxxx xxxx 1xx0 xxxx
*/
DECODE_TABLE (0x0f900090, 0x01000080, arm_cccc_0001_0xx0____1xx0_table),
/*
* Multiply and multiply-accumulate
* cccc 0000 xxxx xxxx xxxx xxxx 1001 xxxx
*/
DECODE_TABLE (0x0f0000f0, 0x00000090, arm_cccc_0000_____1001_table),
/*
* Synchronization primitives
* cccc 0001 xxxx xxxx xxxx xxxx 1001 xxxx
*/
DECODE_TABLE (0x0f0000f0, 0x01000090, arm_cccc_0001_____1001_table),
/*
* Extra load/store instructions
* cccc 000x xxxx xxxx xxxx xxxx 1xx1 xxxx
*/
DECODE_TABLE (0x0e000090, 0x00000090, arm_cccc_000x_____1xx1_table),
/*
* Data-processing (register)
* cccc 000x xxxx xxxx xxxx xxxx xxx0 xxxx
* Data-processing (register-shifted register)
* cccc 000x xxxx xxxx xxxx xxxx 0xx1 xxxx
*/
DECODE_TABLE (0x0e000000, 0x00000000, arm_cccc_000x_table),
/*
* Data-processing (immediate)
* cccc 001x xxxx xxxx xxxx xxxx xxxx xxxx
*/
DECODE_TABLE (0x0e000000, 0x02000000, arm_cccc_001x_table),
/*
* Media instructions
* cccc 011x xxxx xxxx xxxx xxxx xxx1 xxxx
*/
DECODE_TABLE (0x0f000010, 0x06000010, arm_cccc_0110_____xxx1_table),
DECODE_TABLE (0x0f000010, 0x07000010, arm_cccc_0111_____xxx1_table),
/*
* Load/store word and unsigned byte
* cccc 01xx xxxx xxxx xxxx xxxx xxxx xxxx
*/
DECODE_TABLE (0x0c000000, 0x04000000, arm_cccc_01xx_table),
/*
* Block data transfer instructions
* cccc 100x xxxx xxxx xxxx xxxx xxxx xxxx
*/
DECODE_TABLE (0x0e000000, 0x08000000, arm_cccc_100x_table),
/* B cccc 1010 xxxx xxxx xxxx xxxx xxxx xxxx */
/* BL cccc 1011 xxxx xxxx xxxx xxxx xxxx xxxx */
DECODE_SIMULATE (0x0e000000, 0x0a000000, simulate_bbl),
/*
* Supervisor Call, and coprocessor instructions
*/
/* MCRR cccc 1100 0100 xxxx xxxx xxxx xxxx xxxx */
/* MRRC cccc 1100 0101 xxxx xxxx xxxx xxxx xxxx */
/* LDC cccc 110x xxx1 xxxx xxxx xxxx xxxx xxxx */
/* STC cccc 110x xxx0 xxxx xxxx xxxx xxxx xxxx */
/* CDP cccc 1110 xxxx xxxx xxxx xxxx xxx0 xxxx */
/* MCR cccc 1110 xxx0 xxxx xxxx xxxx xxx1 xxxx */
/* MRC cccc 1110 xxx1 xxxx xxxx xxxx xxx1 xxxx */
/* SVC cccc 1111 xxxx xxxx xxxx xxxx xxxx xxxx */
DECODE_REJECT (0x0c000000, 0x0c000000),
DECODE_END
};
static void __kprobes arm_singlestep(struct kprobe *p, struct pt_regs *regs)
{
regs->ARM_pc += 4;
p->ainsn.insn_handler(p, regs);
}
/* Return:
* INSN_REJECTED If instruction is one not allowed to kprobe,
* INSN_GOOD If instruction is supported and uses instruction slot,
* INSN_GOOD_NO_SLOT If instruction is supported but doesn't use its slot.
*
* For instructions we don't want to kprobe (INSN_REJECTED return result):
* These are generally ones that modify the processor state making
* them "hard" to simulate such as switches processor modes or
* make accesses in alternate modes. Any of these could be simulated
* if the work was put into it, but low return considering they
* should also be very rare.
*/
enum kprobe_insn __kprobes
arm_kprobe_decode_insn(kprobe_opcode_t insn, struct arch_specific_insn *asi)
{
asi->insn_singlestep = arm_singlestep;
asi->insn_check_cc = kprobe_condition_checks[insn>>28];
return kprobe_decode_insn(insn, asi, kprobe_decode_arm_table, false);
}

View File

@ -0,0 +1,577 @@
/*
* arch/arm/kernel/kprobes-common.c
*
* Copyright (C) 2011 Jon Medhurst <tixy@yxit.co.uk>.
*
* Some contents moved here from arch/arm/include/asm/kprobes-arm.c which is
* Copyright (C) 2006, 2007 Motorola Inc.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*/
#include <linux/kernel.h>
#include <linux/kprobes.h>
#include "kprobes.h"
#ifndef find_str_pc_offset
/*
* For STR and STM instructions, an ARM core may choose to use either
* a +8 or a +12 displacement from the current instruction's address.
* Whichever value is chosen for a given core, it must be the same for
* both instructions and may not change. This function measures it.
*/
int str_pc_offset;
void __init find_str_pc_offset(void)
{
int addr, scratch, ret;
__asm__ (
"sub %[ret], pc, #4 \n\t"
"str pc, %[addr] \n\t"
"ldr %[scr], %[addr] \n\t"
"sub %[ret], %[scr], %[ret] \n\t"
: [ret] "=r" (ret), [scr] "=r" (scratch), [addr] "+m" (addr));
str_pc_offset = ret;
}
#endif /* !find_str_pc_offset */
#ifndef test_load_write_pc_interworking
bool load_write_pc_interworks;
void __init test_load_write_pc_interworking(void)
{
int arch = cpu_architecture();
BUG_ON(arch == CPU_ARCH_UNKNOWN);
load_write_pc_interworks = arch >= CPU_ARCH_ARMv5T;
}
#endif /* !test_load_write_pc_interworking */
#ifndef test_alu_write_pc_interworking
bool alu_write_pc_interworks;
void __init test_alu_write_pc_interworking(void)
{
int arch = cpu_architecture();
BUG_ON(arch == CPU_ARCH_UNKNOWN);
alu_write_pc_interworks = arch >= CPU_ARCH_ARMv7;
}
#endif /* !test_alu_write_pc_interworking */
void __init arm_kprobe_decode_init(void)
{
find_str_pc_offset();
test_load_write_pc_interworking();
test_alu_write_pc_interworking();
}
static unsigned long __kprobes __check_eq(unsigned long cpsr)
{
return cpsr & PSR_Z_BIT;
}
static unsigned long __kprobes __check_ne(unsigned long cpsr)
{
return (~cpsr) & PSR_Z_BIT;
}
static unsigned long __kprobes __check_cs(unsigned long cpsr)
{
return cpsr & PSR_C_BIT;
}
static unsigned long __kprobes __check_cc(unsigned long cpsr)
{
return (~cpsr) & PSR_C_BIT;
}
static unsigned long __kprobes __check_mi(unsigned long cpsr)
{
return cpsr & PSR_N_BIT;
}
static unsigned long __kprobes __check_pl(unsigned long cpsr)
{
return (~cpsr) & PSR_N_BIT;
}
static unsigned long __kprobes __check_vs(unsigned long cpsr)
{
return cpsr & PSR_V_BIT;
}
static unsigned long __kprobes __check_vc(unsigned long cpsr)
{
return (~cpsr) & PSR_V_BIT;
}
static unsigned long __kprobes __check_hi(unsigned long cpsr)
{
cpsr &= ~(cpsr >> 1); /* PSR_C_BIT &= ~PSR_Z_BIT */
return cpsr & PSR_C_BIT;
}
static unsigned long __kprobes __check_ls(unsigned long cpsr)
{
cpsr &= ~(cpsr >> 1); /* PSR_C_BIT &= ~PSR_Z_BIT */
return (~cpsr) & PSR_C_BIT;
}
static unsigned long __kprobes __check_ge(unsigned long cpsr)
{
cpsr ^= (cpsr << 3); /* PSR_N_BIT ^= PSR_V_BIT */
return (~cpsr) & PSR_N_BIT;
}
static unsigned long __kprobes __check_lt(unsigned long cpsr)
{
cpsr ^= (cpsr << 3); /* PSR_N_BIT ^= PSR_V_BIT */
return cpsr & PSR_N_BIT;
}
static unsigned long __kprobes __check_gt(unsigned long cpsr)
{
unsigned long temp = cpsr ^ (cpsr << 3); /* PSR_N_BIT ^= PSR_V_BIT */
temp |= (cpsr << 1); /* PSR_N_BIT |= PSR_Z_BIT */
return (~temp) & PSR_N_BIT;
}
static unsigned long __kprobes __check_le(unsigned long cpsr)
{
unsigned long temp = cpsr ^ (cpsr << 3); /* PSR_N_BIT ^= PSR_V_BIT */
temp |= (cpsr << 1); /* PSR_N_BIT |= PSR_Z_BIT */
return temp & PSR_N_BIT;
}
static unsigned long __kprobes __check_al(unsigned long cpsr)
{
return true;
}
kprobe_check_cc * const kprobe_condition_checks[16] = {
&__check_eq, &__check_ne, &__check_cs, &__check_cc,
&__check_mi, &__check_pl, &__check_vs, &__check_vc,
&__check_hi, &__check_ls, &__check_ge, &__check_lt,
&__check_gt, &__check_le, &__check_al, &__check_al
};
void __kprobes kprobe_simulate_nop(struct kprobe *p, struct pt_regs *regs)
{
}
void __kprobes kprobe_emulate_none(struct kprobe *p, struct pt_regs *regs)
{
p->ainsn.insn_fn();
}
static void __kprobes simulate_ldm1stm1(struct kprobe *p, struct pt_regs *regs)
{
kprobe_opcode_t insn = p->opcode;
int rn = (insn >> 16) & 0xf;
int lbit = insn & (1 << 20);
int wbit = insn & (1 << 21);
int ubit = insn & (1 << 23);
int pbit = insn & (1 << 24);
long *addr = (long *)regs->uregs[rn];
int reg_bit_vector;
int reg_count;
reg_count = 0;
reg_bit_vector = insn & 0xffff;
while (reg_bit_vector) {
reg_bit_vector &= (reg_bit_vector - 1);
++reg_count;
}
if (!ubit)
addr -= reg_count;
addr += (!pbit == !ubit);
reg_bit_vector = insn & 0xffff;
while (reg_bit_vector) {
int reg = __ffs(reg_bit_vector);
reg_bit_vector &= (reg_bit_vector - 1);
if (lbit)
regs->uregs[reg] = *addr++;
else
*addr++ = regs->uregs[reg];
}
if (wbit) {
if (!ubit)
addr -= reg_count;
addr -= (!pbit == !ubit);
regs->uregs[rn] = (long)addr;
}
}
static void __kprobes simulate_stm1_pc(struct kprobe *p, struct pt_regs *regs)
{
regs->ARM_pc = (long)p->addr + str_pc_offset;
simulate_ldm1stm1(p, regs);
regs->ARM_pc = (long)p->addr + 4;
}
static void __kprobes simulate_ldm1_pc(struct kprobe *p, struct pt_regs *regs)
{
simulate_ldm1stm1(p, regs);
load_write_pc(regs->ARM_pc, regs);
}
static void __kprobes
emulate_generic_r0_12_noflags(struct kprobe *p, struct pt_regs *regs)
{
register void *rregs asm("r1") = regs;
register void *rfn asm("lr") = p->ainsn.insn_fn;
__asm__ __volatile__ (
"stmdb sp!, {%[regs], r11} \n\t"
"ldmia %[regs], {r0-r12} \n\t"
#if __LINUX_ARM_ARCH__ >= 6
"blx %[fn] \n\t"
#else
"str %[fn], [sp, #-4]! \n\t"
"adr lr, 1f \n\t"
"ldr pc, [sp], #4 \n\t"
"1: \n\t"
#endif
"ldr lr, [sp], #4 \n\t" /* lr = regs */
"stmia lr, {r0-r12} \n\t"
"ldr r11, [sp], #4 \n\t"
: [regs] "=r" (rregs), [fn] "=r" (rfn)
: "0" (rregs), "1" (rfn)
: "r0", "r2", "r3", "r4", "r5", "r6", "r7",
"r8", "r9", "r10", "r12", "memory", "cc"
);
}
static void __kprobes
emulate_generic_r2_14_noflags(struct kprobe *p, struct pt_regs *regs)
{
emulate_generic_r0_12_noflags(p, (struct pt_regs *)(regs->uregs+2));
}
static void __kprobes
emulate_ldm_r3_15(struct kprobe *p, struct pt_regs *regs)
{
emulate_generic_r0_12_noflags(p, (struct pt_regs *)(regs->uregs+3));
load_write_pc(regs->ARM_pc, regs);
}
enum kprobe_insn __kprobes
kprobe_decode_ldmstm(kprobe_opcode_t insn, struct arch_specific_insn *asi)
{
kprobe_insn_handler_t *handler = 0;
unsigned reglist = insn & 0xffff;
int is_ldm = insn & 0x100000;
int rn = (insn >> 16) & 0xf;
if (rn <= 12 && (reglist & 0xe000) == 0) {
/* Instruction only uses registers in the range R0..R12 */
handler = emulate_generic_r0_12_noflags;
} else if (rn >= 2 && (reglist & 0x8003) == 0) {
/* Instruction only uses registers in the range R2..R14 */
rn -= 2;
reglist >>= 2;
handler = emulate_generic_r2_14_noflags;
} else if (rn >= 3 && (reglist & 0x0007) == 0) {
/* Instruction only uses registers in the range R3..R15 */
if (is_ldm && (reglist & 0x8000)) {
rn -= 3;
reglist >>= 3;
handler = emulate_ldm_r3_15;
}
}
if (handler) {
/* We can emulate the instruction in (possibly) modified form */
asi->insn[0] = (insn & 0xfff00000) | (rn << 16) | reglist;
asi->insn_handler = handler;
return INSN_GOOD;
}
/* Fallback to slower simulation... */
if (reglist & 0x8000)
handler = is_ldm ? simulate_ldm1_pc : simulate_stm1_pc;
else
handler = simulate_ldm1stm1;
asi->insn_handler = handler;
return INSN_GOOD_NO_SLOT;
}
/*
* Prepare an instruction slot to receive an instruction for emulating.
* This is done by placing a subroutine return after the location where the
* instruction will be placed. We also modify ARM instructions to be
* unconditional as the condition code will already be checked before any
* emulation handler is called.
*/
static kprobe_opcode_t __kprobes
prepare_emulated_insn(kprobe_opcode_t insn, struct arch_specific_insn *asi,
bool thumb)
{
#ifdef CONFIG_THUMB2_KERNEL
if (thumb) {
u16 *thumb_insn = (u16 *)asi->insn;
thumb_insn[1] = 0x4770; /* Thumb bx lr */
thumb_insn[2] = 0x4770; /* Thumb bx lr */
return insn;
}
asi->insn[1] = 0xe12fff1e; /* ARM bx lr */
#else
asi->insn[1] = 0xe1a0f00e; /* mov pc, lr */
#endif
/* Make an ARM instruction unconditional */
if (insn < 0xe0000000)
insn = (insn | 0xe0000000) & ~0x10000000;
return insn;
}
/*
* Write a (probably modified) instruction into the slot previously prepared by
* prepare_emulated_insn
*/
static void __kprobes
set_emulated_insn(kprobe_opcode_t insn, struct arch_specific_insn *asi,
bool thumb)
{
#ifdef CONFIG_THUMB2_KERNEL
if (thumb) {
u16 *ip = (u16 *)asi->insn;
if (is_wide_instruction(insn))
*ip++ = insn >> 16;
*ip++ = insn;
return;
}
#endif
asi->insn[0] = insn;
}
/*
* When we modify the register numbers encoded in an instruction to be emulated,
* the new values come from this define. For ARM and 32-bit Thumb instructions
* this gives...
*
* bit position 16 12 8 4 0
* ---------------+---+---+---+---+---+
* register r2 r0 r1 -- r3
*/
#define INSN_NEW_BITS 0x00020103
/* Each nibble has same value as that at INSN_NEW_BITS bit 16 */
#define INSN_SAMEAS16_BITS 0x22222222
/*
* Validate and modify each of the registers encoded in an instruction.
*
* Each nibble in regs contains a value from enum decode_reg_type. For each
* non-zero value, the corresponding nibble in pinsn is validated and modified
* according to the type.
*/
static bool __kprobes decode_regs(kprobe_opcode_t* pinsn, u32 regs)
{
kprobe_opcode_t insn = *pinsn;
kprobe_opcode_t mask = 0xf; /* Start at least significant nibble */
for (; regs != 0; regs >>= 4, mask <<= 4) {
kprobe_opcode_t new_bits = INSN_NEW_BITS;
switch (regs & 0xf) {
case REG_TYPE_NONE:
/* Nibble not a register, skip to next */
continue;
case REG_TYPE_ANY:
/* Any register is allowed */
break;
case REG_TYPE_SAMEAS16:
/* Replace register with same as at bit position 16 */
new_bits = INSN_SAMEAS16_BITS;
break;
case REG_TYPE_SP:
/* Only allow SP (R13) */
if ((insn ^ 0xdddddddd) & mask)
goto reject;
break;
case REG_TYPE_PC:
/* Only allow PC (R15) */
if ((insn ^ 0xffffffff) & mask)
goto reject;
break;
case REG_TYPE_NOSP:
/* Reject SP (R13) */
if (((insn ^ 0xdddddddd) & mask) == 0)
goto reject;
break;
case REG_TYPE_NOSPPC:
case REG_TYPE_NOSPPCX:
/* Reject SP and PC (R13 and R15) */
if (((insn ^ 0xdddddddd) & 0xdddddddd & mask) == 0)
goto reject;
break;
case REG_TYPE_NOPCWB:
if (!is_writeback(insn))
break; /* No writeback, so any register is OK */
/* fall through... */
case REG_TYPE_NOPC:
case REG_TYPE_NOPCX:
/* Reject PC (R15) */
if (((insn ^ 0xffffffff) & mask) == 0)
goto reject;
break;
}
/* Replace value of nibble with new register number... */
insn &= ~mask;
insn |= new_bits & mask;
}
*pinsn = insn;
return true;
reject:
return false;
}
static const int decode_struct_sizes[NUM_DECODE_TYPES] = {
[DECODE_TYPE_TABLE] = sizeof(struct decode_table),
[DECODE_TYPE_CUSTOM] = sizeof(struct decode_custom),
[DECODE_TYPE_SIMULATE] = sizeof(struct decode_simulate),
[DECODE_TYPE_EMULATE] = sizeof(struct decode_emulate),
[DECODE_TYPE_OR] = sizeof(struct decode_or),
[DECODE_TYPE_REJECT] = sizeof(struct decode_reject)
};
/*
* kprobe_decode_insn operates on data tables in order to decode an ARM
* architecture instruction onto which a kprobe has been placed.
*
* These instruction decoding tables are a concatenation of entries each
* of which consist of one of the following structs:
*
* decode_table
* decode_custom
* decode_simulate
* decode_emulate
* decode_or
* decode_reject
*
* Each of these starts with a struct decode_header which has the following
* fields:
*
* type_regs
* mask
* value
*
* The least significant DECODE_TYPE_BITS of type_regs contains a value
* from enum decode_type, this indicates which of the decode_* structs
* the entry contains. The value DECODE_TYPE_END indicates the end of the
* table.
*
* When the table is parsed, each entry is checked in turn to see if it
* matches the instruction to be decoded using the test:
*
* (insn & mask) == value
*
* If no match is found before the end of the table is reached then decoding
* fails with INSN_REJECTED.
*
* When a match is found, decode_regs() is called to validate and modify each
* of the registers encoded in the instruction; the data it uses to do this
* is (type_regs >> DECODE_TYPE_BITS). A validation failure will cause decoding
* to fail with INSN_REJECTED.
*
* Once the instruction has passed the above tests, further processing
* depends on the type of the table entry's decode struct.
*
*/
int __kprobes
kprobe_decode_insn(kprobe_opcode_t insn, struct arch_specific_insn *asi,
const union decode_item *table, bool thumb)
{
const struct decode_header *h = (struct decode_header *)table;
const struct decode_header *next;
bool matched = false;
insn = prepare_emulated_insn(insn, asi, thumb);
for (;; h = next) {
enum decode_type type = h->type_regs.bits & DECODE_TYPE_MASK;
u32 regs = h->type_regs.bits >> DECODE_TYPE_BITS;
if (type == DECODE_TYPE_END)
return INSN_REJECTED;
next = (struct decode_header *)
((uintptr_t)h + decode_struct_sizes[type]);
if (!matched && (insn & h->mask.bits) != h->value.bits)
continue;
if (!decode_regs(&insn, regs))
return INSN_REJECTED;
switch (type) {
case DECODE_TYPE_TABLE: {
struct decode_table *d = (struct decode_table *)h;
next = (struct decode_header *)d->table.table;
break;
}
case DECODE_TYPE_CUSTOM: {
struct decode_custom *d = (struct decode_custom *)h;
return (*d->decoder.decoder)(insn, asi);
}
case DECODE_TYPE_SIMULATE: {
struct decode_simulate *d = (struct decode_simulate *)h;
asi->insn_handler = d->handler.handler;
return INSN_GOOD_NO_SLOT;
}
case DECODE_TYPE_EMULATE: {
struct decode_emulate *d = (struct decode_emulate *)h;
asi->insn_handler = d->handler.handler;
set_emulated_insn(insn, asi, thumb);
return INSN_GOOD;
}
case DECODE_TYPE_OR:
matched = true;
break;
case DECODE_TYPE_REJECT:
default:
return INSN_REJECTED;
}
}
}

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -28,14 +28,16 @@
#include <asm/traps.h>
#include <asm/cacheflush.h>
#include "kprobes.h"
#define MIN_STACK_SIZE(addr) \
min((unsigned long)MAX_STACK_SIZE, \
(unsigned long)current_thread_info() + THREAD_START_SP - (addr))
#define flush_insns(addr, cnt) \
#define flush_insns(addr, size) \
flush_icache_range((unsigned long)(addr), \
(unsigned long)(addr) + \
sizeof(kprobe_opcode_t) * (cnt))
(size))
/* Used as a marker in ARM_pc to note when we're in a jprobe. */
#define JPROBE_MAGIC_ADDR 0xffffffff
@ -49,16 +51,35 @@ int __kprobes arch_prepare_kprobe(struct kprobe *p)
kprobe_opcode_t insn;
kprobe_opcode_t tmp_insn[MAX_INSN_SIZE];
unsigned long addr = (unsigned long)p->addr;
bool thumb;
kprobe_decode_insn_t *decode_insn;
int is;
if (addr & 0x3 || in_exception_text(addr))
if (in_exception_text(addr))
return -EINVAL;
#ifdef CONFIG_THUMB2_KERNEL
thumb = true;
addr &= ~1; /* Bit 0 would normally be set to indicate Thumb code */
insn = ((u16 *)addr)[0];
if (is_wide_instruction(insn)) {
insn <<= 16;
insn |= ((u16 *)addr)[1];
decode_insn = thumb32_kprobe_decode_insn;
} else
decode_insn = thumb16_kprobe_decode_insn;
#else /* !CONFIG_THUMB2_KERNEL */
thumb = false;
if (addr & 0x3)
return -EINVAL;
insn = *p->addr;
decode_insn = arm_kprobe_decode_insn;
#endif
p->opcode = insn;
p->ainsn.insn = tmp_insn;
switch (arm_kprobe_decode_insn(insn, &p->ainsn)) {
switch ((*decode_insn)(insn, &p->ainsn)) {
case INSN_REJECTED: /* not supported */
return -EINVAL;
@ -68,7 +89,10 @@ int __kprobes arch_prepare_kprobe(struct kprobe *p)
return -ENOMEM;
for (is = 0; is < MAX_INSN_SIZE; ++is)
p->ainsn.insn[is] = tmp_insn[is];
flush_insns(p->ainsn.insn, MAX_INSN_SIZE);
flush_insns(p->ainsn.insn,
sizeof(p->ainsn.insn[0]) * MAX_INSN_SIZE);
p->ainsn.insn_fn = (kprobe_insn_fn_t *)
((uintptr_t)p->ainsn.insn | thumb);
break;
case INSN_GOOD_NO_SLOT: /* instruction doesn't need insn slot */
@ -79,24 +103,88 @@ int __kprobes arch_prepare_kprobe(struct kprobe *p)
return 0;
}
#ifdef CONFIG_THUMB2_KERNEL
/*
* For a 32-bit Thumb breakpoint spanning two memory words we need to take
* special precautions to insert the breakpoint atomically, especially on SMP
* systems. This is achieved by calling this arming function using stop_machine.
*/
static int __kprobes set_t32_breakpoint(void *addr)
{
((u16 *)addr)[0] = KPROBE_THUMB32_BREAKPOINT_INSTRUCTION >> 16;
((u16 *)addr)[1] = KPROBE_THUMB32_BREAKPOINT_INSTRUCTION & 0xffff;
flush_insns(addr, 2*sizeof(u16));
return 0;
}
void __kprobes arch_arm_kprobe(struct kprobe *p)
{
*p->addr = KPROBE_BREAKPOINT_INSTRUCTION;
flush_insns(p->addr, 1);
uintptr_t addr = (uintptr_t)p->addr & ~1; /* Remove any Thumb flag */
if (!is_wide_instruction(p->opcode)) {
*(u16 *)addr = KPROBE_THUMB16_BREAKPOINT_INSTRUCTION;
flush_insns(addr, sizeof(u16));
} else if (addr & 2) {
/* A 32-bit instruction spanning two words needs special care */
stop_machine(set_t32_breakpoint, (void *)addr, &cpu_online_map);
} else {
/* Word aligned 32-bit instruction can be written atomically */
u32 bkp = KPROBE_THUMB32_BREAKPOINT_INSTRUCTION;
#ifndef __ARMEB__ /* Swap halfwords for little-endian */
bkp = (bkp >> 16) | (bkp << 16);
#endif
*(u32 *)addr = bkp;
flush_insns(addr, sizeof(u32));
}
}
#else /* !CONFIG_THUMB2_KERNEL */
void __kprobes arch_arm_kprobe(struct kprobe *p)
{
kprobe_opcode_t insn = p->opcode;
kprobe_opcode_t brkp = KPROBE_ARM_BREAKPOINT_INSTRUCTION;
if (insn >= 0xe0000000)
brkp |= 0xe0000000; /* Unconditional instruction */
else
brkp |= insn & 0xf0000000; /* Copy condition from insn */
*p->addr = brkp;
flush_insns(p->addr, sizeof(p->addr[0]));
}
#endif /* !CONFIG_THUMB2_KERNEL */
/*
* The actual disarming is done here on each CPU and synchronized using
* stop_machine. This synchronization is necessary on SMP to avoid removing
* a probe between the moment the 'Undefined Instruction' exception is raised
* and the moment the exception handler reads the faulting instruction from
* memory.
* memory. It is also needed to atomically set the two half-words of a 32-bit
* Thumb breakpoint.
*/
int __kprobes __arch_disarm_kprobe(void *p)
{
struct kprobe *kp = p;
#ifdef CONFIG_THUMB2_KERNEL
u16 *addr = (u16 *)((uintptr_t)kp->addr & ~1);
kprobe_opcode_t insn = kp->opcode;
unsigned int len;
if (is_wide_instruction(insn)) {
((u16 *)addr)[0] = insn>>16;
((u16 *)addr)[1] = insn;
len = 2*sizeof(u16);
} else {
((u16 *)addr)[0] = insn;
len = sizeof(u16);
}
flush_insns(addr, len);
#else /* !CONFIG_THUMB2_KERNEL */
*kp->addr = kp->opcode;
flush_insns(kp->addr, 1);
flush_insns(kp->addr, sizeof(kp->addr[0]));
#endif
return 0;
}
@ -130,12 +218,24 @@ static void __kprobes set_current_kprobe(struct kprobe *p)
__get_cpu_var(current_kprobe) = p;
}
static void __kprobes singlestep(struct kprobe *p, struct pt_regs *regs,
struct kprobe_ctlblk *kcb)
static void __kprobes
singlestep_skip(struct kprobe *p, struct pt_regs *regs)
{
#ifdef CONFIG_THUMB2_KERNEL
regs->ARM_cpsr = it_advance(regs->ARM_cpsr);
if (is_wide_instruction(p->opcode))
regs->ARM_pc += 4;
else
regs->ARM_pc += 2;
#else
regs->ARM_pc += 4;
if (p->ainsn.insn_check_cc(regs->ARM_cpsr))
p->ainsn.insn_handler(p, regs);
#endif
}
static inline void __kprobes
singlestep(struct kprobe *p, struct pt_regs *regs, struct kprobe_ctlblk *kcb)
{
p->ainsn.insn_singlestep(p, regs);
}
/*
@ -149,11 +249,23 @@ void __kprobes kprobe_handler(struct pt_regs *regs)
{
struct kprobe *p, *cur;
struct kprobe_ctlblk *kcb;
kprobe_opcode_t *addr = (kprobe_opcode_t *)regs->ARM_pc;
kcb = get_kprobe_ctlblk();
cur = kprobe_running();
p = get_kprobe(addr);
#ifdef CONFIG_THUMB2_KERNEL
/*
* First look for a probe which was registered using an address with
* bit 0 set, this is the usual situation for pointers to Thumb code.
* If not found, fallback to looking for one with bit 0 clear.
*/
p = get_kprobe((kprobe_opcode_t *)(regs->ARM_pc | 1));
if (!p)
p = get_kprobe((kprobe_opcode_t *)regs->ARM_pc);
#else /* ! CONFIG_THUMB2_KERNEL */
p = get_kprobe((kprobe_opcode_t *)regs->ARM_pc);
#endif
if (p) {
if (cur) {
@ -173,7 +285,8 @@ void __kprobes kprobe_handler(struct pt_regs *regs)
/* impossible cases */
BUG();
}
} else {
} else if (p->ainsn.insn_check_cc(regs->ARM_cpsr)) {
/* Probe hit and conditional execution check ok. */
set_current_kprobe(p);
kcb->kprobe_status = KPROBE_HIT_ACTIVE;
@ -193,6 +306,13 @@ void __kprobes kprobe_handler(struct pt_regs *regs)
}
reset_current_kprobe();
}
} else {
/*
* Probe hit but conditional execution check failed,
* so just skip the instruction and continue as if
* nothing had happened.
*/
singlestep_skip(p, regs);
}
} else if (cur) {
/* We probably hit a jprobe. Call its break handler. */
@ -300,7 +420,11 @@ void __naked __kprobes kretprobe_trampoline(void)
"bl trampoline_handler \n\t"
"mov lr, r0 \n\t"
"ldmia sp!, {r0 - r11} \n\t"
#ifdef CONFIG_THUMB2_KERNEL
"bx lr \n\t"
#else
"mov pc, lr \n\t"
#endif
: : : "memory");
}
@ -378,11 +502,22 @@ int __kprobes setjmp_pre_handler(struct kprobe *p, struct pt_regs *regs)
struct jprobe *jp = container_of(p, struct jprobe, kp);
struct kprobe_ctlblk *kcb = get_kprobe_ctlblk();
long sp_addr = regs->ARM_sp;
long cpsr;
kcb->jprobe_saved_regs = *regs;
memcpy(kcb->jprobes_stack, (void *)sp_addr, MIN_STACK_SIZE(sp_addr));
regs->ARM_pc = (long)jp->entry;
regs->ARM_cpsr |= PSR_I_BIT;
cpsr = regs->ARM_cpsr | PSR_I_BIT;
#ifdef CONFIG_THUMB2_KERNEL
/* Set correct Thumb state in cpsr */
if (regs->ARM_pc & 1)
cpsr |= PSR_T_BIT;
else
cpsr &= ~PSR_T_BIT;
#endif
regs->ARM_cpsr = cpsr;
preempt_disable();
return 1;
}
@ -404,7 +539,12 @@ void __kprobes jprobe_return(void)
* This is to prevent any simulated instruction from writing
* over the regs when they are accessing the stack.
*/
#ifdef CONFIG_THUMB2_KERNEL
"sub r0, %0, %1 \n\t"
"mov sp, r0 \n\t"
#else
"sub sp, %0, %1 \n\t"
#endif
"ldr r0, ="__stringify(JPROBE_MAGIC_ADDR)"\n\t"
"str %0, [sp, %2] \n\t"
"str r0, [sp, %3] \n\t"
@ -415,15 +555,28 @@ void __kprobes jprobe_return(void)
* Return to the context saved by setjmp_pre_handler
* and restored by longjmp_break_handler.
*/
#ifdef CONFIG_THUMB2_KERNEL
"ldr lr, [sp, %2] \n\t" /* lr = saved sp */
"ldrd r0, r1, [sp, %5] \n\t" /* r0,r1 = saved lr,pc */
"ldr r2, [sp, %4] \n\t" /* r2 = saved psr */
"stmdb lr!, {r0, r1, r2} \n\t" /* push saved lr and */
/* rfe context */
"ldmia sp, {r0 - r12} \n\t"
"mov sp, lr \n\t"
"ldr lr, [sp], #4 \n\t"
"rfeia sp! \n\t"
#else
"ldr r0, [sp, %4] \n\t"
"msr cpsr_cxsf, r0 \n\t"
"ldmia sp, {r0 - pc} \n\t"
#endif
:
: "r" (kcb->jprobe_saved_regs.ARM_sp),
"I" (sizeof(struct pt_regs) * 2),
"J" (offsetof(struct pt_regs, ARM_sp)),
"J" (offsetof(struct pt_regs, ARM_pc)),
"J" (offsetof(struct pt_regs, ARM_cpsr))
"J" (offsetof(struct pt_regs, ARM_cpsr)),
"J" (offsetof(struct pt_regs, ARM_lr))
: "memory", "cc");
}
@ -460,17 +613,44 @@ int __kprobes arch_trampoline_kprobe(struct kprobe *p)
return 0;
}
static struct undef_hook kprobes_break_hook = {
.instr_mask = 0xffffffff,
.instr_val = KPROBE_BREAKPOINT_INSTRUCTION,
#ifdef CONFIG_THUMB2_KERNEL
static struct undef_hook kprobes_thumb16_break_hook = {
.instr_mask = 0xffff,
.instr_val = KPROBE_THUMB16_BREAKPOINT_INSTRUCTION,
.cpsr_mask = MODE_MASK,
.cpsr_val = SVC_MODE,
.fn = kprobe_trap_handler,
};
static struct undef_hook kprobes_thumb32_break_hook = {
.instr_mask = 0xffffffff,
.instr_val = KPROBE_THUMB32_BREAKPOINT_INSTRUCTION,
.cpsr_mask = MODE_MASK,
.cpsr_val = SVC_MODE,
.fn = kprobe_trap_handler,
};
#else /* !CONFIG_THUMB2_KERNEL */
static struct undef_hook kprobes_arm_break_hook = {
.instr_mask = 0x0fffffff,
.instr_val = KPROBE_ARM_BREAKPOINT_INSTRUCTION,
.cpsr_mask = MODE_MASK,
.cpsr_val = SVC_MODE,
.fn = kprobe_trap_handler,
};
#endif /* !CONFIG_THUMB2_KERNEL */
int __init arch_init_kprobes()
{
arm_kprobe_decode_init();
register_undef_hook(&kprobes_break_hook);
#ifdef CONFIG_THUMB2_KERNEL
register_undef_hook(&kprobes_thumb16_break_hook);
register_undef_hook(&kprobes_thumb32_break_hook);
#else
register_undef_hook(&kprobes_arm_break_hook);
#endif
return 0;
}

420
arch/arm/kernel/kprobes.h Normal file
View File

@ -0,0 +1,420 @@
/*
* arch/arm/kernel/kprobes.h
*
* Copyright (C) 2011 Jon Medhurst <tixy@yxit.co.uk>.
*
* Some contents moved here from arch/arm/include/asm/kprobes.h which is
* Copyright (C) 2006, 2007 Motorola Inc.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* General Public License for more details.
*/
#ifndef _ARM_KERNEL_KPROBES_H
#define _ARM_KERNEL_KPROBES_H
/*
* These undefined instructions must be unique and
* reserved solely for kprobes' use.
*/
#define KPROBE_ARM_BREAKPOINT_INSTRUCTION 0x07f001f8
#define KPROBE_THUMB16_BREAKPOINT_INSTRUCTION 0xde18
#define KPROBE_THUMB32_BREAKPOINT_INSTRUCTION 0xf7f0a018
enum kprobe_insn {
INSN_REJECTED,
INSN_GOOD,
INSN_GOOD_NO_SLOT
};
typedef enum kprobe_insn (kprobe_decode_insn_t)(kprobe_opcode_t,
struct arch_specific_insn *);
#ifdef CONFIG_THUMB2_KERNEL
enum kprobe_insn thumb16_kprobe_decode_insn(kprobe_opcode_t,
struct arch_specific_insn *);
enum kprobe_insn thumb32_kprobe_decode_insn(kprobe_opcode_t,
struct arch_specific_insn *);
#else /* !CONFIG_THUMB2_KERNEL */
enum kprobe_insn arm_kprobe_decode_insn(kprobe_opcode_t,
struct arch_specific_insn *);
#endif
void __init arm_kprobe_decode_init(void);
extern kprobe_check_cc * const kprobe_condition_checks[16];
#if __LINUX_ARM_ARCH__ >= 7
/* str_pc_offset is architecturally defined from ARMv7 onwards */
#define str_pc_offset 8
#define find_str_pc_offset()
#else /* __LINUX_ARM_ARCH__ < 7 */
/* We need a run-time check to determine str_pc_offset */
extern int str_pc_offset;
void __init find_str_pc_offset(void);
#endif
/*
* Update ITSTATE after normal execution of an IT block instruction.
*
* The 8 IT state bits are split into two parts in CPSR:
* ITSTATE<1:0> are in CPSR<26:25>
* ITSTATE<7:2> are in CPSR<15:10>
*/
static inline unsigned long it_advance(unsigned long cpsr)
{
if ((cpsr & 0x06000400) == 0) {
/* ITSTATE<2:0> == 0 means end of IT block, so clear IT state */
cpsr &= ~PSR_IT_MASK;
} else {
/* We need to shift left ITSTATE<4:0> */
const unsigned long mask = 0x06001c00; /* Mask ITSTATE<4:0> */
unsigned long it = cpsr & mask;
it <<= 1;
it |= it >> (27 - 10); /* Carry ITSTATE<2> to correct place */
it &= mask;
cpsr &= ~mask;
cpsr |= it;
}
return cpsr;
}
static inline void __kprobes bx_write_pc(long pcv, struct pt_regs *regs)
{
long cpsr = regs->ARM_cpsr;
if (pcv & 0x1) {
cpsr |= PSR_T_BIT;
pcv &= ~0x1;
} else {
cpsr &= ~PSR_T_BIT;
pcv &= ~0x2; /* Avoid UNPREDICTABLE address allignment */
}
regs->ARM_cpsr = cpsr;
regs->ARM_pc = pcv;
}
#if __LINUX_ARM_ARCH__ >= 6
/* Kernels built for >= ARMv6 should never run on <= ARMv5 hardware, so... */
#define load_write_pc_interworks true
#define test_load_write_pc_interworking()
#else /* __LINUX_ARM_ARCH__ < 6 */
/* We need run-time testing to determine if load_write_pc() should interwork. */
extern bool load_write_pc_interworks;
void __init test_load_write_pc_interworking(void);
#endif
static inline void __kprobes load_write_pc(long pcv, struct pt_regs *regs)
{
if (load_write_pc_interworks)
bx_write_pc(pcv, regs);
else
regs->ARM_pc = pcv;
}
#if __LINUX_ARM_ARCH__ >= 7
#define alu_write_pc_interworks true
#define test_alu_write_pc_interworking()
#elif __LINUX_ARM_ARCH__ <= 5
/* Kernels built for <= ARMv5 should never run on >= ARMv6 hardware, so... */
#define alu_write_pc_interworks false
#define test_alu_write_pc_interworking()
#else /* __LINUX_ARM_ARCH__ == 6 */
/* We could be an ARMv6 binary on ARMv7 hardware so we need a run-time check. */
extern bool alu_write_pc_interworks;
void __init test_alu_write_pc_interworking(void);
#endif /* __LINUX_ARM_ARCH__ == 6 */
static inline void __kprobes alu_write_pc(long pcv, struct pt_regs *regs)
{
if (alu_write_pc_interworks)
bx_write_pc(pcv, regs);
else
regs->ARM_pc = pcv;
}
void __kprobes kprobe_simulate_nop(struct kprobe *p, struct pt_regs *regs);
void __kprobes kprobe_emulate_none(struct kprobe *p, struct pt_regs *regs);
enum kprobe_insn __kprobes
kprobe_decode_ldmstm(kprobe_opcode_t insn, struct arch_specific_insn *asi);
/*
* Test if load/store instructions writeback the address register.
* if P (bit 24) == 0 or W (bit 21) == 1
*/
#define is_writeback(insn) ((insn ^ 0x01000000) & 0x01200000)
/*
* The following definitions and macros are used to build instruction
* decoding tables for use by kprobe_decode_insn.
*
* These tables are a concatenation of entries each of which consist of one of
* the decode_* structs. All of the fields in every type of decode structure
* are of the union type decode_item, therefore the entire decode table can be
* viewed as an array of these and declared like:
*
* static const union decode_item table_name[] = {};
*
* In order to construct each entry in the table, macros are used to
* initialise a number of sequential decode_item values in a layout which
* matches the relevant struct. E.g. DECODE_SIMULATE initialise a struct
* decode_simulate by initialising four decode_item objects like this...
*
* {.bits = _type},
* {.bits = _mask},
* {.bits = _value},
* {.handler = _handler},
*
* Initialising a specified member of the union means that the compiler
* will produce a warning if the argument is of an incorrect type.
*
* Below is a list of each of the macros used to initialise entries and a
* description of the action performed when that entry is matched to an
* instruction. A match is found when (instruction & mask) == value.
*
* DECODE_TABLE(mask, value, table)
* Instruction decoding jumps to parsing the new sub-table 'table'.
*
* DECODE_CUSTOM(mask, value, decoder)
* The custom function 'decoder' is called to the complete decoding
* of an instruction.
*
* DECODE_SIMULATE(mask, value, handler)
* Set the probes instruction handler to 'handler', this will be used
* to simulate the instruction when the probe is hit. Decoding returns
* with INSN_GOOD_NO_SLOT.
*
* DECODE_EMULATE(mask, value, handler)
* Set the probes instruction handler to 'handler', this will be used
* to emulate the instruction when the probe is hit. The modified
* instruction (see below) is placed in the probes instruction slot so it
* may be called by the emulation code. Decoding returns with INSN_GOOD.
*
* DECODE_REJECT(mask, value)
* Instruction decoding fails with INSN_REJECTED
*
* DECODE_OR(mask, value)
* This allows the mask/value test of multiple table entries to be
* logically ORed. Once an 'or' entry is matched the decoding action to
* be performed is that of the next entry which isn't an 'or'. E.g.
*
* DECODE_OR (mask1, value1)
* DECODE_OR (mask2, value2)
* DECODE_SIMULATE (mask3, value3, simulation_handler)
*
* This means that if any of the three mask/value pairs match the
* instruction being decoded, then 'simulation_handler' will be used
* for it.
*
* Both the SIMULATE and EMULATE macros have a second form which take an
* additional 'regs' argument.
*
* DECODE_SIMULATEX(mask, value, handler, regs)
* DECODE_EMULATEX (mask, value, handler, regs)
*
* These are used to specify what kind of CPU register is encoded in each of the
* least significant 5 nibbles of the instruction being decoded. The regs value
* is specified using the REGS macro, this takes any of the REG_TYPE_* values
* from enum decode_reg_type as arguments; only the '*' part of the name is
* given. E.g.
*
* REGS(0, ANY, NOPC, 0, ANY)
*
* This indicates an instruction is encoded like:
*
* bits 19..16 ignore
* bits 15..12 any register allowed here
* bits 11.. 8 any register except PC allowed here
* bits 7.. 4 ignore
* bits 3.. 0 any register allowed here
*
* This register specification is checked after a decode table entry is found to
* match an instruction (through the mask/value test). Any invalid register then
* found in the instruction will cause decoding to fail with INSN_REJECTED. In
* the above example this would happen if bits 11..8 of the instruction were
* 1111, indicating R15 or PC.
*
* As well as checking for legal combinations of registers, this data is also
* used to modify the registers encoded in the instructions so that an
* emulation routines can use it. (See decode_regs() and INSN_NEW_BITS.)
*
* Here is a real example which matches ARM instructions of the form
* "AND <Rd>,<Rn>,<Rm>,<shift> <Rs>"
*
* DECODE_EMULATEX (0x0e000090, 0x00000010, emulate_rd12rn16rm0rs8_rwflags,
* REGS(ANY, ANY, NOPC, 0, ANY)),
* ^ ^ ^ ^
* Rn Rd Rs Rm
*
* Decoding the instruction "AND R4, R5, R6, ASL R15" will be rejected because
* Rs == R15
*
* Decoding the instruction "AND R4, R5, R6, ASL R7" will be accepted and the
* instruction will be modified to "AND R0, R2, R3, ASL R1" and then placed into
* the kprobes instruction slot. This can then be called later by the handler
* function emulate_rd12rn16rm0rs8_rwflags in order to simulate the instruction.
*/
enum decode_type {
DECODE_TYPE_END,
DECODE_TYPE_TABLE,
DECODE_TYPE_CUSTOM,
DECODE_TYPE_SIMULATE,
DECODE_TYPE_EMULATE,
DECODE_TYPE_OR,
DECODE_TYPE_REJECT,
NUM_DECODE_TYPES /* Must be last enum */
};
#define DECODE_TYPE_BITS 4
#define DECODE_TYPE_MASK ((1 << DECODE_TYPE_BITS) - 1)
enum decode_reg_type {
REG_TYPE_NONE = 0, /* Not a register, ignore */
REG_TYPE_ANY, /* Any register allowed */
REG_TYPE_SAMEAS16, /* Register should be same as that at bits 19..16 */
REG_TYPE_SP, /* Register must be SP */
REG_TYPE_PC, /* Register must be PC */
REG_TYPE_NOSP, /* Register must not be SP */
REG_TYPE_NOSPPC, /* Register must not be SP or PC */
REG_TYPE_NOPC, /* Register must not be PC */
REG_TYPE_NOPCWB, /* No PC if load/store write-back flag also set */
/* The following types are used when the encoding for PC indicates
* another instruction form. This distiction only matters for test
* case coverage checks.
*/
REG_TYPE_NOPCX, /* Register must not be PC */
REG_TYPE_NOSPPCX, /* Register must not be SP or PC */
/* Alias to allow '0' arg to be used in REGS macro. */
REG_TYPE_0 = REG_TYPE_NONE
};
#define REGS(r16, r12, r8, r4, r0) \
((REG_TYPE_##r16) << 16) + \
((REG_TYPE_##r12) << 12) + \
((REG_TYPE_##r8) << 8) + \
((REG_TYPE_##r4) << 4) + \
(REG_TYPE_##r0)
union decode_item {
u32 bits;
const union decode_item *table;
kprobe_insn_handler_t *handler;
kprobe_decode_insn_t *decoder;
};
#define DECODE_END \
{.bits = DECODE_TYPE_END}
struct decode_header {
union decode_item type_regs;
union decode_item mask;
union decode_item value;
};
#define DECODE_HEADER(_type, _mask, _value, _regs) \
{.bits = (_type) | ((_regs) << DECODE_TYPE_BITS)}, \
{.bits = (_mask)}, \
{.bits = (_value)}
struct decode_table {
struct decode_header header;
union decode_item table;
};
#define DECODE_TABLE(_mask, _value, _table) \
DECODE_HEADER(DECODE_TYPE_TABLE, _mask, _value, 0), \
{.table = (_table)}
struct decode_custom {
struct decode_header header;
union decode_item decoder;
};
#define DECODE_CUSTOM(_mask, _value, _decoder) \
DECODE_HEADER(DECODE_TYPE_CUSTOM, _mask, _value, 0), \
{.decoder = (_decoder)}
struct decode_simulate {
struct decode_header header;
union decode_item handler;
};
#define DECODE_SIMULATEX(_mask, _value, _handler, _regs) \
DECODE_HEADER(DECODE_TYPE_SIMULATE, _mask, _value, _regs), \
{.handler = (_handler)}
#define DECODE_SIMULATE(_mask, _value, _handler) \
DECODE_SIMULATEX(_mask, _value, _handler, 0)
struct decode_emulate {
struct decode_header header;
union decode_item handler;
};
#define DECODE_EMULATEX(_mask, _value, _handler, _regs) \
DECODE_HEADER(DECODE_TYPE_EMULATE, _mask, _value, _regs), \
{.handler = (_handler)}
#define DECODE_EMULATE(_mask, _value, _handler) \
DECODE_EMULATEX(_mask, _value, _handler, 0)
struct decode_or {
struct decode_header header;
};
#define DECODE_OR(_mask, _value) \
DECODE_HEADER(DECODE_TYPE_OR, _mask, _value, 0)
struct decode_reject {
struct decode_header header;
};
#define DECODE_REJECT(_mask, _value) \
DECODE_HEADER(DECODE_TYPE_REJECT, _mask, _value, 0)
int kprobe_decode_insn(kprobe_opcode_t insn, struct arch_specific_insn *asi,
const union decode_item *table, bool thumb16);
#endif /* _ARM_KERNEL_KPROBES_H */

View File

@ -435,7 +435,7 @@ armpmu_reserve_hardware(void)
if (irq >= 0)
free_irq(irq, NULL);
}
release_pmu(pmu_device);
release_pmu(ARM_PMU_DEVICE_CPU);
pmu_device = NULL;
}
@ -454,7 +454,7 @@ armpmu_release_hardware(void)
}
armpmu->stop();
release_pmu(pmu_device);
release_pmu(ARM_PMU_DEVICE_CPU);
pmu_device = NULL;
}
@ -662,6 +662,12 @@ init_hw_perf_events(void)
case 0xC090: /* Cortex-A9 */
armpmu = armv7_a9_pmu_init();
break;
case 0xC050: /* Cortex-A5 */
armpmu = armv7_a5_pmu_init();
break;
case 0xC0F0: /* Cortex-A15 */
armpmu = armv7_a15_pmu_init();
break;
}
/* Intel CPUs [xscale]. */
} else if (0x69 == implementor) {

View File

@ -17,17 +17,23 @@
*/
#ifdef CONFIG_CPU_V7
/* Common ARMv7 event types */
/*
* Common ARMv7 event types
*
* Note: An implementation may not be able to count all of these events
* but the encodings are considered to be `reserved' in the case that
* they are not available.
*/
enum armv7_perf_types {
ARMV7_PERFCTR_PMNC_SW_INCR = 0x00,
ARMV7_PERFCTR_IFETCH_MISS = 0x01,
ARMV7_PERFCTR_ITLB_MISS = 0x02,
ARMV7_PERFCTR_DCACHE_REFILL = 0x03,
ARMV7_PERFCTR_DCACHE_ACCESS = 0x04,
ARMV7_PERFCTR_DCACHE_REFILL = 0x03, /* L1 */
ARMV7_PERFCTR_DCACHE_ACCESS = 0x04, /* L1 */
ARMV7_PERFCTR_DTLB_REFILL = 0x05,
ARMV7_PERFCTR_DREAD = 0x06,
ARMV7_PERFCTR_DWRITE = 0x07,
ARMV7_PERFCTR_INSTR_EXECUTED = 0x08,
ARMV7_PERFCTR_EXC_TAKEN = 0x09,
ARMV7_PERFCTR_EXC_EXECUTED = 0x0A,
ARMV7_PERFCTR_CID_WRITE = 0x0B,
@ -39,21 +45,30 @@ enum armv7_perf_types {
*/
ARMV7_PERFCTR_PC_WRITE = 0x0C,
ARMV7_PERFCTR_PC_IMM_BRANCH = 0x0D,
ARMV7_PERFCTR_PC_PROC_RETURN = 0x0E,
ARMV7_PERFCTR_UNALIGNED_ACCESS = 0x0F,
/* These events are defined by the PMUv2 supplement (ARM DDI 0457A). */
ARMV7_PERFCTR_PC_BRANCH_MIS_PRED = 0x10,
ARMV7_PERFCTR_CLOCK_CYCLES = 0x11,
ARMV7_PERFCTR_PC_BRANCH_MIS_USED = 0x12,
ARMV7_PERFCTR_PC_BRANCH_PRED = 0x12,
ARMV7_PERFCTR_MEM_ACCESS = 0x13,
ARMV7_PERFCTR_L1_ICACHE_ACCESS = 0x14,
ARMV7_PERFCTR_L1_DCACHE_WB = 0x15,
ARMV7_PERFCTR_L2_DCACHE_ACCESS = 0x16,
ARMV7_PERFCTR_L2_DCACHE_REFILL = 0x17,
ARMV7_PERFCTR_L2_DCACHE_WB = 0x18,
ARMV7_PERFCTR_BUS_ACCESS = 0x19,
ARMV7_PERFCTR_MEMORY_ERROR = 0x1A,
ARMV7_PERFCTR_INSTR_SPEC = 0x1B,
ARMV7_PERFCTR_TTBR_WRITE = 0x1C,
ARMV7_PERFCTR_BUS_CYCLES = 0x1D,
ARMV7_PERFCTR_CPU_CYCLES = 0xFF
};
/* ARMv7 Cortex-A8 specific event types */
enum armv7_a8_perf_types {
ARMV7_PERFCTR_INSTR_EXECUTED = 0x08,
ARMV7_PERFCTR_PC_PROC_RETURN = 0x0E,
ARMV7_PERFCTR_WRITE_BUFFER_FULL = 0x40,
ARMV7_PERFCTR_L2_STORE_MERGED = 0x41,
ARMV7_PERFCTR_L2_STORE_BUFF = 0x42,
@ -138,6 +153,39 @@ enum armv7_a9_perf_types {
ARMV7_PERFCTR_PLE_RQST_PROG = 0xA5
};
/* ARMv7 Cortex-A5 specific event types */
enum armv7_a5_perf_types {
ARMV7_PERFCTR_IRQ_TAKEN = 0x86,
ARMV7_PERFCTR_FIQ_TAKEN = 0x87,
ARMV7_PERFCTR_EXT_MEM_RQST = 0xc0,
ARMV7_PERFCTR_NC_EXT_MEM_RQST = 0xc1,
ARMV7_PERFCTR_PREFETCH_LINEFILL = 0xc2,
ARMV7_PERFCTR_PREFETCH_LINEFILL_DROP = 0xc3,
ARMV7_PERFCTR_ENTER_READ_ALLOC = 0xc4,
ARMV7_PERFCTR_READ_ALLOC = 0xc5,
ARMV7_PERFCTR_STALL_SB_FULL = 0xc9,
};
/* ARMv7 Cortex-A15 specific event types */
enum armv7_a15_perf_types {
ARMV7_PERFCTR_L1_DCACHE_READ_ACCESS = 0x40,
ARMV7_PERFCTR_L1_DCACHE_WRITE_ACCESS = 0x41,
ARMV7_PERFCTR_L1_DCACHE_READ_REFILL = 0x42,
ARMV7_PERFCTR_L1_DCACHE_WRITE_REFILL = 0x43,
ARMV7_PERFCTR_L1_DTLB_READ_REFILL = 0x4C,
ARMV7_PERFCTR_L1_DTLB_WRITE_REFILL = 0x4D,
ARMV7_PERFCTR_L2_DCACHE_READ_ACCESS = 0x50,
ARMV7_PERFCTR_L2_DCACHE_WRITE_ACCESS = 0x51,
ARMV7_PERFCTR_L2_DCACHE_READ_REFILL = 0x52,
ARMV7_PERFCTR_L2_DCACHE_WRITE_REFILL = 0x53,
ARMV7_PERFCTR_SPEC_PC_WRITE = 0x76,
};
/*
* Cortex-A8 HW events mapping
*
@ -207,11 +255,6 @@ static const unsigned armv7_a8_perf_cache_map[PERF_COUNT_HW_CACHE_MAX]
},
},
[C(DTLB)] = {
/*
* Only ITLB misses and DTLB refills are supported.
* If users want the DTLB refills misses a raw counter
* must be used.
*/
[C(OP_READ)] = {
[C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
[C(RESULT_MISS)] = ARMV7_PERFCTR_DTLB_REFILL,
@ -337,11 +380,6 @@ static const unsigned armv7_a9_perf_cache_map[PERF_COUNT_HW_CACHE_MAX]
},
},
[C(DTLB)] = {
/*
* Only ITLB misses and DTLB refills are supported.
* If users want the DTLB refills misses a raw counter
* must be used.
*/
[C(OP_READ)] = {
[C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
[C(RESULT_MISS)] = ARMV7_PERFCTR_DTLB_REFILL,
@ -401,6 +439,242 @@ static const unsigned armv7_a9_perf_cache_map[PERF_COUNT_HW_CACHE_MAX]
},
};
/*
* Cortex-A5 HW events mapping
*/
static const unsigned armv7_a5_perf_map[PERF_COUNT_HW_MAX] = {
[PERF_COUNT_HW_CPU_CYCLES] = ARMV7_PERFCTR_CPU_CYCLES,
[PERF_COUNT_HW_INSTRUCTIONS] = ARMV7_PERFCTR_INSTR_EXECUTED,
[PERF_COUNT_HW_CACHE_REFERENCES] = HW_OP_UNSUPPORTED,
[PERF_COUNT_HW_CACHE_MISSES] = HW_OP_UNSUPPORTED,
[PERF_COUNT_HW_BRANCH_INSTRUCTIONS] = ARMV7_PERFCTR_PC_WRITE,
[PERF_COUNT_HW_BRANCH_MISSES] = ARMV7_PERFCTR_PC_BRANCH_MIS_PRED,
[PERF_COUNT_HW_BUS_CYCLES] = HW_OP_UNSUPPORTED,
};
static const unsigned armv7_a5_perf_cache_map[PERF_COUNT_HW_CACHE_MAX]
[PERF_COUNT_HW_CACHE_OP_MAX]
[PERF_COUNT_HW_CACHE_RESULT_MAX] = {
[C(L1D)] = {
[C(OP_READ)] = {
[C(RESULT_ACCESS)]
= ARMV7_PERFCTR_DCACHE_ACCESS,
[C(RESULT_MISS)]
= ARMV7_PERFCTR_DCACHE_REFILL,
},
[C(OP_WRITE)] = {
[C(RESULT_ACCESS)]
= ARMV7_PERFCTR_DCACHE_ACCESS,
[C(RESULT_MISS)]
= ARMV7_PERFCTR_DCACHE_REFILL,
},
[C(OP_PREFETCH)] = {
[C(RESULT_ACCESS)]
= ARMV7_PERFCTR_PREFETCH_LINEFILL,
[C(RESULT_MISS)]
= ARMV7_PERFCTR_PREFETCH_LINEFILL_DROP,
},
},
[C(L1I)] = {
[C(OP_READ)] = {
[C(RESULT_ACCESS)] = ARMV7_PERFCTR_L1_ICACHE_ACCESS,
[C(RESULT_MISS)] = ARMV7_PERFCTR_IFETCH_MISS,
},
[C(OP_WRITE)] = {
[C(RESULT_ACCESS)] = ARMV7_PERFCTR_L1_ICACHE_ACCESS,
[C(RESULT_MISS)] = ARMV7_PERFCTR_IFETCH_MISS,
},
/*
* The prefetch counters don't differentiate between the I
* side and the D side.
*/
[C(OP_PREFETCH)] = {
[C(RESULT_ACCESS)]
= ARMV7_PERFCTR_PREFETCH_LINEFILL,
[C(RESULT_MISS)]
= ARMV7_PERFCTR_PREFETCH_LINEFILL_DROP,
},
},
[C(LL)] = {
[C(OP_READ)] = {
[C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
[C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
},
[C(OP_WRITE)] = {
[C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
[C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
},
[C(OP_PREFETCH)] = {
[C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
[C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
},
},
[C(DTLB)] = {
[C(OP_READ)] = {
[C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
[C(RESULT_MISS)] = ARMV7_PERFCTR_DTLB_REFILL,
},
[C(OP_WRITE)] = {
[C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
[C(RESULT_MISS)] = ARMV7_PERFCTR_DTLB_REFILL,
},
[C(OP_PREFETCH)] = {
[C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
[C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
},
},
[C(ITLB)] = {
[C(OP_READ)] = {
[C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
[C(RESULT_MISS)] = ARMV7_PERFCTR_ITLB_MISS,
},
[C(OP_WRITE)] = {
[C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
[C(RESULT_MISS)] = ARMV7_PERFCTR_ITLB_MISS,
},
[C(OP_PREFETCH)] = {
[C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
[C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
},
},
[C(BPU)] = {
[C(OP_READ)] = {
[C(RESULT_ACCESS)] = ARMV7_PERFCTR_PC_BRANCH_PRED,
[C(RESULT_MISS)]
= ARMV7_PERFCTR_PC_BRANCH_MIS_PRED,
},
[C(OP_WRITE)] = {
[C(RESULT_ACCESS)] = ARMV7_PERFCTR_PC_BRANCH_PRED,
[C(RESULT_MISS)]
= ARMV7_PERFCTR_PC_BRANCH_MIS_PRED,
},
[C(OP_PREFETCH)] = {
[C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
[C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
},
},
};
/*
* Cortex-A15 HW events mapping
*/
static const unsigned armv7_a15_perf_map[PERF_COUNT_HW_MAX] = {
[PERF_COUNT_HW_CPU_CYCLES] = ARMV7_PERFCTR_CPU_CYCLES,
[PERF_COUNT_HW_INSTRUCTIONS] = ARMV7_PERFCTR_INSTR_EXECUTED,
[PERF_COUNT_HW_CACHE_REFERENCES] = HW_OP_UNSUPPORTED,
[PERF_COUNT_HW_CACHE_MISSES] = HW_OP_UNSUPPORTED,
[PERF_COUNT_HW_BRANCH_INSTRUCTIONS] = ARMV7_PERFCTR_SPEC_PC_WRITE,
[PERF_COUNT_HW_BRANCH_MISSES] = ARMV7_PERFCTR_PC_BRANCH_MIS_PRED,
[PERF_COUNT_HW_BUS_CYCLES] = ARMV7_PERFCTR_BUS_CYCLES,
};
static const unsigned armv7_a15_perf_cache_map[PERF_COUNT_HW_CACHE_MAX]
[PERF_COUNT_HW_CACHE_OP_MAX]
[PERF_COUNT_HW_CACHE_RESULT_MAX] = {
[C(L1D)] = {
[C(OP_READ)] = {
[C(RESULT_ACCESS)]
= ARMV7_PERFCTR_L1_DCACHE_READ_ACCESS,
[C(RESULT_MISS)]
= ARMV7_PERFCTR_L1_DCACHE_READ_REFILL,
},
[C(OP_WRITE)] = {
[C(RESULT_ACCESS)]
= ARMV7_PERFCTR_L1_DCACHE_WRITE_ACCESS,
[C(RESULT_MISS)]
= ARMV7_PERFCTR_L1_DCACHE_WRITE_REFILL,
},
[C(OP_PREFETCH)] = {
[C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
[C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
},
},
[C(L1I)] = {
/*
* Not all performance counters differentiate between read
* and write accesses/misses so we're not always strictly
* correct, but it's the best we can do. Writes and reads get
* combined in these cases.
*/
[C(OP_READ)] = {
[C(RESULT_ACCESS)] = ARMV7_PERFCTR_L1_ICACHE_ACCESS,
[C(RESULT_MISS)] = ARMV7_PERFCTR_IFETCH_MISS,
},
[C(OP_WRITE)] = {
[C(RESULT_ACCESS)] = ARMV7_PERFCTR_L1_ICACHE_ACCESS,
[C(RESULT_MISS)] = ARMV7_PERFCTR_IFETCH_MISS,
},
[C(OP_PREFETCH)] = {
[C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
[C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
},
},
[C(LL)] = {
[C(OP_READ)] = {
[C(RESULT_ACCESS)]
= ARMV7_PERFCTR_L2_DCACHE_READ_ACCESS,
[C(RESULT_MISS)]
= ARMV7_PERFCTR_L2_DCACHE_READ_REFILL,
},
[C(OP_WRITE)] = {
[C(RESULT_ACCESS)]
= ARMV7_PERFCTR_L2_DCACHE_WRITE_ACCESS,
[C(RESULT_MISS)]
= ARMV7_PERFCTR_L2_DCACHE_WRITE_REFILL,
},
[C(OP_PREFETCH)] = {
[C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
[C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
},
},
[C(DTLB)] = {
[C(OP_READ)] = {
[C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
[C(RESULT_MISS)]
= ARMV7_PERFCTR_L1_DTLB_READ_REFILL,
},
[C(OP_WRITE)] = {
[C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
[C(RESULT_MISS)]
= ARMV7_PERFCTR_L1_DTLB_WRITE_REFILL,
},
[C(OP_PREFETCH)] = {
[C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
[C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
},
},
[C(ITLB)] = {
[C(OP_READ)] = {
[C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
[C(RESULT_MISS)] = ARMV7_PERFCTR_ITLB_MISS,
},
[C(OP_WRITE)] = {
[C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
[C(RESULT_MISS)] = ARMV7_PERFCTR_ITLB_MISS,
},
[C(OP_PREFETCH)] = {
[C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
[C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
},
},
[C(BPU)] = {
[C(OP_READ)] = {
[C(RESULT_ACCESS)] = ARMV7_PERFCTR_PC_BRANCH_PRED,
[C(RESULT_MISS)]
= ARMV7_PERFCTR_PC_BRANCH_MIS_PRED,
},
[C(OP_WRITE)] = {
[C(RESULT_ACCESS)] = ARMV7_PERFCTR_PC_BRANCH_PRED,
[C(RESULT_MISS)]
= ARMV7_PERFCTR_PC_BRANCH_MIS_PRED,
},
[C(OP_PREFETCH)] = {
[C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
[C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
},
},
};
/*
* Perf Events counters
*/
@ -933,6 +1207,26 @@ static const struct arm_pmu *__init armv7_a9_pmu_init(void)
armv7pmu.num_events = armv7_read_num_pmnc_events();
return &armv7pmu;
}
static const struct arm_pmu *__init armv7_a5_pmu_init(void)
{
armv7pmu.id = ARM_PERF_PMU_ID_CA5;
armv7pmu.name = "ARMv7 Cortex-A5";
armv7pmu.cache_map = &armv7_a5_perf_cache_map;
armv7pmu.event_map = &armv7_a5_perf_map;
armv7pmu.num_events = armv7_read_num_pmnc_events();
return &armv7pmu;
}
static const struct arm_pmu *__init armv7_a15_pmu_init(void)
{
armv7pmu.id = ARM_PERF_PMU_ID_CA15;
armv7pmu.name = "ARMv7 Cortex-A15";
armv7pmu.cache_map = &armv7_a15_perf_cache_map;
armv7pmu.event_map = &armv7_a15_perf_map;
armv7pmu.num_events = armv7_read_num_pmnc_events();
return &armv7pmu;
}
#else
static const struct arm_pmu *__init armv7_a8_pmu_init(void)
{
@ -943,4 +1237,14 @@ static const struct arm_pmu *__init armv7_a9_pmu_init(void)
{
return NULL;
}
static const struct arm_pmu *__init armv7_a5_pmu_init(void)
{
return NULL;
}
static const struct arm_pmu *__init armv7_a15_pmu_init(void)
{
return NULL;
}
#endif /* CONFIG_CPU_V7 */

View File

@ -17,6 +17,7 @@
#include <linux/interrupt.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/of_device.h>
#include <linux/platform_device.h>
#include <asm/pmu.h>
@ -25,36 +26,88 @@ static volatile long pmu_lock;
static struct platform_device *pmu_devices[ARM_NUM_PMU_DEVICES];
static int __devinit pmu_device_probe(struct platform_device *pdev)
static int __devinit pmu_register(struct platform_device *pdev,
enum arm_pmu_type type)
{
if (pdev->id < 0 || pdev->id >= ARM_NUM_PMU_DEVICES) {
if (type < 0 || type >= ARM_NUM_PMU_DEVICES) {
pr_warning("received registration request for unknown "
"device %d\n", pdev->id);
"device %d\n", type);
return -EINVAL;
}
if (pmu_devices[pdev->id])
pr_warning("registering new PMU device type %d overwrites "
"previous registration!\n", pdev->id);
else
pr_info("registered new PMU device of type %d\n",
pdev->id);
if (pmu_devices[type]) {
pr_warning("rejecting duplicate registration of PMU device "
"type %d.", type);
return -ENOSPC;
}
pmu_devices[pdev->id] = pdev;
pr_info("registered new PMU device of type %d\n", type);
pmu_devices[type] = pdev;
return 0;
}
static struct platform_driver pmu_driver = {
#define OF_MATCH_PMU(_name, _type) { \
.compatible = _name, \
.data = (void *)_type, \
}
#define OF_MATCH_CPU(name) OF_MATCH_PMU(name, ARM_PMU_DEVICE_CPU)
static struct of_device_id armpmu_of_device_ids[] = {
OF_MATCH_CPU("arm,cortex-a9-pmu"),
OF_MATCH_CPU("arm,cortex-a8-pmu"),
OF_MATCH_CPU("arm,arm1136-pmu"),
OF_MATCH_CPU("arm,arm1176-pmu"),
{},
};
#define PLAT_MATCH_PMU(_name, _type) { \
.name = _name, \
.driver_data = _type, \
}
#define PLAT_MATCH_CPU(_name) PLAT_MATCH_PMU(_name, ARM_PMU_DEVICE_CPU)
static struct platform_device_id armpmu_plat_device_ids[] = {
PLAT_MATCH_CPU("arm-pmu"),
{},
};
enum arm_pmu_type armpmu_device_type(struct platform_device *pdev)
{
const struct of_device_id *of_id;
const struct platform_device_id *pdev_id;
/* provided by of_device_id table */
if (pdev->dev.of_node) {
of_id = of_match_device(armpmu_of_device_ids, &pdev->dev);
BUG_ON(!of_id);
return (enum arm_pmu_type)of_id->data;
}
/* Provided by platform_device_id table */
pdev_id = platform_get_device_id(pdev);
BUG_ON(!pdev_id);
return pdev_id->driver_data;
}
static int __devinit armpmu_device_probe(struct platform_device *pdev)
{
return pmu_register(pdev, armpmu_device_type(pdev));
}
static struct platform_driver armpmu_driver = {
.driver = {
.name = "arm-pmu",
.of_match_table = armpmu_of_device_ids,
},
.probe = pmu_device_probe,
.probe = armpmu_device_probe,
.id_table = armpmu_plat_device_ids,
};
static int __init register_pmu_driver(void)
{
return platform_driver_register(&pmu_driver);
return platform_driver_register(&armpmu_driver);
}
device_initcall(register_pmu_driver);
@ -77,11 +130,11 @@ reserve_pmu(enum arm_pmu_type device)
EXPORT_SYMBOL_GPL(reserve_pmu);
int
release_pmu(struct platform_device *pdev)
release_pmu(enum arm_pmu_type device)
{
if (WARN_ON(pdev != pmu_devices[pdev->id]))
if (WARN_ON(!pmu_devices[device]))
return -EINVAL;
clear_bit_unlock(pdev->id, &pmu_lock);
clear_bit_unlock(device, &pmu_lock);
return 0;
}
EXPORT_SYMBOL_GPL(release_pmu);

View File

@ -228,34 +228,12 @@ static struct undef_hook thumb_break_hook = {
.fn = break_trap,
};
static int thumb2_break_trap(struct pt_regs *regs, unsigned int instr)
{
unsigned int instr2;
void __user *pc;
/* Check the second half of the instruction. */
pc = (void __user *)(instruction_pointer(regs) + 2);
if (processor_mode(regs) == SVC_MODE) {
instr2 = *(u16 *) pc;
} else {
get_user(instr2, (u16 __user *)pc);
}
if (instr2 == 0xa000) {
ptrace_break(current, regs);
return 0;
} else {
return 1;
}
}
static struct undef_hook thumb2_break_hook = {
.instr_mask = 0xffff,
.instr_val = 0xf7f0,
.instr_mask = 0xffffffff,
.instr_val = 0xf7f0a000,
.cpsr_mask = PSR_T_BIT,
.cpsr_val = PSR_T_BIT,
.fn = thumb2_break_trap,
.fn = break_trap,
};
static int __init ptrace_break_init(void)

View File

@ -343,6 +343,59 @@ static void __init feat_v6_fixup(void)
elf_hwcap &= ~HWCAP_TLS;
}
/*
* cpu_init - initialise one CPU.
*
* cpu_init sets up the per-CPU stacks.
*/
void cpu_init(void)
{
unsigned int cpu = smp_processor_id();
struct stack *stk = &stacks[cpu];
if (cpu >= NR_CPUS) {
printk(KERN_CRIT "CPU%u: bad primary CPU number\n", cpu);
BUG();
}
cpu_proc_init();
/*
* Define the placement constraint for the inline asm directive below.
* In Thumb-2, msr with an immediate value is not allowed.
*/
#ifdef CONFIG_THUMB2_KERNEL
#define PLC "r"
#else
#define PLC "I"
#endif
/*
* setup stacks for re-entrant exception handlers
*/
__asm__ (
"msr cpsr_c, %1\n\t"
"add r14, %0, %2\n\t"
"mov sp, r14\n\t"
"msr cpsr_c, %3\n\t"
"add r14, %0, %4\n\t"
"mov sp, r14\n\t"
"msr cpsr_c, %5\n\t"
"add r14, %0, %6\n\t"
"mov sp, r14\n\t"
"msr cpsr_c, %7"
:
: "r" (stk),
PLC (PSR_F_BIT | PSR_I_BIT | IRQ_MODE),
"I" (offsetof(struct stack, irq[0])),
PLC (PSR_F_BIT | PSR_I_BIT | ABT_MODE),
"I" (offsetof(struct stack, abt[0])),
PLC (PSR_F_BIT | PSR_I_BIT | UND_MODE),
"I" (offsetof(struct stack, und[0])),
PLC (PSR_F_BIT | PSR_I_BIT | SVC_MODE)
: "r14");
}
static void __init setup_processor(void)
{
struct proc_info_list *list;
@ -388,58 +441,7 @@ static void __init setup_processor(void)
feat_v6_fixup();
cacheid_init();
cpu_proc_init();
}
/*
* cpu_init - initialise one CPU.
*
* cpu_init sets up the per-CPU stacks.
*/
void cpu_init(void)
{
unsigned int cpu = smp_processor_id();
struct stack *stk = &stacks[cpu];
if (cpu >= NR_CPUS) {
printk(KERN_CRIT "CPU%u: bad primary CPU number\n", cpu);
BUG();
}
/*
* Define the placement constraint for the inline asm directive below.
* In Thumb-2, msr with an immediate value is not allowed.
*/
#ifdef CONFIG_THUMB2_KERNEL
#define PLC "r"
#else
#define PLC "I"
#endif
/*
* setup stacks for re-entrant exception handlers
*/
__asm__ (
"msr cpsr_c, %1\n\t"
"add r14, %0, %2\n\t"
"mov sp, r14\n\t"
"msr cpsr_c, %3\n\t"
"add r14, %0, %4\n\t"
"mov sp, r14\n\t"
"msr cpsr_c, %5\n\t"
"add r14, %0, %6\n\t"
"mov sp, r14\n\t"
"msr cpsr_c, %7"
:
: "r" (stk),
PLC (PSR_F_BIT | PSR_I_BIT | IRQ_MODE),
"I" (offsetof(struct stack, irq[0])),
PLC (PSR_F_BIT | PSR_I_BIT | ABT_MODE),
"I" (offsetof(struct stack, abt[0])),
PLC (PSR_F_BIT | PSR_I_BIT | UND_MODE),
"I" (offsetof(struct stack, und[0])),
PLC (PSR_F_BIT | PSR_I_BIT | SVC_MODE)
: "r14");
cpu_init();
}
void __init dump_machine_table(void)
@ -915,9 +917,14 @@ void __init setup_arch(char **cmdline_p)
#endif
reserve_crashkernel();
cpu_init();
tcm_init();
#ifdef CONFIG_ZONE_DMA
if (mdesc->dma_zone_size) {
extern unsigned long arm_dma_zone_size;
arm_dma_zone_size = mdesc->dma_zone_size;
}
#endif
#ifdef CONFIG_MULTI_IRQ_HANDLER
handle_arch_irq = mdesc->handle_irq;
#endif
@ -979,6 +986,10 @@ static const char *hwcap_str[] = {
"neon",
"vfpv3",
"vfpv3d16",
"tls",
"vfpv4",
"idiva",
"idivt",
NULL
};

View File

@ -10,64 +10,61 @@
/*
* Save CPU state for a suspend
* r1 = v:p offset
* r3 = virtual return function
* Note: sp is decremented to allocate space for CPU state on stack
* r0-r3,r9,r10,lr corrupted
* r2 = suspend function arg0
* r3 = suspend function
*/
ENTRY(cpu_suspend)
mov r9, lr
ENTRY(__cpu_suspend)
stmfd sp!, {r4 - r11, lr}
#ifdef MULTI_CPU
ldr r10, =processor
mov r2, sp @ current virtual SP
ldr r0, [r10, #CPU_SLEEP_SIZE] @ size of CPU sleep state
ldr r5, [r10, #CPU_SLEEP_SIZE] @ size of CPU sleep state
ldr ip, [r10, #CPU_DO_RESUME] @ virtual resume function
sub sp, sp, r0 @ allocate CPU state on stack
mov r0, sp @ save pointer
#else
ldr r5, =cpu_suspend_size
ldr ip, =cpu_do_resume
#endif
mov r6, sp @ current virtual SP
sub sp, sp, r5 @ allocate CPU state on stack
mov r0, sp @ save pointer to CPU save block
add ip, ip, r1 @ convert resume fn to phys
stmfd sp!, {r1, r2, r3, ip} @ save v:p, virt SP, retfn, phys resume fn
ldr r3, =sleep_save_sp
add r2, sp, r1 @ convert SP to phys
stmfd sp!, {r1, r6, ip} @ save v:p, virt SP, phys resume fn
ldr r5, =sleep_save_sp
add r6, sp, r1 @ convert SP to phys
stmfd sp!, {r2, r3} @ save suspend func arg and pointer
#ifdef CONFIG_SMP
ALT_SMP(mrc p15, 0, lr, c0, c0, 5)
ALT_UP(mov lr, #0)
and lr, lr, #15
str r2, [r3, lr, lsl #2] @ save phys SP
str r6, [r5, lr, lsl #2] @ save phys SP
#else
str r2, [r3] @ save phys SP
str r6, [r5] @ save phys SP
#endif
#ifdef MULTI_CPU
mov lr, pc
ldr pc, [r10, #CPU_DO_SUSPEND] @ save CPU state
#else
mov r2, sp @ current virtual SP
ldr r0, =cpu_suspend_size
sub sp, sp, r0 @ allocate CPU state on stack
mov r0, sp @ save pointer
stmfd sp!, {r1, r2, r3} @ save v:p, virt SP, return fn
ldr r3, =sleep_save_sp
add r2, sp, r1 @ convert SP to phys
#ifdef CONFIG_SMP
ALT_SMP(mrc p15, 0, lr, c0, c0, 5)
ALT_UP(mov lr, #0)
and lr, lr, #15
str r2, [r3, lr, lsl #2] @ save phys SP
#else
str r2, [r3] @ save phys SP
#endif
bl cpu_do_suspend
#endif
@ flush data cache
#ifdef MULTI_CACHE
ldr r10, =cpu_cache
mov lr, r9
mov lr, pc
ldr pc, [r10, #CACHE_FLUSH_KERN_ALL]
#else
mov lr, r9
b __cpuc_flush_kern_all
bl __cpuc_flush_kern_all
#endif
ENDPROC(cpu_suspend)
adr lr, BSYM(cpu_suspend_abort)
ldmfd sp!, {r0, pc} @ call suspend fn
ENDPROC(__cpu_suspend)
.ltorg
cpu_suspend_abort:
ldmia sp!, {r1 - r3} @ pop v:p, virt SP, phys resume fn
mov sp, r2
ldmfd sp!, {r4 - r11, pc}
ENDPROC(cpu_suspend_abort)
/*
* r0 = control register value
* r1 = v:p offset (preserved by cpu_do_resume)
@ -97,7 +94,9 @@ ENDPROC(cpu_resume_turn_mmu_on)
cpu_resume_after_mmu:
str r5, [r2, r4, lsl #2] @ restore old mapping
mcr p15, 0, r0, c1, c0, 0 @ turn on D-cache
mov pc, lr
bl cpu_init @ restore the und/abt/irq banked regs
mov r0, #0 @ return zero on success
ldmfd sp!, {r4 - r11, pc}
ENDPROC(cpu_resume_after_mmu)
/*
@ -120,20 +119,11 @@ ENTRY(cpu_resume)
ldr r0, sleep_save_sp @ stack phys addr
#endif
setmode PSR_I_BIT | PSR_F_BIT | SVC_MODE, r1 @ set SVC, irqs off
#ifdef MULTI_CPU
@ load v:p, stack, return fn, resume fn
ARM( ldmia r0!, {r1, sp, lr, pc} )
THUMB( ldmia r0!, {r1, r2, r3, r4} )
@ load v:p, stack, resume fn
ARM( ldmia r0!, {r1, sp, pc} )
THUMB( ldmia r0!, {r1, r2, r3} )
THUMB( mov sp, r2 )
THUMB( mov lr, r3 )
THUMB( bx r4 )
#else
@ load v:p, stack, return fn
ARM( ldmia r0!, {r1, sp, lr} )
THUMB( ldmia r0!, {r1, r2, lr} )
THUMB( mov sp, r2 )
b cpu_do_resume
#endif
THUMB( bx r3 )
ENDPROC(cpu_resume)
sleep_save_sp:

View File

@ -365,14 +365,21 @@ void __init smp_prepare_cpus(unsigned int max_cpus)
*/
if (max_cpus > ncores)
max_cpus = ncores;
if (max_cpus > 1) {
if (ncores > 1 && max_cpus) {
/*
* Enable the local timer or broadcast device for the
* boot CPU, but only if we have more than one CPU.
*/
percpu_timer_setup();
/*
* Initialise the present map, which describes the set of CPUs
* actually populated at the present time. A platform should
* re-initialize the map in platform_smp_prepare_cpus() if
* present != possible (e.g. physical hotplug).
*/
init_cpu_present(&cpu_possible_map);
/*
* Initialise the SCU if there are more than one CPU
* and let them know where to start.

View File

@ -20,6 +20,7 @@
#define SCU_INVALIDATE 0x0c
#define SCU_FPGA_REVISION 0x10
#ifdef CONFIG_SMP
/*
* Get the number of CPU cores from the SCU configuration
*/
@ -50,6 +51,7 @@ void __init scu_enable(void __iomem *scu_base)
*/
flush_cache_all();
}
#endif
/*
* Set the executing CPUs power mode as defined. This will be in

View File

@ -19,6 +19,8 @@
#include "tcm.h"
static struct gen_pool *tcm_pool;
static bool dtcm_present;
static bool itcm_present;
/* TCM section definitions from the linker */
extern char __itcm_start, __sitcm_text, __eitcm_text;
@ -90,6 +92,18 @@ void tcm_free(void *addr, size_t len)
}
EXPORT_SYMBOL(tcm_free);
bool tcm_dtcm_present(void)
{
return dtcm_present;
}
EXPORT_SYMBOL(tcm_dtcm_present);
bool tcm_itcm_present(void)
{
return itcm_present;
}
EXPORT_SYMBOL(tcm_itcm_present);
static int __init setup_tcm_bank(u8 type, u8 bank, u8 banks,
u32 *offset)
{
@ -134,6 +148,10 @@ static int __init setup_tcm_bank(u8 type, u8 bank, u8 banks,
(tcm_region & 1) ? "" : "not ");
}
/* Not much fun you can do with a size 0 bank */
if (tcm_size == 0)
return 0;
/* Force move the TCM bank to where we want it, enable */
tcm_region = *offset | (tcm_region & 0x00000ffeU) | 1;
@ -165,12 +183,20 @@ void __init tcm_init(void)
u32 tcm_status = read_cpuid_tcmstatus();
u8 dtcm_banks = (tcm_status >> 16) & 0x03;
u8 itcm_banks = (tcm_status & 0x03);
size_t dtcm_code_sz = &__edtcm_data - &__sdtcm_data;
size_t itcm_code_sz = &__eitcm_text - &__sitcm_text;
char *start;
char *end;
char *ram;
int ret;
int i;
/* Values greater than 2 for D/ITCM banks are "reserved" */
if (dtcm_banks > 2)
dtcm_banks = 0;
if (itcm_banks > 2)
itcm_banks = 0;
/* Setup DTCM if present */
if (dtcm_banks > 0) {
for (i = 0; i < dtcm_banks; i++) {
@ -178,6 +204,13 @@ void __init tcm_init(void)
if (ret)
return;
}
/* This means you compiled more code than fits into DTCM */
if (dtcm_code_sz > (dtcm_end - DTCM_OFFSET)) {
pr_info("CPU DTCM: %u bytes of code compiled to "
"DTCM but only %lu bytes of DTCM present\n",
dtcm_code_sz, (dtcm_end - DTCM_OFFSET));
goto no_dtcm;
}
dtcm_res.end = dtcm_end - 1;
request_resource(&iomem_resource, &dtcm_res);
dtcm_iomap[0].length = dtcm_end - DTCM_OFFSET;
@ -186,12 +219,16 @@ void __init tcm_init(void)
start = &__sdtcm_data;
end = &__edtcm_data;
ram = &__dtcm_start;
/* This means you compiled more code than fits into DTCM */
BUG_ON((end - start) > (dtcm_end - DTCM_OFFSET));
memcpy(start, ram, (end-start));
pr_debug("CPU DTCM: copied data from %p - %p\n", start, end);
memcpy(start, ram, dtcm_code_sz);
pr_debug("CPU DTCM: copied data from %p - %p\n",
start, end);
dtcm_present = true;
} else if (dtcm_code_sz) {
pr_info("CPU DTCM: %u bytes of code compiled to DTCM but no "
"DTCM banks present in CPU\n", dtcm_code_sz);
}
no_dtcm:
/* Setup ITCM if present */
if (itcm_banks > 0) {
for (i = 0; i < itcm_banks; i++) {
@ -199,6 +236,13 @@ void __init tcm_init(void)
if (ret)
return;
}
/* This means you compiled more code than fits into ITCM */
if (itcm_code_sz > (itcm_end - ITCM_OFFSET)) {
pr_info("CPU ITCM: %u bytes of code compiled to "
"ITCM but only %lu bytes of ITCM present\n",
itcm_code_sz, (itcm_end - ITCM_OFFSET));
return;
}
itcm_res.end = itcm_end - 1;
request_resource(&iomem_resource, &itcm_res);
itcm_iomap[0].length = itcm_end - ITCM_OFFSET;
@ -207,10 +251,13 @@ void __init tcm_init(void)
start = &__sitcm_text;
end = &__eitcm_text;
ram = &__itcm_start;
/* This means you compiled more code than fits into ITCM */
BUG_ON((end - start) > (itcm_end - ITCM_OFFSET));
memcpy(start, ram, (end-start));
pr_debug("CPU ITCM: copied code from %p - %p\n", start, end);
memcpy(start, ram, itcm_code_sz);
pr_debug("CPU ITCM: copied code from %p - %p\n",
start, end);
itcm_present = true;
} else if (itcm_code_sz) {
pr_info("CPU ITCM: %u bytes of code compiled to ITCM but no "
"ITCM banks present in CPU\n", itcm_code_sz);
}
}
@ -221,7 +268,6 @@ void __init tcm_init(void)
*/
static int __init setup_tcm_pool(void)
{
u32 tcm_status = read_cpuid_tcmstatus();
u32 dtcm_pool_start = (u32) &__edtcm_data;
u32 itcm_pool_start = (u32) &__eitcm_text;
int ret;
@ -236,7 +282,7 @@ static int __init setup_tcm_pool(void)
pr_debug("Setting up TCM memory pool\n");
/* Add the rest of DTCM to the TCM pool */
if (tcm_status & (0x03 << 16)) {
if (dtcm_present) {
if (dtcm_pool_start < dtcm_end) {
ret = gen_pool_add(tcm_pool, dtcm_pool_start,
dtcm_end - dtcm_pool_start, -1);
@ -253,7 +299,7 @@ static int __init setup_tcm_pool(void)
}
/* Add the rest of ITCM to the TCM pool */
if (tcm_status & 0x03) {
if (itcm_present) {
if (itcm_pool_start < itcm_end) {
ret = gen_pool_add(tcm_pool, itcm_pool_start,
itcm_end - itcm_pool_start, -1);

View File

@ -355,9 +355,24 @@ asmlinkage void __exception do_undefinstr(struct pt_regs *regs)
pc = (void __user *)instruction_pointer(regs);
if (processor_mode(regs) == SVC_MODE) {
instr = *(u32 *) pc;
#ifdef CONFIG_THUMB2_KERNEL
if (thumb_mode(regs)) {
instr = ((u16 *)pc)[0];
if (is_wide_instruction(instr)) {
instr <<= 16;
instr |= ((u16 *)pc)[1];
}
} else
#endif
instr = *(u32 *) pc;
} else if (thumb_mode(regs)) {
get_user(instr, (u16 __user *)pc);
if (is_wide_instruction(instr)) {
unsigned int instr2;
get_user(instr2, (u16 __user *)pc+1);
instr <<= 16;
instr |= instr2;
}
} else {
get_user(instr, (u32 __user *)pc);
}

View File

@ -38,57 +38,6 @@ jiffies = jiffies_64 + 4;
SECTIONS
{
#ifdef CONFIG_XIP_KERNEL
. = XIP_VIRT_ADDR(CONFIG_XIP_PHYS_ADDR);
#else
. = PAGE_OFFSET + TEXT_OFFSET;
#endif
.init : { /* Init code and data */
_stext = .;
_sinittext = .;
HEAD_TEXT
INIT_TEXT
ARM_EXIT_KEEP(EXIT_TEXT)
_einittext = .;
ARM_CPU_DISCARD(PROC_INFO)
__arch_info_begin = .;
*(.arch.info.init)
__arch_info_end = .;
__tagtable_begin = .;
*(.taglist.init)
__tagtable_end = .;
#ifdef CONFIG_SMP_ON_UP
__smpalt_begin = .;
*(.alt.smp.init)
__smpalt_end = .;
#endif
__pv_table_begin = .;
*(.pv_table)
__pv_table_end = .;
INIT_SETUP(16)
INIT_CALLS
CON_INITCALL
SECURITY_INITCALL
INIT_RAM_FS
#ifndef CONFIG_XIP_KERNEL
__init_begin = _stext;
INIT_DATA
ARM_EXIT_KEEP(EXIT_DATA)
#endif
}
PERCPU_SECTION(32)
#ifndef CONFIG_XIP_KERNEL
. = ALIGN(PAGE_SIZE);
__init_end = .;
#endif
/*
* unwind exit sections must be discarded before the rest of the
* unwind sections get included.
@ -105,11 +54,23 @@ SECTIONS
#ifndef CONFIG_MMU
*(.fixup)
*(__ex_table)
#endif
#ifndef CONFIG_SMP_ON_UP
*(.alt.smp.init)
#endif
}
#ifdef CONFIG_XIP_KERNEL
. = XIP_VIRT_ADDR(CONFIG_XIP_PHYS_ADDR);
#else
. = PAGE_OFFSET + TEXT_OFFSET;
#endif
.head.text : {
_text = .;
HEAD_TEXT
}
.text : { /* Real text segment */
_text = .; /* Text and read-only data */
_stext = .; /* Text and read-only data */
__exception_text_start = .;
*(.exception.text)
__exception_text_end = .;
@ -122,8 +83,6 @@ SECTIONS
*(.fixup)
#endif
*(.gnu.warning)
*(.rodata)
*(.rodata.*)
*(.glue_7)
*(.glue_7t)
. = ALIGN(4);
@ -152,10 +111,63 @@ SECTIONS
_etext = .; /* End of text and rodata section */
#ifndef CONFIG_XIP_KERNEL
. = ALIGN(PAGE_SIZE);
__init_begin = .;
#endif
INIT_TEXT_SECTION(8)
.exit.text : {
ARM_EXIT_KEEP(EXIT_TEXT)
}
.init.proc.info : {
ARM_CPU_DISCARD(PROC_INFO)
}
.init.arch.info : {
__arch_info_begin = .;
*(.arch.info.init)
__arch_info_end = .;
}
.init.tagtable : {
__tagtable_begin = .;
*(.taglist.init)
__tagtable_end = .;
}
#ifdef CONFIG_SMP_ON_UP
.init.smpalt : {
__smpalt_begin = .;
*(.alt.smp.init)
__smpalt_end = .;
}
#endif
.init.pv_table : {
__pv_table_begin = .;
*(.pv_table)
__pv_table_end = .;
}
.init.data : {
#ifndef CONFIG_XIP_KERNEL
INIT_DATA
#endif
INIT_SETUP(16)
INIT_CALLS
CON_INITCALL
SECURITY_INITCALL
INIT_RAM_FS
}
#ifndef CONFIG_XIP_KERNEL
.exit.data : {
ARM_EXIT_KEEP(EXIT_DATA)
}
#endif
PERCPU_SECTION(32)
#ifdef CONFIG_XIP_KERNEL
__data_loc = ALIGN(4); /* location in binary */
. = PAGE_OFFSET + TEXT_OFFSET;
#else
__init_end = .;
. = ALIGN(THREAD_SIZE);
__data_loc = .;
#endif
@ -270,12 +282,6 @@ SECTIONS
/* Default discards */
DISCARDS
#ifndef CONFIG_SMP_ON_UP
/DISCARD/ : {
*(.alt.smp.init)
}
#endif
}
/*

View File

@ -80,7 +80,3 @@
.macro arch_ret_to_user, tmp1, tmp2
.endm
.macro irq_prio_table
.endm

View File

@ -681,4 +681,5 @@ MACHINE_START(DAVINCI_DA830_EVM, "DaVinci DA830/OMAP-L137/AM17x EVM")
.init_irq = cp_intc_init,
.timer = &davinci_timer,
.init_machine = da830_evm_init,
.dma_zone_size = SZ_128M,
MACHINE_END

View File

@ -1261,4 +1261,5 @@ MACHINE_START(DAVINCI_DA850_EVM, "DaVinci DA850/OMAP-L138/AM18x EVM")
.init_irq = cp_intc_init,
.timer = &davinci_timer,
.init_machine = da850_evm_init,
.dma_zone_size = SZ_128M,
MACHINE_END

View File

@ -356,4 +356,5 @@ MACHINE_START(DAVINCI_DM355_EVM, "DaVinci DM355 EVM")
.init_irq = davinci_irq_init,
.timer = &davinci_timer,
.init_machine = dm355_evm_init,
.dma_zone_size = SZ_128M,
MACHINE_END

View File

@ -275,4 +275,5 @@ MACHINE_START(DM355_LEOPARD, "DaVinci DM355 leopard")
.init_irq = davinci_irq_init,
.timer = &davinci_timer,
.init_machine = dm355_leopard_init,
.dma_zone_size = SZ_128M,
MACHINE_END

View File

@ -617,5 +617,6 @@ MACHINE_START(DAVINCI_DM365_EVM, "DaVinci DM365 EVM")
.init_irq = davinci_irq_init,
.timer = &davinci_timer,
.init_machine = dm365_evm_init,
.dma_zone_size = SZ_128M,
MACHINE_END

View File

@ -717,4 +717,5 @@ MACHINE_START(DAVINCI_EVM, "DaVinci DM644x EVM")
.init_irq = davinci_irq_init,
.timer = &davinci_timer,
.init_machine = davinci_evm_init,
.dma_zone_size = SZ_128M,
MACHINE_END

View File

@ -802,6 +802,7 @@ MACHINE_START(DAVINCI_DM6467_EVM, "DaVinci DM646x EVM")
.init_irq = davinci_irq_init,
.timer = &davinci_timer,
.init_machine = evm_init,
.dma_zone_size = SZ_128M,
MACHINE_END
MACHINE_START(DAVINCI_DM6467TEVM, "DaVinci DM6467T EVM")
@ -810,5 +811,6 @@ MACHINE_START(DAVINCI_DM6467TEVM, "DaVinci DM6467T EVM")
.init_irq = davinci_irq_init,
.timer = &davinci_timer,
.init_machine = evm_init,
.dma_zone_size = SZ_128M,
MACHINE_END

View File

@ -571,4 +571,5 @@ MACHINE_START(MITYOMAPL138, "MityDSP-L138/MityARM-1808")
.init_irq = cp_intc_init,
.timer = &davinci_timer,
.init_machine = mityomapl138_init,
.dma_zone_size = SZ_128M,
MACHINE_END

View File

@ -277,4 +277,5 @@ MACHINE_START(NEUROS_OSD2, "Neuros OSD2")
.init_irq = davinci_irq_init,
.timer = &davinci_timer,
.init_machine = davinci_ntosd2_init,
.dma_zone_size = SZ_128M,
MACHINE_END

View File

@ -343,4 +343,5 @@ MACHINE_START(OMAPL138_HAWKBOARD, "AM18x/OMAP-L138 Hawkboard")
.init_irq = cp_intc_init,
.timer = &davinci_timer,
.init_machine = omapl138_hawk_init,
.dma_zone_size = SZ_128M,
MACHINE_END

View File

@ -156,4 +156,5 @@ MACHINE_START(SFFSDR, "Lyrtech SFFSDR")
.init_irq = davinci_irq_init,
.timer = &davinci_timer,
.init_machine = davinci_sffsdr_init,
.dma_zone_size = SZ_128M,
MACHINE_END

View File

@ -282,4 +282,5 @@ MACHINE_START(TNETV107X, "TNETV107X EVM")
.init_irq = cp_intc_init,
.timer = &davinci_timer,
.init_machine = tnetv107x_evm_board_init,
.dma_zone_size = SZ_128M,
MACHINE_END

View File

@ -46,6 +46,3 @@
#endif
1002:
.endm
.macro irq_prio_table
.endm

View File

@ -41,11 +41,4 @@
*/
#define CONSISTENT_DMA_SIZE (14<<20)
/*
* Restrict DMA-able region to workaround silicon bug. The bug
* restricts buffers available for DMA to video hardware to be
* below 128M
*/
#define ARM_DMA_ZONE_SIZE SZ_128M
#endif /* __ASM_ARCH_MEMORY_H */

View File

@ -154,14 +154,6 @@ void __init smp_init_cpus(void)
void __init platform_smp_prepare_cpus(unsigned int max_cpus)
{
int i;
/*
* Initialise the present map, which describes the set of CPUs
* actually populated at the present time.
*/
for (i = 0; i < max_cpus; i++)
set_cpu_present(i, true);
scu_enable(scu_base_addr());

View File

@ -280,7 +280,7 @@ static struct sleep_save exynos4_l2cc_save[] = {
SAVE_ITEM(S5P_VA_L2CC + L2X0_AUX_CTRL),
};
void exynos4_cpu_suspend(void)
static int exynos4_cpu_suspend(unsigned long arg)
{
unsigned long tmp;
unsigned long mask = 0xFFFFFFFF;

View File

@ -32,28 +32,6 @@
.text
/*
* s3c_cpu_save
*
* entry:
* r1 = v:p offset
*/
ENTRY(s3c_cpu_save)
stmfd sp!, { r3 - r12, lr }
ldr r3, =resume_with_mmu
bl cpu_suspend
ldr r0, =pm_cpu_sleep
ldr r0, [ r0 ]
mov pc, r0
resume_with_mmu:
ldmfd sp!, { r3 - r12, pc }
.ltorg
/*
* sleep magic, to allow the bootloader to check for an valid
* image to resume to. Must be the first word before the

View File

@ -33,4 +33,5 @@ MACHINE_START(H7201, "Hynix GMS30C7201")
.map_io = h720x_map_io,
.init_irq = h720x_init_irq,
.timer = &h7201_timer,
.dma_zone_size = SZ_256M,
MACHINE_END

View File

@ -76,4 +76,5 @@ MACHINE_START(H7202, "Hynix HMS30C7202")
.init_irq = h7202_init_irq,
.timer = &h7202_timer,
.init_machine = init_eval_h7202,
.dma_zone_size = SZ_256M,
MACHINE_END

View File

@ -57,9 +57,6 @@
tst \irqstat, #1 @ bit 0 should be set
.endm
.macro irq_prio_table
.endm
#else
#error hynix processor selection missmatch
#endif

View File

@ -8,11 +8,4 @@
#define __ASM_ARCH_MEMORY_H
#define PLAT_PHYS_OFFSET UL(0x40000000)
/*
* This is the maximum DMA address that can be DMAd to.
* There should not be more than (0xd0000000 - 0xc0000000)
* bytes of RAM.
*/
#define ARM_DMA_ZONE_SIZE SZ_256M
#endif

View File

@ -169,6 +169,9 @@ MACHINE_START(AVILA, "Gateworks Avila Network Platform")
.timer = &ixp4xx_timer,
.boot_params = 0x0100,
.init_machine = avila_init,
#if defined(CONFIG_PCI)
.dma_zone_size = SZ_64M,
#endif
MACHINE_END
/*
@ -184,6 +187,9 @@ MACHINE_START(LOFT, "Giant Shoulder Inc Loft board")
.timer = &ixp4xx_timer,
.boot_params = 0x0100,
.init_machine = avila_init,
#if defined(CONFIG_PCI)
.dma_zone_size = SZ_64M,
#endif
MACHINE_END
#endif

View File

@ -316,6 +316,11 @@ static int abort_handler(unsigned long addr, unsigned int fsr, struct pt_regs *r
}
static int ixp4xx_needs_bounce(struct device *dev, dma_addr_t dma_addr, size_t size)
{
return (dma_addr + size) >= SZ_64M;
}
/*
* Setup DMA mask to 64MB on PCI devices. Ignore all other devices.
*/
@ -324,7 +329,7 @@ static int ixp4xx_pci_platform_notify(struct device *dev)
if(dev->bus == &pci_bus_type) {
*dev->dma_mask = SZ_64M - 1;
dev->coherent_dma_mask = SZ_64M - 1;
dmabounce_register_dev(dev, 2048, 4096);
dmabounce_register_dev(dev, 2048, 4096, ixp4xx_needs_bounce);
}
return 0;
}
@ -337,11 +342,6 @@ static int ixp4xx_pci_platform_notify_remove(struct device *dev)
return 0;
}
int dma_needs_bounce(struct device *dev, dma_addr_t dma_addr, size_t size)
{
return (dev->bus == &pci_bus_type ) && ((dma_addr + size) >= SZ_64M);
}
void __init ixp4xx_pci_preinit(void)
{
unsigned long cpuid = read_cpuid_id();

View File

@ -114,6 +114,9 @@ MACHINE_START(ADI_COYOTE, "ADI Engineering Coyote")
.timer = &ixp4xx_timer,
.boot_params = 0x0100,
.init_machine = coyote_init,
#if defined(CONFIG_PCI)
.dma_zone_size = SZ_64M,
#endif
MACHINE_END
#endif

View File

@ -284,4 +284,7 @@ MACHINE_START(DSMG600, "D-Link DSM-G600 RevA")
.init_irq = ixp4xx_init_irq,
.timer = &dsmg600_timer,
.init_machine = dsmg600_init,
#if defined(CONFIG_PCI)
.dma_zone_size = SZ_64M,
#endif
MACHINE_END

View File

@ -275,5 +275,8 @@ MACHINE_START(FSG, "Freecom FSG-3")
.timer = &ixp4xx_timer,
.boot_params = 0x0100,
.init_machine = fsg_init,
#if defined(CONFIG_PCI)
.dma_zone_size = SZ_64M,
#endif
MACHINE_END

View File

@ -101,5 +101,8 @@ MACHINE_START(GATEWAY7001, "Gateway 7001 AP")
.timer = &ixp4xx_timer,
.boot_params = 0x0100,
.init_machine = gateway7001_init,
#if defined(CONFIG_PCI)
.dma_zone_size = SZ_64M,
#endif
MACHINE_END
#endif

View File

@ -501,4 +501,7 @@ MACHINE_START(GORAMO_MLR, "MultiLink")
.timer = &ixp4xx_timer,
.boot_params = 0x0100,
.init_machine = gmlr_init,
#if defined(CONFIG_PCI)
.dma_zone_size = SZ_64M,
#endif
MACHINE_END

View File

@ -169,6 +169,9 @@ MACHINE_START(GTWX5715, "Gemtek GTWX5715 (Linksys WRV54G)")
.timer = &ixp4xx_timer,
.boot_params = 0x0100,
.init_machine = gtwx5715_init,
#if defined(CONFIG_PCI)
.dma_zone_size = SZ_64M,
#endif
MACHINE_END

View File

@ -14,8 +14,4 @@
*/
#define PLAT_PHYS_OFFSET UL(0x00000000)
#ifdef CONFIG_PCI
#define ARM_DMA_ZONE_SIZE SZ_64M
#endif
#endif

View File

@ -258,6 +258,9 @@ MACHINE_START(IXDP425, "Intel IXDP425 Development Platform")
.timer = &ixp4xx_timer,
.boot_params = 0x0100,
.init_machine = ixdp425_init,
#if defined(CONFIG_PCI)
.dma_zone_size = SZ_64M,
#endif
MACHINE_END
#endif
@ -269,6 +272,9 @@ MACHINE_START(IXDP465, "Intel IXDP465 Development Platform")
.timer = &ixp4xx_timer,
.boot_params = 0x0100,
.init_machine = ixdp425_init,
#if defined(CONFIG_PCI)
.dma_zone_size = SZ_64M,
#endif
MACHINE_END
#endif
@ -280,6 +286,9 @@ MACHINE_START(IXCDP1100, "Intel IXCDP1100 Development Platform")
.timer = &ixp4xx_timer,
.boot_params = 0x0100,
.init_machine = ixdp425_init,
#if defined(CONFIG_PCI)
.dma_zone_size = SZ_64M,
#endif
MACHINE_END
#endif
@ -291,5 +300,8 @@ MACHINE_START(KIXRP435, "Intel KIXRP435 Reference Platform")
.timer = &ixp4xx_timer,
.boot_params = 0x0100,
.init_machine = ixdp425_init,
#if defined(CONFIG_PCI)
.dma_zone_size = SZ_64M,
#endif
MACHINE_END
#endif

View File

@ -319,4 +319,7 @@ MACHINE_START(NAS100D, "Iomega NAS 100d")
.init_irq = ixp4xx_init_irq,
.timer = &ixp4xx_timer,
.init_machine = nas100d_init,
#if defined(CONFIG_PCI)
.dma_zone_size = SZ_64M,
#endif
MACHINE_END

View File

@ -305,4 +305,7 @@ MACHINE_START(NSLU2, "Linksys NSLU2")
.init_irq = ixp4xx_init_irq,
.timer = &nslu2_timer,
.init_machine = nslu2_init,
#if defined(CONFIG_PCI)
.dma_zone_size = SZ_64M,
#endif
MACHINE_END

View File

@ -241,4 +241,7 @@ MACHINE_START(ARCOM_VULCAN, "Arcom/Eurotech Vulcan")
.timer = &ixp4xx_timer,
.boot_params = 0x0100,
.init_machine = vulcan_init,
#if defined(CONFIG_PCI)
.dma_zone_size = SZ_64M,
#endif
MACHINE_END

View File

@ -102,5 +102,8 @@ MACHINE_START(WG302V2, "Netgear WG302 v2 / WAG302 v2")
.timer = &ixp4xx_timer,
.boot_params = 0x0100,
.init_machine = wg302v2_init,
#if defined(CONFIG_PCI)
.dma_zone_size = SZ_64M,
#endif
MACHINE_END
#endif

View File

@ -41,7 +41,3 @@
rsb \irqnr, \irqnr, #31
teq \irqstat, #0
.endm
.macro irq_prio_table
.endm

View File

@ -157,12 +157,4 @@ void __init smp_init_cpus(void)
void __init platform_smp_prepare_cpus(unsigned int max_cpus)
{
int i;
/*
* Initialise the present map, which describes the set of CPUs
* actually populated at the present time.
*/
for (i = 0; i < max_cpus; i++)
set_cpu_present(i, true);
}

View File

@ -286,14 +286,15 @@ void omap3_save_scratchpad_contents(void)
scratchpad_contents.boot_config_ptr = 0x0;
if (cpu_is_omap3630())
scratchpad_contents.public_restore_ptr =
virt_to_phys(get_omap3630_restore_pointer());
virt_to_phys(omap3_restore_3630);
else if (omap_rev() != OMAP3430_REV_ES3_0 &&
omap_rev() != OMAP3430_REV_ES3_1)
scratchpad_contents.public_restore_ptr =
virt_to_phys(get_restore_pointer());
virt_to_phys(omap3_restore);
else
scratchpad_contents.public_restore_ptr =
virt_to_phys(get_es3_restore_pointer());
virt_to_phys(omap3_restore_es3);
if (omap_type() == OMAP2_DEVICE_TYPE_GP)
scratchpad_contents.secure_ram_restore_ptr = 0x0;
else

Some files were not shown because too many files have changed in this diff Show More