hollalinux
/
linux-sg2042
mirror of https://github.com/sophgo/linux-riscv.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
linux-sg2042/arch/arm/mm/proc-macros.S

388 lines
9.1 KiB
ArmAsm
Raw Normal View History

License cleanup: add SPDX GPL-2.0 license identifier to files with no license Many source files in the tree are missing licensing information, which makes it harder for compliance tools to determine the correct license. By default all files without license information are under the default license of the kernel, which is GPL version 2. Update the files which contain no license information with the 'GPL-2.0' SPDX license identifier. The SPDX identifier is a legally binding shorthand, which can be used instead of the full boiler plate text. This patch is based on work done by Thomas Gleixner and Kate Stewart and Philippe Ombredanne. How this work was done: Patches were generated and checked against linux-4.14-rc6 for a subset of the use cases: - file had no licensing information it it. - file was a */uapi/* one with no licensing information in it, - file was a */uapi/* one with existing licensing information, Further patches will be generated in subsequent months to fix up cases where non-standard license headers were used, and references to license had to be inferred by heuristics based on keywords. The analysis to determine which SPDX License Identifier to be applied to a file was done in a spreadsheet of side by side results from of the output of two independent scanners (ScanCode & Windriver) producing SPDX tag:value files created by Philippe Ombredanne. Philippe prepared the base worksheet, and did an initial spot review of a few 1000 files. The 4.13 kernel was the starting point of the analysis with 60,537 files assessed. Kate Stewart did a file by file comparison of the scanner results in the spreadsheet to determine which SPDX license identifier(s) to be applied to the file. She confirmed any determination that was not immediately clear with lawyers working with the Linux Foundation. Criteria used to select files for SPDX license identifier tagging was: - Files considered eligible had to be source code files. - Make and config files were included as candidates if they contained >5 lines of source - File already had some variant of a license header in it (even if <5 lines). All documentation files were explicitly excluded. The following heuristics were used to determine which SPDX license identifiers to apply. - when both scanners couldn't find any license traces, file was considered to have no license information in it, and the top level COPYING file license applied. For non */uapi/* files that summary was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 11139 and resulted in the first patch in this series. If that file was a */uapi/* path one, it was "GPL-2.0 WITH Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 WITH Linux-syscall-note 930 and resulted in the second patch in this series. - if a file had some form of licensing information in it, and was one of the */uapi/* ones, it was denoted with the Linux-syscall-note if any GPL family license was found in the file or had no licensing in it (per prior point). Results summary: SPDX license identifier # files ---------------------------------------------------|------ GPL-2.0 WITH Linux-syscall-note 270 GPL-2.0+ WITH Linux-syscall-note 169 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17 LGPL-2.1+ WITH Linux-syscall-note 15 GPL-1.0+ WITH Linux-syscall-note 14 ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5 LGPL-2.0+ WITH Linux-syscall-note 4 LGPL-2.1 WITH Linux-syscall-note 3 ((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3 ((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1 and that resulted in the third patch in this series. - when the two scanners agreed on the detected license(s), that became the concluded license(s). - when there was disagreement between the two scanners (one detected a license but the other didn't, or they both detected different licenses) a manual inspection of the file occurred. - In most cases a manual inspection of the information in the file resulted in a clear resolution of the license that should apply (and which scanner probably needed to revisit its heuristics). - When it was not immediately clear, the license identifier was confirmed with lawyers working with the Linux Foundation. - If there was any question as to the appropriate license identifier, the file was flagged for further research and to be revisited later in time. In total, over 70 hours of logged manual review was done on the spreadsheet to determine the SPDX license identifiers to apply to the source files by Kate, Philippe, Thomas and, in some cases, confirmation by lawyers working with the Linux Foundation. Kate also obtained a third independent scan of the 4.13 code base from FOSSology, and compared selected files where the other two scanners disagreed against that SPDX file, to see if there was new insights. The Windriver scanner is based on an older version of FOSSology in part, so they are related. Thomas did random spot checks in about 500 files from the spreadsheets for the uapi headers and agreed with SPDX license identifier in the files he inspected. For the non-uapi files Thomas did random spot checks in about 15000 files. In initial set of patches against 4.14-rc6, 3 files were found to have copy/paste license identifier errors, and have been fixed to reflect the correct identifier. Additionally Philippe spent 10 hours this week doing a detailed manual inspection and review of the 12,461 patched files from the initial patch version early this week with: - a full scancode scan run, collecting the matched texts, detected license ids and scores - reviewing anything where there was a license detected (about 500+ files) to ensure that the applied SPDX license was correct - reviewing anything where there was no detection but the patch license was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied SPDX license was correct This produced a worksheet with 20 files needing minor correction. This worksheet was then exported into 3 different .csv files for the different types of files to be modified. These .csv files were then reviewed by Greg. Thomas wrote a script to parse the csv files and add the proper SPDX tag to the file, in the format that the file expected. This script was further refined by Greg based on the output to detect more types of files automatically and to distinguish between header and source .c files (which need different comment types.) Finally Greg ran the script using the .csv files to generate the patches. Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org> Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-11-01 22:07:57 +08:00
/* SPDX-License-Identifier: GPL-2.0 */
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
/*
* We need constants.h for:
* VMA_VM_MM
* VMA_VM_FLAGS
* VM_EXEC
*/
kbuild: arm - use generic asm-offsets.h support Delete obsoleted stuff from arch Makefile and rename constants.h to asm-offsets.h Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
2005-09-10 03:08:59 +08:00
#include <asm/asm-offsets.h>
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
#include <asm/thread_info.h>
ARM: 8606/1: V7M: introduce cache operations This commit implements the cache operation for V7M. It is based on V7 counterpart and differs as follows: - cache operations are memory mapped - only Thumb instruction set is supported - we don't handle user access faults Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com> Tested-by: Andras Szemzo <sza@esh.hu> Tested-by: Joachim Eastwood <manabian@gmail.com> Tested-by: Alexandre TORGUE <alexandre.torgue@st.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2016-08-31 00:30:02 +08:00
#ifdef CONFIG_CPU_V7M
#include <asm/v7m.h>
#endif
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
/*
* vma_vm_mm - get mm pointer from vma pointer (vma->vm_mm)
*/
.macro vma_vm_mm, rd, rn
ldr \rd, [\rn, #VMA_VM_MM]
.endm
/*
* vma_vm_flags - get vma->vm_flags
*/
.macro vma_vm_flags, rd, rn
ldr \rd, [\rn, #VMA_VM_FLAGS]
.endm
/*
* act_mm - get current->active_mm
*/
.macro act_mm, rd
bic \rd, sp, #8128
bic \rd, \rd, #63
ldr \rd, [\rd, #TI_TASK]
ARM: Prepare for randomized task_struct With the new task struct randomization, we can run into a build failure for certain random seeds, which will place fields beyond the allow immediate size in the assembly: arch/arm/kernel/entry-armv.S: Assembler messages: arch/arm/kernel/entry-armv.S:803: Error: bad immediate value for offset (4096) Only two constants in asm-offset.h are affected, and I'm changing both of them here to work correctly in all configurations. One more macro has the problem, but is currently unused, so this removes it instead of adding complexity. Suggested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Arnd Bergmann <arnd@arndb.de> [kees: Adjust commit log slightly] Signed-off-by: Kees Cook <keescook@chromium.org>
2017-07-01 00:03:59 +08:00
.if (TSK_ACTIVE_MM > IMM12_MASK)
add \rd, \rd, #TSK_ACTIVE_MM & ~IMM12_MASK
.endif
ldr \rd, [\rd, #TSK_ACTIVE_MM & IMM12_MASK]
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
.endm
/*
* mmid - get context id from mm pointer (mm->context.id)
ARM: 7649/1: mm: mm->context.id fix for big-endian Since the new ASID code in b5466f8728527a05a493cc4abe9e6f034a1bbaab ("ARM: mm: remove IPI broadcasting on ASID rollover") was changed to use 64bit operations it has broken the BE operation due to an issue with the MM code accessing sub-fields of mm->context.id. When running in BE mode we see the values in mm->context.id are stored with the highest value first, so the LDR in the arch/arm/mm/proc-macros.S reads the wrong part of this field. To resolve this, change the LDR in the mmid macro to load from +4. Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Ben Dooks <ben.dooks@codethink.co.uk> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-02-11 19:25:06 +08:00
* note, this field is 64bit, so in big-endian the two words are swapped too.
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
*/
.macro mmid, rd, rn
ARM: 7649/1: mm: mm->context.id fix for big-endian Since the new ASID code in b5466f8728527a05a493cc4abe9e6f034a1bbaab ("ARM: mm: remove IPI broadcasting on ASID rollover") was changed to use 64bit operations it has broken the BE operation due to an issue with the MM code accessing sub-fields of mm->context.id. When running in BE mode we see the values in mm->context.id are stored with the highest value first, so the LDR in the arch/arm/mm/proc-macros.S reads the wrong part of this field. To resolve this, change the LDR in the mmid macro to load from +4. Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Ben Dooks <ben.dooks@codethink.co.uk> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-02-11 19:25:06 +08:00
#ifdef __ARMEB__
ldr \rd, [\rn, #MM_CONTEXT_ID + 4 ]
#else
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
ldr \rd, [\rn, #MM_CONTEXT_ID]
ARM: 7649/1: mm: mm->context.id fix for big-endian Since the new ASID code in b5466f8728527a05a493cc4abe9e6f034a1bbaab ("ARM: mm: remove IPI broadcasting on ASID rollover") was changed to use 64bit operations it has broken the BE operation due to an issue with the MM code accessing sub-fields of mm->context.id. When running in BE mode we see the values in mm->context.id are stored with the highest value first, so the LDR in the arch/arm/mm/proc-macros.S reads the wrong part of this field. To resolve this, change the LDR in the mmid macro to load from +4. Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Ben Dooks <ben.dooks@codethink.co.uk> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-02-11 19:25:06 +08:00
#endif
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
.endm
/*
* mask_asid - mask the ASID from the context ID
*/
.macro asid, rd, rn
and \rd, \rn, #255
.endm
[ARM] nommu: provide a way for correct control register value selection Most MMU-based CPUs have a restriction on the setting of the data cache enable and mmu enable bits in the control register, whereby if the data cache is enabled, the MMU must also be enabled. Enabling the data cache without the MMU is an invalid combination. However, there are CPUs where the data cache can be enabled without the MMU. In order to allow these CPUs to take advantage of that, provide a method whereby each proc-*.S file defines the control regsiter value for use with nommu (with the MMU disabled.) Later on, when we add support for enabling the MMU on these devices, we can adjust the "crval" macro to also enable the data cache for nommu. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2006-06-29 22:09:57 +08:00
.macro crval, clear, mmuset, ucset
#ifdef CONFIG_MMU
.word \clear
.word \mmuset
#else
.word \clear
.word \ucset
#endif
.endm
[ARM] armv7: add support for ARMv7 cores. This patch adds support for the ARMv7 cores. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2007-05-09 05:27:46 +08:00
/*
ARM: 6527/1: Use CTR instead of CCSIDR for the D-cache line size on ARMv7 The current implementation of the dcache_line_size macro reads the L1 cache size from the CCSIDR register. This, however, is not guaranteed to be the smallest cache line in the cache hierarchy. The patch changes to the macro to use the more architecturally correct CTR register. Reported-by: Kevin Sapp <ksapp@quicinc.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-12-07 23:52:04 +08:00
* dcache_line_size - get the minimum D-cache line size from the CTR register
* on ARMv7.
[ARM] armv7: add support for ARMv7 cores. This patch adds support for the ARMv7 cores. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2007-05-09 05:27:46 +08:00
*/
.macro dcache_line_size, reg, tmp
ARM: 8606/1: V7M: introduce cache operations This commit implements the cache operation for V7M. It is based on V7 counterpart and differs as follows: - cache operations are memory mapped - only Thumb instruction set is supported - we don't handle user access faults Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com> Tested-by: Andras Szemzo <sza@esh.hu> Tested-by: Joachim Eastwood <manabian@gmail.com> Tested-by: Alexandre TORGUE <alexandre.torgue@st.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2016-08-31 00:30:02 +08:00
#ifdef CONFIG_CPU_V7M
movw \tmp, #:lower16:BASEADDR_V7M_SCB + V7M_SCB_CTR
movt \tmp, #:upper16:BASEADDR_V7M_SCB + V7M_SCB_CTR
ldr \tmp, [\tmp]
#else
ARM: 6527/1: Use CTR instead of CCSIDR for the D-cache line size on ARMv7 The current implementation of the dcache_line_size macro reads the L1 cache size from the CCSIDR register. This, however, is not guaranteed to be the smallest cache line in the cache hierarchy. The patch changes to the macro to use the more architecturally correct CTR register. Reported-by: Kevin Sapp <ksapp@quicinc.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-12-07 23:52:04 +08:00
mrc p15, 0, \tmp, c0, c0, 1 @ read ctr
ARM: 8606/1: V7M: introduce cache operations This commit implements the cache operation for V7M. It is based on V7 counterpart and differs as follows: - cache operations are memory mapped - only Thumb instruction set is supported - we don't handle user access faults Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com> Tested-by: Andras Szemzo <sza@esh.hu> Tested-by: Joachim Eastwood <manabian@gmail.com> Tested-by: Alexandre TORGUE <alexandre.torgue@st.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2016-08-31 00:30:02 +08:00
#endif
ARM: 6527/1: Use CTR instead of CCSIDR for the D-cache line size on ARMv7 The current implementation of the dcache_line_size macro reads the L1 cache size from the CCSIDR register. This, however, is not guaranteed to be the smallest cache line in the cache hierarchy. The patch changes to the macro to use the more architecturally correct CTR register. Reported-by: Kevin Sapp <ksapp@quicinc.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-12-07 23:52:04 +08:00
lsr \tmp, \tmp, #16
and \tmp, \tmp, #0xf @ cache line size encoding
mov \reg, #4 @ bytes per word
[ARM] armv7: add support for ARMv7 cores. This patch adds support for the ARMv7 cores. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2007-05-09 05:27:46 +08:00
mov \reg, \reg, lsl \tmp @ actual cache line size
.endm
[ARM] Convert set_pte_ext implementions to macros There are actually only four separate implementations of set_pte_ext. Use assembler macros to insert code for these into the proc-*.S files. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 00:19:08 +08:00
ARM: 6528/1: Use CTR for the I-cache line size on ARMv7 The current implementation of the v7_coherent_*_range function assumes that the D and I cache lines have the same size, which is incorrect architecturally. This patch adds the icache_line_size macro which reads the CTR register. The main loop in v7_coherent_*_range is split in two independent loops or the D and I caches. This also has the performance advantage that the DSB is moved outside the main loop. Reported-by: Kevin Sapp <ksapp@quicinc.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-12-07 23:56:29 +08:00
/*
* icache_line_size - get the minimum I-cache line size from the CTR register
* on ARMv7.
*/
.macro icache_line_size, reg, tmp
ARM: 8606/1: V7M: introduce cache operations This commit implements the cache operation for V7M. It is based on V7 counterpart and differs as follows: - cache operations are memory mapped - only Thumb instruction set is supported - we don't handle user access faults Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com> Tested-by: Andras Szemzo <sza@esh.hu> Tested-by: Joachim Eastwood <manabian@gmail.com> Tested-by: Alexandre TORGUE <alexandre.torgue@st.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2016-08-31 00:30:02 +08:00
#ifdef CONFIG_CPU_V7M
movw \tmp, #:lower16:BASEADDR_V7M_SCB + V7M_SCB_CTR
movt \tmp, #:upper16:BASEADDR_V7M_SCB + V7M_SCB_CTR
ldr \tmp, [\tmp]
#else
ARM: 6528/1: Use CTR for the I-cache line size on ARMv7 The current implementation of the v7_coherent_*_range function assumes that the D and I cache lines have the same size, which is incorrect architecturally. This patch adds the icache_line_size macro which reads the CTR register. The main loop in v7_coherent_*_range is split in two independent loops or the D and I caches. This also has the performance advantage that the DSB is moved outside the main loop. Reported-by: Kevin Sapp <ksapp@quicinc.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-12-07 23:56:29 +08:00
mrc p15, 0, \tmp, c0, c0, 1 @ read ctr
ARM: 8606/1: V7M: introduce cache operations This commit implements the cache operation for V7M. It is based on V7 counterpart and differs as follows: - cache operations are memory mapped - only Thumb instruction set is supported - we don't handle user access faults Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com> Tested-by: Andras Szemzo <sza@esh.hu> Tested-by: Joachim Eastwood <manabian@gmail.com> Tested-by: Alexandre TORGUE <alexandre.torgue@st.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2016-08-31 00:30:02 +08:00
#endif
ARM: 6528/1: Use CTR for the I-cache line size on ARMv7 The current implementation of the v7_coherent_*_range function assumes that the D and I cache lines have the same size, which is incorrect architecturally. This patch adds the icache_line_size macro which reads the CTR register. The main loop in v7_coherent_*_range is split in two independent loops or the D and I caches. This also has the performance advantage that the DSB is moved outside the main loop. Reported-by: Kevin Sapp <ksapp@quicinc.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-12-07 23:56:29 +08:00
and \tmp, \tmp, #0xf @ cache line size encoding
mov \reg, #4 @ bytes per word
mov \reg, \reg, lsl \tmp @ actual cache line size
.endm
[ARM] Convert set_pte_ext implementions to macros There are actually only four separate implementations of set_pte_ext. Use assembler macros to insert code for these into the proc-*.S files. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 00:19:08 +08:00
/*
* Sanity check the PTE configuration for the code below - which makes
Fix common misspellings Fixes generated by 'codespell' and manually reviewed. Signed-off-by: Lucas De Marchi <lucas.demarchi@profusion.mobi>
2011-03-31 09:57:33 +08:00
* certain assumptions about how these bits are laid out.
[ARM] Convert set_pte_ext implementions to macros There are actually only four separate implementations of set_pte_ext. Use assembler macros to insert code for these into the proc-*.S files. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 00:19:08 +08:00
*/
nommu: Add #ifdef CONFIG_MMU around the PTE sanity checks Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2009-07-24 19:35:04 +08:00
#ifdef CONFIG_MMU
[ARM] Convert set_pte_ext implementions to macros There are actually only four separate implementations of set_pte_ext. Use assembler macros to insert code for these into the proc-*.S files. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 00:19:08 +08:00
#if L_PTE_SHARED != PTE_EXT_SHARED
#error PTE shared bit mismatch
#endif
ARM: LPAE: MMU setup for the 3-level page table format This patch adds the MMU initialisation for the LPAE page table format. The swapper_pg_dir size with LPAE is 5 rather than 4 pages. A new proc-v7-3level.S file contains the TTB initialisation, context switch and PTE setting code with the LPAE. The TTBRx split is based on the PAGE_OFFSET with TTBR1 used for the kernel mappings. The 36-bit mappings (supersections) and a few other memory types in mmu.c are conditionally compiled. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2011-11-23 01:30:29 +08:00
#if !defined (CONFIG_ARM_LPAE) && \
(L_PTE_XN+L_PTE_USER+L_PTE_RDONLY+L_PTE_DIRTY+L_PTE_YOUNG+\
arm: drop L_PTE_FILE and pte_file()-related helpers We've replaced remap_file_pages(2) implementation with emulation. Nobody creates non-linear mapping anymore. This patch also adjust __SWP_TYPE_SHIFT, effectively increase size of possible swap file to 128G. Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: Russell King <linux@arm.linux.org.uk> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-02-11 06:10:17 +08:00
L_PTE_PRESENT) > L_PTE_SHARED
[ARM] Convert set_pte_ext implementions to macros There are actually only four separate implementations of set_pte_ext. Use assembler macros to insert code for these into the proc-*.S files. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 00:19:08 +08:00
#error Invalid Linux PTE bit settings
#endif
nommu: Add #ifdef CONFIG_MMU around the PTE sanity checks Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2009-07-24 19:35:04 +08:00
#endif /* CONFIG_MMU */
[ARM] Convert set_pte_ext implementions to macros There are actually only four separate implementations of set_pte_ext. Use assembler macros to insert code for these into the proc-*.S files. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 00:19:08 +08:00
/*
* The ARMv6 and ARMv7 set_pte_ext translation function.
*
* Permission translation:
* YUWD APX AP1 AP0 SVC User
* 0xxx 0 0 0 no acc no acc
* 100x 1 0 1 r/o no acc
* 10x0 1 0 1 r/o no acc
* 1011 0 0 1 r/w no acc
ARM: 6384/1: Remove the domain switching on ARMv6k/v7 CPUs This patch removes the domain switching functionality via the set_fs and __switch_to functions on cores that have a TLS register. Currently, the ioremap and vmalloc areas share the same level 1 page tables and therefore have the same domain (DOMAIN_KERNEL). When the kernel domain is modified from Client to Manager (via the __set_fs or in the __switch_to function), the XN (eXecute Never) bit is overridden and newer CPUs can speculatively prefetch the ioremap'ed memory. Linux performs the kernel domain switching to allow user-specific functions (copy_to/from_user, get/put_user etc.) to access kernel memory. In order for these functions to work with the kernel domain set to Client, the patch modifies the LDRT/STRT and related instructions to the LDR/STR ones. The user pages access rights are also modified for kernel read-only access rather than read/write so that the copy-on-write mechanism still works. CPU_USE_DOMAINS gets disabled only if the hardware has a TLS register (CPU_32v6K is defined) since writing the TLS value to the high vectors page isn't possible. The user addresses passed to the kernel are checked by the access_ok() function so that they do not point to the kernel space. Tested-by: Anton Vorontsov <cbouatmailru@gmail.com> Cc: Tony Lindgren <tony@atomide.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-09-13 23:03:21 +08:00
* 110x 1 1 1 r/o r/o
* 11x0 1 1 1 r/o r/o
ARM: 7954/1: mm: remove remaining domain support from ARMv6 CPU_32v6 currently selects CPU_USE_DOMAINS if CPU_V6 and MMU. This is because ARM 1136 r0pX CPUs lack the v6k extensions, and therefore do not have hardware thread registers. The lack of these registers requires the kernel to update the vectors page at each context switch in order to write a new TLS pointer. This write must be done via the userspace mapping, since aliasing caches can lead to expensive flushing when using kmap. Finally, this requires the vectors page to be mapped r/w for kernel and r/o for user, which has implications for things like put_user which must trigger CoW appropriately when targetting user pages. The upshot of all this is that a v6/v7 kernel makes use of domains to segregate kernel and user memory accesses. This has the nasty side-effect of making device mappings executable, which has been observed to cause subtle bugs on recent cores (e.g. Cortex-A15 performing a speculative instruction fetch from the GIC and acking an interrupt in the process). This patch solves this problem by removing the remaining domain support from ARMv6. A new memory type is added specifically for the vectors page which allows that page (and only that page) to be mapped as user r/o, kernel r/w. All other user r/o pages are mapped also as kernel r/o. Patch co-developed with Russell King. Cc: <stable@vger.kernel.org> Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2014-02-08 02:12:27 +08:00
* 1111 0 1 1 r/w r/w
[ARM] Convert set_pte_ext implementions to macros There are actually only four separate implementations of set_pte_ext. Use assembler macros to insert code for these into the proc-*.S files. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 00:19:08 +08:00
*/
[ARM] Convert ARMv6 and ARMv7 to use new memory types Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 04:07:45 +08:00
.macro armv6_mt_table pfx
\pfx\()_mt_table:
.long 0x00 @ L_PTE_MT_UNCACHED
.long PTE_EXT_TEX(1) @ L_PTE_MT_BUFFERABLE
.long PTE_CACHEABLE @ L_PTE_MT_WRITETHROUGH
.long PTE_CACHEABLE | PTE_BUFFERABLE @ L_PTE_MT_WRITEBACK
.long PTE_BUFFERABLE @ L_PTE_MT_DEV_SHARED
.long 0x00 @ unused
.long 0x00 @ L_PTE_MT_MINICACHE (not present)
.long PTE_EXT_TEX(1) | PTE_CACHEABLE | PTE_BUFFERABLE @ L_PTE_MT_WRITEALLOC
.long 0x00 @ unused
.long PTE_EXT_TEX(1) @ L_PTE_MT_DEV_WC
.long 0x00 @ unused
.long PTE_CACHEABLE | PTE_BUFFERABLE @ L_PTE_MT_DEV_CACHED
.long PTE_EXT_TEX(2) @ L_PTE_MT_DEV_NONSHARED
[ARM] Remove MT_DEVICE_IXP2000 and associated definitions As of the previous commit, MT_DEVICE_IXP2000 encodes to the same PTE bit encoding as MT_DEVICE, so it's now redundant. Convert MT_DEVICE_IXP2000 to use MT_DEVICE instead, and remove its aliases. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 19:42:51 +08:00
.long 0x00 @ unused
[ARM] Convert ARMv6 and ARMv7 to use new memory types Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 04:07:45 +08:00
.long 0x00 @ unused
ARM: 7954/1: mm: remove remaining domain support from ARMv6 CPU_32v6 currently selects CPU_USE_DOMAINS if CPU_V6 and MMU. This is because ARM 1136 r0pX CPUs lack the v6k extensions, and therefore do not have hardware thread registers. The lack of these registers requires the kernel to update the vectors page at each context switch in order to write a new TLS pointer. This write must be done via the userspace mapping, since aliasing caches can lead to expensive flushing when using kmap. Finally, this requires the vectors page to be mapped r/w for kernel and r/o for user, which has implications for things like put_user which must trigger CoW appropriately when targetting user pages. The upshot of all this is that a v6/v7 kernel makes use of domains to segregate kernel and user memory accesses. This has the nasty side-effect of making device mappings executable, which has been observed to cause subtle bugs on recent cores (e.g. Cortex-A15 performing a speculative instruction fetch from the GIC and acking an interrupt in the process). This patch solves this problem by removing the remaining domain support from ARMv6. A new memory type is added specifically for the vectors page which allows that page (and only that page) to be mapped as user r/o, kernel r/w. All other user r/o pages are mapped also as kernel r/o. Patch co-developed with Russell King. Cc: <stable@vger.kernel.org> Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2014-02-08 02:12:27 +08:00
.long PTE_CACHEABLE | PTE_BUFFERABLE | PTE_EXT_APX @ L_PTE_MT_VECTORS
[ARM] Convert ARMv6 and ARMv7 to use new memory types Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 04:07:45 +08:00
.endm
.macro armv6_set_pte_ext pfx
ARM: pgtable: switch order of Linux vs hardware page tables This switches the ordering of the Linux vs hardware page tables in each page, thereby eliminating some of the arithmetic in the page table walks. As we now place the Linux page table at the beginning of the page, we can deal with the offset in the pgt by simply masking it away, along with the other control bits. This also makes the arithmetic all be positive, rather than a mixture. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-11-16 08:16:01 +08:00
str r1, [r0], #2048 @ linux version
[ARM] Convert set_pte_ext implementions to macros There are actually only four separate implementations of set_pte_ext. Use assembler macros to insert code for these into the proc-*.S files. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 00:19:08 +08:00
[ARM] Convert ARMv6 and ARMv7 to use new memory types Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 04:07:45 +08:00
bic r3, r1, #0x000003fc
[ARM] Convert set_pte_ext implementions to macros There are actually only four separate implementations of set_pte_ext. Use assembler macros to insert code for these into the proc-*.S files. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 00:19:08 +08:00
bic r3, r3, #PTE_TYPE_MASK
orr r3, r3, r2
orr r3, r3, #PTE_EXT_AP0 | 2
[ARM] Convert ARMv6 and ARMv7 to use new memory types Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 04:07:45 +08:00
adr ip, \pfx\()_mt_table
and r2, r1, #L_PTE_MT_MASK
ldr r2, [ip, r2]
ARM: pgtable: provide RDONLY page table bit rather than WRITE bit Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-11-16 16:40:36 +08:00
eor r1, r1, #L_PTE_DIRTY
tst r1, #L_PTE_DIRTY|L_PTE_RDONLY
orrne r3, r3, #PTE_EXT_APX
[ARM] Convert set_pte_ext implementions to macros There are actually only four separate implementations of set_pte_ext. Use assembler macros to insert code for these into the proc-*.S files. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 00:19:08 +08:00
tst r1, #L_PTE_USER
orrne r3, r3, #PTE_EXT_AP1
tstne r3, #PTE_EXT_APX
ARM: 7954/1: mm: remove remaining domain support from ARMv6 CPU_32v6 currently selects CPU_USE_DOMAINS if CPU_V6 and MMU. This is because ARM 1136 r0pX CPUs lack the v6k extensions, and therefore do not have hardware thread registers. The lack of these registers requires the kernel to update the vectors page at each context switch in order to write a new TLS pointer. This write must be done via the userspace mapping, since aliasing caches can lead to expensive flushing when using kmap. Finally, this requires the vectors page to be mapped r/w for kernel and r/o for user, which has implications for things like put_user which must trigger CoW appropriately when targetting user pages. The upshot of all this is that a v6/v7 kernel makes use of domains to segregate kernel and user memory accesses. This has the nasty side-effect of making device mappings executable, which has been observed to cause subtle bugs on recent cores (e.g. Cortex-A15 performing a speculative instruction fetch from the GIC and acking an interrupt in the process). This patch solves this problem by removing the remaining domain support from ARMv6. A new memory type is added specifically for the vectors page which allows that page (and only that page) to be mapped as user r/o, kernel r/w. All other user r/o pages are mapped also as kernel r/o. Patch co-developed with Russell King. Cc: <stable@vger.kernel.org> Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2014-02-08 02:12:27 +08:00
@ user read-only -> kernel read-only
bicne r3, r3, #PTE_EXT_AP0
[ARM] Convert set_pte_ext implementions to macros There are actually only four separate implementations of set_pte_ext. Use assembler macros to insert code for these into the proc-*.S files. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 00:19:08 +08:00
ARM: pgtable: invert L_PTE_EXEC to L_PTE_XN The hardware page tables use an XN bit 'execute never'. Historically, we've had a Linux 'execute allow' bit, in the positive sense. Get rid of this artifact as future hardware will continue to have the XN sense. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-11-16 08:23:31 +08:00
tst r1, #L_PTE_XN
orrne r3, r3, #PTE_EXT_XN
[ARM] Convert set_pte_ext implementions to macros There are actually only four separate implementations of set_pte_ext. Use assembler macros to insert code for these into the proc-*.S files. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 00:19:08 +08:00
ARM: 7954/1: mm: remove remaining domain support from ARMv6 CPU_32v6 currently selects CPU_USE_DOMAINS if CPU_V6 and MMU. This is because ARM 1136 r0pX CPUs lack the v6k extensions, and therefore do not have hardware thread registers. The lack of these registers requires the kernel to update the vectors page at each context switch in order to write a new TLS pointer. This write must be done via the userspace mapping, since aliasing caches can lead to expensive flushing when using kmap. Finally, this requires the vectors page to be mapped r/w for kernel and r/o for user, which has implications for things like put_user which must trigger CoW appropriately when targetting user pages. The upshot of all this is that a v6/v7 kernel makes use of domains to segregate kernel and user memory accesses. This has the nasty side-effect of making device mappings executable, which has been observed to cause subtle bugs on recent cores (e.g. Cortex-A15 performing a speculative instruction fetch from the GIC and acking an interrupt in the process). This patch solves this problem by removing the remaining domain support from ARMv6. A new memory type is added specifically for the vectors page which allows that page (and only that page) to be mapped as user r/o, kernel r/w. All other user r/o pages are mapped also as kernel r/o. Patch co-developed with Russell King. Cc: <stable@vger.kernel.org> Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2014-02-08 02:12:27 +08:00
eor r3, r3, r2
[ARM] Convert ARMv6 and ARMv7 to use new memory types Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 04:07:45 +08:00
[ARM] Convert set_pte_ext implementions to macros There are actually only four separate implementations of set_pte_ext. Use assembler macros to insert code for these into the proc-*.S files. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 00:19:08 +08:00
tst r1, #L_PTE_YOUNG
tstne r1, #L_PTE_PRESENT
moveq r3, #0
ARM: mm: introduce present, faulting entries for PAGE_NONE PROT_NONE mappings apply the page protection attributes defined by _P000 which translate to PAGE_NONE for ARM. These attributes specify an XN, RDONLY pte that is inaccessible to userspace. However, on kernels configured without support for domains, such a pte *is* accessible to the kernel and can be read via get_user, allowing tasks to read PROT_NONE pages via syscalls such as read/write over a pipe. This patch introduces a new software pte flag, L_PTE_NONE, that is set to identify faulting, present entries. Signed-off-by: Will Deacon <will.deacon@arm.com>
2012-09-01 12:22:12 +08:00
tstne r1, #L_PTE_NONE
movne r3, #0
[ARM] Convert set_pte_ext implementions to macros There are actually only four separate implementations of set_pte_ext. Use assembler macros to insert code for these into the proc-*.S files. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 00:19:08 +08:00
str r3, [r0]
mcr p15, 0, r0, c7, c10, 1 @ flush_pte
.endm
/*
* The ARMv3, ARMv4 and ARMv5 set_pte_ext translation function,
* covering most CPUs except Xscale and Xscale 3.
*
* Permission translation:
* YUWD AP SVC User
* 0xxx 0x00 no acc no acc
* 100x 0x00 r/o no acc
* 10x0 0x00 r/o no acc
* 1011 0x55 r/w no acc
* 110x 0xaa r/w r/o
* 11x0 0xaa r/w r/o
* 1111 0xff r/w r/w
*/
.macro armv3_set_pte_ext wc_disable=1
ARM: pgtable: switch order of Linux vs hardware page tables This switches the ordering of the Linux vs hardware page tables in each page, thereby eliminating some of the arithmetic in the page table walks. As we now place the Linux page table at the beginning of the page, we can deal with the offset in the pgt by simply masking it away, along with the other control bits. This also makes the arithmetic all be positive, rather than a mixture. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-11-16 08:16:01 +08:00
str r1, [r0], #2048 @ linux version
[ARM] Convert set_pte_ext implementions to macros There are actually only four separate implementations of set_pte_ext. Use assembler macros to insert code for these into the proc-*.S files. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 00:19:08 +08:00
ARM: pgtable: provide RDONLY page table bit rather than WRITE bit Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-11-16 16:40:36 +08:00
eor r3, r1, #L_PTE_PRESENT | L_PTE_YOUNG | L_PTE_DIRTY
[ARM] Convert set_pte_ext implementions to macros There are actually only four separate implementations of set_pte_ext. Use assembler macros to insert code for these into the proc-*.S files. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 00:19:08 +08:00
bic r2, r1, #PTE_SMALL_AP_MASK @ keep C, B bits
bic r2, r2, #PTE_TYPE_MASK
orr r2, r2, #PTE_TYPE_SMALL
tst r3, #L_PTE_USER @ user?
orrne r2, r2, #PTE_SMALL_AP_URO_SRW
ARM: pgtable: provide RDONLY page table bit rather than WRITE bit Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-11-16 16:40:36 +08:00
tst r3, #L_PTE_RDONLY | L_PTE_DIRTY @ write and dirty?
[ARM] Convert set_pte_ext implementions to macros There are actually only four separate implementations of set_pte_ext. Use assembler macros to insert code for these into the proc-*.S files. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 00:19:08 +08:00
orreq r2, r2, #PTE_SMALL_AP_UNO_SRW
tst r3, #L_PTE_PRESENT | L_PTE_YOUNG @ present and young?
movne r2, #0
.if \wc_disable
#ifdef CONFIG_CPU_DCACHE_WRITETHROUGH
tst r2, #PTE_CACHEABLE
bicne r2, r2, #PTE_BUFFERABLE
#endif
.endif
ARM: pgtable: switch order of Linux vs hardware page tables This switches the ordering of the Linux vs hardware page tables in each page, thereby eliminating some of the arithmetic in the page table walks. As we now place the Linux page table at the beginning of the page, we can deal with the offset in the pgt by simply masking it away, along with the other control bits. This also makes the arithmetic all be positive, rather than a mixture. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-11-16 08:16:01 +08:00
str r2, [r0] @ hardware version
[ARM] Convert set_pte_ext implementions to macros There are actually only four separate implementations of set_pte_ext. Use assembler macros to insert code for these into the proc-*.S files. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 00:19:08 +08:00
.endm
/*
* Xscale set_pte_ext translation, split into two halves to cope
* with work-arounds. r3 must be preserved by code between these
* two macros.
*
* Permission translation:
* YUWD AP SVC User
* 0xxx 00 no acc no acc
* 100x 00 r/o no acc
* 10x0 00 r/o no acc
* 1011 01 r/w no acc
* 110x 10 r/w r/o
* 11x0 10 r/w r/o
* 1111 11 r/w r/w
*/
.macro xscale_set_pte_ext_prologue
ARM: pgtable: switch order of Linux vs hardware page tables This switches the ordering of the Linux vs hardware page tables in each page, thereby eliminating some of the arithmetic in the page table walks. As we now place the Linux page table at the beginning of the page, we can deal with the offset in the pgt by simply masking it away, along with the other control bits. This also makes the arithmetic all be positive, rather than a mixture. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-11-16 08:16:01 +08:00
str r1, [r0] @ linux version
[ARM] Convert set_pte_ext implementions to macros There are actually only four separate implementations of set_pte_ext. Use assembler macros to insert code for these into the proc-*.S files. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 00:19:08 +08:00
ARM: pgtable: provide RDONLY page table bit rather than WRITE bit Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-11-16 16:40:36 +08:00
eor r3, r1, #L_PTE_PRESENT | L_PTE_YOUNG | L_PTE_DIRTY
[ARM] Convert set_pte_ext implementions to macros There are actually only four separate implementations of set_pte_ext. Use assembler macros to insert code for these into the proc-*.S files. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 00:19:08 +08:00
bic r2, r1, #PTE_SMALL_AP_MASK @ keep C, B bits
orr r2, r2, #PTE_TYPE_EXT @ extended page
tst r3, #L_PTE_USER @ user?
orrne r2, r2, #PTE_EXT_AP_URO_SRW @ yes -> user r/o, system r/w
ARM: pgtable: provide RDONLY page table bit rather than WRITE bit Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-11-16 16:40:36 +08:00
tst r3, #L_PTE_RDONLY | L_PTE_DIRTY @ write and dirty?
[ARM] Convert set_pte_ext implementions to macros There are actually only four separate implementations of set_pte_ext. Use assembler macros to insert code for these into the proc-*.S files. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 00:19:08 +08:00
orreq r2, r2, #PTE_EXT_AP_UNO_SRW @ yes -> user n/a, system r/w
@ combined with user -> user r/w
.endm
.macro xscale_set_pte_ext_epilogue
tst r3, #L_PTE_PRESENT | L_PTE_YOUNG @ present and young?
movne r2, #0 @ no -> fault
ARM: pgtable: switch order of Linux vs hardware page tables This switches the ordering of the Linux vs hardware page tables in each page, thereby eliminating some of the arithmetic in the page table walks. As we now place the Linux page table at the beginning of the page, we can deal with the offset in the pgt by simply masking it away, along with the other control bits. This also makes the arithmetic all be positive, rather than a mixture. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2010-11-16 08:16:01 +08:00
str r2, [r0, #2048]! @ hardware version
[ARM] Convert set_pte_ext implementions to macros There are actually only four separate implementations of set_pte_ext. Use assembler macros to insert code for these into the proc-*.S files. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2008-09-07 00:19:08 +08:00
mov ip, #0
mcr p15, 0, r0, c7, c10, 1 @ clean L1 D line
mcr p15, 0, ip, c7, c10, 4 @ data write barrier
.endm
ARM: mm: proc-macros: Add generic proc/cache/tlb struct definition macros This patch adds some generic macros to reduce boilerplate when declaring certain common structures in arch/arm/mm/*.S Thanks to Russell King for outlining what the define_processor_functions macro could look like. Signed-off-by: Dave Martin <dave.martin@linaro.org> Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
2011-06-24 00:07:40 +08:00
ARM: bugs: add support for per-processor bug checking Add support for per-processor bug checking - each processor function descriptor gains a function pointer for this check, which must not be an __init function. If non-NULL, this will be called whenever a CPU enters the kernel via which ever path (boot CPU, secondary CPU startup, CPU resuming, etc.) This allows processor specific bug checks to validate that workaround bits are properly enabled by firmware via all entry paths to the kernel. Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk> Reviewed-by: Florian Fainelli <f.fainelli@gmail.com> Boot-tested-by: Tony Lindgren <tony@atomide.com> Reviewed-by: Tony Lindgren <tony@atomide.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com>
2018-05-10 20:07:29 +08:00
.macro define_processor_functions name:req, dabort:req, pabort:req, nommu=0, suspend=0, bugs=0
ARM: ensure that processor vtables is not lost after boot Marek Szyprowski reported problems with CPU hotplug in current kernels. This was tracked down to the processor vtables being located in an init section, and therefore discarded after kernel boot, despite being required after boot to properly initialise the non-boot CPUs. Arrange for these tables to end up in .rodata when required. Reported-by: Marek Szyprowski <m.szyprowski@samsung.com> Tested-by: Krzysztof Kozlowski <krzk@kernel.org> Fixes: 383fb3ee8024 ("ARM: spectre-v2: per-CPU vtables to work around big.Little systems") Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
2018-12-07 00:36:38 +08:00
/*
* If we are building for big.Little with branch predictor hardening,
* we need the processor function tables to remain available after boot.
*/
ARM: fix the cockup in the previous patch The intention in the previous patch was to only place the processor tables in the .rodata section if big.Little was being built and we wanted the branch target hardening, but instead (due to the way it was tested) it ended up always placing the tables into the .rodata section. Although harmless, let's correct this anyway. Fixes: 3a4d0c2172bc ("ARM: ensure that processor vtables is not lost after boot") Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
2018-12-07 21:11:15 +08:00
#if defined(CONFIG_BIG_LITTLE) && defined(CONFIG_HARDEN_BRANCH_PREDICTOR)
ARM: ensure that processor vtables is not lost after boot Marek Szyprowski reported problems with CPU hotplug in current kernels. This was tracked down to the processor vtables being located in an init section, and therefore discarded after kernel boot, despite being required after boot to properly initialise the non-boot CPUs. Arrange for these tables to end up in .rodata when required. Reported-by: Marek Szyprowski <m.szyprowski@samsung.com> Tested-by: Krzysztof Kozlowski <krzk@kernel.org> Fixes: 383fb3ee8024 ("ARM: spectre-v2: per-CPU vtables to work around big.Little systems") Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
2018-12-07 00:36:38 +08:00
.section ".rodata"
#endif
ARM: mm: proc-macros: Add generic proc/cache/tlb struct definition macros This patch adds some generic macros to reduce boilerplate when declaring certain common structures in arch/arm/mm/*.S Thanks to Russell King for outlining what the define_processor_functions macro could look like. Signed-off-by: Dave Martin <dave.martin@linaro.org> Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
2011-06-24 00:07:40 +08:00
.type \name\()_processor_functions, #object
.align 2
ENTRY(\name\()_processor_functions)
.word \dabort
.word \pabort
.word cpu_\name\()_proc_init
ARM: bugs: add support for per-processor bug checking Add support for per-processor bug checking - each processor function descriptor gains a function pointer for this check, which must not be an __init function. If non-NULL, this will be called whenever a CPU enters the kernel via which ever path (boot CPU, secondary CPU startup, CPU resuming, etc.) This allows processor specific bug checks to validate that workaround bits are properly enabled by firmware via all entry paths to the kernel. Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk> Reviewed-by: Florian Fainelli <f.fainelli@gmail.com> Boot-tested-by: Tony Lindgren <tony@atomide.com> Reviewed-by: Tony Lindgren <tony@atomide.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com>
2018-05-10 20:07:29 +08:00
.word \bugs
ARM: mm: proc-macros: Add generic proc/cache/tlb struct definition macros This patch adds some generic macros to reduce boilerplate when declaring certain common structures in arch/arm/mm/*.S Thanks to Russell King for outlining what the define_processor_functions macro could look like. Signed-off-by: Dave Martin <dave.martin@linaro.org> Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
2011-06-24 00:07:40 +08:00
.word cpu_\name\()_proc_fin
.word cpu_\name\()_reset
.word cpu_\name\()_do_idle
.word cpu_\name\()_dcache_clean_area
.word cpu_\name\()_switch_mm
.if \nommu
.word 0
.else
.word cpu_\name\()_set_pte_ext
.endif
.if \suspend
.word cpu_\name\()_suspend_size
ARM: mm: Fix ifdef around cpu_*_do_[suspend, resume] ops Ifdef around cpu_\name\()_do_suspend and cpu_\name\()_do_resume ops in proc-macros.S should check for CONFIG_ARM_CPU_SUSPEND and not CONFIG_PM_SLEEP. Fix it. [ Please note that cpu_v7_do_[suspend,resume] code in proc-v7.S already correctly checks for CONFIG_ARM_CPU_SUSPEND, same is true for functions for other architectures. ] This fix is needed for decoupling suspend/resume and advanced cpuidle support on Exynos platform (next patch fixes build for config with CONFIG_PM_SLEEP=n and CONFIG_ARM_EXYNOS_CPUIDLE=y). If this fix is not present then the following OOPS happens on the first attempt to go into advanced cpuidle mode (AFTR): [ 22.244143] Unable to handle kernel NULL pointer dereference at virtual address 00000000 [ 22.250759] pgd = c0004000 [ 22.253445] [00000000] *pgd=00000000 [ 22.257012] Internal error: Oops: 80000007 [#1] PREEMPT SMP ARM [ 22.262906] Modules linked in: [ 22.265949] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.16.0-next-20140811-dirty #730 [ 22.273757] task: c05dce68 ti: c05d2000 task.ti: c05d2000 [ 22.279139] PC is at 0x0 [ 22.281661] LR is at __cpu_suspend_save+0x4c/0xa8 [ 22.286344] pc : [<00000000>] lr : [<c00125e0>] psr: a0000093 [ 22.286344] sp : c05d3ef4 ip : c05da414 fp : 00000001 [ 22.297799] r10: c05da414 r9 : c0609cb0 r8 : 0000000f [ 22.303008] r7 : c05da444 r6 : 00000038 r5 : ea802c00 r4 : c05d3f14 [ 22.309517] r3 : 00000000 r2 : c05d3f4c r1 : 00000038 r0 : c05d3f20 [ 22.316029] Flags: NzCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment kernel [ 22.323406] Control: 10c5387d Table: 69d5404a DAC: 00000015 [ 22.329135] Process swapper/0 (pid: 0, stack limit = 0xc05d2240) [ 22.335124] Stack: (0xc05d3ef4 to 0xc05d4000) [ 22.339466] 3ee0: ea802c00 00000038 c05d3f4c [ 22.347626] 3f00: 00000000 00000007 c00123bc 00000000 c001d468 6a888000 c05d3f4c 80000000 [ 22.355785] 3f20: 00000007 c003d3a0 0000193d eaf9dde4 eaf9dde4 c02ef0c8 c000969c fffffffe [ 22.363944] 3f40: 00000000 c0037b54 eaf9dbb8 e9d1a380 00000000 c001d468 c0609cb0 00000000 [ 22.372103] 3f60: c0609cb0 c061649e 00000001 c001250c eaf9dbb8 00000001 c0609cb0 c001d618 [ 22.380262] 3f80: c001d5d0 c02ef56c 2d9d2e1e 00000005 eaf9dbb8 c02edcc4 2d9d2e1e 00000005 [ 22.388421] 3fa0: c040446c c05da4ec c040446c eaf9dbb8 c05cfbb0 c004c580 c05dce68 c05b3ae8 [ 22.396580] 3fc0: 00000000 c058bb24 ffffffff ffffffff c058b5e4 00000000 00000000 c05b3ae8 [ 22.404740] 3fe0: c0616994 c05da47c c05b3ae4 c05ddeec 4000406a 40008074 00000000 00000000 [ 22.412909] [<c00125e0>] (__cpu_suspend_save) from [<c00123bc>] (__cpu_suspend+0x5c/0x70) [ 22.421074] [<c00123bc>] (__cpu_suspend) from [<c05d3f4c>] (init_thread_union+0x1f4c/0x2000) [ 22.429479] Code: bad PC value [ 22.432518] ---[ end trace fb90ebf4217d0ad9 ]--- [ 22.437116] Kernel panic - not syncing: Attempted to kill the idle task! [ 22.443800] Rebooting in 5 seconds.. This patch has been tested on Exynos4210 based Origen board. Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com> Acked-by: Kyungmin Park <kyungmin.park@samsung.com> Signed-off-by: Kukjin Kim <kgene.kim@samsung.com>
2014-09-24 01:18:32 +08:00
#ifdef CONFIG_ARM_CPU_SUSPEND
ARM: mm: proc-macros: Add generic proc/cache/tlb struct definition macros This patch adds some generic macros to reduce boilerplate when declaring certain common structures in arch/arm/mm/*.S Thanks to Russell King for outlining what the define_processor_functions macro could look like. Signed-off-by: Dave Martin <dave.martin@linaro.org> Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
2011-06-24 00:07:40 +08:00
.word cpu_\name\()_do_suspend
.word cpu_\name\()_do_resume
ARM: Fix build errors caused by adding generic macros Commit 66a625a (ARM: mm: proc-macros: Add generic proc/cache/tlb struct definition macros) introduced build errors when PM_SLEEP is not enabled. The per-CPU do_suspend/do_resume functions are defined via the preprocessor to constant 0. However, the macros which use these were converted to assembly, resulting in undefined references to these functions. Fix that by moving the ! ifdef section into proc-macros.S and deleting it from all effected proc-*.S files. Acked-by: Dave Martin <dave.martin@linaro.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2011-07-21 21:42:40 +08:00
#else
.word 0
.word 0
#endif
ARM: mm: proc-macros: Add generic proc/cache/tlb struct definition macros This patch adds some generic macros to reduce boilerplate when declaring certain common structures in arch/arm/mm/*.S Thanks to Russell King for outlining what the define_processor_functions macro could look like. Signed-off-by: Dave Martin <dave.martin@linaro.org> Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
2011-06-24 00:07:40 +08:00
.else
.word 0
.word 0
.word 0
.endif
.size \name\()_processor_functions, . - \name\()_processor_functions
ARM: fix the cockup in the previous patch The intention in the previous patch was to only place the processor tables in the .rodata section if big.Little was being built and we wanted the branch target hardening, but instead (due to the way it was tested) it ended up always placing the tables into the .rodata section. Although harmless, let's correct this anyway. Fixes: 3a4d0c2172bc ("ARM: ensure that processor vtables is not lost after boot") Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
2018-12-07 21:11:15 +08:00
#if defined(CONFIG_BIG_LITTLE) && defined(CONFIG_HARDEN_BRANCH_PREDICTOR)
ARM: ensure that processor vtables is not lost after boot Marek Szyprowski reported problems with CPU hotplug in current kernels. This was tracked down to the processor vtables being located in an init section, and therefore discarded after kernel boot, despite being required after boot to properly initialise the non-boot CPUs. Arrange for these tables to end up in .rodata when required. Reported-by: Marek Szyprowski <m.szyprowski@samsung.com> Tested-by: Krzysztof Kozlowski <krzk@kernel.org> Fixes: 383fb3ee8024 ("ARM: spectre-v2: per-CPU vtables to work around big.Little systems") Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
2018-12-07 00:36:38 +08:00
.previous
#endif
ARM: mm: proc-macros: Add generic proc/cache/tlb struct definition macros This patch adds some generic macros to reduce boilerplate when declaring certain common structures in arch/arm/mm/*.S Thanks to Russell King for outlining what the define_processor_functions macro could look like. Signed-off-by: Dave Martin <dave.martin@linaro.org> Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
2011-06-24 00:07:40 +08:00
.endm
.macro define_cache_functions name:req
.align 2
.type \name\()_cache_fns, #object
ENTRY(\name\()_cache_fns)
.long \name\()_flush_icache_all
.long \name\()_flush_kern_cache_all
ARM: mm: implement LoUIS API for cache maintenance ops ARM v7 architecture introduced the concept of cache levels and related control registers. New processors like A7 and A15 embed an L2 unified cache controller that becomes part of the cache level hierarchy. Some operations in the kernel like cpu_suspend and __cpu_disable do not require a flush of the entire cache hierarchy to DRAM but just the cache levels belonging to the Level of Unification Inner Shareable (LoUIS), which in most of ARM v7 systems correspond to L1. The current cache flushing API used in cpu_suspend and __cpu_disable, flush_cache_all(), ends up flushing the whole cache hierarchy since for v7 it cleans and invalidates all cache levels up to Level of Coherency (LoC) which cripples system performance when used in hot paths like hotplug and cpuidle. Therefore a new kernel cache maintenance API must be added to cope with latest ARM system requirements. This patch adds flush_cache_louis() to the ARM kernel cache maintenance API. This function cleans and invalidates all data cache levels up to the Level of Unification Inner Shareable (LoUIS) and invalidates the instruction cache for processors that support it (> v7). This patch also creates an alias of the cache LoUIS function to flush_kern_all for all processor versions prior to v7, so that the current cache flushing behaviour is unchanged for those processors. v7 cache maintenance code implements a cache LoUIS function that cleans and invalidates the D-cache up to LoUIS and invalidates the I-cache, according to the new API. Reviewed-by: Santosh Shilimkar <santosh.shilimkar@ti.com> Reviewed-by: Nicolas Pitre <nico@linaro.org> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Tested-by: Shawn Guo <shawn.guo@linaro.org>
2012-09-06 21:05:13 +08:00
.long \name\()_flush_kern_cache_louis
ARM: mm: proc-macros: Add generic proc/cache/tlb struct definition macros This patch adds some generic macros to reduce boilerplate when declaring certain common structures in arch/arm/mm/*.S Thanks to Russell King for outlining what the define_processor_functions macro could look like. Signed-off-by: Dave Martin <dave.martin@linaro.org> Acked-by: Nicolas Pitre <nicolas.pitre@linaro.org>
2011-06-24 00:07:40 +08:00
.long \name\()_flush_user_cache_all
.long \name\()_flush_user_cache_range
.long \name\()_coherent_kern_range
.long \name\()_coherent_user_range
.long \name\()_flush_kern_dcache_area
.long \name\()_dma_map_area
.long \name\()_dma_unmap_area
.long \name\()_dma_flush_range
.size \name\()_cache_fns, . - \name\()_cache_fns
.endm
.macro define_tlb_functions name:req, flags_up:req, flags_smp
.type \name\()_tlb_fns, #object
ENTRY(\name\()_tlb_fns)
.long \name\()_flush_user_tlb_range
.long \name\()_flush_kern_tlb_range
.ifnb \flags_smp
ALT_SMP(.long \flags_smp )
ALT_UP(.long \flags_up )
.else
.long \flags_up
.endif
.size \name\()_tlb_fns, . - \name\()_tlb_fns
.endm
ARM: 7773/1: PJ4B: Add support for errata 4742 This commit fixes the regression on Armada 370 (the kernal hang during boot) introduced by the commit: "ARM: 7691/1: mm: kill unused TLB_CAN_READ_FROM_L1_CACHE and use ALT_SMP instead". When coming out of either a Wait for Interrupt (WFI) or a Wait for Event (WFE) IDLE states, a specific timing sensitivity exists between the retiring WFI/WFE instructions and the newly issued subsequent instructions. This sensitivity can result in a CPU hang scenario. The workaround is to insert either a Data Synchronization Barrier (DSB) or Data Memory Barrier (DMB) command immediately after the WFI/WFE instruction. This commit was based on the work of Lior Amsalem, but heavily modified to apply the errata fix dynamically according to the processor type thanks to the suggestions of Russell King and Nicolas Pitre. Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com> Reviewed-by: Will Deacon <will.deacon@arm.com> Acked-by: Nicolas Pitre <nico@linaro.org> Tested-by: Willy Tarreau <w@1wt.eu> Cc: <stable@vger.kernel.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-06-23 17:17:11 +08:00
.macro globl_equ x, y
.globl \x
.equ \x, \y
.endm
ARM: 8314/1: replace PROCINFO embedded branch with relative offset This patch replaces the 'branch to setup()' instructions embedded in the PROCINFO structs with the offset to that setup function relative to the base of the struct. This preserves the position independent nature of that field, but uses a data item rather than an instruction. This is mainly done to prevent linker failures on large kernels, where the setup function is out of reach for the branch. Acked-by: Nicolas Pitre <nico@linaro.org> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2015-03-18 14:29:32 +08:00
.macro initfn, func, base
.long \func - \base
.endm
Merge branches 'misc', 'vdso' and 'fixes' into for-next Conflicts: arch/arm/mm/proc-macros.S
2015-04-15 05:28:25 +08:00
ARM: proc-arm94*.S: fix setup function Both ARM946 and ARM940 setup functions were corrupting r1 and r2, which is not permissible - these are used to carry the machine ID and boot data into the kernel, and must be preserved. The code responsible for this was the same in both files: they were using the registers to generate a protection region register value. Fix this by turning this process into a macro, and using that macro in both these files with an alternative register allocation. r0, r3 and r7 can be used for temporary values here. Reported-by: Alex Dumitrache <broscutamaker@gmail.com> Tested-by: Georg Hofstetter <g3gg0.de@gmail.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2015-04-05 06:22:07 +08:00
/*
* Macro to calculate the log2 size for the protection region
* registers. This calculates rd = log2(size) - 1. tmp must
* not be the same register as rd.
*/
.macro pr_sz, rd, size, tmp
mov \tmp, \size, lsr #12
mov \rd, #11
1: movs \tmp, \tmp, lsr #1
addne \rd, \rd, #1
bne 1b
.endm
/*
* Macro to generate a protection region register value
* given a pre-masked address, size, and enable bit.
* Corrupts size.
*/
.macro pr_val, dest, addr, size, enable
pr_sz \dest, \size, \size @ calculate log2(size) - 1
orr \dest, \addr, \dest, lsl #1 @ mask in the region size
orr \dest, \dest, \enable
.endm