But first update the code that uses these facilities with the
new header.
Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Update code that relied on sched.h including various MM types for them.
This will allow us to remove the <linux/mm_types.h> include from <linux/sched.h>.
Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
This was entirely automated, using the script by Al:
PATT='^[[:blank:]]*#[[:blank:]]*include[[:blank:]]*<asm/uaccess.h>'
sed -i -e "s!$PATT!#include <linux/uaccess.h>!" \
$(git grep -l "$PATT"|grep -v ^include/linux/uaccess.h)
to do the replacement at the end of the merge window.
Requested-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Commit 432c6bacbd ("MIPS: Use per-mm page to execute branch delay slot
instructions") accidentally removed use of the MIPS_FPU_EMU_INC_STATS
macro from do_dsemulret, leading to the ds_emul file in debugfs always
returning zero even though we perform delay slot emulations.
Fix this by re-adding the use of the MIPS_FPU_EMU_INC_STATS macro.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Fixes: 432c6bacbd ("MIPS: Use per-mm page to execute branch delay slot instructions")
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/14301/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
In some cases the kernel needs to execute an instruction from the delay
slot of an emulated branch instruction. These cases include:
- Emulated floating point branch instructions (bc1[ft]l?) for systems
which don't include an FPU, or upon which the kernel is run with the
"nofpu" parameter.
- MIPSr6 systems running binaries targeting older revisions of the
architecture, which may include branch instructions whose encodings
are no longer valid in MIPSr6.
Executing instructions from such delay slots is done by writing the
instruction to memory followed by a trap, as part of an "emuframe", and
executing it. This avoids the requirement of an emulator for the entire
MIPS instruction set. Prior to this patch such emuframes are written to
the user stack and executed from there.
This patch moves FP branch delay emuframes off of the user stack and
into a per-mm page. Allocating a page per-mm leaves userland with access
to only what it had access to previously, and compared to other
solutions is relatively simple.
When a thread requires a delay slot emulation, it is allocated a frame.
A thread may only have one frame allocated at any one time, since it may
only ever be executing one instruction at any one time. In order to
ensure that we can free up allocated frame later, its index is recorded
in struct thread_struct. In the typical case, after executing the delay
slot instruction we'll execute a break instruction with the BRK_MEMU
code. This traps back to the kernel & leads to a call to do_dsemulret
which frees the allocated frame & moves the user PC back to the
instruction that would have executed following the emulated branch.
In some cases the delay slot instruction may be invalid, such as a
branch, or may trigger an exception. In these cases the BRK_MEMU break
instruction will not be hit. In order to ensure that frames are freed
this patch introduces dsemul_thread_cleanup() and calls it to free any
allocated frame upon thread exit. If the instruction generated an
exception & leads to a signal being delivered to the thread, or indeed
if a signal simply happens to be delivered to the thread whilst it is
executing from the struct emuframe, then we need to take care to exit
the frame appropriately. This is done by either rolling back the user PC
to the branch or advancing it to the continuation PC prior to signal
delivery, using dsemul_thread_rollback(). If this were not done then a
sigreturn would return to the struct emuframe, and if that frame had
meanwhile been used in response to an emulated branch instruction within
the signal handler then we would execute the wrong user code.
Whilst a user could theoretically place something like a compact branch
to self in a delay slot and cause their thread to become stuck in an
infinite loop with the frame never being deallocated, this would:
- Only affect the users single process.
- Be architecturally invalid since there would be a branch in the
delay slot, which is forbidden.
- Be extremely unlikely to happen by mistake, and provide a program
with no more ability to harm the system than a simple infinite loop
would.
If a thread requires a delay slot emulation & no frame is available to
it (ie. the process has enough other threads that all frames are
currently in use) then the thread joins a waitqueue. It will sleep until
a frame is freed by another thread in the process.
Since we now know whether a thread has an allocated frame due to our
tracking of its index, the cookie field of struct emuframe is removed as
we can be more certain whether we have a valid frame. Since a thread may
only ever have a single frame at any given time, the epc field of struct
emuframe is also removed & the PC to continue from is instead stored in
struct thread_struct. Together these changes simplify & shrink struct
emuframe somewhat, allowing twice as many frames to fit into the page
allocated for them.
The primary benefit of this patch is that we are now free to mark the
user stack non-executable where that is possible.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: Leonid Yegoshin <leonid.yegoshin@imgtec.com>
Cc: Maciej Rozycki <maciej.rozycki@imgtec.com>
Cc: Faraz Shahbazker <faraz.shahbazker@imgtec.com>
Cc: Raghu Gandham <raghu.gandham@imgtec.com>
Cc: Matthew Fortune <matthew.fortune@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/13764/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Remove irrelevant content from the description of the emulation frame in
`mips_dsemul', referring to bare-metal configurations. Update the text,
reflecting the change made with commit ba3049ed40 ("MIPS: Switch FPU
emulator trap to BREAK instruction."), where we switched from using an
address error exception on an unaligned access to the use of a BREAK 514
instruction causing a breakpoint exception instead.
Signed-off-by: Maciej W. Rozycki <macro@imgtec.com>
Cc: Aurelien Jarno <aurelien@aurel32.net>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/12176/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Emulate the microMIPS ADDIUPC instruction directly in `mips_dsemul'. If
executed in the emulation frame, this instruction produces an incorrect
result, because the value of the PC there is not the same as where the
instruction originated.
Reshape code so as to handle all microMIPS cases together.
Signed-off-by: Maciej W. Rozycki <macro@imgtec.com>
Cc: Aurelien Jarno <aurelien@aurel32.net>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/12175/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Complement commit 102cedc32a ("MIPS: microMIPS: Floating point
support.") which introduced microMIPS FPU emulation, but did not adjust
the encoding of the BREAK instruction used to terminate the branch delay
slot emulation frame. Consequently the execution of any such frame is
indeterminate and, depending on CPU configuration, will result in random
code execution or an offending program being terminated with SIGILL.
This is because the regular MIPS BREAK instruction is encoded with the 0
major and the 0xd minor opcode, however in the microMIPS instruction set
this major/minor opcode pair denotes an encoding reserved for the DSP
ASE. Instead the microMIPS BREAK instruction is encoded with the 0
major and the 0x7 minor opcode.
Use the correct BREAK encoding for microMIPS FPU emulation then.
Signed-off-by: Maciej W. Rozycki <macro@imgtec.com>
Cc: Aurelien Jarno <aurelien@aurel32.net>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/12174/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Correct formatting breakage introduced with commit 102cedc32a ("MIPS:
microMIPS: Floating point support."), so that further changes to this
code can be consistent.
Signed-off-by: Maciej W. Rozycki <macro@imgtec.com>
Cc: Aurelien Jarno <aurelien@aurel32.net>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/12173/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Fix an issue introduced with commit 9ab4471c9f ("MIPS: math-emu:
Correct delay-slot exception propagation") where the emulation of a NOP
instruction signals the need to terminate the emulation loop. This in
turn, if the PC has not changed from the entry to the loop, will cause
the kernel to terminate the program with SIGILL.
Consider this program:
static double div(double d)
{
do
d /= 2.0;
while (d > .5);
return d;
}
int main(int argc, char **argv)
{
return div(argc);
}
which gets compiled to the following binary code:
00400490 <main>:
400490: 44840000 mtc1 a0,$f0
400494: 3c020040 lui v0,0x40
400498: d44207f8 ldc1 $f2,2040(v0)
40049c: 46800021 cvt.d.w $f0,$f0
4004a0: 46220002 mul.d $f0,$f0,$f2
4004a4: 4620103c c.lt.d $f2,$f0
4004a8: 4501fffd bc1t 4004a0 <main+0x10>
4004ac: 00000000 nop
4004b0: 4620000d trunc.w.d $f0,$f0
4004b4: 03e00008 jr ra
4004b8: 44020000 mfc1 v0,$f0
4004bc: 00000000 nop
Where the FPU emulator is used, depending on the number of command-line
arguments this code will either run to completion or terminate with
SIGILL.
If no arguments are specified, then BC1T will not be taken, NOP will not
be emulated and code will complete successfully.
If one argument is specified, then BC1T will be taken once and NOP will
be emulated. At this point the entry PC value will be 0x400498 and the
new PC value, set by `mips_dsemul' will be 0x4004a0, the target of BC1T.
The emulation loop will terminate, but SIGILL will not be issued,
because the PC has changed. The FPU emulator will be entered again and
on the second execution BC1T will not be taken, NOP will not be emulated
and code will complete successfully.
If two or more arguments are specified, then the first execution of BC1T
will proceed as above. Upon reentering the FPU emulator the emulation
loop will continue to BC1T, at which point the branch will be taken and
NOP emulated again. At this point however the entry PC value will be
0x4004a0, the same as the target of BC1T. This will make the emulator
conclude that execution has not advanced and therefore an unsupported
FPU instruction has been encountered, and SIGILL will be sent to the
process.
Fix the problem by extending the internal API of `mips_dsemul', making
it return -1 if no delay slot emulation frame has been made, the
instruction has been handled and execution of the emulation loop needs
to continue as if nothing happened. Remove code from `mips_dsemul' to
reproduce steps made by the emulation loop at the conclusion of each
iteration, as those will be reached normally now. Adjust call sites
accordingly. Document the API.
Signed-off-by: Maciej W. Rozycki <macro@imgtec.com>
Cc: Aurelien Jarno <aurelien@aurel32.net>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/12172/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
handle_dsemul does not exist and it's not being used in the code at all
so remove its declaration. The deliberate DS emulation exception is
handled by the do_dsemulret C code.
Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/10950/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Restore EPC at the branch whose delay slot is emulated if the delay-slot
instruction signals. This is so that code in `fpu_emulator_cop1Handler'
does not see EPC having advanced and mistakenly successfully resume
userland execution from the location at the branch target in that case.
Restoring EPC guarantees an immediate exit from the emulation loop and
if EPC hasn't advanced at all since entering the loop, also issuing the
signal reported by the delay-slot instruction.
Signed-off-by: Maciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9701/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Correct a cache coherency regression introduced with be1664c4 [Another
round of fixes for the fp emulator.] for the emulation frame used in
delay-slot emulation.
Two instructions are copied into the frame and as from the commit
referred a cache synchronisation call is made for the second instruction
aka `badinst' of the two only. The `flush_cache_sigtramp' interface is
reused that guarantees that synchronisation will be made for 8 bytes or
2 instructions starting from the address requested, although if cache
lines are wider then a larger area may be synchronised.
Change the call to point to the first of the two instructions aka `emul'
instead, removing unpredictable behaviour resulting from cache
incoherency.
This bug only ever manifested itself on systems implementing 4-byte
cache lines, typically MIPS I systems, causing all kinds of weirdness.
This is because the sequence of two instructions starting from `emul' is
8-byte aligned and for 8-byte or wider cache lines the line synchronised
will span both, so the vast majority of systems have escaped unharmed.
Signed-off-by: Maciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9698/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Delay slot emulation in the FPU emulator is the only kernel user of an
executable stack, it is also very slow. Add a counter so we can see
how many of these emulations are done.
Signed-off-by: David Daney <david.daney@cavium.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/8634/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Add logic needed to do floating point emulation in microMIPS mode.
Signed-off-by: Leonid Yegoshin <Leonid.Yegoshin@imgtec.com>
Signed-off-by: Steven J. Hill <Steven. Hill@imgtec.com>
On SMP systems, the collection of statistics can cause cache line
bouncing in the lines associated with the counters. Also there are
races incrementing the counters on multiple CPUs.
To fix both problems, we collect the statistics in per-CPU variables,
and add them up in the debugfs read operation.
As a test I ran the LTP float_bessel test on a 12 CPU Octeon system.
Without CONFIG_DEBUG_FS : 2602 seconds.
With CONFIG_DEBUG_FS: 2640 seconds.
With non-cpu-local atomic statistics: 14569 seconds.
Signed-off-by: David Daney <ddaney@caviumnetworks.com>
Cc: linux-mips@linux-mips.org
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Arguably using the address error handler has always been ugly. But with
processors that handle unaligned loads and stores in hardware the
current mechanism ceases to work so switch it to a BREAK instruction and
allocate break code 514 to the FPU emulator.
Yoichi Yuasa provided a build fix for CONFIG_BUG=n.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Signed-off-by: Yoichi Yuasa <yoichi_yuasa@tripeaks.co.jp>
Remove includes of <linux/smp_lock.h> where it is not used/needed.
Suggested by Al Viro.
Builds cleanly on x86_64, i386, alpha, ia64, powerpc, sparc,
sparc64, and arm (all 59 defconfigs).
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
VA_TO_REG. Who ever wrote this apparently did enjoy the C Puzzle Book.
ISBN 0201604612, a little old but still fun reading for the next
blackout ;)
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.
Let it rip!
Ingo Molnar