Implement atomic logic ops -- atomic_{or,xor,and}.
These will replace the atomic_{set,clear}_mask functions that are
available on some archs.
Acked-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Firstly, handle zero length calls properly. Believe it or not there
are a few of these happening during early boot.
Next, we can't just drop to a memcpy() call in the forward copy case
where dst <= src. The reason is that the cache initializing stores
used in the Niagara memcpy() implementations can end up clearing out
cache lines before we've sourced their original contents completely.
For example, considering NG4memcpy, the main unrolled loop begins like
this:
load src + 0x00
load src + 0x08
load src + 0x10
load src + 0x18
load src + 0x20
store dst + 0x00
Assume dst is 64 byte aligned and let's say that dst is src - 8 for
this memcpy() call. That store at the end there is the one to the
first line in the cache line, thus clearing the whole line, which thus
clobbers "src + 0x28" before it even gets loaded.
To avoid this, just fall through to a simple copy only mildly
optimized for the case where src and dst are 8 byte aligned and the
length is a multiple of 8 as well. We could get fancy and call
GENmemcpy() but this is good enough for how this thing is actually
used.
Reported-by: David Ahern <david.ahern@oracle.com>
Reported-by: Bob Picco <bpicco@meloft.net>
Signed-off-by: David S. Miller <davem@davemloft.net>
Atomicity between xchg and cmpxchg cannot be guaranteed when xchg is
implemented with a swap and cmpxchg is implemented with locks.
Without this, e.g. mcs_spin_lock and mcs_spin_unlock are broken.
Signed-off-by: Andreas Larsson <andreas@gaisler.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The AES loops in arch/sparc/crypto/aes_glue.c use a scheme where the
key material is preloaded into the FPU registers, and then we loop
over and over doing the crypt operation, reusing those pre-cooked key
registers.
There are intervening blkcipher*() calls between the crypt operation
calls. And those might perform memcpy() and thus also try to use the
FPU.
The sparc64 kernel FPU usage mechanism is designed to allow such
recursive uses, but with a catch.
There has to be a trap between the two FPU using threads of control.
The mechanism works by, when the FPU is already in use by the kernel,
allocating a slot for FPU saving at trap time. Then if, within the
trap handler, we try to use the FPU registers, the pre-trap FPU
register state is saved into the slot. Then at trap return time we
notice this and restore the pre-trap FPU state.
Over the long term there are various more involved ways we can make
this work, but for a quick fix let's take advantage of the fact that
the situation where this happens is very limited.
All sparc64 chips that support the crypto instructiosn also are using
the Niagara4 memcpy routine, and that routine only uses the FPU for
large copies where we can't get the source aligned properly to a
multiple of 8 bytes.
We look to see if the FPU is already in use in this context, and if so
we use the non-large copy path which only uses integer registers.
Furthermore, we also limit this special logic to when we are doing
kernel copy, rather than a user copy.
Signed-off-by: David S. Miller <davem@davemloft.net>
Pull arch atomic cleanups from Ingo Molnar:
"This is a series kept separate from the main locking tree, which
cleans up and improves various details in the atomics type handling:
- Remove the unused atomic_or_long() method
- Consolidate and compress atomic ops implementations between
architectures, to reduce linecount and to make it easier to add new
ops.
- Rewrite generic atomic support to only require cmpxchg() from an
architecture - generate all other methods from that"
* 'locking-arch-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (23 commits)
locking,arch: Use ACCESS_ONCE() instead of cast to volatile in atomic_read()
locking, mips: Fix atomics
locking, sparc64: Fix atomics
locking,arch: Rewrite generic atomic support
locking,arch,xtensa: Fold atomic_ops
locking,arch,sparc: Fold atomic_ops
locking,arch,sh: Fold atomic_ops
locking,arch,powerpc: Fold atomic_ops
locking,arch,parisc: Fold atomic_ops
locking,arch,mn10300: Fold atomic_ops
locking,arch,mips: Fold atomic_ops
locking,arch,metag: Fold atomic_ops
locking,arch,m68k: Fold atomic_ops
locking,arch,m32r: Fold atomic_ops
locking,arch,ia64: Fold atomic_ops
locking,arch,hexagon: Fold atomic_ops
locking,arch,cris: Fold atomic_ops
locking,arch,avr32: Fold atomic_ops
locking,arch,arm64: Fold atomic_ops
locking,arch,arm: Fold atomic_ops
...
The patch folding the atomic ops had a silly fail in the _return primitives.
Fixes: 4f3316c2b5 ("locking,arch,sparc: Fold atomic_ops")
Reported-by: Guenter Roeck <linux@roeck-us.net>
Tested-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Stephen Rothwell <sfr@canb.auug.org.au>
Cc: David S. Miller <davem@davemloft.net>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: sparclinux@vger.kernel.org
Link: http://lkml.kernel.org/r/20140902094016.GD31157@worktop.ger.corp.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
This makes memset follow the standard (instead of returning 0 on success). This
is needed when certain versions of gcc optimizes around memset calls and assume
that the address argument is preserved in %o0.
Signed-off-by: Andreas Larsson <andreas@gaisler.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Many of the atomic op implementations are the same except for one
instruction; fold the lot into a few CPP macros and reduce LoC.
This also prepares for easy addition of new ops.
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Acked-by: David S. Miller <davem@davemloft.net>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Kirill Tkhai <tkhai@yandex.ru>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Sam Ravnborg <sam@ravnborg.org>
Cc: sparclinux@vger.kernel.org
Link: http://lkml.kernel.org/r/20140508135852.825281379@infradead.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Pull sparc updates from David Miller:
1) Add sparc RAM output to /proc/iomem, from Bob Picco.
2) Allow seeks on /dev/mdesc, from Khalid Aziz.
3) Cleanup sparc64 I/O accessors, from Sam Ravnborg.
4) If update_mmu_cache{,_pmd}() is called with an not-valid mapping, do
not insert it into the TLB miss hash tables otherwise we'll
livelock. Based upon work by Christopher Alexander Tobias Schulze.
5) Fix BREAK detection in sunsab driver when no actual characters are
pending, from Christopher Alexander Tobias Schulze.
6) Because we have modules --> openfirmware --> vmalloc ordering of
virtual memory, the lazy VMAP TLB flusher can cons up an invocation
of flush_tlb_kernel_range() that covers the openfirmware address
range. Unfortunately this will flush out the firmware's locked TLB
mapping which causes all kinds of trouble. Just split up the flush
request if this happens, but in the long term the lazy VMAP flusher
should probably be made a little bit smarter.
Based upon work by Christopher Alexander Tobias Schulze.
* git://git.kernel.org/pub/scm/linux/kernel/git/davem/sparc-next:
sparc64: Fix up merge thinko.
sparc: Add "install" target
arch/sparc/math-emu/math_32.c: drop stray break operator
sparc64: ldc_connect() should not return EINVAL when handshake is in progress.
sparc64: Guard against flushing openfirmware mappings.
sunsab: Fix detection of BREAK on sunsab serial console
bbc-i2c: Fix BBC I2C envctrl on SunBlade 2000
sparc64: Do not insert non-valid PTEs into the TSB hash table.
sparc64: avoid code duplication in io_64.h
sparc64: reorder functions in io_64.h
sparc64: drop unused SLOW_DOWN_IO definitions
sparc64: remove macro indirection in io_64.h
sparc64: update IO access functions in PeeCeeI
sparcspkr: use sbus_*() primitives for IO
sparc: Add support for seek and shorter read to /dev/mdesc
sparc: use %s for unaligned panic
drivers/sbus/char: Micro-optimization in display7seg.c
display7seg: Introduce the use of the managed version of kzalloc
sparc64 - add mem to iomem resource
The PeeCeeI.c code used in*() + out*() for IO access.
But these are in little endian and the native (big) endian
result was required which resulted in some bit-shifting.
Shift the code over to use the __raw_*() variants all over.
This simplifies the code as we can drop the calls
to le16_to_cpu() and le32_to_cpu().
And it should be a little faster too.
With this change we now uses the same type of IO access functions
in all of the file.
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Nothing sets function_trace_stop to disable function tracing anymore.
Remove the check for it in the arch code.
Link: http://lkml.kernel.org/r/20140703.211820.1674895115102216877.davem@davemloft.net
Cc: David S. Miller <davem@davemloft.net>
OKed-to-go-through-tracing-tree-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
This is the prevent previous stores from overlapping the block stores
done by the memcpy loop.
Based upon a glibc patch by Jose E. Marchesi
Signed-off-by: David S. Miller <davem@davemloft.net>
Use asm-generic/io.h definitions where applicable.
The inxx() and outxx() methods whcih was duplicated in pcic.c +
leon_pci.c are replaced by a set of static inlins from asm-generic/io.h
iomap.c is replaced by the generic versions, but are still
present to support sparc64.
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Cc: Daniel Hellstrom <daniel@gaisler.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Choose PAGE_OFFSET dynamically based upon cpu type.
Original UltraSPARC-I (spitfire) chips only supported a 44-bit
virtual address space.
Newer chips (T4 and later) support 52-bit virtual addresses
and up to 47-bits of physical memory space.
Therefore we have to adjust PAGE_SIZE dynamically based upon
the capabilities of the chip.
Note that this change alone does not allow us to support > 43-bit
physical memory, to do that we need to re-arrange our page table
support. The current encodings of the pmd_t and pgd_t pointers
restricts us to "32 + 11" == 43 bits.
This change can waste quite a bit of memory for the various tables.
In particular, a future change should work to size and allocate
kern_linear_bitmap[] and sparc64_valid_addr_bitmap[] dynamically.
This isn't easy as we really cannot take a TLB miss when accessing
kern_linear_bitmap[]. We'd have to lock it into the TLB or similar.
Signed-off-by: David S. Miller <davem@davemloft.net>
Acked-by: Bob Picco <bob.picco@oracle.com>
The functions
__down_read
__down_read_trylock
__down_write
__down_write_trylock
__up_read
__up_write
__downgrade_write
are implemented inline, so remove corresponding EXPORT_SYMBOLs
(They lead to compile errors on RT kernel).
Signed-off-by: Kirill Tkhai <tkhai@yandex.ru>
CC: David Miller <davem@davemloft.net>
Signed-off-by: David S. Miller <davem@davemloft.net>
The help text for this config is duplicated across the x86, parisc, and
s390 Kconfig.debug files. Arnd Bergman noted that the help text was
slightly misleading and should be fixed to state that enabling this
option isn't a problem when using pre 4.4 gcc.
To simplify the rewording, consolidate the text into lib/Kconfig.debug
and modify it there to be more explicit about when you should say N to
this config.
Also, make the text a bit more generic by stating that this option
enables compile time checks so we can cover architectures which emit
warnings vs. ones which emit errors. The details of how an
architecture decided to implement the checks isn't as important as the
concept of compile time checking of copy_from_user() calls.
While we're doing this, remove all the copy_from_user_overflow() code
that's duplicated many times and place it into lib/ so that any
architecture supporting this option can get the function for free.
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Ingo Molnar <mingo@kernel.org>
Acked-by: H. Peter Anvin <hpa@zytor.com>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Acked-by: Helge Deller <deller@gmx.de>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Stephen Rothwell <sfr@canb.auug.org.au>
Cc: Chris Metcalf <cmetcalf@tilera.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
srmmu_nocache_bitmap is cleared by bit_map_init(). But bit_map_init()
attempts to clear by memset(), so it can't clear the trailing edge of
bitmap properly on big-endian architecture if the number of bits is not
a multiple of BITS_PER_LONG.
Actually, the number of bits in srmmu_nocache_bitmap is not always
a multiple of BITS_PER_LONG. It is calculated as below:
bitmap_bits = srmmu_nocache_size >> SRMMU_NOCACHE_BITMAP_SHIFT;
srmmu_nocache_size is decided proportionally by the amount of system RAM
and it is rounded to a multiple of PAGE_SIZE. SRMMU_NOCACHE_BITMAP_SHIFT
is defined as (PAGE_SHIFT - 4). So it can only be said that bitmap_bits
is a multiple of 16.
This fixes the problem by using bitmap_clear() instead of memset()
in bit_map_init() and this also uses BITS_TO_LONGS() to calculate correct
size at bitmap allocation time.
Signed-off-by: Akinobu Mita <akinobu.mita@gmail.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: sparclinux@vger.kernel.org
Signed-off-by: David S. Miller <davem@davemloft.net>
Sparc32 already supported it, as a consequence of using the
generic atomic64 implementation. And the sparc64 implementation
is rather trivial.
This allows us to set ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE for all
of sparc, and avoid the annoying warning from lib/atomic64_test.c
Signed-off-by: David S. Miller <davem@davemloft.net>
This adds optimized memset/bzero/page-clear routines for Niagara-4.
We basically can do what powerpc has been able to do for a decade (via
the "dcbz" instruction), which is use cache line clearing stores for
bzero and memsets with a 'c' argument of zero.
As long as we make the cache initializing store to each 32-byte
subblock of the L2 cache line, it works.
As with other Niagara-4 optimized routines, the key is to make sure to
avoid any usage of the %asi register, as reads and writes to it cost
at least 50 cycles.
For the user clear cases, we don't use these new routines, we use the
Niagara-1 variants instead. Those have to use %asi in an unavoidable
way.
A Niagara-4 8K page clear costs just under 600 cycles.
Add definitions of the MRU variants of the cache initializing store
ASIs. By default, cache initializing stores install the line as Least
Recently Used. If we know we're going to use the data immediately
(which is true for page copies and clears) we can use the Most
Recently Used variant, to decrease the likelyhood of the lines being
evicted before they get used.
Signed-off-by: David S. Miller <davem@davemloft.net>
There's a Niagara 2 memcpy fix in this tree and I have
a Kconfig fix from Dave Jones which requires the sparc-next
changes which went upstream yesterday.
Signed-off-by: David S. Miller <davem@davemloft.net>
It gets clobbered by the kernel's VISEntryHalf, so we have to save it
in a different register than the set clobbered by that macro.
The instance in glibc is OK and doesn't have this problem.
Signed-off-by: David S. Miller <davem@davemloft.net>
To use this, an architecture simply needs to:
1) Provide a user_addr_max() implementation via asm/uaccess.h
2) Add "select GENERIC_STRNCPY_FROM_USER" to their arch Kcnfig
3) Remove the existing strncpy_from_user() implementation and symbol
exports their architecture had.
Signed-off-by: David S. Miller <davem@davemloft.net>
Acked-by: David Howells <dhowells@redhat.com>
Hide details of maximum user address calculation in a new
asm/uaccess.h interface named user_addr_max().
Provide little-endian implementation in find_zero(), which should work
but can probably be improved.
Abstrace alignment check behind IS_UNALIGNED() macro.
Kill double-semicolon, noticed by David Howells.
Signed-off-by: David S. Miller <davem@davemloft.net>
Compute a mask that will only have 0x80 in the bytes which
had a zero in them. The formula is:
~(((x & 0x7f7f7f7f) + 0x7f7f7f7f) | x | 0x7f7f7f7f)
In the inner word iteration, we have to compute the "x | 0x7f7f7f7f"
part, so we can reuse that in the above calculation.
Once we have this mask, we perform divide and conquer to find the
highest 0x80 location.
Signed-off-by: David S. Miller <davem@davemloft.net>
Linus removed the end-of-address-space hackery from
fs/namei.c:do_getname() so we really have to validate these edge
conditions and cannot cheat any more (as x86 used to as well).
Move to a common C implementation like x86 did. And if both
src and dst are sufficiently aligned we'll do word at a time
copies and checks as well.
Signed-off-by: David S. Miller <davem@davemloft.net>
Otherwise if no references exist in the static kernel image,
we won't export the symbol properly to modules.
Signed-off-by: David S. Miller <davem@davemloft.net>
Based on copy from microblaze add ucmpdi2 implementation.
This fixes build of niu driver which failed with:
drivers/built-in.o: In function `niu_get_nfc':
niu.c:(.text+0x91494): undefined reference to `__ucmpdi2'
This driver will never be used on a sparc32 system,
but patch added to fix build breakage with all*config builds.
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
For the explicit calls to .udiv/.umul in assembler, I made a
mechanical (read as: safe) transformation. I didn't attempt
to make any simplifications.
In particular, __ndelay and __udelay can be simplified significantly.
Some of the %y reads are unnecessary and these routines have no need
any longer for allocating a register window, they can be leaf
functions.
Signed-off-by: David S. Miller <davem@davemloft.net>
We always have this instruction available, so no need to use
btfixup for it any more.
This also eradicates the whole of atomic_32.S and thus the
__atomic_begin and __atomic_end symbols completely.
Signed-off-by: David S. Miller <davem@davemloft.net>
Both sparc 32-bit's software divide assembler and MPILIB provide
clz_tab[] with identical contents.
Break it out into a seperate object file and select it when
SPARC32 or MPILIB is set.
Reported-by: Al Viro <viro@ZenIV.linux.org.uk>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: James Morris <jmorris@namei.org>
Many architectures don't want to pull in iomap.c,
so they ended up duplicating pci_iomap from that file.
That function isn't trivial, and we are going to modify it
https://lkml.org/lkml/2011/11/14/183
so the duplication hurts.
This reduces the scope of the problem significantly,
by moving pci_iomap to a separate file and
referencing that from all architectures.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
iQEcBAABAgAGBQJPBZXBAAoJECgfDbjSjVRpuuYIAIMD0wE96MuTOSBJX4VG8VAP
UyjL9dsfMRy8CKioQo5/fxpTY07YBCWmNauSSX7pzgcoUKBfYIGn4Z1qwGYsWK9M
CzLs6PXLTugw0FtKobHZl/klRTWEBS6YOUjp9x568rplwF+Ppk7b993uj7eS/g+e
T0mUKzqg4/UavbHd9+W5KgC4drQ5hgtu2WZHoUxBK4umnd3C2G+U82Sthg50o/XU
SC8IGm39K8I36HoIWgXj3Y7nkOP3mQELohOT4ZPiVSmLvGS4i47+ix75anO+8ZvZ
jxHr8RC85IK1Nd89NZhbKOyvx0QQiwoKUZaTwcWXJNSOADzZnM6icdIsodc+Elo=
=ccQZ
-----END PGP SIGNATURE-----
Merge tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost
lib: use generic pci_iomap on all architectures
Many architectures don't want to pull in iomap.c,
so they ended up duplicating pci_iomap from that file.
That function isn't trivial, and we are going to modify it
https://lkml.org/lkml/2011/11/14/183
so the duplication hurts.
This reduces the scope of the problem significantly,
by moving pci_iomap to a separate file and
referencing that from all architectures.
* tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost:
alpha: drop pci_iomap/pci_iounmap from pci-noop.c
mn10300: switch to GENERIC_PCI_IOMAP
mn10300: add missing __iomap markers
frv: switch to GENERIC_PCI_IOMAP
tile: switch to GENERIC_PCI_IOMAP
tile: don't panic on iomap
sparc: switch to GENERIC_PCI_IOMAP
sh: switch to GENERIC_PCI_IOMAP
powerpc: switch to GENERIC_PCI_IOMAP
parisc: switch to GENERIC_PCI_IOMAP
mips: switch to GENERIC_PCI_IOMAP
microblaze: switch to GENERIC_PCI_IOMAP
arm: switch to GENERIC_PCI_IOMAP
alpha: switch to GENERIC_PCI_IOMAP
lib: add GENERIC_PCI_IOMAP
lib: move GENERIC_IOMAP to lib/Kconfig
Fix up trivial conflicts due to changes nearby in arch/{m68k,score}/Kconfig
atomic24 support was used to semaphores in the past - but is no longer used.
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
sparc copied pci_iomap from generic code, probably to avoid
pulling the rest of iomap.c in. Since that's in
a separate file now, we can reuse the common implementation.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Properly return the original destination buffer pointer.
Signed-off-by: David S. Miller <davem@davemloft.net>
Tested-by: Kjetil Oftedal <oftedal@gmail.com>
This is setting things up so that we can correct the return
value, so that it properly returns the original destination
buffer pointer.
Signed-off-by: David S. Miller <davem@davemloft.net>
Tested-by: Kjetil Oftedal <oftedal@gmail.com>
Don't use floating point on Niagara2, use the traditional
plain Niagara code instead.
Unroll Niagara loops to 128 bytes for copy, and 256 bytes
for clear.
Signed-off-by: David S. Miller <davem@davemloft.net>
Should have been done in commit 1af08a1407f4 ("This is in preparation
for more generic atomic").
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>
Cc: Arun Sharma <asharma@fb.com>
Cc: David Miller <davem@davemloft.net>
Cc: "Hans-Christian Egtvedt" <hans-christian.egtvedt@atmel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This allows us to move duplicated code in <asm/atomic.h>
(atomic_inc_not_zero() for now) to <linux/atomic.h>
Signed-off-by: Arun Sharma <asharma@fb.com>
Reviewed-by: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: David Miller <davem@davemloft.net>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Acked-by: Mike Frysinger <vapier@gentoo.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
When we are in the label cc_dword_align, registers %o0 and %o1 have the same last 2 bits,
but it's not guaranteed one of them is zero. So we can get unaligned memory access
in label ccte. Example of parameters which lead to this:
%o0=0x7ff183e9, %o1=0x8e709e7d, %g1=3
With the parameters I had a memory corruption, when the additional 5 bytes were rewritten.
This patch corrects the error.
One comment to the patch. We don't care about the third bit in %o1, because cc_end_cruft
stores word or less.
Signed-off-by: Tkhai Kirill <tkhai@yandex.ru>
Signed-off-by: David S. Miller <davem@davemloft.net>
Use bitmap_set() instead of calling __set_bit() each bit.
Signed-off-by: Akinobu Mita <akinobu.mita@gmail.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: sparclinux@vger.kernel.org
Signed-off-by: David S. Miller <davem@davemloft.net>
SPIN_LOCK_UNLOCK is deprecated. Use the lockdep capable variant
instead.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: David S. Miller <davem@davemloft.net>
As noticed by Mikulas Patocka, the backoff macros don't
completely nop out for UP builds, we still get a
branch always and a delay slot nop.
Fix this by making the branch to the backoff spin loop
selective, then we can nop out the spin loop completely.
Signed-off-by: David S. Miller <davem@davemloft.net>
Simple microoptimizations for sparc64 atomic functions:
Save one instruction by using a delay slot.
Use %g1 instead of %g7, because %g1 is written earlier.
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Basically tip-off the powerpc code, use a 64-bit type and atomic64_t
interfaces for the implementation.
This gets us off of the by-hand asm code I wrote, which frankly I
think probably ruins I-cache hit rates.
The idea was the keep the call chains less deep, but anything taking
the rw-semaphores probably is also calling other stuff and therefore
already has allocated a stack-frame. So no real stack frame savings
ever.
Ben H. has posted patches to make powerpc use 64-bit too and with some
abstractions we can probably use a shared header file somewhere.
With suggestions from Sam Ravnborg.
Signed-off-by: David S. Miller <davem@davemloft.net>
128 bytes is sufficient for the register window save area, but the
calling conventions allow the callee to save up to 6 incoming argument
registers into the stack frame after the register window save area.
This means a minimal stack frame is 176 bytes (128 + (6 * 8)).
This fixes random crashes when using the function tracer.
Reported-by: Frederic Weisbecker <fweisbec@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
It's the only way we'll be able to implement the function
graph tracer properly.
A positive is that we no longer have to worry about the
linker over-optimizing the tail call, since we don't
use a tail call any more.
Signed-off-by: David S. Miller <davem@davemloft.net>
Check function_trace_stop at ftrace_caller
Toss mcount_call and dummy call of ftrace_stub, unnecessary.
Document problems we'll have if the final kernel image link
ever turns on relaxation.
Properly size 'ftrace_call' so it looks right when inspecting
instructions under gdb et al.
Signed-off-by: David S. Miller <davem@davemloft.net>
This mirrors x86 commit 9f0cf4adb6
(x86: Use __builtin_object_size() to validate the buffer size for copy_from_user())
Signed-off-by: David S. Miller <davem@davemloft.net>
Commit 4f70f7a91b
(sparc64: Implement IRQ stacks.) has two bugs.
First, the softirq range check forgets to subtract STACK_BIAS
before comparing with %sp. Next, on failure the wrong label
is jumped to, resulting in a bogus stack being loaded.
Reported-by: Igor Kovalenko <igor.v.kovalenko@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This is based upon a report from Chris Torek and his initial patch.
From Chris's report:
--------------------
This came up in testing kgdb, using the built-in tests -- turn
on CONFIG_KGDB_TESTS, then
echo V1 > /sys/module/kgdbts/parameters/kgdbts
-- but it would affect using kgdb if you were debugging and looking
at bad pointers.
--------------------
When we get a copy_{from,to}_user() request and the %asi is set to
something other than ASI_AIUS (which is userspace) then we branch off
to a routine called memcpy_user_stub(). It just does a straight
memcpy since we are copying from kernel to kernel in this case.
The logic was that since source and destination are both kernel
pointers we don't need to have exception checks.
But for what probe_kernel_{read,write}() is trying to do, we have to
have the checks, otherwise things like kgdb bad kernel pointer
accesses don't do the right thing.
Signed-off-by: David S. Miller <davem@davemloft.net>
This is an implementation of a suggestion made by Chris Torek:
--------------------
Something else I noticed in passing: the EX and EX_LD/EX_ST macros
scattered throughout the various .S files make a fair bit of .fixup
code, all of which does the same thing. At the cost of one symbol
in copy_in_user.S, you could just have one common two-instruction
retl-and-mov-1 fixup that they all share.
--------------------
The following is with a defconfig build:
text data bss dec hex filename
3972767 344024 584449 4901240 4ac978 vmlinux.orig
3968887 344024 584449 4897360 4aba50 vmlinux
Signed-off-by: David S. Miller <davem@davemloft.net>
Previously PeeCeeI.o was a library but it
was always pulled in due to insw and friends being exported
(at least for a modular kernel).
But this resulted in modpost failures if there where no in-kernel
users because then insw & friends were not linked in.
Fix this by including PeeCeeI.o in the kernel unconditionally.
The only drawback for this solution is that a nonmodular kernel
will always include insw & friends no matter if they are in use or not.
Reported-by: Meelis Roos <mroos@linux.ee>
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Move all applicable EXPORT_SYMBOL()s to the file where the respective
symbol is defined.
Removed all the includes that are no longer needed in sparc_ksyms_64.c
Comment all remaining EXPORT_SYMBOL()s in sparc_ksyms_64.c
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Additions by Julian Calaby:
* Moved EXPORT_SYMBOL()s for prom functions to their rightful places.
* Made some minor cleanups to the includes and comments of sparc_ksyms_64.c
* Updated and tidied commit message.
* Rebased patch over sparc-2.6.git HEAD.
* Ensured that all modified files have the correct includes.
Signed-off-by: Julian Calaby <julian.calaby@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Remove the duplicate entries from kernel/sparc_ksyms_*.c
The rationale behind this is that the EXPORT_SYMBOL() should be close to
their definition and we cannot add designate a symbol to be exported in
assembler so at least put it in a file in the same directory.
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Additions by Julian Calaby:
* Rebased over sparc-2.6.git HEAD
Signed-off-by: Julian Calaby <julian.calaby@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Use the new asm/asm.h header to help commonize the
strlen assembler between 32-bit and 64-bit
While we're here, use proper linux/linkage.h macros
instead of by-hand stuff.
Signed-off-by: David S. Miller <davem@davemloft.net>
o Renamed files in sparc64 to <name>_64.S when identical
to sparc32 files.
o iomap.c were equal for sparc32 and sparc64
o adjusted sparc/Makefile now we have only one lib/
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Identical named files renamed to <name>_32.S
Refactored Makefile to prepare for unification.
Linking order was altered slightly - but this is a lib.a file so
it should not matter.
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch removes the CVS keywords that weren't updated for a long time
from comments.
Signed-off-by: Adrian Bunk <bunk@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Almost all implementations of pci_iomap() in the kernel, including the generic
lib/iomap.c one, copies the content of a struct resource into unsigned long's
which will break on 32 bits platforms with 64 bits resources.
This fixes all definitions of pci_iomap() to use resource_size_t. I also
"fixed" the 64bits arch for consistency.
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: <linux-arch@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
ld will generate an unique named section when assembler do not use
"ax" but gcc does. Add the missing annotation.
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Sparc optimized memset (arch/sparc/lib/memset.S) does not fill last
byte of the memory area, if area size is less than 8 bytes and start
address is not word (4-bytes) aligned.
Here is code chunk where bug located:
/* %o0 - memory address, %o1 - size, %g3 - value */
8:
add %o0, 1, %o0
subcc %o1, 1, %o1
bne,a 8b
stb %g3, [%o0 - 1]
This code should write byte every loop iteration, but last time delay
instruction stb is not executed because branch instruction sets
"annul" bit.
Patch replaces bne,a by bne instruction.
Error can be reproduced by simple kernel module:
--------------------
#include <linux/module.h>
#include <linux/config.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <string.h>
static void do_memset(void **p, int size)
{
memset(p, 0x00, size);
}
static int __init memset_test_init(void)
{
char fooc[8];
int *fooi;
memset(fooc, 0xba, sizeof(fooc));
do_memset((void**)(fooc + 3), 1);
fooi = (int*) fooc;
printk("%08X %08X\n", fooi[0], fooi[1]);
return -1;
}
static void __exit memset_test_cleanup(void)
{
return;
}
module_init(memset_test_init);
module_exit(memset_test_cleanup);
MODULE_LICENSE("GPL");
EXPORT_NO_SYMBOLS;
--------------------
Signed-off-by: Alexander Shmelev <ashmelev@task.sun.mcst.ru>
Signed-off-by: David S. Miller <davem@davemloft.net>
Fix 6197fe4d72
arch/sparc/lib/atomic32.c: In function '__cmpxchg_u32':
arch/sparc/lib/atomic32.c:127: error: 'addr' undeclared (first use in this function)
arch/sparc/lib/atomic32.c:127: error: (Each undeclared identifier is reported only once
arch/sparc/lib/atomic32.c:127: error: for each function it appears in.)
I assume this is what was intended..
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: William Irwin <wli@holomorphy.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Remove includes of <linux/smp_lock.h> where it is not used/needed.
Suggested by Al Viro.
Builds cleanly on x86_64, i386, alpha, ia64, powerpc, sparc,
sparc64, and arm (all 59 defconfigs).
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
commit b19cbe2a16 [BRIDGE]: Fix fdb RCU
race
breaks sparc SMP build because atomic_add_unless is not exported.
This patch exports atomic_add_unless and atomic_cmpxchg.
Signed-off-by: Robert Reif <reif@earthlink.net>
Signed-off-by: David S. Miller <davem@davemloft.net>
Recent workqueue changes basically make this a formal requirement.
Also, move atomic32.o from lib-y to obj-y since it exports symbols
to modules.
Signed-off-by: David S. Miller <davem@davemloft.net>
read_trylock() is broken on sparc32 (doesn't build and didn't work
right, actually). Proposed fix:
- make "writer holds lock" distinguishable from "reader tries to grab
lock"
- have __raw_read_trylock() try to acquire the mutex (in LSB of lock),
terminating spin if we see that there's writer holding it. Then do
the rest as we do in read_lock().
Thanks to Ingo for discussion...
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Actually, since we use the same code for all the copying
types in and out of userspace, we check at runtime whether
preemption is disabled.
Signed-off-by: David S. Miller <davem@davemloft.net>
This fixes one build error introduced in sparc with the patch of Oct 30,
resent Nov 4 "[patch 3/5] atomic: atomic_inc_not_zero" I still can't get
sparc to build, but at least it gets further after I remove this line.
Apparently, this change was agreed to by Andrew and Nick on Nov 14, but
everyone thought someone else was doing it.
Signed-off-by: Paul Jackson <pj@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Introduce an atomic_inc_not_zero operation. Make this a special case of
atomic_add_unless because lockless pagecache actually wants
atomic_inc_not_negativeone due to its offset refcount.
Signed-off-by: Nick Piggin <npiggin@suse.de>
Cc: "Paul E. McKenney" <paulmck@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
a) in smp_lock.h #include of sched.h and spinlock.h moved under #ifdef
CONFIG_LOCK_KERNEL.
b) interrupt.h now explicitly pulls sched.h (not via smp_lock.h from
hardirq.h as it used to)
c) in three more places we need changes to compensate for (a) - one place
in arch/sparc needs string.h now, hardirq.h needs forward declaration of
task_struct and preempt.h needs direct include of thread_info.h.
d) thread_info-related helpers in sched.h and thread_info.h put under
ifndef __HAVE_THREAD_FUNCTIONS. Obviously safe.
Signed-off-by: Al Viro <viro@parcelfarce.linux.theplanet.co.uk>
Signed-off-by: Roman Zippel <zippel@linux-m68k.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
From: Al Viro <viro@ZenIV.linux.org.uk>
Instead of playing all of these hand-coded assembler aliasing games,
just translate symbol names in the name space ".sym" to "_Sym" at
module load time.
Signed-off-by: David S. Miller <davem@davemloft.net>
Peter Zijlstra