UbiquitousOS
/
OpenCloudOS-Kernel
mirror of https://gitee.com/OpenCloudOS/OpenCloudOS-Kernel.git
11
0
Fork 0
Code Issues Projects Releases Wiki Activity
OpenCloudOS-Kernel/arch/sparc/lib/atomic_64.S

134 lines
2.8 KiB
ArmAsm
Raw Normal View History

[SPARC64]: Implement atomic backoff. When the cpu count is high and contention hits an atomic object, the processors can synchronize such that some cpus continually get knocked out and cannot complete the atomic update. So implement an exponential backoff when SMP. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-10-16 07:41:44 +08:00
/* atomic.S: These things are too big to do inline.
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
*
sparc: Support atomic64_dec_if_positive properly. Sparc32 already supported it, as a consequence of using the generic atomic64 implementation. And the sparc64 implementation is rather trivial. This allows us to set ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE for all of sparc, and avoid the annoying warning from lib/atomic64_test.c Signed-off-by: David S. Miller <davem@davemloft.net>
2012-11-10 11:37:59 +08:00
* Copyright (C) 1999, 2007 2012 David S. Miller (davem@davemloft.net)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
*/
sparc: Convert some assembler over to linakge.h's ENTRY/ENDPROC Use those, instead of doing it all by hand. Signed-off-by: David S. Miller <davem@davemloft.net>
2012-05-12 11:33:22 +08:00
#include <linux/linkage.h>
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
#include <asm/asi.h>
[SPARC64]: Implement atomic backoff. When the cpu count is high and contention hits an atomic object, the processors can synchronize such that some cpus continually get knocked out and cannot complete the atomic update. So implement an exponential backoff when SMP. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-10-16 07:41:44 +08:00
#include <asm/backoff.h>
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
.text
/* Two versions of the atomic routines, one that
* does not return a value and does not perform
* memory barriers, and a second which returns
* a value and does the barriers.
*/
sparc: Convert some assembler over to linakge.h's ENTRY/ENDPROC Use those, instead of doing it all by hand. Signed-off-by: David S. Miller <davem@davemloft.net>
2012-05-12 11:33:22 +08:00
ENTRY(atomic_add) /* %o0 = increment, %o1 = atomic_ptr */
[SPARC64]: Implement atomic backoff. When the cpu count is high and contention hits an atomic object, the processors can synchronize such that some cpus continually get knocked out and cannot complete the atomic update. So implement an exponential backoff when SMP. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-10-16 07:41:44 +08:00
BACKOFF_SETUP(%o2)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
1: lduw [%o1], %g1
add %g1, %o0, %g7
cas [%o1], %g1, %g7
cmp %g1, %g7
sparc64: Make lock backoff really a NOP on UP builds. As noticed by Mikulas Patocka, the backoff macros don't completely nop out for UP builds, we still get a branch always and a delay slot nop. Fix this by making the branch to the backoff spin loop selective, then we can nop out the spin loop completely. Signed-off-by: David S. Miller <davem@davemloft.net>
2010-08-19 13:53:26 +08:00
bne,pn %icc, BACKOFF_LABEL(2f, 1b)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
nop
retl
nop
[SPARC64]: Implement atomic backoff. When the cpu count is high and contention hits an atomic object, the processors can synchronize such that some cpus continually get knocked out and cannot complete the atomic update. So implement an exponential backoff when SMP. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-10-16 07:41:44 +08:00
2: BACKOFF_SPIN(%o2, %o3, 1b)
sparc: Convert some assembler over to linakge.h's ENTRY/ENDPROC Use those, instead of doing it all by hand. Signed-off-by: David S. Miller <davem@davemloft.net>
2012-05-12 11:33:22 +08:00
ENDPROC(atomic_add)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
sparc: Convert some assembler over to linakge.h's ENTRY/ENDPROC Use those, instead of doing it all by hand. Signed-off-by: David S. Miller <davem@davemloft.net>
2012-05-12 11:33:22 +08:00
ENTRY(atomic_sub) /* %o0 = decrement, %o1 = atomic_ptr */
[SPARC64]: Implement atomic backoff. When the cpu count is high and contention hits an atomic object, the processors can synchronize such that some cpus continually get knocked out and cannot complete the atomic update. So implement an exponential backoff when SMP. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-10-16 07:41:44 +08:00
BACKOFF_SETUP(%o2)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
1: lduw [%o1], %g1
sub %g1, %o0, %g7
cas [%o1], %g1, %g7
cmp %g1, %g7
sparc64: Make lock backoff really a NOP on UP builds. As noticed by Mikulas Patocka, the backoff macros don't completely nop out for UP builds, we still get a branch always and a delay slot nop. Fix this by making the branch to the backoff spin loop selective, then we can nop out the spin loop completely. Signed-off-by: David S. Miller <davem@davemloft.net>
2010-08-19 13:53:26 +08:00
bne,pn %icc, BACKOFF_LABEL(2f, 1b)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
nop
retl
nop
[SPARC64]: Implement atomic backoff. When the cpu count is high and contention hits an atomic object, the processors can synchronize such that some cpus continually get knocked out and cannot complete the atomic update. So implement an exponential backoff when SMP. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-10-16 07:41:44 +08:00
2: BACKOFF_SPIN(%o2, %o3, 1b)
sparc: Convert some assembler over to linakge.h's ENTRY/ENDPROC Use those, instead of doing it all by hand. Signed-off-by: David S. Miller <davem@davemloft.net>
2012-05-12 11:33:22 +08:00
ENDPROC(atomic_sub)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
sparc: Convert some assembler over to linakge.h's ENTRY/ENDPROC Use those, instead of doing it all by hand. Signed-off-by: David S. Miller <davem@davemloft.net>
2012-05-12 11:33:22 +08:00
ENTRY(atomic_add_ret) /* %o0 = increment, %o1 = atomic_ptr */
[SPARC64]: Implement atomic backoff. When the cpu count is high and contention hits an atomic object, the processors can synchronize such that some cpus continually get knocked out and cannot complete the atomic update. So implement an exponential backoff when SMP. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-10-16 07:41:44 +08:00
BACKOFF_SETUP(%o2)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
1: lduw [%o1], %g1
add %g1, %o0, %g7
cas [%o1], %g1, %g7
cmp %g1, %g7
sparc64: Make lock backoff really a NOP on UP builds. As noticed by Mikulas Patocka, the backoff macros don't completely nop out for UP builds, we still get a branch always and a delay slot nop. Fix this by making the branch to the backoff spin loop selective, then we can nop out the spin loop completely. Signed-off-by: David S. Miller <davem@davemloft.net>
2010-08-19 13:53:26 +08:00
bne,pn %icc, BACKOFF_LABEL(2f, 1b)
sparc64: simple microoptimizations for atomic functions Simple microoptimizations for sparc64 atomic functions: Save one instruction by using a delay slot. Use %g1 instead of %g7, because %g1 is written earlier. Signed-off-by: Mikulas Patocka <mpatocka@redhat.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2010-08-18 16:03:37 +08:00
add %g1, %o0, %g1
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
retl
sparc64: simple microoptimizations for atomic functions Simple microoptimizations for sparc64 atomic functions: Save one instruction by using a delay slot. Use %g1 instead of %g7, because %g1 is written earlier. Signed-off-by: Mikulas Patocka <mpatocka@redhat.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2010-08-18 16:03:37 +08:00
sra %g1, 0, %o0
[SPARC64]: Implement atomic backoff. When the cpu count is high and contention hits an atomic object, the processors can synchronize such that some cpus continually get knocked out and cannot complete the atomic update. So implement an exponential backoff when SMP. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-10-16 07:41:44 +08:00
2: BACKOFF_SPIN(%o2, %o3, 1b)
sparc: Convert some assembler over to linakge.h's ENTRY/ENDPROC Use those, instead of doing it all by hand. Signed-off-by: David S. Miller <davem@davemloft.net>
2012-05-12 11:33:22 +08:00
ENDPROC(atomic_add_ret)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
sparc: Convert some assembler over to linakge.h's ENTRY/ENDPROC Use those, instead of doing it all by hand. Signed-off-by: David S. Miller <davem@davemloft.net>
2012-05-12 11:33:22 +08:00
ENTRY(atomic_sub_ret) /* %o0 = decrement, %o1 = atomic_ptr */
[SPARC64]: Implement atomic backoff. When the cpu count is high and contention hits an atomic object, the processors can synchronize such that some cpus continually get knocked out and cannot complete the atomic update. So implement an exponential backoff when SMP. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-10-16 07:41:44 +08:00
BACKOFF_SETUP(%o2)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
1: lduw [%o1], %g1
sub %g1, %o0, %g7
cas [%o1], %g1, %g7
cmp %g1, %g7
sparc64: Make lock backoff really a NOP on UP builds. As noticed by Mikulas Patocka, the backoff macros don't completely nop out for UP builds, we still get a branch always and a delay slot nop. Fix this by making the branch to the backoff spin loop selective, then we can nop out the spin loop completely. Signed-off-by: David S. Miller <davem@davemloft.net>
2010-08-19 13:53:26 +08:00
bne,pn %icc, BACKOFF_LABEL(2f, 1b)
sparc64: simple microoptimizations for atomic functions Simple microoptimizations for sparc64 atomic functions: Save one instruction by using a delay slot. Use %g1 instead of %g7, because %g1 is written earlier. Signed-off-by: Mikulas Patocka <mpatocka@redhat.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2010-08-18 16:03:37 +08:00
sub %g1, %o0, %g1
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
retl
sparc64: simple microoptimizations for atomic functions Simple microoptimizations for sparc64 atomic functions: Save one instruction by using a delay slot. Use %g1 instead of %g7, because %g1 is written earlier. Signed-off-by: Mikulas Patocka <mpatocka@redhat.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2010-08-18 16:03:37 +08:00
sra %g1, 0, %o0
[SPARC64]: Implement atomic backoff. When the cpu count is high and contention hits an atomic object, the processors can synchronize such that some cpus continually get knocked out and cannot complete the atomic update. So implement an exponential backoff when SMP. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-10-16 07:41:44 +08:00
2: BACKOFF_SPIN(%o2, %o3, 1b)
sparc: Convert some assembler over to linakge.h's ENTRY/ENDPROC Use those, instead of doing it all by hand. Signed-off-by: David S. Miller <davem@davemloft.net>
2012-05-12 11:33:22 +08:00
ENDPROC(atomic_sub_ret)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
sparc: Convert some assembler over to linakge.h's ENTRY/ENDPROC Use those, instead of doing it all by hand. Signed-off-by: David S. Miller <davem@davemloft.net>
2012-05-12 11:33:22 +08:00
ENTRY(atomic64_add) /* %o0 = increment, %o1 = atomic_ptr */
[SPARC64]: Implement atomic backoff. When the cpu count is high and contention hits an atomic object, the processors can synchronize such that some cpus continually get knocked out and cannot complete the atomic update. So implement an exponential backoff when SMP. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-10-16 07:41:44 +08:00
BACKOFF_SETUP(%o2)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
1: ldx [%o1], %g1
add %g1, %o0, %g7
casx [%o1], %g1, %g7
cmp %g1, %g7
sparc64: Make lock backoff really a NOP on UP builds. As noticed by Mikulas Patocka, the backoff macros don't completely nop out for UP builds, we still get a branch always and a delay slot nop. Fix this by making the branch to the backoff spin loop selective, then we can nop out the spin loop completely. Signed-off-by: David S. Miller <davem@davemloft.net>
2010-08-19 13:53:26 +08:00
bne,pn %xcc, BACKOFF_LABEL(2f, 1b)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
nop
retl
nop
[SPARC64]: Implement atomic backoff. When the cpu count is high and contention hits an atomic object, the processors can synchronize such that some cpus continually get knocked out and cannot complete the atomic update. So implement an exponential backoff when SMP. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-10-16 07:41:44 +08:00
2: BACKOFF_SPIN(%o2, %o3, 1b)
sparc: Convert some assembler over to linakge.h's ENTRY/ENDPROC Use those, instead of doing it all by hand. Signed-off-by: David S. Miller <davem@davemloft.net>
2012-05-12 11:33:22 +08:00
ENDPROC(atomic64_add)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
sparc: Convert some assembler over to linakge.h's ENTRY/ENDPROC Use those, instead of doing it all by hand. Signed-off-by: David S. Miller <davem@davemloft.net>
2012-05-12 11:33:22 +08:00
ENTRY(atomic64_sub) /* %o0 = decrement, %o1 = atomic_ptr */
[SPARC64]: Implement atomic backoff. When the cpu count is high and contention hits an atomic object, the processors can synchronize such that some cpus continually get knocked out and cannot complete the atomic update. So implement an exponential backoff when SMP. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-10-16 07:41:44 +08:00
BACKOFF_SETUP(%o2)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
1: ldx [%o1], %g1
sub %g1, %o0, %g7
casx [%o1], %g1, %g7
cmp %g1, %g7
sparc64: Make lock backoff really a NOP on UP builds. As noticed by Mikulas Patocka, the backoff macros don't completely nop out for UP builds, we still get a branch always and a delay slot nop. Fix this by making the branch to the backoff spin loop selective, then we can nop out the spin loop completely. Signed-off-by: David S. Miller <davem@davemloft.net>
2010-08-19 13:53:26 +08:00
bne,pn %xcc, BACKOFF_LABEL(2f, 1b)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
nop
retl
nop
[SPARC64]: Implement atomic backoff. When the cpu count is high and contention hits an atomic object, the processors can synchronize such that some cpus continually get knocked out and cannot complete the atomic update. So implement an exponential backoff when SMP. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-10-16 07:41:44 +08:00
2: BACKOFF_SPIN(%o2, %o3, 1b)
sparc: Convert some assembler over to linakge.h's ENTRY/ENDPROC Use those, instead of doing it all by hand. Signed-off-by: David S. Miller <davem@davemloft.net>
2012-05-12 11:33:22 +08:00
ENDPROC(atomic64_sub)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
sparc: Convert some assembler over to linakge.h's ENTRY/ENDPROC Use those, instead of doing it all by hand. Signed-off-by: David S. Miller <davem@davemloft.net>
2012-05-12 11:33:22 +08:00
ENTRY(atomic64_add_ret) /* %o0 = increment, %o1 = atomic_ptr */
[SPARC64]: Implement atomic backoff. When the cpu count is high and contention hits an atomic object, the processors can synchronize such that some cpus continually get knocked out and cannot complete the atomic update. So implement an exponential backoff when SMP. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-10-16 07:41:44 +08:00
BACKOFF_SETUP(%o2)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
1: ldx [%o1], %g1
add %g1, %o0, %g7
casx [%o1], %g1, %g7
cmp %g1, %g7
sparc64: Make lock backoff really a NOP on UP builds. As noticed by Mikulas Patocka, the backoff macros don't completely nop out for UP builds, we still get a branch always and a delay slot nop. Fix this by making the branch to the backoff spin loop selective, then we can nop out the spin loop completely. Signed-off-by: David S. Miller <davem@davemloft.net>
2010-08-19 13:53:26 +08:00
bne,pn %xcc, BACKOFF_LABEL(2f, 1b)
[SPARC64]: Avoid membar instructions in delay slots. In particular, avoid membar instructions in the delay slot of a jmpl instruction. UltraSPARC-I, II, IIi, and IIe have a bug, documented in the UltraSPARC-IIi User's Manual, Appendix K, Erratum 51 The long and short of it is that if the IMU unit misses on a branch or jmpl, and there is a store buffer synchronizing membar in the delay slot, the chip can stop fetching instructions. If interrupts are enabled or some other trap is enabled, the chip will unwedge itself, but performance will suffer. We already had a workaround for this bug in a few spots, but it's better to have the entire tree sanitized for this rule. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-06-28 06:42:04 +08:00
nop
sparc64: simple microoptimizations for atomic functions Simple microoptimizations for sparc64 atomic functions: Save one instruction by using a delay slot. Use %g1 instead of %g7, because %g1 is written earlier. Signed-off-by: Mikulas Patocka <mpatocka@redhat.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2010-08-18 16:03:37 +08:00
retl
add %g1, %o0, %o0
[SPARC64]: Implement atomic backoff. When the cpu count is high and contention hits an atomic object, the processors can synchronize such that some cpus continually get knocked out and cannot complete the atomic update. So implement an exponential backoff when SMP. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-10-16 07:41:44 +08:00
2: BACKOFF_SPIN(%o2, %o3, 1b)
sparc: Convert some assembler over to linakge.h's ENTRY/ENDPROC Use those, instead of doing it all by hand. Signed-off-by: David S. Miller <davem@davemloft.net>
2012-05-12 11:33:22 +08:00
ENDPROC(atomic64_add_ret)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
sparc: Convert some assembler over to linakge.h's ENTRY/ENDPROC Use those, instead of doing it all by hand. Signed-off-by: David S. Miller <davem@davemloft.net>
2012-05-12 11:33:22 +08:00
ENTRY(atomic64_sub_ret) /* %o0 = decrement, %o1 = atomic_ptr */
[SPARC64]: Implement atomic backoff. When the cpu count is high and contention hits an atomic object, the processors can synchronize such that some cpus continually get knocked out and cannot complete the atomic update. So implement an exponential backoff when SMP. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-10-16 07:41:44 +08:00
BACKOFF_SETUP(%o2)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
1: ldx [%o1], %g1
sub %g1, %o0, %g7
casx [%o1], %g1, %g7
cmp %g1, %g7
sparc64: Make lock backoff really a NOP on UP builds. As noticed by Mikulas Patocka, the backoff macros don't completely nop out for UP builds, we still get a branch always and a delay slot nop. Fix this by making the branch to the backoff spin loop selective, then we can nop out the spin loop completely. Signed-off-by: David S. Miller <davem@davemloft.net>
2010-08-19 13:53:26 +08:00
bne,pn %xcc, BACKOFF_LABEL(2f, 1b)
[SPARC64]: Avoid membar instructions in delay slots. In particular, avoid membar instructions in the delay slot of a jmpl instruction. UltraSPARC-I, II, IIi, and IIe have a bug, documented in the UltraSPARC-IIi User's Manual, Appendix K, Erratum 51 The long and short of it is that if the IMU unit misses on a branch or jmpl, and there is a store buffer synchronizing membar in the delay slot, the chip can stop fetching instructions. If interrupts are enabled or some other trap is enabled, the chip will unwedge itself, but performance will suffer. We already had a workaround for this bug in a few spots, but it's better to have the entire tree sanitized for this rule. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-06-28 06:42:04 +08:00
nop
sparc64: simple microoptimizations for atomic functions Simple microoptimizations for sparc64 atomic functions: Save one instruction by using a delay slot. Use %g1 instead of %g7, because %g1 is written earlier. Signed-off-by: Mikulas Patocka <mpatocka@redhat.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2010-08-18 16:03:37 +08:00
retl
sub %g1, %o0, %o0
[SPARC64]: Implement atomic backoff. When the cpu count is high and contention hits an atomic object, the processors can synchronize such that some cpus continually get knocked out and cannot complete the atomic update. So implement an exponential backoff when SMP. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-10-16 07:41:44 +08:00
2: BACKOFF_SPIN(%o2, %o3, 1b)
sparc: Convert some assembler over to linakge.h's ENTRY/ENDPROC Use those, instead of doing it all by hand. Signed-off-by: David S. Miller <davem@davemloft.net>
2012-05-12 11:33:22 +08:00
ENDPROC(atomic64_sub_ret)
sparc: Support atomic64_dec_if_positive properly. Sparc32 already supported it, as a consequence of using the generic atomic64 implementation. And the sparc64 implementation is rather trivial. This allows us to set ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE for all of sparc, and avoid the annoying warning from lib/atomic64_test.c Signed-off-by: David S. Miller <davem@davemloft.net>
2012-11-10 11:37:59 +08:00
ENTRY(atomic64_dec_if_positive) /* %o0 = atomic_ptr */
BACKOFF_SETUP(%o2)
1: ldx [%o0], %g1
brlez,pn %g1, 3f
sub %g1, 1, %g7
casx [%o0], %g1, %g7
cmp %g1, %g7
bne,pn %xcc, BACKOFF_LABEL(2f, 1b)
nop
3: retl
sub %g1, 1, %o0
2: BACKOFF_SPIN(%o2, %o3, 1b)
ENDPROC(atomic64_dec_if_positive)