Commit Graph

3 Commits

Author SHA1 Message Date
Ingo Molnar c5905afb0e static keys: Introduce 'struct static_key', static_key_true()/false() and static_key_slow_[inc|dec]()
So here's a boot tested patch on top of Jason's series that does
all the cleanups I talked about and turns jump labels into a
more intuitive to use facility. It should also address the
various misconceptions and confusions that surround jump labels.

Typical usage scenarios:

        #include <linux/static_key.h>

        struct static_key key = STATIC_KEY_INIT_TRUE;

        if (static_key_false(&key))
                do unlikely code
        else
                do likely code

Or:

        if (static_key_true(&key))
                do likely code
        else
                do unlikely code

The static key is modified via:

        static_key_slow_inc(&key);
        ...
        static_key_slow_dec(&key);

The 'slow' prefix makes it abundantly clear that this is an
expensive operation.

I've updated all in-kernel code to use this everywhere. Note
that I (intentionally) have not pushed through the rename
blindly through to the lowest levels: the actual jump-label
patching arch facility should be named like that, so we want to
decouple jump labels from the static-key facility a bit.

On non-jump-label enabled architectures static keys default to
likely()/unlikely() branches.

Signed-off-by: Ingo Molnar <mingo@elte.hu>
Acked-by: Jason Baron <jbaron@redhat.com>
Acked-by: Steven Rostedt <rostedt@goodmis.org>
Cc: a.p.zijlstra@chello.nl
Cc: mathieu.desnoyers@efficios.com
Cc: davem@davemloft.net
Cc: ddaney.cavm@gmail.com
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Link: http://lkml.kernel.org/r/20120222085809.GA26397@elte.hu
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2012-02-24 10:05:59 +01:00
Anton Blanchard c113a3aee2 powerpc: Jump label misalignment causes oops at boot
I hit an oops at boot on the first instruction of timer_cpu_notify:

NIP [c000000000722f88] .timer_cpu_notify+0x0/0x388

The code should look like:

c000000000722f78:       eb e9 00 30     ld      r31,48(r9)
c000000000722f7c:       2f bf 00 00     cmpdi   cr7,r31,0
c000000000722f80:       40 9e ff 44     bne+    cr7,c000000000722ec4
c000000000722f84:       4b ff ff 74     b       c000000000722ef8

c000000000722f88 <.timer_cpu_notify>:
c000000000722f88:       7c 08 02 a6     mflr    r0
c000000000722f8c:       2f a4 00 07     cmpdi   cr7,r4,7
c000000000722f90:       fb c1 ff f0     std     r30,-16(r1)
c000000000722f94:       fb 61 ff d8     std     r27,-40(r1)

But the oops output shows:

eb61ffd8 eb81ffe0 eba1ffe8 ebc1fff0 7c0803a6 ebe1fff8 4e800020
00000000 ebe90030 c0000000 00ad0a28 00000000 2fa40007 fbc1fff0 fb61ffd8

So we scribbled over our instructions with c000000000ad0a28, which
is an address inside the jump_table ELF section.

It turns out the jump_table section is only aligned to 8 bytes but
we are aligning our entries within the section to 16 bytes. This
means our entries are offset from the table:

c000000000acd4a8 <__start___jump_table>:
        ...
c000000000ad0a10:       c0 00 00 00     lfs     f0,0(0)
c000000000ad0a14:       00 70 cd 5c     .long 0x70cd5c
c000000000ad0a18:       c0 00 00 00     lfs     f0,0(0)
c000000000ad0a1c:       00 70 cd 90     .long 0x70cd90
c000000000ad0a20:       c0 00 00 00     lfs     f0,0(0)
c000000000ad0a24:       00 ac a4 20     .long 0xaca420

And the jump table sort code gets very confused and writes into the
wrong spot. Remove the alignment, and also remove the padding since
we it saves some space and we shouldn't need it.

Signed-off-by: Anton Blanchard <anton@samba.org>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2011-08-05 14:47:55 +10:00
Michael Ellerman ac5f89c7d8 powerpc: Add jump label support
This patch adds support for the new "jump label" feature.

Unlike x86 and sparc we just merrily patch the code with no locks etc,
as far as I know this is safe, but I'm not really sure what the x86/sparc
code is protecting against so maybe it's not.

I also don't see any reason for us to implement the poke_early() routine,
even though sparc does.

[BenH: Updated the patch to upstream generic changes]

Signed-off-by: Michael Ellerman <michael@ellerman.id.au>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2011-07-01 13:48:55 +10:00