[PATCH] rcu: add a prefetch() in rcu_do_batch()
On some workloads, (for example when lot of close() syscalls are done), RCU qlen can be quite large, and RCU heads are no longer in cpu cache when rcu_do_batch() is called. This patch adds a prefetch() in rcu_do_batch() to give CPU a hint to bring back cache lines containing 'struct rcu_head's. Most list manipulations macros include prefetch(), but not open coded ones (at least with current C compilers :) ) I got a nice speedup on a trivial benchmark (3.48 us per iteration instead of 3.95 us on a 1.6 GHz Pentium-M) while (1) { pipe(p); close(fd[0]); close(fd[1]);} Signed-off-by: Eric Dumazet <dada1@cosmosbay.com> Cc: "Paul E. McKenney" <paulmck@us.ibm.com> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This commit is contained in:
parent
9a0efbb879
commit
1c69d921ed
|
@ -235,12 +235,14 @@ static void rcu_do_batch(struct rcu_data *rdp)
|
||||||
|
|
||||||
list = rdp->donelist;
|
list = rdp->donelist;
|
||||||
while (list) {
|
while (list) {
|
||||||
next = rdp->donelist = list->next;
|
next = list->next;
|
||||||
|
prefetch(next);
|
||||||
list->func(list);
|
list->func(list);
|
||||||
list = next;
|
list = next;
|
||||||
if (++count >= rdp->blimit)
|
if (++count >= rdp->blimit)
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
|
rdp->donelist = list;
|
||||||
|
|
||||||
local_irq_disable();
|
local_irq_disable();
|
||||||
rdp->qlen -= count;
|
rdp->qlen -= count;
|
||||||
|
|
Loading…
Reference in New Issue