2019-05-27 14:55:06 +08:00
|
|
|
/* SPDX-License-Identifier: GPL-2.0-or-later */
|
2016-03-26 05:22:08 +08:00
|
|
|
/*
|
|
|
|
|
* A generic stack depot implementation
|
|
|
|
|
*
|
|
|
|
|
* Author: Alexander Potapenko <glider@google.com>
|
|
|
|
|
* Copyright (C) 2016 Google, Inc.
|
|
|
|
|
*
|
|
|
|
|
* Based on code by Dmitry Chernenkov.
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
#ifndef _LINUX_STACKDEPOT_H
|
|
|
|
|
#define _LINUX_STACKDEPOT_H
|
|
|
|
|
|
lib/stackdepot: include gfp.h
Patch series "stackdepot, kasan, workqueue: Avoid expanding stackdepot
slabs when holding raw_spin_lock", v2.
Shuah Khan reported [1]:
| When CONFIG_PROVE_RAW_LOCK_NESTING=y and CONFIG_KASAN are enabled,
| kasan_record_aux_stack() runs into "BUG: Invalid wait context" when
| it tries to allocate memory attempting to acquire spinlock in page
| allocation code while holding workqueue pool raw_spinlock.
|
| There are several instances of this problem when block layer tries
| to __queue_work(). Call trace from one of these instances is below:
|
| kblockd_mod_delayed_work_on()
| mod_delayed_work_on()
| __queue_delayed_work()
| __queue_work() (rcu_read_lock, raw_spin_lock pool->lock held)
| insert_work()
| kasan_record_aux_stack()
| kasan_save_stack()
| stack_depot_save()
| alloc_pages()
| __alloc_pages()
| get_page_from_freelist()
| rm_queue()
| rm_queue_pcplist()
| local_lock_irqsave(&pagesets.lock, flags);
| [ BUG: Invalid wait context triggered ]
PROVE_RAW_LOCK_NESTING is pointing out that (on RT kernels) the locking
rules are being violated. More generally, memory is being allocated
from a non-preemptive context (raw_spin_lock'd c-s) where it is not
allowed.
To properly fix this, we must prevent stackdepot from replenishing its
"stack slab" pool if memory allocations cannot be done in the current
context: it's a bug to use either GFP_ATOMIC nor GFP_NOWAIT in certain
non-preemptive contexts, including raw_spin_locks (see gfp.h and commit
ab00db216c9c7).
The only downside is that saving a stack trace may fail if: stackdepot
runs out of space AND the same stack trace has not been recorded before.
I expect this to be unlikely, and a simple experiment (boot the kernel)
didn't result in any failure to record stack trace from insert_work().
The series includes a few minor fixes to stackdepot that I noticed in
preparing the series. It then introduces __stack_depot_save(), which
exposes the option to force stackdepot to not allocate any memory.
Finally, KASAN is changed to use the new stackdepot interface and
provide kasan_record_aux_stack_noalloc(), which is then used by
workqueue code.
[1] https://lkml.kernel.org/r/20210902200134.25603-1-skhan@linuxfoundation.org
This patch (of 6):
<linux/stackdepot.h> refers to gfp_t, but doesn't include gfp.h.
Fix it by including <linux/gfp.h>.
Link: https://lkml.kernel.org/r/20210913112609.2651084-1-elver@google.com
Link: https://lkml.kernel.org/r/20210913112609.2651084-2-elver@google.com
Signed-off-by: Marco Elver <elver@google.com>
Tested-by: Shuah Khan <skhan@linuxfoundation.org>
Acked-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Reviewed-by: Andrey Konovalov <andreyknvl@gmail.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Walter Wu <walter-zh.wu@mediatek.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Vijayanand Jitta <vjitta@codeaurora.org>
Cc: Vinayak Menon <vinmenon@codeaurora.org>
Cc: "Gustavo A. R. Silva" <gustavoars@kernel.org>
Cc: Taras Madan <tarasmadan@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2021-11-06 04:35:33 +08:00
|
|
|
#include <linux/gfp.h>
|
|
|
|
|
|
2016-03-26 05:22:08 +08:00
|
|
|
typedef u32 depot_stack_handle_t;
|
|
|
|
|
|
2021-11-06 04:35:39 +08:00
|
|
|
depot_stack_handle_t __stack_depot_save(unsigned long *entries,
|
|
|
|
|
unsigned int nr_entries,
|
|
|
|
|
gfp_t gfp_flags, bool can_alloc);
|
|
|
|
|
|
2019-04-25 17:44:56 +08:00
|
|
|
depot_stack_handle_t stack_depot_save(unsigned long *entries,
|
|
|
|
|
unsigned int nr_entries, gfp_t gfp_flags);
|
2016-03-26 05:22:08 +08:00
|
|
|
|
2019-04-25 17:44:56 +08:00
|
|
|
unsigned int stack_depot_fetch(depot_stack_handle_t handle,
|
|
|
|
|
unsigned long **entries);
|
2016-03-26 05:22:08 +08:00
|
|
|
|
2021-02-26 09:21:27 +08:00
|
|
|
#ifdef CONFIG_STACKDEPOT
|
|
|
|
|
int stack_depot_init(void);
|
|
|
|
|
#else
|
|
|
|
|
static inline int stack_depot_init(void)
|
|
|
|
|
{
|
|
|
|
|
return 0;
|
|
|
|
|
}
|
|
|
|
|
#endif /* CONFIG_STACKDEPOT */
|
|
|
|
|
|
2016-03-26 05:22:08 +08:00
|
|
|
#endif
|