2019-04-19 05:35:54 +08:00
|
|
|
===============================
|
|
|
|
Documentation for /proc/sys/vm/
|
|
|
|
===============================
|
2005-04-17 06:20:36 +08:00
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
kernel version 2.6.29
|
2005-04-17 06:20:36 +08:00
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
Copyright (c) 1998, 1999, Rik van Riel <riel@nl.linux.org>
|
|
|
|
|
|
|
|
Copyright (c) 2008 Peter W. Morreale <pmorreale@novell.com>
|
|
|
|
|
|
|
|
For general info and legal blurb, please look in index.rst.
|
|
|
|
|
|
|
|
------------------------------------------------------------------------------
|
2005-04-17 06:20:36 +08:00
|
|
|
|
|
|
|
This file contains the documentation for the sysctl files in
|
2009-01-16 05:50:42 +08:00
|
|
|
/proc/sys/vm and is valid for Linux kernel version 2.6.29.
|
2005-04-17 06:20:36 +08:00
|
|
|
|
|
|
|
The files in this directory can be used to tune the operation
|
|
|
|
of the virtual memory (VM) subsystem of the Linux kernel and
|
|
|
|
the writeout of dirty data to disk.
|
|
|
|
|
|
|
|
Default values and initialization routines for most of these
|
|
|
|
files can be found in mm/swap.c.
|
|
|
|
|
|
|
|
Currently, these files are in /proc/sys/vm:
|
2009-01-16 05:50:42 +08:00
|
|
|
|
2013-04-30 06:08:11 +08:00
|
|
|
- admin_reserve_kbytes
|
2010-05-25 05:32:28 +08:00
|
|
|
- compact_memory
|
2020-10-22 14:54:03 +08:00
|
|
|
- compaction_proactiveness
|
2015-04-16 07:13:20 +08:00
|
|
|
- compact_unevictable_allowed
|
2009-01-16 05:50:42 +08:00
|
|
|
- dirty_background_bytes
|
2005-04-17 06:20:36 +08:00
|
|
|
- dirty_background_ratio
|
2009-01-16 05:50:42 +08:00
|
|
|
- dirty_bytes
|
2005-04-17 06:20:36 +08:00
|
|
|
- dirty_expire_centisecs
|
2009-01-16 05:50:42 +08:00
|
|
|
- dirty_ratio
|
2018-06-19 07:59:18 +08:00
|
|
|
- dirtytime_expire_seconds
|
2005-04-17 06:20:36 +08:00
|
|
|
- dirty_writeback_centisecs
|
2009-01-16 05:50:42 +08:00
|
|
|
- drop_caches
|
2010-05-25 05:32:31 +08:00
|
|
|
- extfrag_threshold
|
2020-10-22 14:54:03 +08:00
|
|
|
- highmem_is_dirtyable
|
2009-01-16 05:50:42 +08:00
|
|
|
- hugetlb_shm_group
|
|
|
|
- laptop_mode
|
|
|
|
- legacy_va_layout
|
|
|
|
- lowmem_reserve_ratio
|
2005-04-17 06:20:36 +08:00
|
|
|
- max_map_count
|
2009-09-16 17:50:15 +08:00
|
|
|
- memory_failure_early_kill
|
|
|
|
- memory_failure_recovery
|
2005-04-17 06:20:36 +08:00
|
|
|
- min_free_kbytes
|
2006-09-26 14:31:52 +08:00
|
|
|
- min_slab_ratio
|
2009-01-16 05:50:42 +08:00
|
|
|
- min_unmapped_ratio
|
|
|
|
- mmap_min_addr
|
mm: mmap: add new /proc tunable for mmap_base ASLR
Address Space Layout Randomization (ASLR) provides a barrier to
exploitation of user-space processes in the presence of security
vulnerabilities by making it more difficult to find desired code/data
which could help an attack. This is done by adding a random offset to
the location of regions in the process address space, with a greater
range of potential offset values corresponding to better protection/a
larger search-space for brute force, but also to greater potential for
fragmentation.
The offset added to the mmap_base address, which provides the basis for
the majority of the mappings for a process, is set once on process exec
in arch_pick_mmap_layout() and is done via hard-coded per-arch values,
which reflect, hopefully, the best compromise for all systems. The
trade-off between increased entropy in the offset value generation and
the corresponding increased variability in address space fragmentation
is not absolute, however, and some platforms may tolerate higher amounts
of entropy. This patch introduces both new Kconfig values and a sysctl
interface which may be used to change the amount of entropy used for
offset generation on a system.
The direct motivation for this change was in response to the
libstagefright vulnerabilities that affected Android, specifically to
information provided by Google's project zero at:
http://googleprojectzero.blogspot.com/2015/09/stagefrightened.html
The attack presented therein, by Google's project zero, specifically
targeted the limited randomness used to generate the offset added to the
mmap_base address in order to craft a brute-force-based attack.
Concretely, the attack was against the mediaserver process, which was
limited to respawning every 5 seconds, on an arm device. The hard-coded
8 bits used resulted in an average expected success rate of defeating
the mmap ASLR after just over 10 minutes (128 tries at 5 seconds a
piece). With this patch, and an accompanying increase in the entropy
value to 16 bits, the same attack would take an average expected time of
over 45 hours (32768 tries), which makes it both less feasible and more
likely to be noticed.
The introduced Kconfig and sysctl options are limited by per-arch
minimum and maximum values, the minimum of which was chosen to match the
current hard-coded value and the maximum of which was chosen so as to
give the greatest flexibility without generating an invalid mmap_base
address, generally a 3-4 bits less than the number of bits in the
user-space accessible virtual address space.
When decided whether or not to change the default value, a system
developer should consider that mmap_base address could be placed
anywhere up to 2^(value) bits away from the non-randomized location,
which would introduce variable-sized areas above and below the mmap_base
address such that the maximum vm_area_struct size may be reduced,
preventing very large allocations.
This patch (of 4):
ASLR only uses as few as 8 bits to generate the random offset for the
mmap base address on 32 bit architectures. This value was chosen to
prevent a poorly chosen value from dividing the address space in such a
way as to prevent large allocations. This may not be an issue on all
platforms. Allow the specification of a minimum number of bits so that
platforms desiring greater ASLR protection may determine where to place
the trade-off.
Signed-off-by: Daniel Cashman <dcashman@google.com>
Cc: Russell King <linux@arm.linux.org.uk>
Acked-by: Kees Cook <keescook@chromium.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Don Zickus <dzickus@redhat.com>
Cc: Eric W. Biederman <ebiederm@xmission.com>
Cc: Heinrich Schuchardt <xypron.glpk@gmx.de>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: David Rientjes <rientjes@google.com>
Cc: Mark Salyzyn <salyzyn@android.com>
Cc: Jeff Vander Stoep <jeffv@google.com>
Cc: Nick Kralevich <nnk@google.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Hector Marco-Gisbert <hecmargi@upv.es>
Cc: Borislav Petkov <bp@suse.de>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-15 07:19:53 +08:00
|
|
|
- mmap_rnd_bits
|
|
|
|
- mmap_rnd_compat_bits
|
2007-12-18 08:20:25 +08:00
|
|
|
- nr_hugepages
|
2018-07-20 21:35:00 +08:00
|
|
|
- nr_hugepages_mempolicy
|
2007-12-18 08:20:25 +08:00
|
|
|
- nr_overcommit_hugepages
|
2009-01-16 05:50:42 +08:00
|
|
|
- nr_trim_pages (only if CONFIG_MMU=n)
|
|
|
|
- numa_zonelist_order
|
|
|
|
- oom_dump_tasks
|
|
|
|
- oom_kill_allocating_task
|
2014-01-22 07:49:14 +08:00
|
|
|
- overcommit_kbytes
|
2009-01-16 05:50:42 +08:00
|
|
|
- overcommit_memory
|
|
|
|
- overcommit_ratio
|
|
|
|
- page-cluster
|
|
|
|
- panic_on_oom
|
2021-06-29 10:42:24 +08:00
|
|
|
- percpu_pagelist_high_fraction
|
2009-01-16 05:50:42 +08:00
|
|
|
- stat_interval
|
2016-05-20 08:12:50 +08:00
|
|
|
- stat_refresh
|
2017-11-16 09:38:22 +08:00
|
|
|
- numa_stat
|
2009-01-16 05:50:42 +08:00
|
|
|
- swappiness
|
userfaultfd/sysctl: add vm.unprivileged_userfaultfd
Userfaultfd can be misued to make it easier to exploit existing
use-after-free (and similar) bugs that might otherwise only make a
short window or race condition available. By using userfaultfd to
stall a kernel thread, a malicious program can keep some state that it
wrote, stable for an extended period, which it can then access using an
existing exploit. While it doesn't cause the exploit itself, and while
it's not the only thing that can stall a kernel thread when accessing a
memory location, it's one of the few that never needs privilege.
We can add a flag, allowing userfaultfd to be restricted, so that in
general it won't be useable by arbitrary user programs, but in
environments that require userfaultfd it can be turned back on.
Add a global sysctl knob "vm.unprivileged_userfaultfd" to control
whether userfaultfd is allowed by unprivileged users. When this is
set to zero, only privileged users (root user, or users with the
CAP_SYS_PTRACE capability) will be able to use the userfaultfd
syscalls.
Andrea said:
: The only difference between the bpf sysctl and the userfaultfd sysctl
: this way is that the bpf sysctl adds the CAP_SYS_ADMIN capability
: requirement, while userfaultfd adds the CAP_SYS_PTRACE requirement,
: because the userfaultfd monitor is more likely to need CAP_SYS_PTRACE
: already if it's doing other kind of tracking on processes runtime, in
: addition of userfaultfd. In other words both syscalls works only for
: root, when the two sysctl are opt-in set to 1.
[dgilbert@redhat.com: changelog additions]
[akpm@linux-foundation.org: documentation tweak, per Mike]
Link: http://lkml.kernel.org/r/20190319030722.12441-2-peterx@redhat.com
Signed-off-by: Peter Xu <peterx@redhat.com>
Suggested-by: Andrea Arcangeli <aarcange@redhat.com>
Suggested-by: Mike Rapoport <rppt@linux.ibm.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
Reviewed-by: Andrea Arcangeli <aarcange@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Luis Chamberlain <mcgrof@kernel.org>
Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
Cc: Maya Gokhale <gokhale2@llnl.gov>
Cc: Jerome Glisse <jglisse@redhat.com>
Cc: Pavel Emelyanov <xemul@virtuozzo.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Martin Cracauer <cracauer@cons.org>
Cc: Denis Plotnikov <dplotnikov@virtuozzo.com>
Cc: Marty McFadden <mcfadden8@llnl.gov>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: "Kirill A . Shutemov" <kirill@shutemov.name>
Cc: "Dr . David Alan Gilbert" <dgilbert@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2019-05-14 08:16:41 +08:00
|
|
|
- unprivileged_userfaultfd
|
mm: limit growth of 3% hardcoded other user reserve
Add user_reserve_kbytes knob.
Limit the growth of the memory reserved for other user processes to
min(3% current process size, user_reserve_pages). Only about 8MB is
necessary to enable recovery in the default mode, and only a few hundred
MB are required even when overcommit is disabled.
user_reserve_pages defaults to min(3% free pages, 128MB)
I arrived at 128MB by taking the max VSZ of sshd, login, bash, and top ...
then adding the RSS of each.
This only affects OVERCOMMIT_NEVER mode.
Background
1. user reserve
__vm_enough_memory reserves a hardcoded 3% of the current process size for
other applications when overcommit is disabled. This was done so that a
user could recover if they launched a memory hogging process. Without the
reserve, a user would easily run into a message such as:
bash: fork: Cannot allocate memory
2. admin reserve
Additionally, a hardcoded 3% of free memory is reserved for root in both
overcommit 'guess' and 'never' modes. This was intended to prevent a
scenario where root-cant-log-in and perform recovery operations.
Note that this reserve shrinks, and doesn't guarantee a useful reserve.
Motivation
The two hardcoded memory reserves should be updated to account for current
memory sizes.
Also, the admin reserve would be more useful if it didn't shrink too much.
When the current code was originally written, 1GB was considered
"enterprise". Now the 3% reserve can grow to multiple GB on large memory
systems, and it only needs to be a few hundred MB at most to enable a user
or admin to recover a system with an unwanted memory hogging process.
I've found that reducing these reserves is especially beneficial for a
specific type of application load:
* single application system
* one or few processes (e.g. one per core)
* allocating all available memory
* not initializing every page immediately
* long running
I've run scientific clusters with this sort of load. A long running job
sometimes failed many hours (weeks of CPU time) into a calculation. They
weren't initializing all of their memory immediately, and they weren't
using calloc, so I put systems into overcommit 'never' mode. These
clusters run diskless and have no swap.
However, with the current reserves, a user wishing to allocate as much
memory as possible to one process may be prevented from using, for
example, almost 2GB out of 32GB.
The effect is less, but still significant when a user starts a job with
one process per core. I have repeatedly seen a set of processes
requesting the same amount of memory fail because one of them could not
allocate the amount of memory a user would expect to be able to allocate.
For example, Message Passing Interfce (MPI) processes, one per core. And
it is similar for other parallel programming frameworks.
Changing this reserve code will make the overcommit never mode more useful
by allowing applications to allocate nearly all of the available memory.
Also, the new admin_reserve_kbytes will be safer than the current behavior
since the hardcoded 3% of available memory reserve can shrink to something
useless in the case where applications have grabbed all available memory.
Risks
* "bash: fork: Cannot allocate memory"
The downside of the first patch-- which creates a tunable user reserve
that is only used in overcommit 'never' mode--is that an admin can set
it so low that a user may not be able to kill their process, even if
they already have a shell prompt.
Of course, a user can get in the same predicament with the current 3%
reserve--they just have to launch processes until 3% becomes negligible.
* root-cant-log-in problem
The second patch, adding the tunable rootuser_reserve_pages, allows
the admin to shoot themselves in the foot by setting it too small. They
can easily get the system into a state where root-can't-log-in.
However, the new admin_reserve_kbytes will be safer than the current
behavior since the hardcoded 3% of available memory reserve can shrink
to something useless in the case where applications have grabbed all
available memory.
Alternatives
* Memory cgroups provide a more flexible way to limit application memory.
Not everyone wants to set up cgroups or deal with their overhead.
* We could create a fourth overcommit mode which provides smaller reserves.
The size of useful reserves may be drastically different depending
on the whether the system is embedded or enterprise.
* Force users to initialize all of their memory or use calloc.
Some users don't want/expect the system to overcommit when they malloc.
Overcommit 'never' mode is for this scenario, and it should work well.
The new user and admin reserve tunables are simple to use, with low
overhead compared to cgroups. The patches preserve current behavior where
3% of memory is less than 128MB, except that the admin reserve doesn't
shrink to an unusable size under pressure. The code allows admins to tune
for embedded and enterprise usage.
FAQ
* How is the root-cant-login problem addressed?
What happens if admin_reserve_pages is set to 0?
Root is free to shoot themselves in the foot by setting
admin_reserve_kbytes too low.
On x86_64, the minimum useful reserve is:
8MB for overcommit 'guess'
128MB for overcommit 'never'
admin_reserve_pages defaults to min(3% free memory, 8MB)
So, anyone switching to 'never' mode needs to adjust
admin_reserve_pages.
* How do you calculate a minimum useful reserve?
A user or the admin needs enough memory to login and perform
recovery operations, which includes, at a minimum:
sshd or login + bash (or some other shell) + top (or ps, kill, etc.)
For overcommit 'guess', we can sum resident set sizes (RSS)
because we only need enough memory to handle what the recovery
programs will typically use. On x86_64 this is about 8MB.
For overcommit 'never', we can take the max of their virtual sizes (VSZ)
and add the sum of their RSS. We use VSZ instead of RSS because mode
forces us to ensure we can fulfill all of the requested memory allocations--
even if the programs only use a fraction of what they ask for.
On x86_64 this is about 128MB.
When swap is enabled, reserves are useful even when they are as
small as 10MB, regardless of overcommit mode.
When both swap and overcommit are disabled, then the admin should
tune the reserves higher to be absolutley safe. Over 230MB each
was safest in my testing.
* What happens if user_reserve_pages is set to 0?
Note, this only affects overcomitt 'never' mode.
Then a user will be able to allocate all available memory minus
admin_reserve_kbytes.
However, they will easily see a message such as:
"bash: fork: Cannot allocate memory"
And they won't be able to recover/kill their application.
The admin should be able to recover the system if
admin_reserve_kbytes is set appropriately.
* What's the difference between overcommit 'guess' and 'never'?
"Guess" allows an allocation if there are enough free + reclaimable
pages. It has a hardcoded 3% of free pages reserved for root.
"Never" allows an allocation if there is enough swap + a configurable
percentage (default is 50) of physical RAM. It has a hardcoded 3% of
free pages reserved for root, like "Guess" mode. It also has a
hardcoded 3% of the current process size reserved for additional
applications.
* Why is overcommit 'guess' not suitable even when an app eventually
writes to every page? It takes free pages, file pages, available
swap pages, reclaimable slab pages into consideration. In other words,
these are all pages available, then why isn't overcommit suitable?
Because it only looks at the present state of the system. It
does not take into account the memory that other applications have
malloced, but haven't initialized yet. It overcommits the system.
Test Summary
There was little change in behavior in the default overcommit 'guess'
mode with swap enabled before and after the patch. This was expected.
Systems run most predictably (i.e. no oom kills) in overcommit 'never'
mode with swap enabled. This also allowed the most memory to be allocated
to a user application.
Overcommit 'guess' mode without swap is a bad idea. It is easy to
crash the system. None of the other tested combinations crashed.
This matches my experience on the Roadrunner supercomputer.
Without the tunable user reserve, a system in overcommit 'never' mode
and without swap does not allow the admin to recover, although the
admin can.
With the new tunable reserves, a system in overcommit 'never' mode
and without swap can be configured to:
1. maximize user-allocatable memory, running close to the edge of
recoverability
2. maximize recoverability, sacrificing allocatable memory to
ensure that a user cannot take down a system
Test Description
Fedora 18 VM - 4 x86_64 cores, 5725MB RAM, 4GB Swap
System is booted into multiuser console mode, with unnecessary services
turned off. Caches were dropped before each test.
Hogs are user memtester processes that attempt to allocate all free memory
as reported by /proc/meminfo
In overcommit 'never' mode, memory_ratio=100
Test Results
3.9.0-rc1-mm1
Overcommit | Swap | Hogs | MB Got/Wanted | OOMs | User Recovery | Admin Recovery
---------- ---- ---- ------------- ---- ------------- --------------
guess yes 1 5432/5432 no yes yes
guess yes 4 5444/5444 1 yes yes
guess no 1 5302/5449 no yes yes
guess no 4 - crash no no
never yes 1 5460/5460 1 yes yes
never yes 4 5460/5460 1 yes yes
never no 1 5218/5432 no no yes
never no 4 5203/5448 no no yes
3.9.0-rc1-mm1-tunablereserves
User and Admin Recovery show their respective reserves, if applicable.
Overcommit | Swap | Hogs | MB Got/Wanted | OOMs | User Recovery | Admin Recovery
---------- ---- ---- ------------- ---- ------------- --------------
guess yes 1 5419/5419 no - yes 8MB yes
guess yes 4 5436/5436 1 - yes 8MB yes
guess no 1 5440/5440 * - yes 8MB yes
guess no 4 - crash - no 8MB no
* process would successfully mlock, then the oom killer would pick it
never yes 1 5446/5446 no 10MB yes 20MB yes
never yes 4 5456/5456 no 10MB yes 20MB yes
never no 1 5387/5429 no 128MB no 8MB barely
never no 1 5323/5428 no 226MB barely 8MB barely
never no 1 5323/5428 no 226MB barely 8MB barely
never no 1 5359/5448 no 10MB no 10MB barely
never no 1 5323/5428 no 0MB no 10MB barely
never no 1 5332/5428 no 0MB no 50MB yes
never no 1 5293/5429 no 0MB no 90MB yes
never no 1 5001/5427 no 230MB yes 338MB yes
never no 4* 4998/5424 no 230MB yes 338MB yes
* more memtesters were launched, able to allocate approximately another 100MB
Future Work
- Test larger memory systems.
- Test an embedded image.
- Test other architectures.
- Time malloc microbenchmarks.
- Would it be useful to be able to set overcommit policy for
each memory cgroup?
- Some lines are slightly above 80 chars.
Perhaps define a macro to convert between pages and kb?
Other places in the kernel do this.
[akpm@linux-foundation.org: coding-style fixes]
[akpm@linux-foundation.org: make init_user_reserve() static]
Signed-off-by: Andrew Shewmaker <agshew@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-04-30 06:08:10 +08:00
|
|
|
- user_reserve_kbytes
|
2009-01-16 05:50:42 +08:00
|
|
|
- vfs_cache_pressure
|
mm: reclaim small amounts of memory when an external fragmentation event occurs
An external fragmentation event was previously described as
When the page allocator fragments memory, it records the event using
the mm_page_alloc_extfrag event. If the fallback_order is smaller
than a pageblock order (order-9 on 64-bit x86) then it's considered
an event that will cause external fragmentation issues in the future.
The kernel reduces the probability of such events by increasing the
watermark sizes by calling set_recommended_min_free_kbytes early in the
lifetime of the system. This works reasonably well in general but if
there are enough sparsely populated pageblocks then the problem can still
occur as enough memory is free overall and kswapd stays asleep.
This patch introduces a watermark_boost_factor sysctl that allows a zone
watermark to be temporarily boosted when an external fragmentation causing
events occurs. The boosting will stall allocations that would decrease
free memory below the boosted low watermark and kswapd is woken if the
calling context allows to reclaim an amount of memory relative to the size
of the high watermark and the watermark_boost_factor until the boost is
cleared. When kswapd finishes, it wakes kcompactd at the pageblock order
to clean some of the pageblocks that may have been affected by the
fragmentation event. kswapd avoids any writeback, slab shrinkage and swap
from reclaim context during this operation to avoid excessive system
disruption in the name of fragmentation avoidance. Care is taken so that
kswapd will do normal reclaim work if the system is really low on memory.
This was evaluated using the same workloads as "mm, page_alloc: Spread
allocations across zones before introducing fragmentation".
1-socket Skylake machine
config-global-dhp__workload_thpfioscale XFS (no special madvise)
4 fio threads, 1 THP allocating thread
--------------------------------------
4.20-rc3 extfrag events < order 9: 804694
4.20-rc3+patch: 408912 (49% reduction)
4.20-rc3+patch1-4: 18421 (98% reduction)
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Amean fault-base-1 653.58 ( 0.00%) 652.71 ( 0.13%)
Amean fault-huge-1 0.00 ( 0.00%) 178.93 * -99.00%*
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Percentage huge-1 0.00 ( 0.00%) 5.12 ( 100.00%)
Note that external fragmentation causing events are massively reduced by
this path whether in comparison to the previous kernel or the vanilla
kernel. The fault latency for huge pages appears to be increased but that
is only because THP allocations were successful with the patch applied.
1-socket Skylake machine
global-dhp__workload_thpfioscale-madvhugepage-xfs (MADV_HUGEPAGE)
-----------------------------------------------------------------
4.20-rc3 extfrag events < order 9: 291392
4.20-rc3+patch: 191187 (34% reduction)
4.20-rc3+patch1-4: 13464 (95% reduction)
thpfioscale Fault Latencies
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Min fault-base-1 912.00 ( 0.00%) 905.00 ( 0.77%)
Min fault-huge-1 127.00 ( 0.00%) 135.00 ( -6.30%)
Amean fault-base-1 1467.55 ( 0.00%) 1481.67 ( -0.96%)
Amean fault-huge-1 1127.11 ( 0.00%) 1063.88 * 5.61%*
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Percentage huge-1 77.64 ( 0.00%) 83.46 ( 7.49%)
As before, massive reduction in external fragmentation events, some jitter
on latencies and an increase in THP allocation success rates.
2-socket Haswell machine
config-global-dhp__workload_thpfioscale XFS (no special madvise)
4 fio threads, 5 THP allocating threads
----------------------------------------------------------------
4.20-rc3 extfrag events < order 9: 215698
4.20-rc3+patch: 200210 (7% reduction)
4.20-rc3+patch1-4: 14263 (93% reduction)
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Amean fault-base-5 1346.45 ( 0.00%) 1306.87 ( 2.94%)
Amean fault-huge-5 3418.60 ( 0.00%) 1348.94 ( 60.54%)
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Percentage huge-5 0.78 ( 0.00%) 7.91 ( 910.64%)
There is a 93% reduction in fragmentation causing events, there is a big
reduction in the huge page fault latency and allocation success rate is
higher.
2-socket Haswell machine
global-dhp__workload_thpfioscale-madvhugepage-xfs (MADV_HUGEPAGE)
-----------------------------------------------------------------
4.20-rc3 extfrag events < order 9: 166352
4.20-rc3+patch: 147463 (11% reduction)
4.20-rc3+patch1-4: 11095 (93% reduction)
thpfioscale Fault Latencies
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Amean fault-base-5 6217.43 ( 0.00%) 7419.67 * -19.34%*
Amean fault-huge-5 3163.33 ( 0.00%) 3263.80 ( -3.18%)
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Percentage huge-5 95.14 ( 0.00%) 87.98 ( -7.53%)
There is a large reduction in fragmentation events with some jitter around
the latencies and success rates. As before, the high THP allocation
success rate does mean the system is under a lot of pressure. However, as
the fragmentation events are reduced, it would be expected that the
long-term allocation success rate would be higher.
Link: http://lkml.kernel.org/r/20181123114528.28802-5-mgorman@techsingularity.net
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Zi Yan <zi.yan@cs.rutgers.edu>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-12-28 16:35:52 +08:00
|
|
|
- watermark_boost_factor
|
2016-07-12 18:05:59 +08:00
|
|
|
- watermark_scale_factor
|
2009-01-16 05:50:42 +08:00
|
|
|
- zone_reclaim_mode
|
|
|
|
|
2005-04-17 06:20:36 +08:00
|
|
|
|
2013-04-30 06:08:11 +08:00
|
|
|
admin_reserve_kbytes
|
2019-04-19 05:35:54 +08:00
|
|
|
====================
|
2013-04-30 06:08:11 +08:00
|
|
|
|
|
|
|
The amount of free memory in the system that should be reserved for users
|
|
|
|
with the capability cap_sys_admin.
|
|
|
|
|
|
|
|
admin_reserve_kbytes defaults to min(3% of free pages, 8MB)
|
|
|
|
|
|
|
|
That should provide enough for the admin to log in and kill a process,
|
|
|
|
if necessary, under the default overcommit 'guess' mode.
|
|
|
|
|
|
|
|
Systems running under overcommit 'never' should increase this to account
|
|
|
|
for the full Virtual Memory Size of programs used to recover. Otherwise,
|
|
|
|
root may not be able to log in to recover the system.
|
|
|
|
|
|
|
|
How do you calculate a minimum useful reserve?
|
|
|
|
|
|
|
|
sshd or login + bash (or some other shell) + top (or ps, kill, etc.)
|
|
|
|
|
|
|
|
For overcommit 'guess', we can sum resident set sizes (RSS).
|
|
|
|
On x86_64 this is about 8MB.
|
|
|
|
|
|
|
|
For overcommit 'never', we can take the max of their virtual sizes (VSZ)
|
|
|
|
and add the sum of their RSS.
|
|
|
|
On x86_64 this is about 128MB.
|
|
|
|
|
|
|
|
Changing this takes effect whenever an application requests memory.
|
|
|
|
|
|
|
|
|
2010-05-25 05:32:28 +08:00
|
|
|
compact_memory
|
2019-04-19 05:35:54 +08:00
|
|
|
==============
|
2010-05-25 05:32:28 +08:00
|
|
|
|
|
|
|
Available only when CONFIG_COMPACTION is set. When 1 is written to the file,
|
|
|
|
all zones are compacted such that free memory is available in contiguous
|
|
|
|
blocks where possible. This can be important for example in the allocation of
|
|
|
|
huge pages although processes will also directly compact memory as required.
|
|
|
|
|
mm: proactive compaction
For some applications, we need to allocate almost all memory as hugepages.
However, on a running system, higher-order allocations can fail if the
memory is fragmented. Linux kernel currently does on-demand compaction as
we request more hugepages, but this style of compaction incurs very high
latency. Experiments with one-time full memory compaction (followed by
hugepage allocations) show that kernel is able to restore a highly
fragmented memory state to a fairly compacted memory state within <1 sec
for a 32G system. Such data suggests that a more proactive compaction can
help us allocate a large fraction of memory as hugepages keeping
allocation latencies low.
For a more proactive compaction, the approach taken here is to define a
new sysctl called 'vm.compaction_proactiveness' which dictates bounds for
external fragmentation which kcompactd tries to maintain.
The tunable takes a value in range [0, 100], with a default of 20.
Note that a previous version of this patch [1] was found to introduce too
many tunables (per-order extfrag{low, high}), but this one reduces them to
just one sysctl. Also, the new tunable is an opaque value instead of
asking for specific bounds of "external fragmentation", which would have
been difficult to estimate. The internal interpretation of this opaque
value allows for future fine-tuning.
Currently, we use a simple translation from this tunable to [low, high]
"fragmentation score" thresholds (low=100-proactiveness, high=low+10%).
The score for a node is defined as weighted mean of per-zone external
fragmentation. A zone's present_pages determines its weight.
To periodically check per-node score, we reuse per-node kcompactd threads,
which are woken up every 500 milliseconds to check the same. If a node's
score exceeds its high threshold (as derived from user-provided
proactiveness value), proactive compaction is started until its score
reaches its low threshold value. By default, proactiveness is set to 20,
which implies threshold values of low=80 and high=90.
This patch is largely based on ideas from Michal Hocko [2]. See also the
LWN article [3].
Performance data
================
System: x64_64, 1T RAM, 80 CPU threads.
Kernel: 5.6.0-rc3 + this patch
echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/defrag
Before starting the driver, the system was fragmented from a userspace
program that allocates all memory and then for each 2M aligned section,
frees 3/4 of base pages using munmap. The workload is mainly anonymous
userspace pages, which are easy to move around. I intentionally avoided
unmovable pages in this test to see how much latency we incur when
hugepage allocations hit direct compaction.
1. Kernel hugepage allocation latencies
With the system in such a fragmented state, a kernel driver then allocates
as many hugepages as possible and measures allocation latency:
(all latency values are in microseconds)
- With vanilla 5.6.0-rc3
percentile latency
–––––––––– –––––––
5 7894
10 9496
25 12561
30 15295
40 18244
50 21229
60 27556
75 30147
80 31047
90 32859
95 33799
Total 2M hugepages allocated = 383859 (749G worth of hugepages out of 762G
total free => 98% of free memory could be allocated as hugepages)
- With 5.6.0-rc3 + this patch, with proactiveness=20
sysctl -w vm.compaction_proactiveness=20
percentile latency
–––––––––– –––––––
5 2
10 2
25 3
30 3
40 3
50 4
60 4
75 4
80 4
90 5
95 429
Total 2M hugepages allocated = 384105 (750G worth of hugepages out of 762G
total free => 98% of free memory could be allocated as hugepages)
2. JAVA heap allocation
In this test, we first fragment memory using the same method as for (1).
Then, we start a Java process with a heap size set to 700G and request the
heap to be allocated with THP hugepages. We also set THP to madvise to
allow hugepage backing of this heap.
/usr/bin/time
java -Xms700G -Xmx700G -XX:+UseTransparentHugePages -XX:+AlwaysPreTouch
The above command allocates 700G of Java heap using hugepages.
- With vanilla 5.6.0-rc3
17.39user 1666.48system 27:37.89elapsed
- With 5.6.0-rc3 + this patch, with proactiveness=20
8.35user 194.58system 3:19.62elapsed
Elapsed time remains around 3:15, as proactiveness is further increased.
Note that proactive compaction happens throughout the runtime of these
workloads. The situation of one-time compaction, sufficient to supply
hugepages for following allocation stream, can probably happen for more
extreme proactiveness values, like 80 or 90.
In the above Java workload, proactiveness is set to 20. The test starts
with a node's score of 80 or higher, depending on the delay between the
fragmentation step and starting the benchmark, which gives more-or-less
time for the initial round of compaction. As t he benchmark consumes
hugepages, node's score quickly rises above the high threshold (90) and
proactive compaction starts again, which brings down the score to the low
threshold level (80). Repeat.
bpftrace also confirms proactive compaction running 20+ times during the
runtime of this Java benchmark. kcompactd threads consume 100% of one of
the CPUs while it tries to bring a node's score within thresholds.
Backoff behavior
================
Above workloads produce a memory state which is easy to compact. However,
if memory is filled with unmovable pages, proactive compaction should
essentially back off. To test this aspect:
- Created a kernel driver that allocates almost all memory as hugepages
followed by freeing first 3/4 of each hugepage.
- Set proactiveness=40
- Note that proactive_compact_node() is deferred maximum number of times
with HPAGE_FRAG_CHECK_INTERVAL_MSEC of wait between each check
(=> ~30 seconds between retries).
[1] https://patchwork.kernel.org/patch/11098289/
[2] https://lore.kernel.org/linux-mm/20161230131412.GI13301@dhcp22.suse.cz/
[3] https://lwn.net/Articles/817905/
Signed-off-by: Nitin Gupta <nigupta@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Tested-by: Oleksandr Natalenko <oleksandr@redhat.com>
Reviewed-by: Vlastimil Babka <vbabka@suse.cz>
Reviewed-by: Khalid Aziz <khalid.aziz@oracle.com>
Reviewed-by: Oleksandr Natalenko <oleksandr@redhat.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Khalid Aziz <khalid.aziz@oracle.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mel Gorman <mgorman@techsingularity.net>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Nitin Gupta <ngupta@nitingupta.dev>
Cc: Oleksandr Natalenko <oleksandr@redhat.com>
Link: http://lkml.kernel.org/r/20200616204527.19185-1-nigupta@nvidia.com
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-08-12 09:31:00 +08:00
|
|
|
compaction_proactiveness
|
|
|
|
========================
|
|
|
|
|
|
|
|
This tunable takes a value in the range [0, 100] with a default value of
|
|
|
|
20. This tunable determines how aggressively compaction is done in the
|
|
|
|
background. Setting it to 0 disables proactive compaction.
|
|
|
|
|
|
|
|
Note that compaction has a non-trivial system-wide impact as pages
|
|
|
|
belonging to different processes are moved around, which could also lead
|
|
|
|
to latency spikes in unsuspecting applications. The kernel employs
|
|
|
|
various heuristics to avoid wasting CPU cycles if it detects that
|
|
|
|
proactive compaction is not being effective.
|
|
|
|
|
|
|
|
Be careful when setting it to extreme values like 100, as that may
|
|
|
|
cause excessive background compaction activity.
|
2010-05-25 05:32:28 +08:00
|
|
|
|
2015-04-16 07:13:20 +08:00
|
|
|
compact_unevictable_allowed
|
2019-04-19 05:35:54 +08:00
|
|
|
===========================
|
2015-04-16 07:13:20 +08:00
|
|
|
|
|
|
|
Available only when CONFIG_COMPACTION is set. When set to 1, compaction is
|
|
|
|
allowed to examine the unevictable lru (mlocked pages) for pages to compact.
|
|
|
|
This should be used on systems where stalls for minor page faults are an
|
|
|
|
acceptable trade for large contiguous free memory. Set to 0 to prevent
|
|
|
|
compaction from moving pages that are unevictable. Default value is 1.
|
2020-04-02 12:10:42 +08:00
|
|
|
On CONFIG_PREEMPT_RT the default value is 0 in order to avoid a page fault, due
|
Documentation: fix multiple typos found in the admin-guide subdirectory
Fix thirty five typos in dm-integrity.rst, dm-raid.rst, dm-zoned.rst,
verity.rst, writecache.rst, tsx_async_abort.rst, md.rst, bttv.rst,
dvb_references.rst, frontend-cardlist.rst, gspca-cardlist.rst, ipu3.rst,
remote-controller.rst, mm/index.rst, numaperf.rst, userfaultfd.rst,
module-signing.rst, imx-ddr.rst, intel-speed-select.rst,
intel_pstate.rst, ramoops.rst, abi.rst, kernel.rst, vm.rst
Signed-off-by: Andrew Klychkov <andrew.a.klychkov@gmail.com>
Link: https://lore.kernel.org/r/20201204072848.GA49895@spblnx124.lan
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2020-12-04 15:28:48 +08:00
|
|
|
to compaction, which would block the task from becoming active until the fault
|
2020-04-02 12:10:42 +08:00
|
|
|
is resolved.
|
2015-04-16 07:13:20 +08:00
|
|
|
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
dirty_background_bytes
|
2019-04-19 05:35:54 +08:00
|
|
|
======================
|
2005-04-17 06:20:36 +08:00
|
|
|
|
2012-07-25 23:12:01 +08:00
|
|
|
Contains the amount of dirty memory at which the background kernel
|
|
|
|
flusher threads will start writeback.
|
2005-04-17 06:20:36 +08:00
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
Note:
|
|
|
|
dirty_background_bytes is the counterpart of dirty_background_ratio. Only
|
|
|
|
one of them may be specified at a time. When one sysctl is written it is
|
|
|
|
immediately taken into account to evaluate the dirty memory limits and the
|
|
|
|
other appears as 0 when read.
|
2005-04-17 06:20:36 +08:00
|
|
|
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
dirty_background_ratio
|
2019-04-19 05:35:54 +08:00
|
|
|
======================
|
2005-04-17 06:20:36 +08:00
|
|
|
|
2013-11-13 07:08:30 +08:00
|
|
|
Contains, as a percentage of total available memory that contains free pages
|
|
|
|
and reclaimable pages, the number of pages at which the background kernel
|
|
|
|
flusher threads will start writing out dirty data.
|
|
|
|
|
2015-09-18 14:10:55 +08:00
|
|
|
The total available memory is not equal to total system memory.
|
2005-04-17 06:20:36 +08:00
|
|
|
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
dirty_bytes
|
2019-04-19 05:35:54 +08:00
|
|
|
===========
|
2009-01-16 05:50:42 +08:00
|
|
|
|
|
|
|
Contains the amount of dirty memory at which a process generating disk writes
|
|
|
|
will itself start writeback.
|
|
|
|
|
2010-10-28 06:33:31 +08:00
|
|
|
Note: dirty_bytes is the counterpart of dirty_ratio. Only one of them may be
|
|
|
|
specified at a time. When one sysctl is written it is immediately taken into
|
|
|
|
account to evaluate the dirty memory limits and the other appears as 0 when
|
|
|
|
read.
|
2005-04-17 06:20:36 +08:00
|
|
|
|
2009-05-01 06:08:57 +08:00
|
|
|
Note: the minimum value allowed for dirty_bytes is two pages (in bytes); any
|
|
|
|
value lower than this limit will be ignored and the old configuration will be
|
|
|
|
retained.
|
|
|
|
|
2005-04-17 06:20:36 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
dirty_expire_centisecs
|
2019-04-19 05:35:54 +08:00
|
|
|
======================
|
2005-04-17 06:20:36 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
This tunable is used to define when dirty data is old enough to be eligible
|
2012-07-25 23:12:01 +08:00
|
|
|
for writeout by the kernel flusher threads. It is expressed in 100'ths
|
|
|
|
of a second. Data which has been dirty in-memory for longer than this
|
|
|
|
interval will be written out next time a flusher thread wakes up.
|
2009-01-16 05:50:42 +08:00
|
|
|
|
|
|
|
|
|
|
|
dirty_ratio
|
2019-04-19 05:35:54 +08:00
|
|
|
===========
|
2009-01-16 05:50:42 +08:00
|
|
|
|
2013-11-13 07:08:30 +08:00
|
|
|
Contains, as a percentage of total available memory that contains free pages
|
|
|
|
and reclaimable pages, the number of pages at which a process which is
|
|
|
|
generating disk writes will itself start writing out dirty data.
|
|
|
|
|
2015-09-18 14:10:55 +08:00
|
|
|
The total available memory is not equal to total system memory.
|
2005-04-17 06:20:36 +08:00
|
|
|
|
|
|
|
|
2018-06-19 07:59:18 +08:00
|
|
|
dirtytime_expire_seconds
|
2019-04-19 05:35:54 +08:00
|
|
|
========================
|
2018-06-19 07:59:18 +08:00
|
|
|
|
|
|
|
When a lazytime inode is constantly having its pages dirtied, the inode with
|
|
|
|
an updated timestamp will never get chance to be written out. And, if the
|
|
|
|
only thing that has happened on the file system is a dirtytime inode caused
|
|
|
|
by an atime update, a worker will be scheduled to make sure that inode
|
|
|
|
eventually gets pushed out to disk. This tunable is used to define when dirty
|
|
|
|
inode is old enough to be eligible for writeback by the kernel flusher threads.
|
|
|
|
And, it is also used as the interval to wakeup dirtytime_writeback thread.
|
|
|
|
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
dirty_writeback_centisecs
|
2019-04-19 05:35:54 +08:00
|
|
|
=========================
|
2005-04-17 06:20:36 +08:00
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
The kernel flusher threads will periodically wake up and write `old` data
|
2009-01-16 05:50:42 +08:00
|
|
|
out to disk. This tunable expresses the interval between those wakeups, in
|
|
|
|
100'ths of a second.
|
2005-04-17 06:20:36 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
Setting this to zero disables periodic writeback altogether.
|
2005-04-17 06:20:36 +08:00
|
|
|
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
drop_caches
|
2019-04-19 05:35:54 +08:00
|
|
|
===========
|
2005-04-17 06:20:36 +08:00
|
|
|
|
2014-04-04 05:48:19 +08:00
|
|
|
Writing to this will cause the kernel to drop clean caches, as well as
|
|
|
|
reclaimable slab objects like dentries and inodes. Once dropped, their
|
|
|
|
memory becomes free.
|
2005-04-17 06:20:36 +08:00
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
To free pagecache::
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
echo 1 > /proc/sys/vm/drop_caches
|
2019-04-19 05:35:54 +08:00
|
|
|
|
|
|
|
To free reclaimable slab objects (includes dentries and inodes)::
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
echo 2 > /proc/sys/vm/drop_caches
|
2019-04-19 05:35:54 +08:00
|
|
|
|
|
|
|
To free slab objects and pagecache::
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
echo 3 > /proc/sys/vm/drop_caches
|
2005-04-17 06:20:36 +08:00
|
|
|
|
2014-04-04 05:48:19 +08:00
|
|
|
This is a non-destructive operation and will not free any dirty objects.
|
|
|
|
To increase the number of objects freed by this operation, the user may run
|
2019-04-19 05:35:54 +08:00
|
|
|
`sync` prior to writing to /proc/sys/vm/drop_caches. This will minimize the
|
2014-04-04 05:48:19 +08:00
|
|
|
number of dirty objects on the system and create more candidates to be
|
|
|
|
dropped.
|
|
|
|
|
|
|
|
This file is not a means to control the growth of the various kernel caches
|
|
|
|
(inodes, dentries, pagecache, etc...) These objects are automatically
|
|
|
|
reclaimed by the kernel when memory is needed elsewhere on the system.
|
|
|
|
|
|
|
|
Use of this file can cause performance problems. Since it discards cached
|
|
|
|
objects, it may cost a significant amount of I/O and CPU to recreate the
|
|
|
|
dropped objects, especially if they were under heavy use. Because of this,
|
|
|
|
use outside of a testing or debugging environment is not recommended.
|
|
|
|
|
|
|
|
You may see informational messages in your kernel log when this file is
|
2019-04-19 05:35:54 +08:00
|
|
|
used::
|
2014-04-04 05:48:19 +08:00
|
|
|
|
|
|
|
cat (1234): drop_caches: 3
|
|
|
|
|
|
|
|
These are informational only. They do not mean that anything is wrong
|
2019-01-12 00:14:10 +08:00
|
|
|
with your system. To disable them, echo 4 (bit 2) into drop_caches.
|
2005-04-17 06:20:36 +08:00
|
|
|
|
|
|
|
|
2010-05-25 05:32:31 +08:00
|
|
|
extfrag_threshold
|
2019-04-19 05:35:54 +08:00
|
|
|
=================
|
2010-05-25 05:32:31 +08:00
|
|
|
|
|
|
|
This parameter affects whether the kernel will compact memory or direct
|
2015-07-14 13:35:11 +08:00
|
|
|
reclaim to satisfy a high-order allocation. The extfrag/extfrag_index file in
|
|
|
|
debugfs shows what the fragmentation index for each order is in each zone in
|
|
|
|
the system. Values tending towards 0 imply allocations would fail due to lack
|
|
|
|
of memory, values towards 1000 imply failures are due to fragmentation and -1
|
|
|
|
implies that the allocation will succeed as long as watermarks are met.
|
2010-05-25 05:32:31 +08:00
|
|
|
|
|
|
|
The kernel will not compact memory in a zone if the
|
|
|
|
fragmentation index is <= extfrag_threshold. The default value is 500.
|
|
|
|
|
|
|
|
|
2017-07-11 06:49:38 +08:00
|
|
|
highmem_is_dirtyable
|
2019-04-19 05:35:54 +08:00
|
|
|
====================
|
2017-07-11 06:49:38 +08:00
|
|
|
|
|
|
|
Available only for systems with CONFIG_HIGHMEM enabled (32b systems).
|
|
|
|
|
|
|
|
This parameter controls whether the high memory is considered for dirty
|
|
|
|
writers throttling. This is not the case by default which means that
|
|
|
|
only the amount of memory directly visible/usable by the kernel can
|
|
|
|
be dirtied. As a result, on systems with a large amount of memory and
|
|
|
|
lowmem basically depleted writers might be throttled too early and
|
|
|
|
streaming writes can get very slow.
|
|
|
|
|
|
|
|
Changing the value to non zero would allow more memory to be dirtied
|
|
|
|
and thus allow writers to write more data which can be flushed to the
|
|
|
|
storage more effectively. Note this also comes with a risk of pre-mature
|
|
|
|
OOM killer because some writers (e.g. direct block device writes) can
|
|
|
|
only use the low memory and they can fill it up with dirty data without
|
|
|
|
any throttling.
|
|
|
|
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
hugetlb_shm_group
|
2019-04-19 05:35:54 +08:00
|
|
|
=================
|
2006-01-08 17:00:40 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
hugetlb_shm_group contains group id that is allowed to create SysV
|
|
|
|
shared memory segment using hugetlb page.
|
2006-01-08 17:00:40 +08:00
|
|
|
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
laptop_mode
|
2019-04-19 05:35:54 +08:00
|
|
|
===========
|
2006-01-19 09:42:32 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
laptop_mode is a knob that controls "laptop mode". All the things that are
|
2019-06-14 02:07:43 +08:00
|
|
|
controlled by this knob are discussed in Documentation/admin-guide/laptops/laptop-mode.rst.
|
2006-01-19 09:42:32 +08:00
|
|
|
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
legacy_va_layout
|
2019-04-19 05:35:54 +08:00
|
|
|
================
|
2006-02-01 19:05:34 +08:00
|
|
|
|
2010-06-28 19:59:28 +08:00
|
|
|
If non-zero, this sysctl disables the new 32-bit mmap layout - the kernel
|
2009-01-16 05:50:42 +08:00
|
|
|
will use the legacy (2.4) layout for all processes.
|
2006-02-01 19:05:34 +08:00
|
|
|
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
lowmem_reserve_ratio
|
2019-04-19 05:35:54 +08:00
|
|
|
====================
|
2009-01-16 05:50:42 +08:00
|
|
|
|
|
|
|
For some specialised workloads on highmem machines it is dangerous for
|
|
|
|
the kernel to allow process memory to be allocated from the "lowmem"
|
|
|
|
zone. This is because that memory could then be pinned via the mlock()
|
|
|
|
system call, or by unavailability of swapspace.
|
|
|
|
|
|
|
|
And on large highmem machines this lack of reclaimable lowmem memory
|
|
|
|
can be fatal.
|
|
|
|
|
|
|
|
So the Linux page allocator has a mechanism which prevents allocations
|
2019-04-19 05:35:54 +08:00
|
|
|
which *could* use highmem from using too much lowmem. This means that
|
2009-01-16 05:50:42 +08:00
|
|
|
a certain amount of lowmem is defended from the possibility of being
|
|
|
|
captured into pinned user memory.
|
|
|
|
|
|
|
|
(The same argument applies to the old 16 megabyte ISA DMA region. This
|
|
|
|
mechanism will also defend that region from allocations which could use
|
|
|
|
highmem or lowmem).
|
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
The `lowmem_reserve_ratio` tunable determines how aggressive the kernel is
|
2009-01-16 05:50:42 +08:00
|
|
|
in defending these lower zones.
|
|
|
|
|
|
|
|
If you have a machine which uses highmem or ISA DMA and your
|
|
|
|
applications are using mlock(), or if you are running with no swap then
|
|
|
|
you probably should change the lowmem_reserve_ratio setting.
|
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
The lowmem_reserve_ratio is an array. You can see them by reading this file::
|
|
|
|
|
|
|
|
% cat /proc/sys/vm/lowmem_reserve_ratio
|
|
|
|
256 256 32
|
2009-01-16 05:50:42 +08:00
|
|
|
|
|
|
|
But, these values are not used directly. The kernel calculates # of protection
|
|
|
|
pages for each zones from them. These are shown as array of protection pages
|
|
|
|
in /proc/zoneinfo like followings. (This is an example of x86-64 box).
|
2019-04-19 05:35:54 +08:00
|
|
|
Each zone has an array of protection pages like this::
|
|
|
|
|
|
|
|
Node 0, zone DMA
|
|
|
|
pages free 1355
|
|
|
|
min 3
|
|
|
|
low 3
|
|
|
|
high 4
|
2009-01-16 05:50:42 +08:00
|
|
|
:
|
|
|
|
:
|
2019-04-19 05:35:54 +08:00
|
|
|
numa_other 0
|
|
|
|
protection: (0, 2004, 2004, 2004)
|
2009-01-16 05:50:42 +08:00
|
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
2019-04-19 05:35:54 +08:00
|
|
|
pagesets
|
|
|
|
cpu: 0 pcp: 0
|
|
|
|
:
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
These protections are added to score to judge whether this zone should be used
|
|
|
|
for page allocation or should be reclaimed.
|
|
|
|
|
|
|
|
In this example, if normal pages (index=2) are required to this DMA zone and
|
2009-06-17 06:32:12 +08:00
|
|
|
watermark[WMARK_HIGH] is used for watermark, the kernel judges this zone should
|
|
|
|
not be used because pages_free(1355) is smaller than watermark + protection[2]
|
2009-01-16 05:50:42 +08:00
|
|
|
(4 + 2004 = 2008). If this protection value is 0, this zone would be used for
|
|
|
|
normal page requirement. If requirement is DMA zone(index=0), protection[0]
|
|
|
|
(=0) is used.
|
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
zone[i]'s protection[j] is calculated by following expression::
|
2009-01-16 05:50:42 +08:00
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
(i < j):
|
|
|
|
zone[i]->protection[j]
|
|
|
|
= (total sums of managed_pages from zone[i+1] to zone[j] on the node)
|
|
|
|
/ lowmem_reserve_ratio[i];
|
|
|
|
(i = j):
|
|
|
|
(should not be protected. = 0;
|
|
|
|
(i > j):
|
|
|
|
(not necessary, but looks 0)
|
2009-01-16 05:50:42 +08:00
|
|
|
|
|
|
|
The default values of lowmem_reserve_ratio[i] are
|
2019-04-19 05:35:54 +08:00
|
|
|
|
|
|
|
=== ====================================
|
2009-01-16 05:50:42 +08:00
|
|
|
256 (if zone[i] means DMA or DMA32 zone)
|
2019-04-19 05:35:54 +08:00
|
|
|
32 (others)
|
|
|
|
=== ====================================
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
As above expression, they are reciprocal number of ratio.
|
2015-09-09 06:04:10 +08:00
|
|
|
256 means 1/256. # of protection pages becomes about "0.39%" of total managed
|
2009-01-16 05:50:42 +08:00
|
|
|
pages of higher zones on the node.
|
|
|
|
|
|
|
|
If you would like to protect more pages, smaller values are effective.
|
2018-04-11 07:30:11 +08:00
|
|
|
The minimum value is 1 (1/1 -> 100%). The value less than 1 completely
|
|
|
|
disables protection of the pages.
|
2006-02-01 19:05:34 +08:00
|
|
|
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
max_map_count:
|
2019-04-19 05:35:54 +08:00
|
|
|
==============
|
2006-01-19 09:42:32 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
This file contains the maximum number of memory map areas a process
|
|
|
|
may have. Memory map areas are used as a side-effect of calling
|
2017-02-25 06:58:47 +08:00
|
|
|
malloc, directly by mmap, mprotect, and madvise, and also when loading
|
|
|
|
shared libraries.
|
2006-01-19 09:42:32 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
While most applications need less than a thousand maps, certain
|
|
|
|
programs, particularly malloc debuggers, may consume lots of them,
|
|
|
|
e.g., up to one or two maps per allocation.
|
2006-06-23 17:03:13 +08:00
|
|
|
|
2020-12-10 16:21:34 +08:00
|
|
|
The default value is 65530.
|
2006-07-03 15:24:13 +08:00
|
|
|
|
2009-09-16 17:50:15 +08:00
|
|
|
|
|
|
|
memory_failure_early_kill:
|
2019-04-19 05:35:54 +08:00
|
|
|
==========================
|
2009-09-16 17:50:15 +08:00
|
|
|
|
|
|
|
Control how to kill processes when uncorrected memory error (typically
|
|
|
|
a 2bit error in a memory module) is detected in the background by hardware
|
|
|
|
that cannot be handled by the kernel. In some cases (like the page
|
|
|
|
still having a valid copy on disk) the kernel will handle the failure
|
|
|
|
transparently without affecting any applications. But if there is
|
|
|
|
no other uptodate copy of the data it will kill to prevent any data
|
|
|
|
corruptions from propagating.
|
|
|
|
|
|
|
|
1: Kill all processes that have the corrupted and not reloadable page mapped
|
|
|
|
as soon as the corruption is detected. Note this is not supported
|
|
|
|
for a few types of pages, like kernel internally allocated data or
|
|
|
|
the swap cache, but works for the majority of user pages.
|
|
|
|
|
|
|
|
0: Only unmap the corrupted page from all processes and only kill a process
|
|
|
|
who tries to access it.
|
|
|
|
|
|
|
|
The kill is done using a catchable SIGBUS with BUS_MCEERR_AO, so processes can
|
|
|
|
handle this if they want to.
|
|
|
|
|
|
|
|
This is only active on architectures/platforms with advanced machine
|
|
|
|
check handling and depends on the hardware capabilities.
|
|
|
|
|
|
|
|
Applications can override this setting individually with the PR_MCE_KILL prctl
|
|
|
|
|
|
|
|
|
|
|
|
memory_failure_recovery
|
2019-04-19 05:35:54 +08:00
|
|
|
=======================
|
2009-09-16 17:50:15 +08:00
|
|
|
|
|
|
|
Enable memory failure recovery (when supported by the platform)
|
|
|
|
|
|
|
|
1: Attempt recovery.
|
|
|
|
|
|
|
|
0: Always panic on a memory failure.
|
|
|
|
|
2006-07-03 15:24:13 +08:00
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
min_free_kbytes
|
|
|
|
===============
|
2006-07-03 15:24:13 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
This is used to force the Linux VM to keep a minimum number
|
2009-06-17 06:32:12 +08:00
|
|
|
of kilobytes free. The VM uses this number to compute a
|
|
|
|
watermark[WMARK_MIN] value for each lowmem zone in the system.
|
|
|
|
Each lowmem zone gets a number of reserved free pages based
|
|
|
|
proportionally on its size.
|
2009-01-16 05:50:42 +08:00
|
|
|
|
|
|
|
Some minimal amount of memory is needed to satisfy PF_MEMALLOC
|
|
|
|
allocations; if you set this to lower than 1024KB, your system will
|
|
|
|
become subtly broken, and prone to deadlock under high loads.
|
|
|
|
|
|
|
|
Setting this too high will OOM your machine instantly.
|
2006-07-03 15:24:13 +08:00
|
|
|
|
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
min_slab_ratio
|
|
|
|
==============
|
2006-09-26 14:31:52 +08:00
|
|
|
|
|
|
|
This is available only on NUMA kernels.
|
|
|
|
|
|
|
|
A percentage of the total pages in each zone. On Zone reclaim
|
|
|
|
(fallback from the local zone occurs) slabs will be reclaimed if more
|
|
|
|
than this percentage of pages in a zone are reclaimable slab pages.
|
|
|
|
This insures that the slab growth stays under control even in NUMA
|
|
|
|
systems that rarely perform global reclaim.
|
|
|
|
|
|
|
|
The default is 5 percent.
|
|
|
|
|
|
|
|
Note that slab reclaim is triggered in a per zone / node fashion.
|
|
|
|
The process of reclaiming slab memory is currently not node specific
|
|
|
|
and may not be fast.
|
|
|
|
|
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
min_unmapped_ratio
|
|
|
|
==================
|
2006-06-23 17:03:13 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
This is available only on NUMA kernels.
|
2006-06-23 17:03:13 +08:00
|
|
|
|
vmscan: properly account for the number of page cache pages zone_reclaim() can reclaim
A bug was brought to my attention against a distro kernel but it affects
mainline and I believe problems like this have been reported in various
guises on the mailing lists although I don't have specific examples at the
moment.
The reported problem was that malloc() stalled for a long time (minutes in
some cases) if a large tmpfs mount was occupying a large percentage of
memory overall. The pages did not get cleaned or reclaimed by
zone_reclaim() because the zone_reclaim_mode was unsuitable, but the lists
are uselessly scanned frequencly making the CPU spin at near 100%.
This patchset intends to address that bug and bring the behaviour of
zone_reclaim() more in line with expectations which were noticed during
investigation. It is based on top of mmotm and takes advantage of
Kosaki's work with respect to zone_reclaim().
Patch 1 fixes the heuristics that zone_reclaim() uses to determine if the
scan should go ahead. The broken heuristic is what was causing the
malloc() stall as it uselessly scanned the LRU constantly. Currently,
zone_reclaim is assuming zone_reclaim_mode is 1 and historically it
could not deal with tmpfs pages at all. This fixes up the heuristic so
that an unnecessary scan is more likely to be correctly avoided.
Patch 2 notes that zone_reclaim() returning a failure automatically means
the zone is marked full. This is not always true. It could have
failed because the GFP mask or zone_reclaim_mode were unsuitable.
Patch 3 introduces a counter zreclaim_failed that will increment each
time the zone_reclaim scan-avoidance heuristics fail. If that
counter is rapidly increasing, then zone_reclaim_mode should be
set to 0 as a temporarily resolution and a bug reported because
the scan-avoidance heuristic is still broken.
This patch:
On NUMA machines, the administrator can configure zone_reclaim_mode that
is a more targetted form of direct reclaim. On machines with large NUMA
distances for example, a zone_reclaim_mode defaults to 1 meaning that
clean unmapped pages will be reclaimed if the zone watermarks are not
being met.
There is a heuristic that determines if the scan is worthwhile but the
problem is that the heuristic is not being properly applied and is
basically assuming zone_reclaim_mode is 1 if it is enabled. The lack of
proper detection can manfiest as high CPU usage as the LRU list is scanned
uselessly.
Historically, once enabled it was depending on NR_FILE_PAGES which may
include swapcache pages that the reclaim_mode cannot deal with. Patch
vmscan-change-the-number-of-the-unmapped-files-in-zone-reclaim.patch by
Kosaki Motohiro noted that zone_page_state(zone, NR_FILE_PAGES) included
pages that were not file-backed such as swapcache and made a calculation
based on the inactive, active and mapped files. This is far superior when
zone_reclaim==1 but if RECLAIM_SWAP is set, then NR_FILE_PAGES is a
reasonable starting figure.
This patch alters how zone_reclaim() works out how many pages it might be
able to reclaim given the current reclaim_mode. If RECLAIM_SWAP is set in
the reclaim_mode it will either consider NR_FILE_PAGES as potential
candidates or else use NR_{IN}ACTIVE}_PAGES-NR_FILE_MAPPED to discount
swapcache and other non-file-backed pages. If RECLAIM_WRITE is not set,
then NR_FILE_DIRTY number of pages are not candidates. If RECLAIM_SWAP is
not set, then NR_FILE_MAPPED are not.
[kosaki.motohiro@jp.fujitsu.com: Estimate unmapped pages minus tmpfs pages]
[fengguang.wu@intel.com: Fix underflow problem in Kosaki's estimate]
Signed-off-by: Mel Gorman <mel@csn.ul.ie>
Reviewed-by: Rik van Riel <riel@redhat.com>
Acked-by: Christoph Lameter <cl@linux-foundation.org>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Wu Fengguang <fengguang.wu@intel.com>
Cc: <stable@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2009-06-17 06:33:20 +08:00
|
|
|
This is a percentage of the total pages in each zone. Zone reclaim will
|
|
|
|
only occur if more than this percentage of pages are in a state that
|
|
|
|
zone_reclaim_mode allows to be reclaimed.
|
|
|
|
|
|
|
|
If zone_reclaim_mode has the value 4 OR'd, then the percentage is compared
|
|
|
|
against all file-backed unmapped pages including swapcache pages and tmpfs
|
|
|
|
files. Otherwise, only unmapped pages backed by normal files but not tmpfs
|
|
|
|
files and similar are considered.
|
2007-05-07 05:49:59 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
The default is 1 percent.
|
2006-06-23 17:03:13 +08:00
|
|
|
|
2007-05-07 05:49:59 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
mmap_min_addr
|
2019-04-19 05:35:54 +08:00
|
|
|
=============
|
2007-06-29 03:55:21 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
This file indicates the amount of address space which a user process will
|
tree-wide: fix assorted typos all over the place
That is "success", "unknown", "through", "performance", "[re|un]mapping"
, "access", "default", "reasonable", "[con]currently", "temperature"
, "channel", "[un]used", "application", "example","hierarchy", "therefore"
, "[over|under]flow", "contiguous", "threshold", "enough" and others.
Signed-off-by: André Goddard Rosa <andre.goddard@gmail.com>
Signed-off-by: Jiri Kosina <jkosina@suse.cz>
2009-11-14 23:09:05 +08:00
|
|
|
be restricted from mmapping. Since kernel null dereference bugs could
|
2009-01-16 05:50:42 +08:00
|
|
|
accidentally operate based on the information in the first couple of pages
|
|
|
|
of memory userspace processes should not be allowed to write to them. By
|
|
|
|
default this value is set to 0 and no protections will be enforced by the
|
|
|
|
security module. Setting this value to something like 64k will allow the
|
|
|
|
vast majority of applications to work correctly and provide defense in depth
|
|
|
|
against future potential kernel bugs.
|
2007-10-17 14:25:56 +08:00
|
|
|
|
oom: add sysctl to enable task memory dump
Adds a new sysctl, 'oom_dump_tasks', that enables the kernel to produce a
dump of all system tasks (excluding kernel threads) when performing an
OOM-killing. Information includes pid, uid, tgid, vm size, rss, cpu,
oom_adj score, and name.
This is helpful for determining why there was an OOM condition and which
rogue task caused it.
It is configurable so that large systems, such as those with several
thousand tasks, do not incur a performance penalty associated with dumping
data they may not desire.
If an OOM was triggered as a result of a memory controller, the tasklist
shall be filtered to exclude tasks that are not a member of the same
cgroup.
Cc: Andrea Arcangeli <andrea@suse.de>
Cc: Christoph Lameter <clameter@sgi.com>
Cc: Balbir Singh <balbir@linux.vnet.ibm.com>
Signed-off-by: David Rientjes <rientjes@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-02-07 16:14:07 +08:00
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
mmap_rnd_bits
|
|
|
|
=============
|
mm: mmap: add new /proc tunable for mmap_base ASLR
Address Space Layout Randomization (ASLR) provides a barrier to
exploitation of user-space processes in the presence of security
vulnerabilities by making it more difficult to find desired code/data
which could help an attack. This is done by adding a random offset to
the location of regions in the process address space, with a greater
range of potential offset values corresponding to better protection/a
larger search-space for brute force, but also to greater potential for
fragmentation.
The offset added to the mmap_base address, which provides the basis for
the majority of the mappings for a process, is set once on process exec
in arch_pick_mmap_layout() and is done via hard-coded per-arch values,
which reflect, hopefully, the best compromise for all systems. The
trade-off between increased entropy in the offset value generation and
the corresponding increased variability in address space fragmentation
is not absolute, however, and some platforms may tolerate higher amounts
of entropy. This patch introduces both new Kconfig values and a sysctl
interface which may be used to change the amount of entropy used for
offset generation on a system.
The direct motivation for this change was in response to the
libstagefright vulnerabilities that affected Android, specifically to
information provided by Google's project zero at:
http://googleprojectzero.blogspot.com/2015/09/stagefrightened.html
The attack presented therein, by Google's project zero, specifically
targeted the limited randomness used to generate the offset added to the
mmap_base address in order to craft a brute-force-based attack.
Concretely, the attack was against the mediaserver process, which was
limited to respawning every 5 seconds, on an arm device. The hard-coded
8 bits used resulted in an average expected success rate of defeating
the mmap ASLR after just over 10 minutes (128 tries at 5 seconds a
piece). With this patch, and an accompanying increase in the entropy
value to 16 bits, the same attack would take an average expected time of
over 45 hours (32768 tries), which makes it both less feasible and more
likely to be noticed.
The introduced Kconfig and sysctl options are limited by per-arch
minimum and maximum values, the minimum of which was chosen to match the
current hard-coded value and the maximum of which was chosen so as to
give the greatest flexibility without generating an invalid mmap_base
address, generally a 3-4 bits less than the number of bits in the
user-space accessible virtual address space.
When decided whether or not to change the default value, a system
developer should consider that mmap_base address could be placed
anywhere up to 2^(value) bits away from the non-randomized location,
which would introduce variable-sized areas above and below the mmap_base
address such that the maximum vm_area_struct size may be reduced,
preventing very large allocations.
This patch (of 4):
ASLR only uses as few as 8 bits to generate the random offset for the
mmap base address on 32 bit architectures. This value was chosen to
prevent a poorly chosen value from dividing the address space in such a
way as to prevent large allocations. This may not be an issue on all
platforms. Allow the specification of a minimum number of bits so that
platforms desiring greater ASLR protection may determine where to place
the trade-off.
Signed-off-by: Daniel Cashman <dcashman@google.com>
Cc: Russell King <linux@arm.linux.org.uk>
Acked-by: Kees Cook <keescook@chromium.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Don Zickus <dzickus@redhat.com>
Cc: Eric W. Biederman <ebiederm@xmission.com>
Cc: Heinrich Schuchardt <xypron.glpk@gmx.de>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: David Rientjes <rientjes@google.com>
Cc: Mark Salyzyn <salyzyn@android.com>
Cc: Jeff Vander Stoep <jeffv@google.com>
Cc: Nick Kralevich <nnk@google.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Hector Marco-Gisbert <hecmargi@upv.es>
Cc: Borislav Petkov <bp@suse.de>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-15 07:19:53 +08:00
|
|
|
|
|
|
|
This value can be used to select the number of bits to use to
|
|
|
|
determine the random offset to the base address of vma regions
|
|
|
|
resulting from mmap allocations on architectures which support
|
|
|
|
tuning address space randomization. This value will be bounded
|
|
|
|
by the architecture's minimum and maximum supported values.
|
|
|
|
|
|
|
|
This value can be changed after boot using the
|
|
|
|
/proc/sys/vm/mmap_rnd_bits tunable
|
|
|
|
|
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
mmap_rnd_compat_bits
|
|
|
|
====================
|
mm: mmap: add new /proc tunable for mmap_base ASLR
Address Space Layout Randomization (ASLR) provides a barrier to
exploitation of user-space processes in the presence of security
vulnerabilities by making it more difficult to find desired code/data
which could help an attack. This is done by adding a random offset to
the location of regions in the process address space, with a greater
range of potential offset values corresponding to better protection/a
larger search-space for brute force, but also to greater potential for
fragmentation.
The offset added to the mmap_base address, which provides the basis for
the majority of the mappings for a process, is set once on process exec
in arch_pick_mmap_layout() and is done via hard-coded per-arch values,
which reflect, hopefully, the best compromise for all systems. The
trade-off between increased entropy in the offset value generation and
the corresponding increased variability in address space fragmentation
is not absolute, however, and some platforms may tolerate higher amounts
of entropy. This patch introduces both new Kconfig values and a sysctl
interface which may be used to change the amount of entropy used for
offset generation on a system.
The direct motivation for this change was in response to the
libstagefright vulnerabilities that affected Android, specifically to
information provided by Google's project zero at:
http://googleprojectzero.blogspot.com/2015/09/stagefrightened.html
The attack presented therein, by Google's project zero, specifically
targeted the limited randomness used to generate the offset added to the
mmap_base address in order to craft a brute-force-based attack.
Concretely, the attack was against the mediaserver process, which was
limited to respawning every 5 seconds, on an arm device. The hard-coded
8 bits used resulted in an average expected success rate of defeating
the mmap ASLR after just over 10 minutes (128 tries at 5 seconds a
piece). With this patch, and an accompanying increase in the entropy
value to 16 bits, the same attack would take an average expected time of
over 45 hours (32768 tries), which makes it both less feasible and more
likely to be noticed.
The introduced Kconfig and sysctl options are limited by per-arch
minimum and maximum values, the minimum of which was chosen to match the
current hard-coded value and the maximum of which was chosen so as to
give the greatest flexibility without generating an invalid mmap_base
address, generally a 3-4 bits less than the number of bits in the
user-space accessible virtual address space.
When decided whether or not to change the default value, a system
developer should consider that mmap_base address could be placed
anywhere up to 2^(value) bits away from the non-randomized location,
which would introduce variable-sized areas above and below the mmap_base
address such that the maximum vm_area_struct size may be reduced,
preventing very large allocations.
This patch (of 4):
ASLR only uses as few as 8 bits to generate the random offset for the
mmap base address on 32 bit architectures. This value was chosen to
prevent a poorly chosen value from dividing the address space in such a
way as to prevent large allocations. This may not be an issue on all
platforms. Allow the specification of a minimum number of bits so that
platforms desiring greater ASLR protection may determine where to place
the trade-off.
Signed-off-by: Daniel Cashman <dcashman@google.com>
Cc: Russell King <linux@arm.linux.org.uk>
Acked-by: Kees Cook <keescook@chromium.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Don Zickus <dzickus@redhat.com>
Cc: Eric W. Biederman <ebiederm@xmission.com>
Cc: Heinrich Schuchardt <xypron.glpk@gmx.de>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: David Rientjes <rientjes@google.com>
Cc: Mark Salyzyn <salyzyn@android.com>
Cc: Jeff Vander Stoep <jeffv@google.com>
Cc: Nick Kralevich <nnk@google.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Hector Marco-Gisbert <hecmargi@upv.es>
Cc: Borislav Petkov <bp@suse.de>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-15 07:19:53 +08:00
|
|
|
|
|
|
|
This value can be used to select the number of bits to use to
|
|
|
|
determine the random offset to the base address of vma regions
|
|
|
|
resulting from mmap allocations for applications run in
|
|
|
|
compatibility mode on architectures which support tuning address
|
|
|
|
space randomization. This value will be bounded by the
|
|
|
|
architecture's minimum and maximum supported values.
|
|
|
|
|
|
|
|
This value can be changed after boot using the
|
|
|
|
/proc/sys/vm/mmap_rnd_compat_bits tunable
|
|
|
|
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
nr_hugepages
|
2019-04-19 05:35:54 +08:00
|
|
|
============
|
oom: add sysctl to enable task memory dump
Adds a new sysctl, 'oom_dump_tasks', that enables the kernel to produce a
dump of all system tasks (excluding kernel threads) when performing an
OOM-killing. Information includes pid, uid, tgid, vm size, rss, cpu,
oom_adj score, and name.
This is helpful for determining why there was an OOM condition and which
rogue task caused it.
It is configurable so that large systems, such as those with several
thousand tasks, do not incur a performance penalty associated with dumping
data they may not desire.
If an OOM was triggered as a result of a memory controller, the tasklist
shall be filtered to exclude tasks that are not a member of the same
cgroup.
Cc: Andrea Arcangeli <andrea@suse.de>
Cc: Christoph Lameter <clameter@sgi.com>
Cc: Balbir Singh <balbir@linux.vnet.ibm.com>
Signed-off-by: David Rientjes <rientjes@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-02-07 16:14:07 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
Change the minimum size of the hugepage pool.
|
oom: add sysctl to enable task memory dump
Adds a new sysctl, 'oom_dump_tasks', that enables the kernel to produce a
dump of all system tasks (excluding kernel threads) when performing an
OOM-killing. Information includes pid, uid, tgid, vm size, rss, cpu,
oom_adj score, and name.
This is helpful for determining why there was an OOM condition and which
rogue task caused it.
It is configurable so that large systems, such as those with several
thousand tasks, do not incur a performance penalty associated with dumping
data they may not desire.
If an OOM was triggered as a result of a memory controller, the tasklist
shall be filtered to exclude tasks that are not a member of the same
cgroup.
Cc: Andrea Arcangeli <andrea@suse.de>
Cc: Christoph Lameter <clameter@sgi.com>
Cc: Balbir Singh <balbir@linux.vnet.ibm.com>
Signed-off-by: David Rientjes <rientjes@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-02-07 16:14:07 +08:00
|
|
|
|
2018-04-18 16:07:49 +08:00
|
|
|
See Documentation/admin-guide/mm/hugetlbpage.rst
|
oom: add sysctl to enable task memory dump
Adds a new sysctl, 'oom_dump_tasks', that enables the kernel to produce a
dump of all system tasks (excluding kernel threads) when performing an
OOM-killing. Information includes pid, uid, tgid, vm size, rss, cpu,
oom_adj score, and name.
This is helpful for determining why there was an OOM condition and which
rogue task caused it.
It is configurable so that large systems, such as those with several
thousand tasks, do not incur a performance penalty associated with dumping
data they may not desire.
If an OOM was triggered as a result of a memory controller, the tasklist
shall be filtered to exclude tasks that are not a member of the same
cgroup.
Cc: Andrea Arcangeli <andrea@suse.de>
Cc: Christoph Lameter <clameter@sgi.com>
Cc: Balbir Singh <balbir@linux.vnet.ibm.com>
Signed-off-by: David Rientjes <rientjes@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-02-07 16:14:07 +08:00
|
|
|
|
2018-07-20 21:35:00 +08:00
|
|
|
|
|
|
|
nr_hugepages_mempolicy
|
2019-04-19 05:35:54 +08:00
|
|
|
======================
|
2018-07-20 21:35:00 +08:00
|
|
|
|
|
|
|
Change the size of the hugepage pool at run-time on a specific
|
|
|
|
set of NUMA nodes.
|
|
|
|
|
|
|
|
See Documentation/admin-guide/mm/hugetlbpage.rst
|
|
|
|
|
oom: add sysctl to enable task memory dump
Adds a new sysctl, 'oom_dump_tasks', that enables the kernel to produce a
dump of all system tasks (excluding kernel threads) when performing an
OOM-killing. Information includes pid, uid, tgid, vm size, rss, cpu,
oom_adj score, and name.
This is helpful for determining why there was an OOM condition and which
rogue task caused it.
It is configurable so that large systems, such as those with several
thousand tasks, do not incur a performance penalty associated with dumping
data they may not desire.
If an OOM was triggered as a result of a memory controller, the tasklist
shall be filtered to exclude tasks that are not a member of the same
cgroup.
Cc: Andrea Arcangeli <andrea@suse.de>
Cc: Christoph Lameter <clameter@sgi.com>
Cc: Balbir Singh <balbir@linux.vnet.ibm.com>
Signed-off-by: David Rientjes <rientjes@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-02-07 16:14:07 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
nr_overcommit_hugepages
|
2019-04-19 05:35:54 +08:00
|
|
|
=======================
|
oom: add sysctl to enable task memory dump
Adds a new sysctl, 'oom_dump_tasks', that enables the kernel to produce a
dump of all system tasks (excluding kernel threads) when performing an
OOM-killing. Information includes pid, uid, tgid, vm size, rss, cpu,
oom_adj score, and name.
This is helpful for determining why there was an OOM condition and which
rogue task caused it.
It is configurable so that large systems, such as those with several
thousand tasks, do not incur a performance penalty associated with dumping
data they may not desire.
If an OOM was triggered as a result of a memory controller, the tasklist
shall be filtered to exclude tasks that are not a member of the same
cgroup.
Cc: Andrea Arcangeli <andrea@suse.de>
Cc: Christoph Lameter <clameter@sgi.com>
Cc: Balbir Singh <balbir@linux.vnet.ibm.com>
Signed-off-by: David Rientjes <rientjes@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-02-07 16:14:07 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
Change the maximum size of the hugepage pool. The maximum is
|
|
|
|
nr_hugepages + nr_overcommit_hugepages.
|
2007-10-17 14:25:56 +08:00
|
|
|
|
2018-04-18 16:07:49 +08:00
|
|
|
See Documentation/admin-guide/mm/hugetlbpage.rst
|
2007-10-17 14:25:56 +08:00
|
|
|
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
nr_trim_pages
|
2019-04-19 05:35:54 +08:00
|
|
|
=============
|
2007-06-29 03:55:21 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
This is available only on NOMMU kernels.
|
|
|
|
|
|
|
|
This value adjusts the excess page trimming behaviour of power-of-2 aligned
|
|
|
|
NOMMU mmap allocations.
|
|
|
|
|
|
|
|
A value of 0 disables trimming of allocations entirely, while a value of 1
|
|
|
|
trims excess pages aggressively. Any value >= 1 acts as the watermark where
|
|
|
|
trimming of allocations is initiated.
|
|
|
|
|
|
|
|
The default value is 1.
|
|
|
|
|
2020-06-23 21:31:36 +08:00
|
|
|
See Documentation/admin-guide/mm/nommu-mmap.rst for more information.
|
2007-06-29 03:55:21 +08:00
|
|
|
|
2007-07-16 14:38:01 +08:00
|
|
|
|
|
|
|
numa_zonelist_order
|
2019-04-19 05:35:54 +08:00
|
|
|
===================
|
2007-07-16 14:38:01 +08:00
|
|
|
|
2017-09-07 07:20:13 +08:00
|
|
|
This sysctl is only for NUMA and it is deprecated. Anything but
|
|
|
|
Node order will fail!
|
|
|
|
|
2007-07-16 14:38:01 +08:00
|
|
|
'where the memory is allocated from' is controlled by zonelists.
|
2019-04-19 05:35:54 +08:00
|
|
|
|
2007-07-16 14:38:01 +08:00
|
|
|
(This documentation ignores ZONE_HIGHMEM/ZONE_DMA32 for simple explanation.
|
2019-04-19 05:35:54 +08:00
|
|
|
you may be able to read ZONE_DMA as ZONE_DMA32...)
|
2007-07-16 14:38:01 +08:00
|
|
|
|
|
|
|
In non-NUMA case, a zonelist for GFP_KERNEL is ordered as following.
|
|
|
|
ZONE_NORMAL -> ZONE_DMA
|
|
|
|
This means that a memory allocation request for GFP_KERNEL will
|
|
|
|
get memory from ZONE_DMA only when ZONE_NORMAL is not available.
|
|
|
|
|
|
|
|
In NUMA case, you can think of following 2 types of order.
|
2019-04-19 05:35:54 +08:00
|
|
|
Assume 2 node NUMA and below is zonelist of Node(0)'s GFP_KERNEL::
|
2007-07-16 14:38:01 +08:00
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
(A) Node(0) ZONE_NORMAL -> Node(0) ZONE_DMA -> Node(1) ZONE_NORMAL
|
|
|
|
(B) Node(0) ZONE_NORMAL -> Node(1) ZONE_NORMAL -> Node(0) ZONE_DMA.
|
2007-07-16 14:38:01 +08:00
|
|
|
|
|
|
|
Type(A) offers the best locality for processes on Node(0), but ZONE_DMA
|
|
|
|
will be used before ZONE_NORMAL exhaustion. This increases possibility of
|
|
|
|
out-of-memory(OOM) of ZONE_DMA because ZONE_DMA is tend to be small.
|
|
|
|
|
|
|
|
Type(B) cannot offer the best locality but is more robust against OOM of
|
|
|
|
the DMA zone.
|
|
|
|
|
|
|
|
Type(A) is called as "Node" order. Type (B) is "Zone" order.
|
|
|
|
|
|
|
|
"Node order" orders the zonelists by node, then by zone within each node.
|
2011-04-06 17:09:55 +08:00
|
|
|
Specify "[Nn]ode" for node order
|
2007-07-16 14:38:01 +08:00
|
|
|
|
|
|
|
"Zone Order" orders the zonelists by zone type, then by node within each
|
2011-04-06 17:09:55 +08:00
|
|
|
zone. Specify "[Zz]one" for zone order.
|
2007-07-16 14:38:01 +08:00
|
|
|
|
2016-04-29 07:19:11 +08:00
|
|
|
Specify "[Dd]efault" to request automatic configuration.
|
|
|
|
|
|
|
|
On 32-bit, the Normal zone needs to be preserved for allocations accessible
|
|
|
|
by the kernel, so "zone" order will be selected.
|
|
|
|
|
|
|
|
On 64-bit, devices that require DMA32/DMA are relatively rare, so "node"
|
|
|
|
order will be selected.
|
|
|
|
|
|
|
|
Default order is recommended unless this is causing problems for your
|
|
|
|
system/application.
|
2007-12-18 08:20:25 +08:00
|
|
|
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
oom_dump_tasks
|
2019-04-19 05:35:54 +08:00
|
|
|
==============
|
2007-12-18 08:20:25 +08:00
|
|
|
|
mm: account pmd page tables to the process
Dave noticed that unprivileged process can allocate significant amount of
memory -- >500 MiB on x86_64 -- and stay unnoticed by oom-killer and
memory cgroup. The trick is to allocate a lot of PMD page tables. Linux
kernel doesn't account PMD tables to the process, only PTE.
The use-cases below use few tricks to allocate a lot of PMD page tables
while keeping VmRSS and VmPTE low. oom_score for the process will be 0.
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>
#include <sys/prctl.h>
#define PUD_SIZE (1UL << 30)
#define PMD_SIZE (1UL << 21)
#define NR_PUD 130000
int main(void)
{
char *addr = NULL;
unsigned long i;
prctl(PR_SET_THP_DISABLE);
for (i = 0; i < NR_PUD ; i++) {
addr = mmap(addr + PUD_SIZE, PUD_SIZE, PROT_WRITE|PROT_READ,
MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
if (addr == MAP_FAILED) {
perror("mmap");
break;
}
*addr = 'x';
munmap(addr, PMD_SIZE);
mmap(addr, PMD_SIZE, PROT_WRITE|PROT_READ,
MAP_ANONYMOUS|MAP_PRIVATE|MAP_FIXED, -1, 0);
if (addr == MAP_FAILED)
perror("re-mmap"), exit(1);
}
printf("PID %d consumed %lu KiB in PMD page tables\n",
getpid(), i * 4096 >> 10);
return pause();
}
The patch addresses the issue by account PMD tables to the process the
same way we account PTE.
The main place where PMD tables is accounted is __pmd_alloc() and
free_pmd_range(). But there're few corner cases:
- HugeTLB can share PMD page tables. The patch handles by accounting
the table to all processes who share it.
- x86 PAE pre-allocates few PMD tables on fork.
- Architectures with FIRST_USER_ADDRESS > 0. We need to adjust sanity
check on exit(2).
Accounting only happens on configuration where PMD page table's level is
present (PMD is not folded). As with nr_ptes we use per-mm counter. The
counter value is used to calculate baseline for badness score by
oom-killer.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Reported-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Hugh Dickins <hughd@google.com>
Reviewed-by: Cyrill Gorcunov <gorcunov@openvz.org>
Cc: Pavel Emelyanov <xemul@openvz.org>
Cc: David Rientjes <rientjes@google.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-02-12 07:26:50 +08:00
|
|
|
Enables a system-wide task dump (excluding kernel threads) to be produced
|
|
|
|
when the kernel performs an OOM-killing and includes such information as
|
2017-11-16 09:35:40 +08:00
|
|
|
pid, uid, tgid, vm size, rss, pgtables_bytes, swapents, oom_score_adj
|
|
|
|
score, and name. This is helpful to determine why the OOM killer was
|
|
|
|
invoked, to identify the rogue task that caused it, and to determine why
|
|
|
|
the OOM killer chose the task it did to kill.
|
2007-12-18 08:20:25 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
If this is set to zero, this information is suppressed. On very
|
|
|
|
large systems with thousands of tasks it may not be feasible to dump
|
|
|
|
the memory state information for each one. Such systems should not
|
|
|
|
be forced to incur a performance penalty in OOM conditions when the
|
|
|
|
information may not be desired.
|
|
|
|
|
|
|
|
If this is set to non-zero, this information is shown whenever the
|
|
|
|
OOM killer actually kills a memory-hogging task.
|
|
|
|
|
2010-08-10 08:18:53 +08:00
|
|
|
The default value is 1 (enabled).
|
2007-12-18 08:20:25 +08:00
|
|
|
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
oom_kill_allocating_task
|
2019-04-19 05:35:54 +08:00
|
|
|
========================
|
2007-12-18 08:20:25 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
This enables or disables killing the OOM-triggering task in
|
|
|
|
out-of-memory situations.
|
2007-12-18 08:20:25 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
If this is set to zero, the OOM killer will scan through the entire
|
|
|
|
tasklist and select a task based on heuristics to kill. This normally
|
|
|
|
selects a rogue memory-hogging task that frees up a large amount of
|
|
|
|
memory when killed.
|
|
|
|
|
|
|
|
If this is set to non-zero, the OOM killer simply kills the task that
|
|
|
|
triggered the out-of-memory condition. This avoids the expensive
|
|
|
|
tasklist scan.
|
|
|
|
|
|
|
|
If panic_on_oom is selected, it takes precedence over whatever value
|
|
|
|
is used in oom_kill_allocating_task.
|
|
|
|
|
|
|
|
The default value is 0.
|
2009-01-08 20:04:47 +08:00
|
|
|
|
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
overcommit_kbytes
|
|
|
|
=================
|
2014-01-22 07:49:14 +08:00
|
|
|
|
|
|
|
When overcommit_memory is set to 2, the committed address space is not
|
|
|
|
permitted to exceed swap plus this amount of physical RAM. See below.
|
|
|
|
|
|
|
|
Note: overcommit_kbytes is the counterpart of overcommit_ratio. Only one
|
|
|
|
of them may be specified at a time. Setting one disables the other (which
|
|
|
|
then appears as 0 when read).
|
|
|
|
|
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
overcommit_memory
|
|
|
|
=================
|
2009-01-08 20:04:47 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
This value contains a flag that enables memory overcommitment.
|
2009-01-08 20:04:47 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
When this flag is 0, the kernel attempts to estimate the amount
|
|
|
|
of free memory left when userspace requests more memory.
|
2009-01-08 20:04:47 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
When this flag is 1, the kernel pretends there is always enough
|
|
|
|
memory until it actually runs out.
|
2009-01-08 20:04:47 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
When this flag is 2, the kernel uses a "never overcommit"
|
|
|
|
policy that attempts to prevent any overcommit of memory.
|
mm: limit growth of 3% hardcoded other user reserve
Add user_reserve_kbytes knob.
Limit the growth of the memory reserved for other user processes to
min(3% current process size, user_reserve_pages). Only about 8MB is
necessary to enable recovery in the default mode, and only a few hundred
MB are required even when overcommit is disabled.
user_reserve_pages defaults to min(3% free pages, 128MB)
I arrived at 128MB by taking the max VSZ of sshd, login, bash, and top ...
then adding the RSS of each.
This only affects OVERCOMMIT_NEVER mode.
Background
1. user reserve
__vm_enough_memory reserves a hardcoded 3% of the current process size for
other applications when overcommit is disabled. This was done so that a
user could recover if they launched a memory hogging process. Without the
reserve, a user would easily run into a message such as:
bash: fork: Cannot allocate memory
2. admin reserve
Additionally, a hardcoded 3% of free memory is reserved for root in both
overcommit 'guess' and 'never' modes. This was intended to prevent a
scenario where root-cant-log-in and perform recovery operations.
Note that this reserve shrinks, and doesn't guarantee a useful reserve.
Motivation
The two hardcoded memory reserves should be updated to account for current
memory sizes.
Also, the admin reserve would be more useful if it didn't shrink too much.
When the current code was originally written, 1GB was considered
"enterprise". Now the 3% reserve can grow to multiple GB on large memory
systems, and it only needs to be a few hundred MB at most to enable a user
or admin to recover a system with an unwanted memory hogging process.
I've found that reducing these reserves is especially beneficial for a
specific type of application load:
* single application system
* one or few processes (e.g. one per core)
* allocating all available memory
* not initializing every page immediately
* long running
I've run scientific clusters with this sort of load. A long running job
sometimes failed many hours (weeks of CPU time) into a calculation. They
weren't initializing all of their memory immediately, and they weren't
using calloc, so I put systems into overcommit 'never' mode. These
clusters run diskless and have no swap.
However, with the current reserves, a user wishing to allocate as much
memory as possible to one process may be prevented from using, for
example, almost 2GB out of 32GB.
The effect is less, but still significant when a user starts a job with
one process per core. I have repeatedly seen a set of processes
requesting the same amount of memory fail because one of them could not
allocate the amount of memory a user would expect to be able to allocate.
For example, Message Passing Interfce (MPI) processes, one per core. And
it is similar for other parallel programming frameworks.
Changing this reserve code will make the overcommit never mode more useful
by allowing applications to allocate nearly all of the available memory.
Also, the new admin_reserve_kbytes will be safer than the current behavior
since the hardcoded 3% of available memory reserve can shrink to something
useless in the case where applications have grabbed all available memory.
Risks
* "bash: fork: Cannot allocate memory"
The downside of the first patch-- which creates a tunable user reserve
that is only used in overcommit 'never' mode--is that an admin can set
it so low that a user may not be able to kill their process, even if
they already have a shell prompt.
Of course, a user can get in the same predicament with the current 3%
reserve--they just have to launch processes until 3% becomes negligible.
* root-cant-log-in problem
The second patch, adding the tunable rootuser_reserve_pages, allows
the admin to shoot themselves in the foot by setting it too small. They
can easily get the system into a state where root-can't-log-in.
However, the new admin_reserve_kbytes will be safer than the current
behavior since the hardcoded 3% of available memory reserve can shrink
to something useless in the case where applications have grabbed all
available memory.
Alternatives
* Memory cgroups provide a more flexible way to limit application memory.
Not everyone wants to set up cgroups or deal with their overhead.
* We could create a fourth overcommit mode which provides smaller reserves.
The size of useful reserves may be drastically different depending
on the whether the system is embedded or enterprise.
* Force users to initialize all of their memory or use calloc.
Some users don't want/expect the system to overcommit when they malloc.
Overcommit 'never' mode is for this scenario, and it should work well.
The new user and admin reserve tunables are simple to use, with low
overhead compared to cgroups. The patches preserve current behavior where
3% of memory is less than 128MB, except that the admin reserve doesn't
shrink to an unusable size under pressure. The code allows admins to tune
for embedded and enterprise usage.
FAQ
* How is the root-cant-login problem addressed?
What happens if admin_reserve_pages is set to 0?
Root is free to shoot themselves in the foot by setting
admin_reserve_kbytes too low.
On x86_64, the minimum useful reserve is:
8MB for overcommit 'guess'
128MB for overcommit 'never'
admin_reserve_pages defaults to min(3% free memory, 8MB)
So, anyone switching to 'never' mode needs to adjust
admin_reserve_pages.
* How do you calculate a minimum useful reserve?
A user or the admin needs enough memory to login and perform
recovery operations, which includes, at a minimum:
sshd or login + bash (or some other shell) + top (or ps, kill, etc.)
For overcommit 'guess', we can sum resident set sizes (RSS)
because we only need enough memory to handle what the recovery
programs will typically use. On x86_64 this is about 8MB.
For overcommit 'never', we can take the max of their virtual sizes (VSZ)
and add the sum of their RSS. We use VSZ instead of RSS because mode
forces us to ensure we can fulfill all of the requested memory allocations--
even if the programs only use a fraction of what they ask for.
On x86_64 this is about 128MB.
When swap is enabled, reserves are useful even when they are as
small as 10MB, regardless of overcommit mode.
When both swap and overcommit are disabled, then the admin should
tune the reserves higher to be absolutley safe. Over 230MB each
was safest in my testing.
* What happens if user_reserve_pages is set to 0?
Note, this only affects overcomitt 'never' mode.
Then a user will be able to allocate all available memory minus
admin_reserve_kbytes.
However, they will easily see a message such as:
"bash: fork: Cannot allocate memory"
And they won't be able to recover/kill their application.
The admin should be able to recover the system if
admin_reserve_kbytes is set appropriately.
* What's the difference between overcommit 'guess' and 'never'?
"Guess" allows an allocation if there are enough free + reclaimable
pages. It has a hardcoded 3% of free pages reserved for root.
"Never" allows an allocation if there is enough swap + a configurable
percentage (default is 50) of physical RAM. It has a hardcoded 3% of
free pages reserved for root, like "Guess" mode. It also has a
hardcoded 3% of the current process size reserved for additional
applications.
* Why is overcommit 'guess' not suitable even when an app eventually
writes to every page? It takes free pages, file pages, available
swap pages, reclaimable slab pages into consideration. In other words,
these are all pages available, then why isn't overcommit suitable?
Because it only looks at the present state of the system. It
does not take into account the memory that other applications have
malloced, but haven't initialized yet. It overcommits the system.
Test Summary
There was little change in behavior in the default overcommit 'guess'
mode with swap enabled before and after the patch. This was expected.
Systems run most predictably (i.e. no oom kills) in overcommit 'never'
mode with swap enabled. This also allowed the most memory to be allocated
to a user application.
Overcommit 'guess' mode without swap is a bad idea. It is easy to
crash the system. None of the other tested combinations crashed.
This matches my experience on the Roadrunner supercomputer.
Without the tunable user reserve, a system in overcommit 'never' mode
and without swap does not allow the admin to recover, although the
admin can.
With the new tunable reserves, a system in overcommit 'never' mode
and without swap can be configured to:
1. maximize user-allocatable memory, running close to the edge of
recoverability
2. maximize recoverability, sacrificing allocatable memory to
ensure that a user cannot take down a system
Test Description
Fedora 18 VM - 4 x86_64 cores, 5725MB RAM, 4GB Swap
System is booted into multiuser console mode, with unnecessary services
turned off. Caches were dropped before each test.
Hogs are user memtester processes that attempt to allocate all free memory
as reported by /proc/meminfo
In overcommit 'never' mode, memory_ratio=100
Test Results
3.9.0-rc1-mm1
Overcommit | Swap | Hogs | MB Got/Wanted | OOMs | User Recovery | Admin Recovery
---------- ---- ---- ------------- ---- ------------- --------------
guess yes 1 5432/5432 no yes yes
guess yes 4 5444/5444 1 yes yes
guess no 1 5302/5449 no yes yes
guess no 4 - crash no no
never yes 1 5460/5460 1 yes yes
never yes 4 5460/5460 1 yes yes
never no 1 5218/5432 no no yes
never no 4 5203/5448 no no yes
3.9.0-rc1-mm1-tunablereserves
User and Admin Recovery show their respective reserves, if applicable.
Overcommit | Swap | Hogs | MB Got/Wanted | OOMs | User Recovery | Admin Recovery
---------- ---- ---- ------------- ---- ------------- --------------
guess yes 1 5419/5419 no - yes 8MB yes
guess yes 4 5436/5436 1 - yes 8MB yes
guess no 1 5440/5440 * - yes 8MB yes
guess no 4 - crash - no 8MB no
* process would successfully mlock, then the oom killer would pick it
never yes 1 5446/5446 no 10MB yes 20MB yes
never yes 4 5456/5456 no 10MB yes 20MB yes
never no 1 5387/5429 no 128MB no 8MB barely
never no 1 5323/5428 no 226MB barely 8MB barely
never no 1 5323/5428 no 226MB barely 8MB barely
never no 1 5359/5448 no 10MB no 10MB barely
never no 1 5323/5428 no 0MB no 10MB barely
never no 1 5332/5428 no 0MB no 50MB yes
never no 1 5293/5429 no 0MB no 90MB yes
never no 1 5001/5427 no 230MB yes 338MB yes
never no 4* 4998/5424 no 230MB yes 338MB yes
* more memtesters were launched, able to allocate approximately another 100MB
Future Work
- Test larger memory systems.
- Test an embedded image.
- Test other architectures.
- Time malloc microbenchmarks.
- Would it be useful to be able to set overcommit policy for
each memory cgroup?
- Some lines are slightly above 80 chars.
Perhaps define a macro to convert between pages and kb?
Other places in the kernel do this.
[akpm@linux-foundation.org: coding-style fixes]
[akpm@linux-foundation.org: make init_user_reserve() static]
Signed-off-by: Andrew Shewmaker <agshew@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-04-30 06:08:10 +08:00
|
|
|
Note that user_reserve_kbytes affects this policy.
|
2009-01-08 20:04:47 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
This feature can be very useful because there are a lot of
|
|
|
|
programs that malloc() huge amounts of memory "just-in-case"
|
|
|
|
and don't use much of it.
|
|
|
|
|
|
|
|
The default value is 0.
|
|
|
|
|
2018-03-22 03:22:47 +08:00
|
|
|
See Documentation/vm/overcommit-accounting.rst and
|
2018-08-22 12:53:20 +08:00
|
|
|
mm/util.c::__vm_enough_memory() for more information.
|
2009-01-16 05:50:42 +08:00
|
|
|
|
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
overcommit_ratio
|
|
|
|
================
|
2009-01-16 05:50:42 +08:00
|
|
|
|
|
|
|
When overcommit_memory is set to 2, the committed address
|
|
|
|
space is not permitted to exceed swap plus this percentage
|
|
|
|
of physical RAM. See above.
|
|
|
|
|
|
|
|
|
|
|
|
page-cluster
|
2019-04-19 05:35:54 +08:00
|
|
|
============
|
2009-01-16 05:50:42 +08:00
|
|
|
|
2012-08-01 07:41:46 +08:00
|
|
|
page-cluster controls the number of pages up to which consecutive pages
|
|
|
|
are read in from swap in a single attempt. This is the swap counterpart
|
|
|
|
to page cache readahead.
|
|
|
|
The mentioned consecutivity is not in terms of virtual/physical addresses,
|
|
|
|
but consecutive on swap space - that means they were swapped out together.
|
2009-01-16 05:50:42 +08:00
|
|
|
|
|
|
|
It is a logarithmic value - setting it to zero means "1 page", setting
|
|
|
|
it to 1 means "2 pages", setting it to 2 means "4 pages", etc.
|
2012-08-01 07:41:46 +08:00
|
|
|
Zero disables swap readahead completely.
|
2009-01-16 05:50:42 +08:00
|
|
|
|
|
|
|
The default value is three (eight pages at a time). There may be some
|
|
|
|
small benefits in tuning this to a different value if your workload is
|
|
|
|
swap-intensive.
|
|
|
|
|
2012-08-01 07:41:46 +08:00
|
|
|
Lower values mean lower latencies for initial faults, but at the same time
|
|
|
|
extra faults and I/O delays for following faults if they would have been part of
|
|
|
|
that consecutive pages readahead would have brought in.
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
|
|
|
|
panic_on_oom
|
2019-04-19 05:35:54 +08:00
|
|
|
============
|
2009-01-16 05:50:42 +08:00
|
|
|
|
|
|
|
This enables or disables panic on out-of-memory feature.
|
|
|
|
|
|
|
|
If this is set to 0, the kernel will kill some rogue process,
|
|
|
|
called oom_killer. Usually, oom_killer can kill rogue processes and
|
|
|
|
system will survive.
|
|
|
|
|
|
|
|
If this is set to 1, the kernel panics when out-of-memory happens.
|
|
|
|
However, if a process limits using nodes by mempolicy/cpusets,
|
|
|
|
and those nodes become memory exhaustion status, one process
|
|
|
|
may be killed by oom-killer. No panic occurs in this case.
|
|
|
|
Because other nodes' memory may be free. This means system total status
|
|
|
|
may be not fatal yet.
|
|
|
|
|
|
|
|
If this is set to 2, the kernel panics compulsorily even on the
|
memcg: handle panic_on_oom=always case
Presently, if panic_on_oom=2, the whole system panics even if the oom
happend in some special situation (as cpuset, mempolicy....). Then,
panic_on_oom=2 means painc_on_oom_always.
Now, memcg doesn't check panic_on_oom flag. This patch adds a check.
BTW, how it's useful ?
kdump+panic_on_oom=2 is the last tool to investigate what happens in
oom-ed system. When a task is killed, the sysytem recovers and there will
be few hint to know what happnes. In mission critical system, oom should
never happen. Then, panic_on_oom=2+kdump is useful to avoid next OOM by
knowing precise information via snapshot.
TODO:
- For memcg, it's for isolate system's memory usage, oom-notiifer and
freeze_at_oom (or rest_at_oom) should be implemented. Then, management
daemon can do similar jobs (as kdump) or taking snapshot per cgroup.
Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: Balbir Singh <balbir@linux.vnet.ibm.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Nick Piggin <npiggin@suse.de>
Reviewed-by: Daisuke Nishimura <nishimura@mxp.nes.nec.co.jp>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2010-03-11 07:22:32 +08:00
|
|
|
above-mentioned. Even oom happens under memory cgroup, the whole
|
|
|
|
system panics.
|
2009-01-16 05:50:42 +08:00
|
|
|
|
|
|
|
The default value is 0.
|
2019-04-19 05:35:54 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
1 and 2 are for failover of clustering. Please select either
|
|
|
|
according to your policy of failover.
|
2019-04-19 05:35:54 +08:00
|
|
|
|
memcg: handle panic_on_oom=always case
Presently, if panic_on_oom=2, the whole system panics even if the oom
happend in some special situation (as cpuset, mempolicy....). Then,
panic_on_oom=2 means painc_on_oom_always.
Now, memcg doesn't check panic_on_oom flag. This patch adds a check.
BTW, how it's useful ?
kdump+panic_on_oom=2 is the last tool to investigate what happens in
oom-ed system. When a task is killed, the sysytem recovers and there will
be few hint to know what happnes. In mission critical system, oom should
never happen. Then, panic_on_oom=2+kdump is useful to avoid next OOM by
knowing precise information via snapshot.
TODO:
- For memcg, it's for isolate system's memory usage, oom-notiifer and
freeze_at_oom (or rest_at_oom) should be implemented. Then, management
daemon can do similar jobs (as kdump) or taking snapshot per cgroup.
Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: Balbir Singh <balbir@linux.vnet.ibm.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Nick Piggin <npiggin@suse.de>
Reviewed-by: Daisuke Nishimura <nishimura@mxp.nes.nec.co.jp>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2010-03-11 07:22:32 +08:00
|
|
|
panic_on_oom=2+kdump gives you very strong tool to investigate
|
|
|
|
why oom happens. You can get snapshot.
|
2009-01-16 05:50:42 +08:00
|
|
|
|
|
|
|
|
2021-06-29 10:42:24 +08:00
|
|
|
percpu_pagelist_high_fraction
|
|
|
|
=============================
|
|
|
|
|
|
|
|
This is the fraction of pages in each zone that are can be stored to
|
|
|
|
per-cpu page lists. It is an upper boundary that is divided depending
|
|
|
|
on the number of online CPUs. The min value for this is 8 which means
|
|
|
|
that we do not allow more than 1/8th of pages in each zone to be stored
|
|
|
|
on per-cpu page lists. This entry only changes the value of hot per-cpu
|
|
|
|
page lists. A user can specify a number like 100 to allocate 1/100th of
|
|
|
|
each zone between per-cpu lists.
|
|
|
|
|
|
|
|
The batch value of each per-cpu page list remains the same regardless of
|
|
|
|
the value of the high fraction so allocation latencies are unaffected.
|
|
|
|
|
|
|
|
The initial value is zero. Kernel uses this value to set the high pcp->high
|
|
|
|
mark based on the low watermark for the zone and the number of local
|
|
|
|
online CPUs. If the user writes '0' to this sysctl, it will revert to
|
|
|
|
this default behavior.
|
|
|
|
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
stat_interval
|
2019-04-19 05:35:54 +08:00
|
|
|
=============
|
2009-01-16 05:50:42 +08:00
|
|
|
|
|
|
|
The time interval between which vm statistics are updated. The default
|
|
|
|
is 1 second.
|
|
|
|
|
|
|
|
|
2016-05-20 08:12:50 +08:00
|
|
|
stat_refresh
|
2019-04-19 05:35:54 +08:00
|
|
|
============
|
2016-05-20 08:12:50 +08:00
|
|
|
|
|
|
|
Any read or write (by root only) flushes all the per-cpu vm statistics
|
|
|
|
into their global totals, for more accurate reports when testing
|
|
|
|
e.g. cat /proc/sys/vm/stat_refresh /proc/meminfo
|
|
|
|
|
|
|
|
As a side-effect, it also checks for negative totals (elsewhere reported
|
|
|
|
as 0) and "fails" with EINVAL if any are found, with a warning in dmesg.
|
|
|
|
(At time of writing, a few stats are known sometimes to be found negative,
|
|
|
|
with no ill effects: errors and warnings on these stats are suppressed.)
|
|
|
|
|
|
|
|
|
2017-11-16 09:38:22 +08:00
|
|
|
numa_stat
|
2019-04-19 05:35:54 +08:00
|
|
|
=========
|
2017-11-16 09:38:22 +08:00
|
|
|
|
|
|
|
This interface allows runtime configuration of numa statistics.
|
|
|
|
|
|
|
|
When page allocation performance becomes a bottleneck and you can tolerate
|
|
|
|
some possible tool breakage and decreased numa counter precision, you can
|
2019-04-19 05:35:54 +08:00
|
|
|
do::
|
|
|
|
|
2017-11-16 09:38:22 +08:00
|
|
|
echo 0 > /proc/sys/vm/numa_stat
|
|
|
|
|
|
|
|
When page allocation performance is not a bottleneck and you want all
|
2019-04-19 05:35:54 +08:00
|
|
|
tooling to work, you can do::
|
|
|
|
|
2017-11-16 09:38:22 +08:00
|
|
|
echo 1 > /proc/sys/vm/numa_stat
|
|
|
|
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
swappiness
|
2019-04-19 05:35:54 +08:00
|
|
|
==========
|
2009-01-16 05:50:42 +08:00
|
|
|
|
2020-06-04 07:02:37 +08:00
|
|
|
This control is used to define the rough relative IO cost of swapping
|
|
|
|
and filesystem paging, as a value between 0 and 200. At 100, the VM
|
|
|
|
assumes equal IO cost and will thus apply memory pressure to the page
|
|
|
|
cache and swap-backed pages equally; lower values signify more
|
|
|
|
expensive swap IO, higher values indicates cheaper.
|
|
|
|
|
|
|
|
Keep in mind that filesystem IO patterns under memory pressure tend to
|
|
|
|
be more efficient than swap's random IO. An optimal value will require
|
|
|
|
experimentation and will also be workload-dependent.
|
2009-01-16 05:50:42 +08:00
|
|
|
|
|
|
|
The default value is 60.
|
|
|
|
|
2020-06-04 07:02:37 +08:00
|
|
|
For in-memory swap, like zram or zswap, as well as hybrid setups that
|
|
|
|
have swap on faster devices than the filesystem, values beyond 100 can
|
|
|
|
be considered. For example, if the random IO against the swap device
|
|
|
|
is on average 2x faster than IO from the filesystem, swappiness should
|
|
|
|
be 133 (x + 2x = 200, 2x = 133.33).
|
|
|
|
|
|
|
|
At 0, the kernel will not initiate swap until the amount of free and
|
|
|
|
file-backed pages is less than the high watermark in a zone.
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
|
userfaultfd/sysctl: add vm.unprivileged_userfaultfd
Userfaultfd can be misued to make it easier to exploit existing
use-after-free (and similar) bugs that might otherwise only make a
short window or race condition available. By using userfaultfd to
stall a kernel thread, a malicious program can keep some state that it
wrote, stable for an extended period, which it can then access using an
existing exploit. While it doesn't cause the exploit itself, and while
it's not the only thing that can stall a kernel thread when accessing a
memory location, it's one of the few that never needs privilege.
We can add a flag, allowing userfaultfd to be restricted, so that in
general it won't be useable by arbitrary user programs, but in
environments that require userfaultfd it can be turned back on.
Add a global sysctl knob "vm.unprivileged_userfaultfd" to control
whether userfaultfd is allowed by unprivileged users. When this is
set to zero, only privileged users (root user, or users with the
CAP_SYS_PTRACE capability) will be able to use the userfaultfd
syscalls.
Andrea said:
: The only difference between the bpf sysctl and the userfaultfd sysctl
: this way is that the bpf sysctl adds the CAP_SYS_ADMIN capability
: requirement, while userfaultfd adds the CAP_SYS_PTRACE requirement,
: because the userfaultfd monitor is more likely to need CAP_SYS_PTRACE
: already if it's doing other kind of tracking on processes runtime, in
: addition of userfaultfd. In other words both syscalls works only for
: root, when the two sysctl are opt-in set to 1.
[dgilbert@redhat.com: changelog additions]
[akpm@linux-foundation.org: documentation tweak, per Mike]
Link: http://lkml.kernel.org/r/20190319030722.12441-2-peterx@redhat.com
Signed-off-by: Peter Xu <peterx@redhat.com>
Suggested-by: Andrea Arcangeli <aarcange@redhat.com>
Suggested-by: Mike Rapoport <rppt@linux.ibm.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
Reviewed-by: Andrea Arcangeli <aarcange@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Luis Chamberlain <mcgrof@kernel.org>
Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
Cc: Maya Gokhale <gokhale2@llnl.gov>
Cc: Jerome Glisse <jglisse@redhat.com>
Cc: Pavel Emelyanov <xemul@virtuozzo.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Martin Cracauer <cracauer@cons.org>
Cc: Denis Plotnikov <dplotnikov@virtuozzo.com>
Cc: Marty McFadden <mcfadden8@llnl.gov>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: "Kirill A . Shutemov" <kirill@shutemov.name>
Cc: "Dr . David Alan Gilbert" <dgilbert@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2019-05-14 08:16:41 +08:00
|
|
|
unprivileged_userfaultfd
|
2019-04-19 05:35:54 +08:00
|
|
|
========================
|
userfaultfd/sysctl: add vm.unprivileged_userfaultfd
Userfaultfd can be misued to make it easier to exploit existing
use-after-free (and similar) bugs that might otherwise only make a
short window or race condition available. By using userfaultfd to
stall a kernel thread, a malicious program can keep some state that it
wrote, stable for an extended period, which it can then access using an
existing exploit. While it doesn't cause the exploit itself, and while
it's not the only thing that can stall a kernel thread when accessing a
memory location, it's one of the few that never needs privilege.
We can add a flag, allowing userfaultfd to be restricted, so that in
general it won't be useable by arbitrary user programs, but in
environments that require userfaultfd it can be turned back on.
Add a global sysctl knob "vm.unprivileged_userfaultfd" to control
whether userfaultfd is allowed by unprivileged users. When this is
set to zero, only privileged users (root user, or users with the
CAP_SYS_PTRACE capability) will be able to use the userfaultfd
syscalls.
Andrea said:
: The only difference between the bpf sysctl and the userfaultfd sysctl
: this way is that the bpf sysctl adds the CAP_SYS_ADMIN capability
: requirement, while userfaultfd adds the CAP_SYS_PTRACE requirement,
: because the userfaultfd monitor is more likely to need CAP_SYS_PTRACE
: already if it's doing other kind of tracking on processes runtime, in
: addition of userfaultfd. In other words both syscalls works only for
: root, when the two sysctl are opt-in set to 1.
[dgilbert@redhat.com: changelog additions]
[akpm@linux-foundation.org: documentation tweak, per Mike]
Link: http://lkml.kernel.org/r/20190319030722.12441-2-peterx@redhat.com
Signed-off-by: Peter Xu <peterx@redhat.com>
Suggested-by: Andrea Arcangeli <aarcange@redhat.com>
Suggested-by: Mike Rapoport <rppt@linux.ibm.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
Reviewed-by: Andrea Arcangeli <aarcange@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Luis Chamberlain <mcgrof@kernel.org>
Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
Cc: Maya Gokhale <gokhale2@llnl.gov>
Cc: Jerome Glisse <jglisse@redhat.com>
Cc: Pavel Emelyanov <xemul@virtuozzo.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Martin Cracauer <cracauer@cons.org>
Cc: Denis Plotnikov <dplotnikov@virtuozzo.com>
Cc: Marty McFadden <mcfadden8@llnl.gov>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: "Kirill A . Shutemov" <kirill@shutemov.name>
Cc: "Dr . David Alan Gilbert" <dgilbert@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2019-05-14 08:16:41 +08:00
|
|
|
|
userfaultfd: add user-mode only option to unprivileged_userfaultfd sysctl knob
With this change, when the knob is set to 0, it allows unprivileged users
to call userfaultfd, like when it is set to 1, but with the restriction
that page faults from only user-mode can be handled. In this mode, an
unprivileged user (without SYS_CAP_PTRACE capability) must pass
UFFD_USER_MODE_ONLY to userfaultd or the API will fail with EPERM.
This enables administrators to reduce the likelihood that an attacker with
access to userfaultfd can delay faulting kernel code to widen timing
windows for other exploits.
The default value of this knob is changed to 0. This is required for
correct functioning of pipe mutex. However, this will fail postcopy live
migration, which will be unnoticeable to the VM guests. To avoid this,
set 'vm.userfault = 1' in /sys/sysctl.conf.
The main reason this change is desirable as in the short term is that the
Android userland will behave as with the sysctl set to zero. So without
this commit, any Linux binary using userfaultfd to manage its memory would
behave differently if run within the Android userland. For more details,
refer to Andrea's reply [1].
[1] https://lore.kernel.org/lkml/20200904033438.GI9411@redhat.com/
Link: https://lkml.kernel.org/r/20201120030411.2690816-3-lokeshgidra@google.com
Signed-off-by: Lokesh Gidra <lokeshgidra@google.com>
Reviewed-by: Andrea Arcangeli <aarcange@redhat.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Peter Xu <peterx@redhat.com>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Stephen Smalley <stephen.smalley.work@gmail.com>
Cc: Eric Biggers <ebiggers@kernel.org>
Cc: Daniel Colascione <dancol@dancol.org>
Cc: "Joel Fernandes (Google)" <joel@joelfernandes.org>
Cc: Kalesh Singh <kaleshsingh@google.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Jeff Vander Stoep <jeffv@google.com>
Cc: <calin@google.com>
Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
Cc: Shaohua Li <shli@fb.com>
Cc: Jerome Glisse <jglisse@redhat.com>
Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Mel Gorman <mgorman@techsingularity.net>
Cc: Nitin Gupta <nigupta@nvidia.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Iurii Zaikin <yzaikin@google.com>
Cc: Luis Chamberlain <mcgrof@kernel.org>
Cc: Daniel Colascione <dancol@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-12-15 11:13:54 +08:00
|
|
|
This flag controls the mode in which unprivileged users can use the
|
|
|
|
userfaultfd system calls. Set this to 0 to restrict unprivileged users
|
|
|
|
to handle page faults in user mode only. In this case, users without
|
|
|
|
SYS_CAP_PTRACE must pass UFFD_USER_MODE_ONLY in order for userfaultfd to
|
|
|
|
succeed. Prohibiting use of userfaultfd for handling faults from kernel
|
|
|
|
mode may make certain vulnerabilities more difficult to exploit.
|
userfaultfd/sysctl: add vm.unprivileged_userfaultfd
Userfaultfd can be misued to make it easier to exploit existing
use-after-free (and similar) bugs that might otherwise only make a
short window or race condition available. By using userfaultfd to
stall a kernel thread, a malicious program can keep some state that it
wrote, stable for an extended period, which it can then access using an
existing exploit. While it doesn't cause the exploit itself, and while
it's not the only thing that can stall a kernel thread when accessing a
memory location, it's one of the few that never needs privilege.
We can add a flag, allowing userfaultfd to be restricted, so that in
general it won't be useable by arbitrary user programs, but in
environments that require userfaultfd it can be turned back on.
Add a global sysctl knob "vm.unprivileged_userfaultfd" to control
whether userfaultfd is allowed by unprivileged users. When this is
set to zero, only privileged users (root user, or users with the
CAP_SYS_PTRACE capability) will be able to use the userfaultfd
syscalls.
Andrea said:
: The only difference between the bpf sysctl and the userfaultfd sysctl
: this way is that the bpf sysctl adds the CAP_SYS_ADMIN capability
: requirement, while userfaultfd adds the CAP_SYS_PTRACE requirement,
: because the userfaultfd monitor is more likely to need CAP_SYS_PTRACE
: already if it's doing other kind of tracking on processes runtime, in
: addition of userfaultfd. In other words both syscalls works only for
: root, when the two sysctl are opt-in set to 1.
[dgilbert@redhat.com: changelog additions]
[akpm@linux-foundation.org: documentation tweak, per Mike]
Link: http://lkml.kernel.org/r/20190319030722.12441-2-peterx@redhat.com
Signed-off-by: Peter Xu <peterx@redhat.com>
Suggested-by: Andrea Arcangeli <aarcange@redhat.com>
Suggested-by: Mike Rapoport <rppt@linux.ibm.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
Reviewed-by: Andrea Arcangeli <aarcange@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Luis Chamberlain <mcgrof@kernel.org>
Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
Cc: Maya Gokhale <gokhale2@llnl.gov>
Cc: Jerome Glisse <jglisse@redhat.com>
Cc: Pavel Emelyanov <xemul@virtuozzo.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Martin Cracauer <cracauer@cons.org>
Cc: Denis Plotnikov <dplotnikov@virtuozzo.com>
Cc: Marty McFadden <mcfadden8@llnl.gov>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: "Kirill A . Shutemov" <kirill@shutemov.name>
Cc: "Dr . David Alan Gilbert" <dgilbert@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2019-05-14 08:16:41 +08:00
|
|
|
|
userfaultfd: add user-mode only option to unprivileged_userfaultfd sysctl knob
With this change, when the knob is set to 0, it allows unprivileged users
to call userfaultfd, like when it is set to 1, but with the restriction
that page faults from only user-mode can be handled. In this mode, an
unprivileged user (without SYS_CAP_PTRACE capability) must pass
UFFD_USER_MODE_ONLY to userfaultd or the API will fail with EPERM.
This enables administrators to reduce the likelihood that an attacker with
access to userfaultfd can delay faulting kernel code to widen timing
windows for other exploits.
The default value of this knob is changed to 0. This is required for
correct functioning of pipe mutex. However, this will fail postcopy live
migration, which will be unnoticeable to the VM guests. To avoid this,
set 'vm.userfault = 1' in /sys/sysctl.conf.
The main reason this change is desirable as in the short term is that the
Android userland will behave as with the sysctl set to zero. So without
this commit, any Linux binary using userfaultfd to manage its memory would
behave differently if run within the Android userland. For more details,
refer to Andrea's reply [1].
[1] https://lore.kernel.org/lkml/20200904033438.GI9411@redhat.com/
Link: https://lkml.kernel.org/r/20201120030411.2690816-3-lokeshgidra@google.com
Signed-off-by: Lokesh Gidra <lokeshgidra@google.com>
Reviewed-by: Andrea Arcangeli <aarcange@redhat.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Peter Xu <peterx@redhat.com>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Stephen Smalley <stephen.smalley.work@gmail.com>
Cc: Eric Biggers <ebiggers@kernel.org>
Cc: Daniel Colascione <dancol@dancol.org>
Cc: "Joel Fernandes (Google)" <joel@joelfernandes.org>
Cc: Kalesh Singh <kaleshsingh@google.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Jeff Vander Stoep <jeffv@google.com>
Cc: <calin@google.com>
Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
Cc: Shaohua Li <shli@fb.com>
Cc: Jerome Glisse <jglisse@redhat.com>
Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Mel Gorman <mgorman@techsingularity.net>
Cc: Nitin Gupta <nigupta@nvidia.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Iurii Zaikin <yzaikin@google.com>
Cc: Luis Chamberlain <mcgrof@kernel.org>
Cc: Daniel Colascione <dancol@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-12-15 11:13:54 +08:00
|
|
|
Set this to 1 to allow unprivileged users to use the userfaultfd system
|
|
|
|
calls without any restrictions.
|
|
|
|
|
|
|
|
The default value is 0.
|
userfaultfd/sysctl: add vm.unprivileged_userfaultfd
Userfaultfd can be misued to make it easier to exploit existing
use-after-free (and similar) bugs that might otherwise only make a
short window or race condition available. By using userfaultfd to
stall a kernel thread, a malicious program can keep some state that it
wrote, stable for an extended period, which it can then access using an
existing exploit. While it doesn't cause the exploit itself, and while
it's not the only thing that can stall a kernel thread when accessing a
memory location, it's one of the few that never needs privilege.
We can add a flag, allowing userfaultfd to be restricted, so that in
general it won't be useable by arbitrary user programs, but in
environments that require userfaultfd it can be turned back on.
Add a global sysctl knob "vm.unprivileged_userfaultfd" to control
whether userfaultfd is allowed by unprivileged users. When this is
set to zero, only privileged users (root user, or users with the
CAP_SYS_PTRACE capability) will be able to use the userfaultfd
syscalls.
Andrea said:
: The only difference between the bpf sysctl and the userfaultfd sysctl
: this way is that the bpf sysctl adds the CAP_SYS_ADMIN capability
: requirement, while userfaultfd adds the CAP_SYS_PTRACE requirement,
: because the userfaultfd monitor is more likely to need CAP_SYS_PTRACE
: already if it's doing other kind of tracking on processes runtime, in
: addition of userfaultfd. In other words both syscalls works only for
: root, when the two sysctl are opt-in set to 1.
[dgilbert@redhat.com: changelog additions]
[akpm@linux-foundation.org: documentation tweak, per Mike]
Link: http://lkml.kernel.org/r/20190319030722.12441-2-peterx@redhat.com
Signed-off-by: Peter Xu <peterx@redhat.com>
Suggested-by: Andrea Arcangeli <aarcange@redhat.com>
Suggested-by: Mike Rapoport <rppt@linux.ibm.com>
Reviewed-by: Mike Rapoport <rppt@linux.ibm.com>
Reviewed-by: Andrea Arcangeli <aarcange@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Luis Chamberlain <mcgrof@kernel.org>
Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
Cc: Maya Gokhale <gokhale2@llnl.gov>
Cc: Jerome Glisse <jglisse@redhat.com>
Cc: Pavel Emelyanov <xemul@virtuozzo.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Martin Cracauer <cracauer@cons.org>
Cc: Denis Plotnikov <dplotnikov@virtuozzo.com>
Cc: Marty McFadden <mcfadden8@llnl.gov>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: "Kirill A . Shutemov" <kirill@shutemov.name>
Cc: "Dr . David Alan Gilbert" <dgilbert@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2019-05-14 08:16:41 +08:00
|
|
|
|
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
user_reserve_kbytes
|
|
|
|
===================
|
mm: limit growth of 3% hardcoded other user reserve
Add user_reserve_kbytes knob.
Limit the growth of the memory reserved for other user processes to
min(3% current process size, user_reserve_pages). Only about 8MB is
necessary to enable recovery in the default mode, and only a few hundred
MB are required even when overcommit is disabled.
user_reserve_pages defaults to min(3% free pages, 128MB)
I arrived at 128MB by taking the max VSZ of sshd, login, bash, and top ...
then adding the RSS of each.
This only affects OVERCOMMIT_NEVER mode.
Background
1. user reserve
__vm_enough_memory reserves a hardcoded 3% of the current process size for
other applications when overcommit is disabled. This was done so that a
user could recover if they launched a memory hogging process. Without the
reserve, a user would easily run into a message such as:
bash: fork: Cannot allocate memory
2. admin reserve
Additionally, a hardcoded 3% of free memory is reserved for root in both
overcommit 'guess' and 'never' modes. This was intended to prevent a
scenario where root-cant-log-in and perform recovery operations.
Note that this reserve shrinks, and doesn't guarantee a useful reserve.
Motivation
The two hardcoded memory reserves should be updated to account for current
memory sizes.
Also, the admin reserve would be more useful if it didn't shrink too much.
When the current code was originally written, 1GB was considered
"enterprise". Now the 3% reserve can grow to multiple GB on large memory
systems, and it only needs to be a few hundred MB at most to enable a user
or admin to recover a system with an unwanted memory hogging process.
I've found that reducing these reserves is especially beneficial for a
specific type of application load:
* single application system
* one or few processes (e.g. one per core)
* allocating all available memory
* not initializing every page immediately
* long running
I've run scientific clusters with this sort of load. A long running job
sometimes failed many hours (weeks of CPU time) into a calculation. They
weren't initializing all of their memory immediately, and they weren't
using calloc, so I put systems into overcommit 'never' mode. These
clusters run diskless and have no swap.
However, with the current reserves, a user wishing to allocate as much
memory as possible to one process may be prevented from using, for
example, almost 2GB out of 32GB.
The effect is less, but still significant when a user starts a job with
one process per core. I have repeatedly seen a set of processes
requesting the same amount of memory fail because one of them could not
allocate the amount of memory a user would expect to be able to allocate.
For example, Message Passing Interfce (MPI) processes, one per core. And
it is similar for other parallel programming frameworks.
Changing this reserve code will make the overcommit never mode more useful
by allowing applications to allocate nearly all of the available memory.
Also, the new admin_reserve_kbytes will be safer than the current behavior
since the hardcoded 3% of available memory reserve can shrink to something
useless in the case where applications have grabbed all available memory.
Risks
* "bash: fork: Cannot allocate memory"
The downside of the first patch-- which creates a tunable user reserve
that is only used in overcommit 'never' mode--is that an admin can set
it so low that a user may not be able to kill their process, even if
they already have a shell prompt.
Of course, a user can get in the same predicament with the current 3%
reserve--they just have to launch processes until 3% becomes negligible.
* root-cant-log-in problem
The second patch, adding the tunable rootuser_reserve_pages, allows
the admin to shoot themselves in the foot by setting it too small. They
can easily get the system into a state where root-can't-log-in.
However, the new admin_reserve_kbytes will be safer than the current
behavior since the hardcoded 3% of available memory reserve can shrink
to something useless in the case where applications have grabbed all
available memory.
Alternatives
* Memory cgroups provide a more flexible way to limit application memory.
Not everyone wants to set up cgroups or deal with their overhead.
* We could create a fourth overcommit mode which provides smaller reserves.
The size of useful reserves may be drastically different depending
on the whether the system is embedded or enterprise.
* Force users to initialize all of their memory or use calloc.
Some users don't want/expect the system to overcommit when they malloc.
Overcommit 'never' mode is for this scenario, and it should work well.
The new user and admin reserve tunables are simple to use, with low
overhead compared to cgroups. The patches preserve current behavior where
3% of memory is less than 128MB, except that the admin reserve doesn't
shrink to an unusable size under pressure. The code allows admins to tune
for embedded and enterprise usage.
FAQ
* How is the root-cant-login problem addressed?
What happens if admin_reserve_pages is set to 0?
Root is free to shoot themselves in the foot by setting
admin_reserve_kbytes too low.
On x86_64, the minimum useful reserve is:
8MB for overcommit 'guess'
128MB for overcommit 'never'
admin_reserve_pages defaults to min(3% free memory, 8MB)
So, anyone switching to 'never' mode needs to adjust
admin_reserve_pages.
* How do you calculate a minimum useful reserve?
A user or the admin needs enough memory to login and perform
recovery operations, which includes, at a minimum:
sshd or login + bash (or some other shell) + top (or ps, kill, etc.)
For overcommit 'guess', we can sum resident set sizes (RSS)
because we only need enough memory to handle what the recovery
programs will typically use. On x86_64 this is about 8MB.
For overcommit 'never', we can take the max of their virtual sizes (VSZ)
and add the sum of their RSS. We use VSZ instead of RSS because mode
forces us to ensure we can fulfill all of the requested memory allocations--
even if the programs only use a fraction of what they ask for.
On x86_64 this is about 128MB.
When swap is enabled, reserves are useful even when they are as
small as 10MB, regardless of overcommit mode.
When both swap and overcommit are disabled, then the admin should
tune the reserves higher to be absolutley safe. Over 230MB each
was safest in my testing.
* What happens if user_reserve_pages is set to 0?
Note, this only affects overcomitt 'never' mode.
Then a user will be able to allocate all available memory minus
admin_reserve_kbytes.
However, they will easily see a message such as:
"bash: fork: Cannot allocate memory"
And they won't be able to recover/kill their application.
The admin should be able to recover the system if
admin_reserve_kbytes is set appropriately.
* What's the difference between overcommit 'guess' and 'never'?
"Guess" allows an allocation if there are enough free + reclaimable
pages. It has a hardcoded 3% of free pages reserved for root.
"Never" allows an allocation if there is enough swap + a configurable
percentage (default is 50) of physical RAM. It has a hardcoded 3% of
free pages reserved for root, like "Guess" mode. It also has a
hardcoded 3% of the current process size reserved for additional
applications.
* Why is overcommit 'guess' not suitable even when an app eventually
writes to every page? It takes free pages, file pages, available
swap pages, reclaimable slab pages into consideration. In other words,
these are all pages available, then why isn't overcommit suitable?
Because it only looks at the present state of the system. It
does not take into account the memory that other applications have
malloced, but haven't initialized yet. It overcommits the system.
Test Summary
There was little change in behavior in the default overcommit 'guess'
mode with swap enabled before and after the patch. This was expected.
Systems run most predictably (i.e. no oom kills) in overcommit 'never'
mode with swap enabled. This also allowed the most memory to be allocated
to a user application.
Overcommit 'guess' mode without swap is a bad idea. It is easy to
crash the system. None of the other tested combinations crashed.
This matches my experience on the Roadrunner supercomputer.
Without the tunable user reserve, a system in overcommit 'never' mode
and without swap does not allow the admin to recover, although the
admin can.
With the new tunable reserves, a system in overcommit 'never' mode
and without swap can be configured to:
1. maximize user-allocatable memory, running close to the edge of
recoverability
2. maximize recoverability, sacrificing allocatable memory to
ensure that a user cannot take down a system
Test Description
Fedora 18 VM - 4 x86_64 cores, 5725MB RAM, 4GB Swap
System is booted into multiuser console mode, with unnecessary services
turned off. Caches were dropped before each test.
Hogs are user memtester processes that attempt to allocate all free memory
as reported by /proc/meminfo
In overcommit 'never' mode, memory_ratio=100
Test Results
3.9.0-rc1-mm1
Overcommit | Swap | Hogs | MB Got/Wanted | OOMs | User Recovery | Admin Recovery
---------- ---- ---- ------------- ---- ------------- --------------
guess yes 1 5432/5432 no yes yes
guess yes 4 5444/5444 1 yes yes
guess no 1 5302/5449 no yes yes
guess no 4 - crash no no
never yes 1 5460/5460 1 yes yes
never yes 4 5460/5460 1 yes yes
never no 1 5218/5432 no no yes
never no 4 5203/5448 no no yes
3.9.0-rc1-mm1-tunablereserves
User and Admin Recovery show their respective reserves, if applicable.
Overcommit | Swap | Hogs | MB Got/Wanted | OOMs | User Recovery | Admin Recovery
---------- ---- ---- ------------- ---- ------------- --------------
guess yes 1 5419/5419 no - yes 8MB yes
guess yes 4 5436/5436 1 - yes 8MB yes
guess no 1 5440/5440 * - yes 8MB yes
guess no 4 - crash - no 8MB no
* process would successfully mlock, then the oom killer would pick it
never yes 1 5446/5446 no 10MB yes 20MB yes
never yes 4 5456/5456 no 10MB yes 20MB yes
never no 1 5387/5429 no 128MB no 8MB barely
never no 1 5323/5428 no 226MB barely 8MB barely
never no 1 5323/5428 no 226MB barely 8MB barely
never no 1 5359/5448 no 10MB no 10MB barely
never no 1 5323/5428 no 0MB no 10MB barely
never no 1 5332/5428 no 0MB no 50MB yes
never no 1 5293/5429 no 0MB no 90MB yes
never no 1 5001/5427 no 230MB yes 338MB yes
never no 4* 4998/5424 no 230MB yes 338MB yes
* more memtesters were launched, able to allocate approximately another 100MB
Future Work
- Test larger memory systems.
- Test an embedded image.
- Test other architectures.
- Time malloc microbenchmarks.
- Would it be useful to be able to set overcommit policy for
each memory cgroup?
- Some lines are slightly above 80 chars.
Perhaps define a macro to convert between pages and kb?
Other places in the kernel do this.
[akpm@linux-foundation.org: coding-style fixes]
[akpm@linux-foundation.org: make init_user_reserve() static]
Signed-off-by: Andrew Shewmaker <agshew@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-04-30 06:08:10 +08:00
|
|
|
|
2015-01-02 11:03:19 +08:00
|
|
|
When overcommit_memory is set to 2, "never overcommit" mode, reserve
|
mm: limit growth of 3% hardcoded other user reserve
Add user_reserve_kbytes knob.
Limit the growth of the memory reserved for other user processes to
min(3% current process size, user_reserve_pages). Only about 8MB is
necessary to enable recovery in the default mode, and only a few hundred
MB are required even when overcommit is disabled.
user_reserve_pages defaults to min(3% free pages, 128MB)
I arrived at 128MB by taking the max VSZ of sshd, login, bash, and top ...
then adding the RSS of each.
This only affects OVERCOMMIT_NEVER mode.
Background
1. user reserve
__vm_enough_memory reserves a hardcoded 3% of the current process size for
other applications when overcommit is disabled. This was done so that a
user could recover if they launched a memory hogging process. Without the
reserve, a user would easily run into a message such as:
bash: fork: Cannot allocate memory
2. admin reserve
Additionally, a hardcoded 3% of free memory is reserved for root in both
overcommit 'guess' and 'never' modes. This was intended to prevent a
scenario where root-cant-log-in and perform recovery operations.
Note that this reserve shrinks, and doesn't guarantee a useful reserve.
Motivation
The two hardcoded memory reserves should be updated to account for current
memory sizes.
Also, the admin reserve would be more useful if it didn't shrink too much.
When the current code was originally written, 1GB was considered
"enterprise". Now the 3% reserve can grow to multiple GB on large memory
systems, and it only needs to be a few hundred MB at most to enable a user
or admin to recover a system with an unwanted memory hogging process.
I've found that reducing these reserves is especially beneficial for a
specific type of application load:
* single application system
* one or few processes (e.g. one per core)
* allocating all available memory
* not initializing every page immediately
* long running
I've run scientific clusters with this sort of load. A long running job
sometimes failed many hours (weeks of CPU time) into a calculation. They
weren't initializing all of their memory immediately, and they weren't
using calloc, so I put systems into overcommit 'never' mode. These
clusters run diskless and have no swap.
However, with the current reserves, a user wishing to allocate as much
memory as possible to one process may be prevented from using, for
example, almost 2GB out of 32GB.
The effect is less, but still significant when a user starts a job with
one process per core. I have repeatedly seen a set of processes
requesting the same amount of memory fail because one of them could not
allocate the amount of memory a user would expect to be able to allocate.
For example, Message Passing Interfce (MPI) processes, one per core. And
it is similar for other parallel programming frameworks.
Changing this reserve code will make the overcommit never mode more useful
by allowing applications to allocate nearly all of the available memory.
Also, the new admin_reserve_kbytes will be safer than the current behavior
since the hardcoded 3% of available memory reserve can shrink to something
useless in the case where applications have grabbed all available memory.
Risks
* "bash: fork: Cannot allocate memory"
The downside of the first patch-- which creates a tunable user reserve
that is only used in overcommit 'never' mode--is that an admin can set
it so low that a user may not be able to kill their process, even if
they already have a shell prompt.
Of course, a user can get in the same predicament with the current 3%
reserve--they just have to launch processes until 3% becomes negligible.
* root-cant-log-in problem
The second patch, adding the tunable rootuser_reserve_pages, allows
the admin to shoot themselves in the foot by setting it too small. They
can easily get the system into a state where root-can't-log-in.
However, the new admin_reserve_kbytes will be safer than the current
behavior since the hardcoded 3% of available memory reserve can shrink
to something useless in the case where applications have grabbed all
available memory.
Alternatives
* Memory cgroups provide a more flexible way to limit application memory.
Not everyone wants to set up cgroups or deal with their overhead.
* We could create a fourth overcommit mode which provides smaller reserves.
The size of useful reserves may be drastically different depending
on the whether the system is embedded or enterprise.
* Force users to initialize all of their memory or use calloc.
Some users don't want/expect the system to overcommit when they malloc.
Overcommit 'never' mode is for this scenario, and it should work well.
The new user and admin reserve tunables are simple to use, with low
overhead compared to cgroups. The patches preserve current behavior where
3% of memory is less than 128MB, except that the admin reserve doesn't
shrink to an unusable size under pressure. The code allows admins to tune
for embedded and enterprise usage.
FAQ
* How is the root-cant-login problem addressed?
What happens if admin_reserve_pages is set to 0?
Root is free to shoot themselves in the foot by setting
admin_reserve_kbytes too low.
On x86_64, the minimum useful reserve is:
8MB for overcommit 'guess'
128MB for overcommit 'never'
admin_reserve_pages defaults to min(3% free memory, 8MB)
So, anyone switching to 'never' mode needs to adjust
admin_reserve_pages.
* How do you calculate a minimum useful reserve?
A user or the admin needs enough memory to login and perform
recovery operations, which includes, at a minimum:
sshd or login + bash (or some other shell) + top (or ps, kill, etc.)
For overcommit 'guess', we can sum resident set sizes (RSS)
because we only need enough memory to handle what the recovery
programs will typically use. On x86_64 this is about 8MB.
For overcommit 'never', we can take the max of their virtual sizes (VSZ)
and add the sum of their RSS. We use VSZ instead of RSS because mode
forces us to ensure we can fulfill all of the requested memory allocations--
even if the programs only use a fraction of what they ask for.
On x86_64 this is about 128MB.
When swap is enabled, reserves are useful even when they are as
small as 10MB, regardless of overcommit mode.
When both swap and overcommit are disabled, then the admin should
tune the reserves higher to be absolutley safe. Over 230MB each
was safest in my testing.
* What happens if user_reserve_pages is set to 0?
Note, this only affects overcomitt 'never' mode.
Then a user will be able to allocate all available memory minus
admin_reserve_kbytes.
However, they will easily see a message such as:
"bash: fork: Cannot allocate memory"
And they won't be able to recover/kill their application.
The admin should be able to recover the system if
admin_reserve_kbytes is set appropriately.
* What's the difference between overcommit 'guess' and 'never'?
"Guess" allows an allocation if there are enough free + reclaimable
pages. It has a hardcoded 3% of free pages reserved for root.
"Never" allows an allocation if there is enough swap + a configurable
percentage (default is 50) of physical RAM. It has a hardcoded 3% of
free pages reserved for root, like "Guess" mode. It also has a
hardcoded 3% of the current process size reserved for additional
applications.
* Why is overcommit 'guess' not suitable even when an app eventually
writes to every page? It takes free pages, file pages, available
swap pages, reclaimable slab pages into consideration. In other words,
these are all pages available, then why isn't overcommit suitable?
Because it only looks at the present state of the system. It
does not take into account the memory that other applications have
malloced, but haven't initialized yet. It overcommits the system.
Test Summary
There was little change in behavior in the default overcommit 'guess'
mode with swap enabled before and after the patch. This was expected.
Systems run most predictably (i.e. no oom kills) in overcommit 'never'
mode with swap enabled. This also allowed the most memory to be allocated
to a user application.
Overcommit 'guess' mode without swap is a bad idea. It is easy to
crash the system. None of the other tested combinations crashed.
This matches my experience on the Roadrunner supercomputer.
Without the tunable user reserve, a system in overcommit 'never' mode
and without swap does not allow the admin to recover, although the
admin can.
With the new tunable reserves, a system in overcommit 'never' mode
and without swap can be configured to:
1. maximize user-allocatable memory, running close to the edge of
recoverability
2. maximize recoverability, sacrificing allocatable memory to
ensure that a user cannot take down a system
Test Description
Fedora 18 VM - 4 x86_64 cores, 5725MB RAM, 4GB Swap
System is booted into multiuser console mode, with unnecessary services
turned off. Caches were dropped before each test.
Hogs are user memtester processes that attempt to allocate all free memory
as reported by /proc/meminfo
In overcommit 'never' mode, memory_ratio=100
Test Results
3.9.0-rc1-mm1
Overcommit | Swap | Hogs | MB Got/Wanted | OOMs | User Recovery | Admin Recovery
---------- ---- ---- ------------- ---- ------------- --------------
guess yes 1 5432/5432 no yes yes
guess yes 4 5444/5444 1 yes yes
guess no 1 5302/5449 no yes yes
guess no 4 - crash no no
never yes 1 5460/5460 1 yes yes
never yes 4 5460/5460 1 yes yes
never no 1 5218/5432 no no yes
never no 4 5203/5448 no no yes
3.9.0-rc1-mm1-tunablereserves
User and Admin Recovery show their respective reserves, if applicable.
Overcommit | Swap | Hogs | MB Got/Wanted | OOMs | User Recovery | Admin Recovery
---------- ---- ---- ------------- ---- ------------- --------------
guess yes 1 5419/5419 no - yes 8MB yes
guess yes 4 5436/5436 1 - yes 8MB yes
guess no 1 5440/5440 * - yes 8MB yes
guess no 4 - crash - no 8MB no
* process would successfully mlock, then the oom killer would pick it
never yes 1 5446/5446 no 10MB yes 20MB yes
never yes 4 5456/5456 no 10MB yes 20MB yes
never no 1 5387/5429 no 128MB no 8MB barely
never no 1 5323/5428 no 226MB barely 8MB barely
never no 1 5323/5428 no 226MB barely 8MB barely
never no 1 5359/5448 no 10MB no 10MB barely
never no 1 5323/5428 no 0MB no 10MB barely
never no 1 5332/5428 no 0MB no 50MB yes
never no 1 5293/5429 no 0MB no 90MB yes
never no 1 5001/5427 no 230MB yes 338MB yes
never no 4* 4998/5424 no 230MB yes 338MB yes
* more memtesters were launched, able to allocate approximately another 100MB
Future Work
- Test larger memory systems.
- Test an embedded image.
- Test other architectures.
- Time malloc microbenchmarks.
- Would it be useful to be able to set overcommit policy for
each memory cgroup?
- Some lines are slightly above 80 chars.
Perhaps define a macro to convert between pages and kb?
Other places in the kernel do this.
[akpm@linux-foundation.org: coding-style fixes]
[akpm@linux-foundation.org: make init_user_reserve() static]
Signed-off-by: Andrew Shewmaker <agshew@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-04-30 06:08:10 +08:00
|
|
|
min(3% of current process size, user_reserve_kbytes) of free memory.
|
|
|
|
This is intended to prevent a user from starting a single memory hogging
|
|
|
|
process, such that they cannot recover (kill the hog).
|
|
|
|
|
|
|
|
user_reserve_kbytes defaults to min(3% of the current process size, 128MB).
|
|
|
|
|
|
|
|
If this is reduced to zero, then the user will be allowed to allocate
|
|
|
|
all free memory with a single process, minus admin_reserve_kbytes.
|
|
|
|
Any subsequent attempts to execute a command will result in
|
|
|
|
"fork: Cannot allocate memory".
|
|
|
|
|
|
|
|
Changing this takes effect whenever an application requests memory.
|
|
|
|
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
vfs_cache_pressure
|
2019-04-19 05:35:54 +08:00
|
|
|
==================
|
2009-01-16 05:50:42 +08:00
|
|
|
|
2014-06-05 07:11:03 +08:00
|
|
|
This percentage value controls the tendency of the kernel to reclaim
|
|
|
|
the memory which is used for caching of directory and inode objects.
|
2009-01-16 05:50:42 +08:00
|
|
|
|
|
|
|
At the default value of vfs_cache_pressure=100 the kernel will attempt to
|
|
|
|
reclaim dentries and inodes at a "fair" rate with respect to pagecache and
|
|
|
|
swapcache reclaim. Decreasing vfs_cache_pressure causes the kernel to prefer
|
2009-09-22 08:01:40 +08:00
|
|
|
to retain dentry and inode caches. When vfs_cache_pressure=0, the kernel will
|
|
|
|
never reclaim dentries and inodes due to memory pressure and this can easily
|
|
|
|
lead to out-of-memory conditions. Increasing vfs_cache_pressure beyond 100
|
2009-01-16 05:50:42 +08:00
|
|
|
causes the kernel to prefer to reclaim dentries and inodes.
|
|
|
|
|
2014-06-05 07:11:03 +08:00
|
|
|
Increasing vfs_cache_pressure significantly beyond 100 may have negative
|
|
|
|
performance impact. Reclaim code needs to take various locks to find freeable
|
|
|
|
directory and inode objects. With vfs_cache_pressure=1000, it will look for
|
|
|
|
ten times more freeable objects than there are.
|
|
|
|
|
2016-03-18 05:19:14 +08:00
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
watermark_boost_factor
|
|
|
|
======================
|
mm: reclaim small amounts of memory when an external fragmentation event occurs
An external fragmentation event was previously described as
When the page allocator fragments memory, it records the event using
the mm_page_alloc_extfrag event. If the fallback_order is smaller
than a pageblock order (order-9 on 64-bit x86) then it's considered
an event that will cause external fragmentation issues in the future.
The kernel reduces the probability of such events by increasing the
watermark sizes by calling set_recommended_min_free_kbytes early in the
lifetime of the system. This works reasonably well in general but if
there are enough sparsely populated pageblocks then the problem can still
occur as enough memory is free overall and kswapd stays asleep.
This patch introduces a watermark_boost_factor sysctl that allows a zone
watermark to be temporarily boosted when an external fragmentation causing
events occurs. The boosting will stall allocations that would decrease
free memory below the boosted low watermark and kswapd is woken if the
calling context allows to reclaim an amount of memory relative to the size
of the high watermark and the watermark_boost_factor until the boost is
cleared. When kswapd finishes, it wakes kcompactd at the pageblock order
to clean some of the pageblocks that may have been affected by the
fragmentation event. kswapd avoids any writeback, slab shrinkage and swap
from reclaim context during this operation to avoid excessive system
disruption in the name of fragmentation avoidance. Care is taken so that
kswapd will do normal reclaim work if the system is really low on memory.
This was evaluated using the same workloads as "mm, page_alloc: Spread
allocations across zones before introducing fragmentation".
1-socket Skylake machine
config-global-dhp__workload_thpfioscale XFS (no special madvise)
4 fio threads, 1 THP allocating thread
--------------------------------------
4.20-rc3 extfrag events < order 9: 804694
4.20-rc3+patch: 408912 (49% reduction)
4.20-rc3+patch1-4: 18421 (98% reduction)
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Amean fault-base-1 653.58 ( 0.00%) 652.71 ( 0.13%)
Amean fault-huge-1 0.00 ( 0.00%) 178.93 * -99.00%*
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Percentage huge-1 0.00 ( 0.00%) 5.12 ( 100.00%)
Note that external fragmentation causing events are massively reduced by
this path whether in comparison to the previous kernel or the vanilla
kernel. The fault latency for huge pages appears to be increased but that
is only because THP allocations were successful with the patch applied.
1-socket Skylake machine
global-dhp__workload_thpfioscale-madvhugepage-xfs (MADV_HUGEPAGE)
-----------------------------------------------------------------
4.20-rc3 extfrag events < order 9: 291392
4.20-rc3+patch: 191187 (34% reduction)
4.20-rc3+patch1-4: 13464 (95% reduction)
thpfioscale Fault Latencies
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Min fault-base-1 912.00 ( 0.00%) 905.00 ( 0.77%)
Min fault-huge-1 127.00 ( 0.00%) 135.00 ( -6.30%)
Amean fault-base-1 1467.55 ( 0.00%) 1481.67 ( -0.96%)
Amean fault-huge-1 1127.11 ( 0.00%) 1063.88 * 5.61%*
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Percentage huge-1 77.64 ( 0.00%) 83.46 ( 7.49%)
As before, massive reduction in external fragmentation events, some jitter
on latencies and an increase in THP allocation success rates.
2-socket Haswell machine
config-global-dhp__workload_thpfioscale XFS (no special madvise)
4 fio threads, 5 THP allocating threads
----------------------------------------------------------------
4.20-rc3 extfrag events < order 9: 215698
4.20-rc3+patch: 200210 (7% reduction)
4.20-rc3+patch1-4: 14263 (93% reduction)
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Amean fault-base-5 1346.45 ( 0.00%) 1306.87 ( 2.94%)
Amean fault-huge-5 3418.60 ( 0.00%) 1348.94 ( 60.54%)
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Percentage huge-5 0.78 ( 0.00%) 7.91 ( 910.64%)
There is a 93% reduction in fragmentation causing events, there is a big
reduction in the huge page fault latency and allocation success rate is
higher.
2-socket Haswell machine
global-dhp__workload_thpfioscale-madvhugepage-xfs (MADV_HUGEPAGE)
-----------------------------------------------------------------
4.20-rc3 extfrag events < order 9: 166352
4.20-rc3+patch: 147463 (11% reduction)
4.20-rc3+patch1-4: 11095 (93% reduction)
thpfioscale Fault Latencies
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Amean fault-base-5 6217.43 ( 0.00%) 7419.67 * -19.34%*
Amean fault-huge-5 3163.33 ( 0.00%) 3263.80 ( -3.18%)
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Percentage huge-5 95.14 ( 0.00%) 87.98 ( -7.53%)
There is a large reduction in fragmentation events with some jitter around
the latencies and success rates. As before, the high THP allocation
success rate does mean the system is under a lot of pressure. However, as
the fragmentation events are reduced, it would be expected that the
long-term allocation success rate would be higher.
Link: http://lkml.kernel.org/r/20181123114528.28802-5-mgorman@techsingularity.net
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Zi Yan <zi.yan@cs.rutgers.edu>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-12-28 16:35:52 +08:00
|
|
|
|
|
|
|
This factor controls the level of reclaim when memory is being fragmented.
|
|
|
|
It defines the percentage of the high watermark of a zone that will be
|
|
|
|
reclaimed if pages of different mobility are being mixed within pageblocks.
|
|
|
|
The intent is that compaction has less work to do in the future and to
|
|
|
|
increase the success rate of future high-order allocations such as SLUB
|
|
|
|
allocations, THP and hugetlbfs pages.
|
|
|
|
|
2019-04-26 13:23:51 +08:00
|
|
|
To make it sensible with respect to the watermark_scale_factor
|
|
|
|
parameter, the unit is in fractions of 10,000. The default value of
|
2021-06-29 10:42:58 +08:00
|
|
|
15,000 means that up to 150% of the high watermark will be reclaimed in the
|
|
|
|
event of a pageblock being mixed due to fragmentation. The level of reclaim
|
|
|
|
is determined by the number of fragmentation events that occurred in the
|
|
|
|
recent past. If this value is smaller than a pageblock then a pageblocks
|
|
|
|
worth of pages will be reclaimed (e.g. 2MB on 64-bit x86). A boost factor
|
|
|
|
of 0 will disable the feature.
|
mm: reclaim small amounts of memory when an external fragmentation event occurs
An external fragmentation event was previously described as
When the page allocator fragments memory, it records the event using
the mm_page_alloc_extfrag event. If the fallback_order is smaller
than a pageblock order (order-9 on 64-bit x86) then it's considered
an event that will cause external fragmentation issues in the future.
The kernel reduces the probability of such events by increasing the
watermark sizes by calling set_recommended_min_free_kbytes early in the
lifetime of the system. This works reasonably well in general but if
there are enough sparsely populated pageblocks then the problem can still
occur as enough memory is free overall and kswapd stays asleep.
This patch introduces a watermark_boost_factor sysctl that allows a zone
watermark to be temporarily boosted when an external fragmentation causing
events occurs. The boosting will stall allocations that would decrease
free memory below the boosted low watermark and kswapd is woken if the
calling context allows to reclaim an amount of memory relative to the size
of the high watermark and the watermark_boost_factor until the boost is
cleared. When kswapd finishes, it wakes kcompactd at the pageblock order
to clean some of the pageblocks that may have been affected by the
fragmentation event. kswapd avoids any writeback, slab shrinkage and swap
from reclaim context during this operation to avoid excessive system
disruption in the name of fragmentation avoidance. Care is taken so that
kswapd will do normal reclaim work if the system is really low on memory.
This was evaluated using the same workloads as "mm, page_alloc: Spread
allocations across zones before introducing fragmentation".
1-socket Skylake machine
config-global-dhp__workload_thpfioscale XFS (no special madvise)
4 fio threads, 1 THP allocating thread
--------------------------------------
4.20-rc3 extfrag events < order 9: 804694
4.20-rc3+patch: 408912 (49% reduction)
4.20-rc3+patch1-4: 18421 (98% reduction)
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Amean fault-base-1 653.58 ( 0.00%) 652.71 ( 0.13%)
Amean fault-huge-1 0.00 ( 0.00%) 178.93 * -99.00%*
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Percentage huge-1 0.00 ( 0.00%) 5.12 ( 100.00%)
Note that external fragmentation causing events are massively reduced by
this path whether in comparison to the previous kernel or the vanilla
kernel. The fault latency for huge pages appears to be increased but that
is only because THP allocations were successful with the patch applied.
1-socket Skylake machine
global-dhp__workload_thpfioscale-madvhugepage-xfs (MADV_HUGEPAGE)
-----------------------------------------------------------------
4.20-rc3 extfrag events < order 9: 291392
4.20-rc3+patch: 191187 (34% reduction)
4.20-rc3+patch1-4: 13464 (95% reduction)
thpfioscale Fault Latencies
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Min fault-base-1 912.00 ( 0.00%) 905.00 ( 0.77%)
Min fault-huge-1 127.00 ( 0.00%) 135.00 ( -6.30%)
Amean fault-base-1 1467.55 ( 0.00%) 1481.67 ( -0.96%)
Amean fault-huge-1 1127.11 ( 0.00%) 1063.88 * 5.61%*
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Percentage huge-1 77.64 ( 0.00%) 83.46 ( 7.49%)
As before, massive reduction in external fragmentation events, some jitter
on latencies and an increase in THP allocation success rates.
2-socket Haswell machine
config-global-dhp__workload_thpfioscale XFS (no special madvise)
4 fio threads, 5 THP allocating threads
----------------------------------------------------------------
4.20-rc3 extfrag events < order 9: 215698
4.20-rc3+patch: 200210 (7% reduction)
4.20-rc3+patch1-4: 14263 (93% reduction)
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Amean fault-base-5 1346.45 ( 0.00%) 1306.87 ( 2.94%)
Amean fault-huge-5 3418.60 ( 0.00%) 1348.94 ( 60.54%)
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Percentage huge-5 0.78 ( 0.00%) 7.91 ( 910.64%)
There is a 93% reduction in fragmentation causing events, there is a big
reduction in the huge page fault latency and allocation success rate is
higher.
2-socket Haswell machine
global-dhp__workload_thpfioscale-madvhugepage-xfs (MADV_HUGEPAGE)
-----------------------------------------------------------------
4.20-rc3 extfrag events < order 9: 166352
4.20-rc3+patch: 147463 (11% reduction)
4.20-rc3+patch1-4: 11095 (93% reduction)
thpfioscale Fault Latencies
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Amean fault-base-5 6217.43 ( 0.00%) 7419.67 * -19.34%*
Amean fault-huge-5 3163.33 ( 0.00%) 3263.80 ( -3.18%)
4.20.0-rc3 4.20.0-rc3
lowzone-v5r8 boost-v5r8
Percentage huge-5 95.14 ( 0.00%) 87.98 ( -7.53%)
There is a large reduction in fragmentation events with some jitter around
the latencies and success rates. As before, the high THP allocation
success rate does mean the system is under a lot of pressure. However, as
the fragmentation events are reduced, it would be expected that the
long-term allocation success rate would be higher.
Link: http://lkml.kernel.org/r/20181123114528.28802-5-mgorman@techsingularity.net
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Zi Yan <zi.yan@cs.rutgers.edu>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-12-28 16:35:52 +08:00
|
|
|
|
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
watermark_scale_factor
|
|
|
|
======================
|
2016-03-18 05:19:14 +08:00
|
|
|
|
|
|
|
This factor controls the aggressiveness of kswapd. It defines the
|
|
|
|
amount of memory left in a node/system before kswapd is woken up and
|
|
|
|
how much memory needs to be free before kswapd goes back to sleep.
|
|
|
|
|
|
|
|
The unit is in fractions of 10,000. The default value of 10 means the
|
|
|
|
distances between watermarks are 0.1% of the available memory in the
|
|
|
|
node/system. The maximum value is 1000, or 10% of memory.
|
|
|
|
|
|
|
|
A high rate of threads entering direct reclaim (allocstall) or kswapd
|
|
|
|
going to sleep prematurely (kswapd_low_wmark_hit_quickly) can indicate
|
|
|
|
that the number of free pages kswapd maintains for latency reasons is
|
|
|
|
too small for the allocation bursts occurring in the system. This knob
|
|
|
|
can then be used to tune kswapd aggressiveness accordingly.
|
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
zone_reclaim_mode
|
|
|
|
=================
|
2009-01-16 05:50:42 +08:00
|
|
|
|
|
|
|
Zone_reclaim_mode allows someone to set more or less aggressive approaches to
|
|
|
|
reclaim memory when a zone runs out of memory. If it is set to zero then no
|
|
|
|
zone reclaim occurs. Allocations will be satisfied from other zones / nodes
|
|
|
|
in the system.
|
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
This is value OR'ed together of
|
2009-01-16 05:50:42 +08:00
|
|
|
|
2019-04-19 05:35:54 +08:00
|
|
|
= ===================================
|
|
|
|
1 Zone reclaim on
|
|
|
|
2 Zone reclaim writes dirty pages out
|
|
|
|
4 Zone reclaim swaps pages
|
|
|
|
= ===================================
|
2009-01-16 05:50:42 +08:00
|
|
|
|
2014-06-05 07:07:14 +08:00
|
|
|
zone_reclaim_mode is disabled by default. For file servers or workloads
|
|
|
|
that benefit from having their data cached, zone_reclaim_mode should be
|
|
|
|
left disabled as the caching effect is likely to be more important than
|
2009-01-16 05:50:42 +08:00
|
|
|
data locality.
|
|
|
|
|
2021-02-25 04:09:15 +08:00
|
|
|
Consider enabling one or more zone_reclaim mode bits if it's known that the
|
|
|
|
workload is partitioned such that each partition fits within a NUMA node
|
|
|
|
and that accessing remote memory would cause a measurable performance
|
|
|
|
reduction. The page allocator will take additional actions before
|
|
|
|
allocating off node pages.
|
2014-06-05 07:07:14 +08:00
|
|
|
|
2009-01-16 05:50:42 +08:00
|
|
|
Allowing zone reclaim to write out pages stops processes that are
|
|
|
|
writing large amounts of data from dirtying pages on other nodes. Zone
|
|
|
|
reclaim will write out dirty pages if a zone fills up and so effectively
|
|
|
|
throttle the process. This may decrease the performance of a single process
|
|
|
|
since it cannot use all of system memory to buffer the outgoing writes
|
|
|
|
anymore but it preserve the memory on other nodes so that the performance
|
|
|
|
of other processes running on other nodes will not be affected.
|
|
|
|
|
|
|
|
Allowing regular swap effectively restricts allocations to the local
|
|
|
|
node unless explicitly overridden by memory policies or cpuset
|
|
|
|
configurations.
|