Remove some goto's that made the logic here a little more tortuous than
necessary.
Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: Neil Brown <neilb@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
We shouldn't check for replays until after checking whether the open owner is
confirmed. Clients are allowed to reuse openowners without bumping the seqid.
Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: Neil Brown <neilb@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
We need to make sure open reclaims are marked confirmed immediately so that we
can handle replays even if they fail (e.g. with a seqid-incrementing error).
(See 8.1.8.)
Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: Neil Brown <neilb@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Make sure we get a directory when we look up the recovery directory.
Thanks to Christoph Hellwig for the bug report.
Based on feedback from Christoph and others, we may remove the need for this
lookup and just pass in a file descriptor from userspace instead, and/or
completely move the directory handling to userspace. For now we're just
fixing the obvious bugs.
Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: Neil Brown <neilb@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
We should be opening this directory RDONLY, not RDWR.
Thanks to Christoph Hellwig for the bug report.
Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: Neil Brown <neilb@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Allow mechanisms to return more varied errors on the context creation
downcall.
Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: Neil Brown <neilb@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
We require the server's gssd to create a completed context before asking the
kernel to send a final context init reply. However, gssd could be buggy, or
under some bizarre circumstances we might purge the context from our cache
before we get the chance to use it here.
Handle this case by returning GSS_S_NO_CONTEXT to the client.
Also move the relevant code here to a separate function rather than nesting
excessively.
Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: Neil Brown <neilb@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Kerberos context initiation is handled in a single round trip, but other
mechanisms (including spkm3) may require more, so we need to handle the
GSS_S_CONTINUE case in svcauth_gss_accept. Send a null verifier.
Signed-off-by: Andy Adamson <andros@citi.umich.edu>
Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: Neil Brown <neilb@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Simple, useful debugging printk: print the number of each op as we process it.
Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: Neil Brown <neilb@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Fix some bad logic.
Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: Neil Brown <neilb@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
It's confusing having both release_stateowner() and release_state_owner().
And as it turns out, release_state_owner() is short and only called from one
place; so just remove it.
Also note the confirmed check is superfluous there--preprocess_seqid_op
already check this.
And remove a redundant comment and a superfluous line assignment while we're
at it.
Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: Neil Brown <neilb@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
One of the things that's confusing about nfsd4_lock is that the lk_stateowner
field could be set to either of two different lockowners: the open owner or
the lock owner. Rename to lk_replay_owner and add a comment to make it clear
that it's used for whichever stateowner has its sequence id bumped for replay
detection.
Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: Neil Brown <neilb@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
release_state_owner also puts the lock owner on the close_lru. There's no
need for that, though; replays of the failed lock would be handled by the
openowner not the lockowner.
Also consolidate the cleanup a bit, fixing leaks that can happen if errors
occur between the time a new lock owner is allocated and the lock is done.
Remove a comment and dprintk that look a little redundant.
Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: Neil Brown <neilb@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Logic fixes for LOCK and UNLOCK.
- Move the permission check on the current file handle outside of
nfs4_lock_state()
- remove the file manager fl_release_private calls; fl_ops is not set.
Signed-off-by: Andy Adamson <andros@citi.umich.edu>
Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: Neil Brown <neilb@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
The server code currently keeps track of the destination address on every
request so that it can reply using the same address. However we forget to do
that in the case of a deferred request. Remedy this oversight. >From folks
at PolyServe.
Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: Neil Brown <neilb@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
These are both called from two places close together. I could rearrange that
code so there is only one call site, but just removing the 'inline' is
probably best.
Signed-off-by: Neil Brown <neilb@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Change nfsd_sync_dir to return an error if ->sync fails, and pass that error
up through the stack. This involves a number of rearrangements of error
paths, and care to distinguish between Linux -errno numbers and NFSERR
numbers.
In the 'create' routines, we continue with the 'setattr' even if a previous
sync_dir failed.
This patch is quite different from Takashi's in a few ways, but there is still
a strong lineage.
Signed-off-by: Neil Brown <neilb@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Set the correct type and creator for symlinks.
Signed-off-by: Roman Zippel <zippel@linux-m68k.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
HFS+ also requires the correct creation date so recent version of OS X
recognize it as link.
Improve link handling:
- if something is wrong with the link, ignore the link attribute and treat
it as regular file (this also fixes a missing unlock during lookup).
- check for incorrect link counts during unlink.
Signed-off-by: Roman Zippel <zippel@linux-m68k.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Read the correct ctime from disk (it was written but never read for some
reason). Read also creation date, which is used in the next patch. (Problem
found by Olivier Castan <olivier.castan@certa.ssi.gouv.fr>)
Signed-off-by: Roman Zippel <zippel@linux-m68k.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Add support for HFSX, which allows for case-sensitive filenames.
Signed-off-by: Roman Zippel <zippel@linux-m68k.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Add the log level and a "hfs: " prefix to all kernel prints.
Signed-off-by: Roman Zippel <zippel@linux-m68k.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Add the log level and a "hfs: " prefix to all kernel prints. (HFS and HFS+
will use the same prefix, as they share some code and could be merged at some
point.)
Signed-off-by: Roman Zippel <zippel@linux-m68k.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
All standard system calls should be declared in include/linux/syscalls.h.
Add some of the new additions that were previously missed.
Signed-off-by: Arnd Bergmann <arndb@de.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Currently, a negative policy argument passed into the
'sys_sched_setscheduler()' system call, will return with success. However,
the manpage for 'sys_sched_setscheduler' says:
EINVAL The scheduling policy is not one of the recognized policies, or the
parameter p does not make sense for the policy.
Signed-off-by: Jason Baron <jbaron@redhat.com>
Acked-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
v9fs mmap support was originally removed from v9fs at Al Viro's request,
but recently there have been requests from folks who want readpage
functionality (primarily to enable execution of files mounted via 9P).
This patch adds readpage support (but not writepage which contained most of
the objectionable code). It passes fsx-linux (and other regressions) so it
should be relatively safe.
Signed-off-by: Eric Van Hensbergen <ericvh@gmail.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Correct a bit of whitespace problems while working here.
Signed-off-by: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Cc: Jeff Dike <jdike@addtoit.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
When the user specifies both a COW file and its backing file, if the previous
backing file is not found, currently UML tries again to use it and fails.
This can be corrected by changing same_backing_files() return value in that
case, so that the caller will try to change the COW file to point to the new
location, as already done in other cases.
Additionally, given the change in the meaning of the func, change its name,
invert its return value, so all values are inverted except when
stat(from_cow,&buf2) fails. And add some comments and two minor bugfixes -
remove a fd leak (return err rather than goto out) and a repeated check.
Tested well.
Signed-off-by: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Cc: Jeff Dike <jdike@addtoit.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
*) mark as "EXPERIMENTAL" various items that either aren't very stable or
that are actively crashing the setup of users which don't really need them
(i.e. HIGHMEM and 3-level pagetables on x86 - nobody needs either,
everybody reports "I'm using it and getting trouble").
*) move net/Kconfig near to the rest of network configurations, and
drivers/block/Kconfig near "Block layer" submenu.
*) it's useless and doesn't work well to force NETDEVICES on and to disable
the prompt like it's done. Better remove the attempt, and change that to a
simple "default y if UML".
*) drop the warning about "report problems about HPPFS" - it's redundant
anyway, as that's the usual procedure, and HPPFS users are especially
technical (i.e. they know reporting bugs is _good_).
Signed-off-by: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Cc: Jeff Dike <jdike@addtoit.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Ugly trick to help make malloc not sleeping - we can't do anything else. But
this is not yet optimal, since spinlock don't trigger in_atomic() when
preemption is disabled.
Also, even if ugly, this was already used in one place, and was even more
bogus. Fix it.
Signed-off-by: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Cc: Jeff Dike <jdike@addtoit.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
In a previous patch I shifted an allocation to being atomic.
In this patch, a better but more intrusive solution is implemented, i.e. hold
the lock only when really needing it, especially not over pipe operations, nor
over the culprit allocation.
Additionally, while at it, add a missing kfree in the failure path, and make
sure that if we fail in forking, write_sigio_pid is -1 and not, say, -ENOMEM.
And fix whitespace, at least for things I was touching anyway.
Signed-off-by: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Cc: Jeff Dike <jdike@addtoit.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
In this error path, when the interface has had a problem, we call dev_close(),
which is disallowed for two reasons:
*) takes again the UML internal spinlock, inside the ->stop method of this
device
*) can be called in process context only, while we're in interrupt context.
I've also thought that calling dev_close() may be a wrong policy to follow,
but it's not up to me to decide that.
However, we may end up with multiple dev_close() queued on the same device.
But the initial test for (dev->flags & IFF_UP) makes this harmless, though -
and dev_close() is supposed to care about races with itself. So there's no
harm in delaying the shutdown, IMHO.
Something to mark the interface as "going to shutdown" would be appreciated,
but dev_deactivate has the same problems as dev_close(), so we can't use it
either.
Signed-off-by: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Cc: Jeff Dike <jdike@addtoit.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Pre-clear transport-specific private structure before passing it down.
In fact, I just got a slab corruption and kernel panic on exit because kfree()
was called on a pointer which probably was never allocated, BUT hadn't been
set to NULL by the driver.
As the code is full of such errors, I've decided for now to go the safe way
(we're talking about drivers), and to do the simple thing. I'm also starting
to fix drivers, and already sent a patch for the daemon transport.
Signed-off-by: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Cc: Jeff Dike <jdike@addtoit.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Avoid uninitialized data in the daemon_data structure. I used this transport
before doing proper setup before-hand, and I got some very nice SLAB
corruption due to freeing crap pointers. So just make sure to clear
everything when appropriate.
Signed-off-by: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Cc: Jeff Dike <jdike@addtoit.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
I added this line to share this file with UML, but now it's no longer
shared so remove this useless leftover.
Signed-off-by: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Acked-by: Jeff Dike <jdike@addtoit.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Some fixes to make softints work in tt mode.
Signed-off-by: Jeff Dike <jdike@addtoit.com>
Cc: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Now that we are doing soft interrupts, there's no point in using sigsetjmp and
siglongjmp. Using setjmp and longjmp saves a sigprocmask on every jump.
Signed-off-by: Jeff Dike <jdike@addtoit.com>
Cc: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This patch implements soft interrupts. Interrupt enabling and disabling no
longer map to sigprocmask. Rather, a flag is set indicating whether
interrupts may be handled. If a signal comes in and interrupts are marked as
OK, then it is handled normally. If interrupts are marked as off, then the
signal handler simply returns after noting that a signal needs handling. When
interrupts are enabled later on, this pending signals flag is checked, and the
IRQ handlers are called at that point.
The point of this is to reduce the cost of local_irq_save et al, since they
are very much more common than the signals that they are enabling and
disabling. Soft interrupts produce a speed-up of ~25% on a kernel build.
Subtleties -
UML uses sigsetjmp/siglongjmp to switch contexts. sigsetjmp has been
wrapped in a save_flags-like macro which remembers the interrupt state at
setjmp time, and restores it when it is longjmp-ed back to.
The enable_signals function has to loop because the IRQ handler
disables interrupts before returning. enable_signals has to return with
signals enabled, and signals may come in between the disabling and the
return to enable_signals. So, it loops for as long as there are pending
signals, ensuring that signals are enabled when it finally returns, and
that there are no pending signals that need to be dealt with.
Signed-off-by: Jeff Dike <jdike@addtoit.com>
Cc: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Stop using global variables to hold the file descriptor and offset used to map
the skas0 stubs. Instead, calculate them using the page physical addresses.
Signed-off-by: Jeff Dike <jdike@addtoit.com>
Cc: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
The serial UML OS-abstraction layer patch (um/kernel/skas dir).
This moves all systemcalls from skas/process.c file under os-Linux dir and
join skas/process.c and skas/process_kern.c files.
Signed-off-by: Gennady Sharapov <gennady.v.sharapov@intel.com>
Signed-off-by: Jeff Dike <jdike@addtoit.com>
Cc: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
The serial UML OS-abstraction layer patch (um/kernel/skas dir).
This moves all systemcalls from skas/mem_user.c file under os-Linux dir and
join skas/mem_user.c and skas/mem.c files.
Signed-off-by: Gennady Sharapov <gennady.v.sharapov@intel.com>
Signed-off-by: Jeff Dike <jdike@addtoit.com>
Cc: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
The serial UML OS-abstraction layer patch (um/kernel dir).
This moves skas headers to arch/um/include.
Signed-off-by: Gennady Sharapov <Gennady.V.Sharapov@intel.com>
Signed-off-by: Jeff Dike <jdike@addtoit.com>
Cc: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Current implementation of boot_timer_handler isn't usable for s390. So I
changed its name to do_boot_timer_handler, taking (struct sigcontext *)sc as
argument. do_boot_timer_handler is called from new boot_timer_handler() in
arch/um/os-Linux/signal.c, which uses the same mechanisms as other signal
handler to find out sigcontext pointer.
Signed-off-by: Bodo Stroesser <bstroesser@fujitsu-siemens.com>
Signed-off-by: Jeff Dike <jdike@addtoit.com>
Cc: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
The serial UML OS-abstraction layer patch (um/kernel dir).
This moves all systemcalls from time.c file under os-Linux dir and joins
time.c and tine_kernel.c files
Signed-off-by: Gennady Sharapov <Gennady.V.Sharapov@intel.com>
Signed-off-by: Jeff Dike <jdike@addtoit.com>
Cc: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
The serial UML OS-abstraction layer patch (um/kernel dir).
This moves all systemcalls from user_util.c file under os-Linux dir
Signed-off-by: Gennady Sharapov <Gennady.V.Sharapov@intel.com>
Signed-off-by: Jeff Dike <jdike@addtoit.com>
Cc: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
s390 doesn't have a LDT. So MM_COPY_SEGMENTS will not be supported on s390.
The only user of MM_COPY_SEGMENTS is new_mm(), but that's no longer useful, as
arch/sys-i386/ldt.c defines init_new_ldt(), which is called immediately after
new_mm(). So we should copy host's LDT in init_new_ldt(), if /proc/mm is
available, to have this subarch specific call in subarch code.
Signed-off-by: Bodo Stroesser <bstroesser@fujitsu-siemens.com>
Signed-off-by: Jeff Dike <jdike@addtoit.com>
Cc: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Add implementations of the write* and __raw_write* functions. __raw_writel is
needed by lib/iocopy.c, which shouldn't be used in UML, but which is
unconditionally linked in anyway.
Signed-off-by: Jeff Dike <jdike@addtoit.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Move the interrupt check from slab_node into ___cache_alloc and adds an
"unlikely()" to avoid pipeline stalls on some architectures.
Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This patch fixes a regression in 2.6.14 against 2.6.13 that causes an
imbalance in memory allocation during bootup.
The slab allocator in 2.6.13 is not numa aware and simply calls
alloc_pages(). This means that memory policies may control the behavior of
alloc_pages(). During bootup the memory policy is set to MPOL_INTERLEAVE
resulting in the spreading out of allocations during bootup over all
available nodes. The slab allocator in 2.6.13 has only a single list of
slab pages. As a result the per cpu slab cache and the spinlock controlled
page lists may contain slab entries from off node memory. The slab
allocator in 2.6.13 makes no effort to discern the locality of an entry on
its lists.
The NUMA aware slab allocator in 2.6.14 controls locality of the slab pages
explicitly by calling alloc_pages_node(). The NUMA slab allocator manages
slab entries by having lists of available slab pages for each node. The
per cpu slab cache can only contain slab entries associated with the node
local to the processor. This guarantees that the default allocation mode
of the slab allocator always assigns local memory if available.
Setting MPOL_INTERLEAVE as a default policy during bootup has no effect
anymore. In 2.6.14 all node unspecific slab allocations are performed on
the boot processor. This means that most of key data structures are
allocated on one node. Most processors will have to refer to these
structures making the boot node a potential bottleneck. This may reduce
performance and cause unnecessary memory pressure on the boot node.
This patch implements NUMA policies in the slab layer. There is the need
of explicit application of NUMA memory policies by the slab allcator itself
since the NUMA slab allocator does no longer let the page_allocator control
locality.
The check for policies is made directly at the beginning of __cache_alloc
using current->mempolicy. The memory policy is already frequently checked
by the page allocator (alloc_page_vma() and alloc_page_current()). So it
is highly likely that the cacheline is present. For MPOL_INTERLEAVE
kmalloc() will spread out each request to one node after another so that an
equal distribution of allocations can be obtained during bootup.
It is not possible to push the policy check to lower layers of the NUMA
slab allocator since the per cpu caches are now only containing slab
entries from the current node. If the policy says that the local node is
not to be preferred or forbidden then there is no point in checking the
slab cache or local list of slab pages. The allocation better be directed
immediately to the lists containing slab entries for the allowed set of
nodes.
This way of applying policy also fixes another strange behavior in 2.6.13.
alloc_pages() is controlled by the memory allocation policy of the current
process. It could therefore be that one process is running with
MPOL_INTERLEAVE and would f.e. obtain a new page following that policy
since no slab entries are in the lists anymore. A page can typically be
used for multiple slab entries but lets say that the current process is
only using one. The other entries are then added to the slab lists. These
are now non local entries in the slab lists despite of the possible
availability of local pages that would provide faster access and increase
the performance of the application.
Another process without MPOL_INTERLEAVE may now run and expect a local slab
entry from kmalloc(). However, there are still these free slab entries
from the off node page obtained from the other process via MPOL_INTERLEAVE
in the cache. The process will then get an off node slab entry although
other slab entries may be available that are local to that process. This
means that the policy if one process may contaminate the locality of the
slab caches for other processes.
This patch in effect insures that a per process policy is followed for the
allocation of slab entries and that there cannot be a memory policy
influence from one process to another. A process with default policy will
always get a local slab entry if one is available. And the process using
memory policies will get its memory arranged as requested. Off-node slab
allocation will require the use of spinlocks and will make the use of per
cpu caches not possible. A process using memory policies to redirect
allocations offnode will have to cope with additional lock overhead in
addition to the latency added by the need to access a remote slab entry.
Changes V1->V2
- Remove #ifdef CONFIG_NUMA by moving forward declaration into
prior #ifdef CONFIG_NUMA section.
- Give the function determining the node number to use a saner
name.
Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
J. Bruce Fields