This call may have resulted to local_tlb_flush_range returning with
interrupts disabled resulting in excessive interrupt latency.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
The commit a923660d786a53e78834b19062f7af2535f7f8ad accidently
prevents TX49 from using CDEX. Use build_dst_pref() only if prefetch
for store was really available.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Provide a custom copy_user_highpage() to deal with aliasing issues on
MIPS. It uses kmap_coherent() to map an user page for kernel with same
color. Rewrite copy_to_user_page() and copy_from_user_page() with the
new interfaces to avoid extra cache flushing.
The main part of this patch was originally written by Ralf Baechle;
Atushi Nemoto did the the debugging.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This patch makes paging_init() use highend_pfn/highstart_pfn globals.
It removes the need of 'high' local which was needed only by HIGHMEM config.
More important perhaps, it fixes a bug when HIGHMEM is set but there's
actually no physical highmem (highend_pfn = 0)
Signed-off-by: Franck Bui-Huu <fbuihuu@gmail.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
On a CPU with aliases the IDE core needs to flush caches in the special
IDE variants of insw, insl etc. If IDE support is built as a module this
will only work if local_flush_data_cache_page happens is exported as a
module.
As per policy export local_flush_data_cache_page as GPL symbol only.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Pass struct dev pointer to dma_cache_sync()
dma_cache_sync() is ill-designed in that it does not have a struct device
pointer argument which makes proper support for systems that consist of a
mix of coherent and non-coherent DMA devices hard. Change dma_cache_sync
to take a struct device pointer as first argument and fix all its callers
to pass it.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Cc: James Bottomley <James.Bottomley@steeleye.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Greg KH <greg@kroah.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
dma_is_consistent() is ill-designed in that it does not have a struct
device pointer argument which makes proper support for systems that consist
of a mix of coherent and non-coherent DMA devices hard. Change
dma_is_consistent to take a struct device pointer as first argument and fix
the sole caller to pass it.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Cc: James Bottomley <James.Bottomley@steeleye.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Greg KH <greg@kroah.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Introduce pagefault_{disable,enable}() and use these where previously we did
manual preempt increments/decrements to make the pagefault handler do the
atomic thing.
Currently they still rely on the increased preempt count, but do not rely on
the disabled preemption, this might go away in the future.
(NOTE: the extra barrier() in pagefault_disable might fix some holes on
machines which have too many registers for their own good)
[heiko.carstens@de.ibm.com: s390 fix]
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Acked-by: Nick Piggin <npiggin@suse.de>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
r4k_flush_cache_all() and r4k_flush_cache_mm() case: these are noop if
the CPU did not have dc_aliases. It would mean we do not need to care
about icache here.
r4k_flush_cache_range case: if r4k_flush_cache_mm() did not need to
care about icache, it would be same for this function.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
It allows caller of this function to not care about CKSEG0/XKPHYS
address mixes. It's now automatically done by free_init_pages().
We can now safely remove hack needed by 64 bit kernels with
CONFIG_BUILD_ELF64=n in free_initmem().
Signed-off-by: Franck Bui-Huu <fbuihuu@gmail.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
__pa() was used by virt_to_page() and virt_addr_valid(). These
latter are used when kernel is initialised so __pa() is not
appropriate, we use virt_to_phys() instead.
Futhermore __pa() is going to take care of CKSEG0/XKPHYS
address mix for 64 bit kernels. This makes __pa() more complex
than virt_to_phys() and this extra work is not needed by
virt_to_page() and virt_addr_valid().
Eventually it consolidates virt_to_phys() prototype by making
its argument 'const'. this avoids some warnings that was due
to some virt_to_page() usages which pass const pointer.
Signed-off-by: Franck Bui-Huu <fbuihuu@gmail.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Removing flush_icache_page a while ago broke SB1 which was using an empty
flush_data_cache_page function. This glues things well enough so a more
efficient but also more intrusive solution can be found later.
Signed-Off-By: Thiemo Seufer <ths@networkno.de>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
This fixes the
start_kernel(): bug: interrupts were enabled early
messages.
Signed-off-by: Manish Lachwani <mlachwani@mvista.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
The current implementation uses a sequence of a cacheflush and a copy.
This is racy in case of a multithreaded debuggee and renders GDB
virtually unusable.
Aside this fixes a performance hog rendering access to /proc/cmdline very
slow and resulting in a enough cache stalls for the 34K AP/SP programming
model to make the bare metal code on the non-Linux VPE miss RT deadlines.
The main part of this patch was originally written by Ralf Baechle;
Atushi Nemoto did the the debugging.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
With the existing prototype the following code:
const void __iomem *io = ioremap();
x = readb(io);
iounmap(io);
did result in a warning.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* git://git.kernel.org/pub/scm/linux/kernel/git/bunk/trivial: (39 commits)
Add missing maintainer countries in CREDITS
Fix bytes <-> kilobytes typo in Kconfig for ramdisk
fix a typo in Documentation/pi-futex.txt
BUG_ON conversion for fs/xfs/
BUG_ON() conversion in fs/nfsd/
BUG_ON conversion for fs/reiserfs
BUG_ON cleanups in arch/i386
BUG_ON cleanup in drivers/net/tokenring/
BUG_ON cleanup for drivers/md/
kerneldoc-typo in led-class.c
debugfs: spelling fix
rcutorture: Fix incorrect description of default for nreaders parameter
parport: Remove space in function calls
Michal Wronski: update contact info
Spelling fix: "control" instead of "cotrol"
reboot parameter in Documentation/kernel-parameters.txt
Fix copy&waste bug in comment in scripts/kernel-doc
remove duplicate "until" from kernel/workqueue.c
ite_gpio fix tabbage
fix file specification in comments
...
Fixed trivial path conflicts due to removed files:
arch/mips/dec/boot/decstation.c, drivers/char/ite_gpio.c
The patch below corrects multiple occurances of "the the"
typos across several files, both in source comments and KConfig files.
There is no actual code changed, only text. Note this only affects the /arch
directory, and I believe I could find many more elsewhere. :)
Signed-off-by: Adrian Bunk <bunk@stusta.de>
Fix build error due to stacktrace API change. Now save_stack_trace()
tries to save all kernel context, including interrupts and exception.
Also some asm code are changed a bit so that we can detect the end of
current context easily.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
This is an updated version of Eric Biederman's is_init() patch.
(http://lkml.org/lkml/2006/2/6/280). It applies cleanly to 2.6.18-rc3 and
replaces a few more instances of ->pid == 1 with is_init().
Further, is_init() checks pid and thus removes dependency on Eric's other
patches for now.
Eric's original description:
There are a lot of places in the kernel where we test for init
because we give it special properties. Most significantly init
must not die. This results in code all over the kernel test
->pid == 1.
Introduce is_init to capture this case.
With multiple pid spaces for all of the cases affected we are
looking for only the first process on the system, not some other
process that has pid == 1.
Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
Signed-off-by: Sukadev Bhattiprolu <sukadev@us.ibm.com>
Cc: Dave Hansen <haveblue@us.ibm.com>
Cc: Serge Hallyn <serue@us.ibm.com>
Cc: Cedric Le Goater <clg@fr.ibm.com>
Cc: <lxc-devel@lists.sourceforge.net>
Acked-by: Paul Mackerras <paulus@samba.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
With more recent compilers inline doesn't necessarily means a function
will always be inlined. So leave that decission to the compiler and
make the function as __init.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
c-r4k.c and c-sb1.c use drop_mmu_context() to flush virtually tagged
I-caches, but this does not work for flushing other task's icache. This
is for example triggered by copy_to_user_page() called from ptrace(2).
Use indexed flush for such cases.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Fix array initialization in lots of arches
The number of zones may now be reduced from 4 to 2 for many arches. Fix the
array initialization for the zones array for all architectures so that it is
not initializing a fixed number of elements.
Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
With sparsemem, pfn should be checked by pfn_valid() before pfn_to_page().
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
With some memory model other than FLATMEM, the single node can
contains some holes so there might be many invalid pages. For
example, with two 256M memory and one 256M hole, some variables
(num_physpage, totalpages, nr_kernel_pages, nr_all_pages, etc.) will
indicate that there are 768MB on this system. This is not desired
because, for example, alloc_large_system_hash() allocates too many
entries.
Use free_area_init_node() with counted zholes_size[] instead of
free_area_init().
For num_physpages, use number of ram pages instead of max_low_pfn.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Move memory_present() in arch/mips/kernel/setup.c. When using sparsemem
extreme, this function does an allocate for bootmem. This would always
fail since init_bootmem hasn't been called yet.
Move memory_present after free_bootmem. This only marks actual memory
ranges as present instead of the entire address space.
Signed-off-by: Chad Reese <creese@caviumnetworks.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Save the Config.OD bit from being clobbered by coherency_setup(). This
bit, when set, fixes various errata in the early steppings of Au1x00
SOCs. Unfortunately, the bit was write-only on the most early of them.
In addition, also restore the bit after a wakeup from sleep.
Signed-off-by: Sergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
A proper fix would involve introducing the notion of shared caches but
at this stage of 2.6.17 that's going to be too intrusive and not needed
for current hardware; aside I think some discussion will be needed.
So for now on the affected SMP configurations which happen to suffer from
cache aliases we make use of the fact that a single cache will be shared
by all processors. This solves the deadlock issue and will improve
performance by getting rid of the smp_call_function overhead.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Nothing exciting; Linux just didn't know it yet so this is most adding
a value to a case statement.
Signed-off-by: Chris Dearman <chris@mips.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Fix the cache index value in tx49_blast_icache32_page_indexed().
This is a damage by de62893bc0 commit.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Just about every architecture defines some macros to do operations on pfns.
They're all virtually identical. This patch consolidates all of them.
One minor glitch is that at least i386 uses them in a very skeletal header
file. To keep away from #include dependency hell, I stuck the new
definitions in a new, isolated header.
Of all of the implementations, sh64 is the only one that varied by a bit.
It used some masks to ensure that any sign-extension got ripped away before
the arithmetic is done. This has been posted to that sh64 maintainers and
the development list.
Compiles on x86, x86_64, ia64 and ppc64.
Signed-off-by: Dave Hansen <haveblue@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
set_page_count usage outside mm/ is limited to setting the refcount to 1.
Remove set_page_count from outside mm/, and replace those users with
init_page_count() and set_page_refcounted().
This allows more debug checking, and tighter control on how code is allowed
to play around with page->_count.
Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Have an explicit mm call to split higher order pages into individual pages.
Should help to avoid bugs and be more explicit about the code's intention.
Signed-off-by: Nick Piggin <npiggin@suse.de>
Cc: Russell King <rmk@arm.linux.org.uk>
Cc: David Howells <dhowells@redhat.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Chris Zankel <chris@zankel.net>
Signed-off-by: Yoichi Yuasa <yoichi_yuasa@tripeaks.co.jp>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
The TX49XX has the prefetch instruction. It supports only Pref_Load
(hint 0). Actually changes in this patch except for Kconfig are not
have any effects, I added these changes to prevent misuse of unsupported
hints.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Basically identical to c-r4k.c, so maintaining one is really enough.
Signed-off-by: Thiemo Seufer <ths@networkno.de>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
This option is no longer usable with supported compilers. It will be
replaced by usage of -msym32 in a separate patch.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
If dcache_size != icache_size or dcache_size != scache_size, or
set-associative cache, icache/scache does not flushed properly. Make
blast_?cache_page_indexed() masks its index value correctly. Also,
use physical address for physically indexed pcache/scache.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
When a CPU has no scache, the scache flushing functions currently
aren't getting initialized and the NULL pointer is eventually called
as a function. Initialize the scache flushing functions as a noop
when there's no scache.
Initial patch by me and most of the debugging done by Martin Michlmayr.
Signed-off-by: Martin Michlmayr <tbm@cyrius.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Add blast_xxx_range(), protected_blast_xxx_range() etc. for common
use. They are built by __BUILD_BLAST_CACHE_RANGE().
Use protected_cache_op() macro for various protected_ routines.
Output code should be logically same.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
I'm pretty sure that the CKSEG0 bits are wrong, but I did need to
cover that region - because the SB-1 kernel links at 0xffffffff80100000
or so, disassembly and printing static variables don't work unless the
debugger can read that region.
Signed-off-by: Daniel Jacobowitz <dan@codesourcery.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Page count should be initialized to 1 on each of the MIPS empty zero pages,
to avoid a bad_page warning whenever one of them is freed from all mappings.
Signed-off-by: Hugh Dickins <hugh@veritas.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
First step in pushing down the page_table_lock. init_mm.page_table_lock has
been used throughout the architectures (usually for ioremap): not to serialize
kernel address space allocation (that's usually vmlist_lock), but because
pud_alloc,pmd_alloc,pte_alloc_kernel expect caller holds it.
Reverse that: don't lock or unlock init_mm.page_table_lock in any of the
architectures; instead rely on pud_alloc,pmd_alloc,pte_alloc_kernel to take
and drop it when allocating a new one, to check lest a racing task already
did. Similarly no page_table_lock in vmalloc's map_vm_area.
Some temporary ugliness in __pud_alloc and __pmd_alloc: since they also handle
user mms, which are converted only by a later patch, for now they have to lock
differently according to whether or not it's init_mm.
If sources get muddled, there's a danger that an arch source taking
init_mm.page_table_lock will be mixed with common source also taking it (or
neither take it). So break the rules and make another change, which should
break the build for such a mismatch: remove the redundant mm arg from
pte_alloc_kernel (ppc64 scrapped its distinct ioremap_mm in 2.6.13).
Exceptions: arm26 used pte_alloc_kernel on user mm, now pte_alloc_map; ia64
used pte_alloc_map on init_mm, now pte_alloc_kernel; parisc had bad args to
pmd_alloc and pte_alloc_kernel in unused USE_HPPA_IOREMAP code; ppc64
map_io_page forgot to unlock on failure; ppc mmu_mapin_ram and ppc64 im_free
took page_table_lock for no good reason.
Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Cacheflush(0, 0, 0) was crashing the system. This is because
flush_icache_range(start, end) tries to flushing whole address space
(0 - ~0UL) if both start and end are zero.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
This may have made sense on a paranoid day with pass 1 BCM1250 processors
that were throwing cache error exception left and right for no good
reason. On modern silicion that hardly makes sense and the code had
gotten just an obscurity ...
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
tx39_flush_cache_range() does nothing if !cpu_has_dc_aliases. It should
flush d-cache and invalidate i-cache since the TX39(H2) has separate I/D
cache.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Ralf Baechle