hollalinux
/
linux-sg2042
mirror of https://github.com/sophgo/linux-riscv.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'akpm' (fixes from Andrew Morton) 

Merge patch-bomb from Andrew Morton:
 - part of OCFS2 (review is laggy again)
 - procfs
 - slab
 - all of MM
 - zram, zbud
 - various other random things: arch, filesystems.

* emailed patches from Andrew Morton <akpm@linux-foundation.org>: (164 commits)
  nosave: consolidate __nosave_{begin,end} in <asm/sections.h>
  include/linux/screen_info.h: remove unused ORIG_* macros
  kernel/sys.c: compat sysinfo syscall: fix undefined behavior
  kernel/sys.c: whitespace fixes
  acct: eliminate compile warning
  kernel/async.c: switch to pr_foo()
  include/linux/blkdev.h: use NULL instead of zero
  include/linux/kernel.h: deduplicate code implementing clamp* macros
  include/linux/kernel.h: rewrite min3, max3 and clamp using min and max
  alpha: use Kbuild logic to include <asm-generic/sections.h>
  frv: remove deprecated IRQF_DISABLED
  frv: remove unused cpuinfo_frv and friends to fix future build error
  zbud: avoid accessing last unused freelist
  zsmalloc: simplify init_zspage free obj linking
  mm/zsmalloc.c: correct comment for fullness group computation
  zram: use notify_free to account all free notifications
  zram: report maximum used memory
  zram: zram memory size limitation
  zsmalloc: change return value unit of zs_get_total_size_bytes
  zsmalloc: move pages_allocated to zs_pool
  ...
This commit is contained in:
Linus Torvalds 2014-10-09 22:26:14 -04:00
parent b528392669 7f8998c7ae
commit 0cf744bc7a
177 changed files with 4109 additions and 2872 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
Documentation/ABI/stable/sysfs-devices-node
View File

@ -85,14 +85,6 @@ Description:
will be compacted. When it completes, memory will be freed
into blocks which have as many contiguous pages as possible
What: /sys/devices/system/node/nodeX/scan_unevictable_pages
Date: October 2008
Contact: Lee Schermerhorn <lee.schermerhorn@hp.com>
Description:
When set, it triggers scanning the node's unevictable lists
and move any pages that have become evictable onto the respective
zone's inactive list. See mm/vmscan.c
What: /sys/devices/system/node/nodeX/hugepages/hugepages-<size>/
Date: December 2009
Contact: Lee Schermerhorn <lee.schermerhorn@hp.com>

32
Documentation/ABI/testing/sysfs-block-zram
View File

@ -77,11 +77,14 @@ What: /sys/block/zram<id>/notify_free
Date: August 2010
Contact: Nitin Gupta <ngupta@vflare.org>
Description:
The notify_free file is read-only and specifies the number of
swap slot free notifications received by this device. These
notifications are sent to a swap block device when a swap slot
is freed. This statistic is applicable only when this disk is
being used as a swap disk.
The notify_free file is read-only. Depending on device usage
scenario it may account a) the number of pages freed because
of swap slot free notifications or b) the number of pages freed
because of REQ_DISCARD requests sent by bio. The former ones
are sent to a swap block device when a swap slot is freed, which
implies that this disk is being used as a swap disk. The latter
ones are sent by filesystem mounted with discard option,
whenever some data blocks are getting discarded.
What: /sys/block/zram<id>/zero_pages
Date: August 2010
@ -119,3 +122,22 @@ Description:
efficiency can be calculated using compr_data_size and this
statistic.
Unit: bytes
What: /sys/block/zram<id>/mem_used_max
Date: August 2014
Contact: Minchan Kim <minchan@kernel.org>
Description:
The mem_used_max file is read/write and specifies the amount
of maximum memory zram have consumed to store compressed data.
For resetting the value, you should write "0". Otherwise,
you could see -EINVAL.
Unit: bytes
What: /sys/block/zram<id>/mem_limit
Date: August 2014
Contact: Minchan Kim <minchan@kernel.org>
Description:
The mem_limit file is read/write and specifies the maximum
amount of memory ZRAM can use to store the compressed data. The
limit could be changed in run time and "0" means disable the
limit. No limit is the initial state. Unit: bytes

8
Documentation/ABI/testing/sysfs-devices-memory
View File

@ -61,6 +61,14 @@ Users: hotplug memory remove tools
http://www.ibm.com/developerworks/wikis/display/LinuxP/powerpc-utils
What: /sys/devices/system/memory/memoryX/valid_zones
Date: July 2014
Contact: Zhang Zhen <zhenzhang.zhang@huawei.com>
Description:
The file /sys/devices/system/memory/memoryX/valid_zones is
read-only and is designed to show which zone this memory
block can be onlined to.
What: /sys/devices/system/memoryX/nodeY
Date: October 2009
Contact: Linux Memory Management list <linux-mm@kvack.org>

25
Documentation/blockdev/zram.txt
View File

@ -74,14 +74,30 @@ There is little point creating a zram of greater than twice the size of memory
since we expect a 2:1 compression ratio. Note that zram uses about 0.1% of the
size of the disk when not in use so a huge zram is wasteful.
5) Activate:
5) Set memory limit: Optional
Set memory limit by writing the value to sysfs node 'mem_limit'.
The value can be either in bytes or you can use mem suffixes.
In addition, you could change the value in runtime.
Examples:
# limit /dev/zram0 with 50MB memory
echo $((50*1024*1024)) > /sys/block/zram0/mem_limit
# Using mem suffixes
echo 256K > /sys/block/zram0/mem_limit
echo 512M > /sys/block/zram0/mem_limit
echo 1G > /sys/block/zram0/mem_limit
# To disable memory limit
echo 0 > /sys/block/zram0/mem_limit
6) Activate:
mkswap /dev/zram0
swapon /dev/zram0
mkfs.ext4 /dev/zram1
mount /dev/zram1 /tmp
6) Stats:
7) Stats:
Per-device statistics are exported as various nodes under
/sys/block/zram<id>/
disksize
@ -95,12 +111,13 @@ size of the disk when not in use so a huge zram is wasteful.
orig_data_size
compr_data_size
mem_used_total
mem_used_max
7) Deactivate:
8) Deactivate:
swapoff /dev/zram0
umount /dev/zram1
8) Reset:
9) Reset:
Write any positive value to 'reset' sysfs node
echo 1 > /sys/block/zram0/reset
echo 1 > /sys/block/zram1/reset

17
Documentation/kernel-parameters.txt
View File

@ -656,7 +656,8 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
Sets the size of kernel global memory area for
contiguous memory allocations and optionally the
placement constraint by the physical address range of
memory allocations. For more information, see
memory allocations. A value of 0 disables CMA
altogether. For more information, see
include/linux/dma-contiguous.h
cmo_free_hint= [PPC] Format: { yes | no }
@ -3158,6 +3159,13 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
slram= [HW,MTD]
slab_nomerge [MM]
Disable merging of slabs with similar size. May be
necessary if there is some reason to distinguish
allocs to different slabs. Debug options disable
merging on their own.
For more information see Documentation/vm/slub.txt.
slab_max_order= [MM, SLAB]
Determines the maximum allowed order for slabs.
A high setting may cause OOMs due to memory
@ -3193,11 +3201,8 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
For more information see Documentation/vm/slub.txt.
slub_nomerge [MM, SLUB]
Disable merging of slabs with similar size. May be
necessary if there is some reason to distinguish
allocs to different slabs. Debug options disable
merging on their own.
For more information see Documentation/vm/slub.txt.
Same with slab_nomerge. This is supported for legacy.
See slab_nomerge for more information.
smart2= [HW]
Format: <io1>[,<io2>[,...,<io8>]]

11
Documentation/memory-hotplug.txt
View File

@ -155,6 +155,7 @@ Under each memory block, you can see 4 files:
/sys/devices/system/memory/memoryXXX/phys_device
/sys/devices/system/memory/memoryXXX/state
/sys/devices/system/memory/memoryXXX/removable
/sys/devices/system/memory/memoryXXX/valid_zones
'phys_index' : read-only and contains memory block id, same as XXX.
'state' : read-write
@ -170,6 +171,15 @@ Under each memory block, you can see 4 files:
block is removable and a value of 0 indicates that
it is not removable. A memory block is removable only if
every section in the block is removable.
'valid_zones' : read-only: designed to show which zones this memory block
can be onlined to.
The first column shows it's default zone.
"memory6/valid_zones: Normal Movable" shows this memoryblock
can be onlined to ZONE_NORMAL by default and to ZONE_MOVABLE
by online_movable.
"memory7/valid_zones: Movable Normal" shows this memoryblock
can be onlined to ZONE_MOVABLE by default and to ZONE_NORMAL
by online_kernel.
NOTE:
These directories/files appear after physical memory hotplug phase.
@ -408,7 +418,6 @@ node if necessary.
- allowing memory hot-add to ZONE_MOVABLE. maybe we need some switch like
sysctl or new control file.
- showing memory block and physical device relationship.
- showing memory block is under ZONE_MOVABLE or not
- test and make it better memory offlining.
- support HugeTLB page migration and offlining.
- memmap removing at memory offline.

1
arch/alpha/include/asm/Kbuild
View File

@ -8,4 +8,5 @@ generic-y += irq_work.h
generic-y += mcs_spinlock.h
generic-y += preempt.h
generic-y += scatterlist.h
generic-y += sections.h
generic-y += trace_clock.h

7
arch/alpha/include/asm/sections.h
View File

@ -1,7 +0,0 @@
#ifndef _ALPHA_SECTIONS_H
#define _ALPHA_SECTIONS_H
/* nothing to see, move along */
#include <asm-generic/sections.h>
#endif

6
arch/arm/Kconfig
View File

@ -14,6 +14,7 @@ config ARM
select CLONE_BACKWARDS
select CPU_PM if (SUSPEND || CPU_IDLE)
select DCACHE_WORD_ACCESS if HAVE_EFFICIENT_UNALIGNED_ACCESS
select GENERIC_ALLOCATOR
select GENERIC_ATOMIC64 if (CPU_V7M || CPU_V6 || !CPU_32v6K || !AEABI)
select GENERIC_CLOCKEVENTS_BROADCAST if SMP
select GENERIC_IDLE_POLL_SETUP
@ -61,6 +62,7 @@ config ARM
select HAVE_PERF_EVENTS
select HAVE_PERF_REGS
select HAVE_PERF_USER_STACK_DUMP
select HAVE_RCU_TABLE_FREE if (SMP && ARM_LPAE)
select HAVE_REGS_AND_STACK_ACCESS_API
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_UID16
@ -1659,6 +1661,10 @@ config ARCH_SELECT_MEMORY_MODEL
config HAVE_ARCH_PFN_VALID
def_bool ARCH_HAS_HOLES_MEMORYMODEL || !SPARSEMEM
config HAVE_GENERIC_RCU_GUP
def_bool y
depends on ARM_LPAE
config HIGHMEM
bool "High Memory Support"
depends on MMU

2
arch/arm/include/asm/pgtable-2level.h
View File

@ -182,6 +182,8 @@ static inline pmd_t *pmd_offset(pud_t *pud, unsigned long addr)
#define pmd_addr_end(addr,end) (end)
#define set_pte_ext(ptep,pte,ext) cpu_set_pte_ext(ptep,pte,ext)
#define pte_special(pte) (0)
static inline pte_t pte_mkspecial(pte_t pte) { return pte; }
/*
* We don't have huge page support for short descriptors, for the moment

15
arch/arm/include/asm/pgtable-3level.h
View File

@ -213,10 +213,19 @@ static inline pmd_t *pmd_offset(pud_t *pud, unsigned long addr)
#define pmd_isclear(pmd, val) (!(pmd_val(pmd) & (val)))
#define pmd_young(pmd) (pmd_isset((pmd), PMD_SECT_AF))
#define pte_special(pte) (pte_isset((pte), L_PTE_SPECIAL))
static inline pte_t pte_mkspecial(pte_t pte)
{
pte_val(pte) |= L_PTE_SPECIAL;
return pte;
}
#define __HAVE_ARCH_PTE_SPECIAL
#define __HAVE_ARCH_PMD_WRITE
#define pmd_write(pmd) (pmd_isclear((pmd), L_PMD_SECT_RDONLY))
#define pmd_dirty(pmd) (pmd_isset((pmd), L_PMD_SECT_DIRTY))
#define pud_page(pud) pmd_page(__pmd(pud_val(pud)))
#define pud_write(pud) pmd_write(__pmd(pud_val(pud)))
#define pmd_hugewillfault(pmd) (!pmd_young(pmd) || !pmd_write(pmd))
#define pmd_thp_or_huge(pmd) (pmd_huge(pmd) || pmd_trans_huge(pmd))
@ -224,6 +233,12 @@ static inline pmd_t *pmd_offset(pud_t *pud, unsigned long addr)
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
#define pmd_trans_huge(pmd) (pmd_val(pmd) && !pmd_table(pmd))
#define pmd_trans_splitting(pmd) (pmd_isset((pmd), L_PMD_SECT_SPLITTING))
#ifdef CONFIG_HAVE_RCU_TABLE_FREE
#define __HAVE_ARCH_PMDP_SPLITTING_FLUSH
void pmdp_splitting_flush(struct vm_area_struct *vma, unsigned long address,
pmd_t *pmdp);
#endif
#endif
#define PMD_BIT_FUNC(fn,op) \

6
arch/arm/include/asm/pgtable.h
View File

@ -226,7 +226,6 @@ static inline pte_t *pmd_page_vaddr(pmd_t pmd)
#define pte_dirty(pte) (pte_isset((pte), L_PTE_DIRTY))
#define pte_young(pte) (pte_isset((pte), L_PTE_YOUNG))
#define pte_exec(pte) (pte_isclear((pte), L_PTE_XN))
#define pte_special(pte) (0)
#define pte_valid_user(pte) \
(pte_valid(pte) && pte_isset((pte), L_PTE_USER) && pte_young(pte))
@ -245,7 +244,8 @@ static inline void set_pte_at(struct mm_struct *mm, unsigned long addr,
unsigned long ext = 0;
if (addr < TASK_SIZE && pte_valid_user(pteval)) {
__sync_icache_dcache(pteval);
if (!pte_special(pteval))
__sync_icache_dcache(pteval);
ext |= PTE_EXT_NG;
}
@ -264,8 +264,6 @@ PTE_BIT_FUNC(mkyoung, |= L_PTE_YOUNG);
PTE_BIT_FUNC(mkexec, &= ~L_PTE_XN);
PTE_BIT_FUNC(mknexec, |= L_PTE_XN);
static inline pte_t pte_mkspecial(pte_t pte) { return pte; }
static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
{
const pteval_t mask = L_PTE_XN | L_PTE_RDONLY | L_PTE_USER |

38
arch/arm/include/asm/tlb.h
View File

@ -35,12 +35,39 @@
#define MMU_GATHER_BUNDLE 8
#ifdef CONFIG_HAVE_RCU_TABLE_FREE
static inline void __tlb_remove_table(void *_table)
{
free_page_and_swap_cache((struct page *)_table);
}
struct mmu_table_batch {
struct rcu_head rcu;
unsigned int nr;
void *tables[0];
};
#define MAX_TABLE_BATCH \
((PAGE_SIZE - sizeof(struct mmu_table_batch)) / sizeof(void *))
extern void tlb_table_flush(struct mmu_gather *tlb);
extern void tlb_remove_table(struct mmu_gather *tlb, void *table);
#define tlb_remove_entry(tlb, entry) tlb_remove_table(tlb, entry)
#else
#define tlb_remove_entry(tlb, entry) tlb_remove_page(tlb, entry)
#endif /* CONFIG_HAVE_RCU_TABLE_FREE */
/*
* TLB handling. This allows us to remove pages from the page
* tables, and efficiently handle the TLB issues.
*/
struct mmu_gather {
struct mm_struct *mm;
#ifdef CONFIG_HAVE_RCU_TABLE_FREE
struct mmu_table_batch *batch;
unsigned int need_flush;
#endif
unsigned int fullmm;
struct vm_area_struct *vma;
unsigned long start, end;
@ -101,6 +128,9 @@ static inline void __tlb_alloc_page(struct mmu_gather *tlb)
static inline void tlb_flush_mmu_tlbonly(struct mmu_gather *tlb)
{
tlb_flush(tlb);
#ifdef CONFIG_HAVE_RCU_TABLE_FREE
tlb_table_flush(tlb);
#endif
}
static inline void tlb_flush_mmu_free(struct mmu_gather *tlb)
@ -129,6 +159,10 @@ tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm, unsigned long start
tlb->pages = tlb->local;
tlb->nr = 0;
__tlb_alloc_page(tlb);
#ifdef CONFIG_HAVE_RCU_TABLE_FREE
tlb->batch = NULL;
#endif
}
static inline void
@ -205,7 +239,7 @@ static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte,
tlb_add_flush(tlb, addr + SZ_1M);
#endif
tlb_remove_page(tlb, pte);
tlb_remove_entry(tlb, pte);
}
static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmdp,
@ -213,7 +247,7 @@ static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmdp,
{
#ifdef CONFIG_ARM_LPAE
tlb_add_flush(tlb, addr);
tlb_remove_page(tlb, virt_to_page(pmdp));
tlb_remove_entry(tlb, virt_to_page(pmdp));
#endif
}

3
arch/arm/kernel/hibernate.c
View File

@ -21,8 +21,7 @@
#include <asm/idmap.h>
#include <asm/suspend.h>
#include <asm/memory.h>
extern const void __nosave_begin, __nosave_end;
#include <asm/sections.h>
int pfn_is_nosave(unsigned long pfn)
{

204
arch/arm/mm/dma-mapping.c
View File

@ -12,6 +12,7 @@
#include <linux/bootmem.h>
#include <linux/module.h>
#include <linux/mm.h>
#include <linux/genalloc.h>
#include <linux/gfp.h>
#include <linux/errno.h>
#include <linux/list.h>
@ -298,57 +299,29 @@ static void *
__dma_alloc_remap(struct page *page, size_t size, gfp_t gfp, pgprot_t prot,
const void *caller)
{
struct vm_struct *area;
unsigned long addr;
/*
* DMA allocation can be mapped to user space, so lets
* set VM_USERMAP flags too.
*/
area = get_vm_area_caller(size, VM_ARM_DMA_CONSISTENT | VM_USERMAP,
caller);
if (!area)
return NULL;
addr = (unsigned long)area->addr;
area->phys_addr = __pfn_to_phys(page_to_pfn(page));
if (ioremap_page_range(addr, addr + size, area->phys_addr, prot)) {
vunmap((void *)addr);
return NULL;
}
return (void *)addr;
return dma_common_contiguous_remap(page, size,
VM_ARM_DMA_CONSISTENT | VM_USERMAP,
prot, caller);
}
static void __dma_free_remap(void *cpu_addr, size_t size)
{
unsigned int flags = VM_ARM_DMA_CONSISTENT | VM_USERMAP;
struct vm_struct *area = find_vm_area(cpu_addr);
if (!area || (area->flags & flags) != flags) {
WARN(1, "trying to free invalid coherent area: %p\n", cpu_addr);
return;
}
unmap_kernel_range((unsigned long)cpu_addr, size);
vunmap(cpu_addr);
dma_common_free_remap(cpu_addr, size,
VM_ARM_DMA_CONSISTENT | VM_USERMAP);
}
#define DEFAULT_DMA_COHERENT_POOL_SIZE SZ_256K
static struct gen_pool *atomic_pool;
struct dma_pool {
size_t size;
spinlock_t lock;
unsigned long *bitmap;
unsigned long nr_pages;
void *vaddr;
struct page **pages;
};
static struct dma_pool atomic_pool = {
.size = DEFAULT_DMA_COHERENT_POOL_SIZE,
};
static size_t atomic_pool_size = DEFAULT_DMA_COHERENT_POOL_SIZE;
static int __init early_coherent_pool(char *p)
{
atomic_pool.size = memparse(p, &p);
atomic_pool_size = memparse(p, &p);
return 0;
}
early_param("coherent_pool", early_coherent_pool);
@ -358,14 +331,14 @@ void __init init_dma_coherent_pool_size(unsigned long size)
/*
* Catch any attempt to set the pool size too late.
*/
BUG_ON(atomic_pool.vaddr);
BUG_ON(atomic_pool);
/*
* Set architecture specific coherent pool size only if
* it has not been changed by kernel command line parameter.
*/
if (atomic_pool.size == DEFAULT_DMA_COHERENT_POOL_SIZE)
atomic_pool.size = size;
if (atomic_pool_size == DEFAULT_DMA_COHERENT_POOL_SIZE)
atomic_pool_size = size;
}
/*
@ -373,52 +346,44 @@ void __init init_dma_coherent_pool_size(unsigned long size)
*/
static int __init atomic_pool_init(void)
{
struct dma_pool *pool = &atomic_pool;
pgprot_t prot = pgprot_dmacoherent(PAGE_KERNEL);
gfp_t gfp = GFP_KERNEL | GFP_DMA;
unsigned long nr_pages = pool->size >> PAGE_SHIFT;
unsigned long *bitmap;
struct page *page;
struct page **pages;
void *ptr;
int bitmap_size = BITS_TO_LONGS(nr_pages) * sizeof(long);
bitmap = kzalloc(bitmap_size, GFP_KERNEL);
if (!bitmap)
goto no_bitmap;
pages = kzalloc(nr_pages * sizeof(struct page *), GFP_KERNEL);
if (!pages)
goto no_pages;
atomic_pool = gen_pool_create(PAGE_SHIFT, -1);
if (!atomic_pool)
goto out;
if (dev_get_cma_area(NULL))
ptr = __alloc_from_contiguous(NULL, pool->size, prot, &page,
atomic_pool_init);
ptr = __alloc_from_contiguous(NULL, atomic_pool_size, prot,
&page, atomic_pool_init);
else
ptr = __alloc_remap_buffer(NULL, pool->size, gfp, prot, &page,
atomic_pool_init);
ptr = __alloc_remap_buffer(NULL, atomic_pool_size, gfp, prot,
&page, atomic_pool_init);
if (ptr) {
int i;
int ret;
for (i = 0; i < nr_pages; i++)
pages[i] = page + i;
ret = gen_pool_add_virt(atomic_pool, (unsigned long)ptr,
page_to_phys(page),
atomic_pool_size, -1);
if (ret)
goto destroy_genpool;
spin_lock_init(&pool->lock);
pool->vaddr = ptr;
pool->pages = pages;
pool->bitmap = bitmap;
pool->nr_pages = nr_pages;
pr_info("DMA: preallocated %u KiB pool for atomic coherent allocations\n",
(unsigned)pool->size / 1024);
gen_pool_set_algo(atomic_pool,
gen_pool_first_fit_order_align,
(void *)PAGE_SHIFT);
pr_info("DMA: preallocated %zd KiB pool for atomic coherent allocations\n",
atomic_pool_size / 1024);
return 0;
}
kfree(pages);
no_pages:
kfree(bitmap);
no_bitmap:
pr_err("DMA: failed to allocate %u KiB pool for atomic coherent allocation\n",
(unsigned)pool->size / 1024);
destroy_genpool:
gen_pool_destroy(atomic_pool);
atomic_pool = NULL;
out:
pr_err("DMA: failed to allocate %zx KiB pool for atomic coherent allocation\n",
atomic_pool_size / 1024);
return -ENOMEM;
}
/*
@ -522,76 +487,36 @@ static void *__alloc_remap_buffer(struct device *dev, size_t size, gfp_t gfp,
static void *__alloc_from_pool(size_t size, struct page **ret_page)
{
struct dma_pool *pool = &atomic_pool;
unsigned int count = PAGE_ALIGN(size) >> PAGE_SHIFT;
unsigned int pageno;
unsigned long flags;
unsigned long val;
void *ptr = NULL;
unsigned long align_mask;
if (!pool->vaddr) {
if (!atomic_pool) {
WARN(1, "coherent pool not initialised!\n");
return NULL;
}
/*
* Align the region allocation - allocations from pool are rather
* small, so align them to their order in pages, minimum is a page
* size. This helps reduce fragmentation of the DMA space.
*/
align_mask = (1 << get_order(size)) - 1;
val = gen_pool_alloc(atomic_pool, size);
if (val) {
phys_addr_t phys = gen_pool_virt_to_phys(atomic_pool, val);
spin_lock_irqsave(&pool->lock, flags);
pageno = bitmap_find_next_zero_area(pool->bitmap, pool->nr_pages,
0, count, align_mask);
if (pageno < pool->nr_pages) {
bitmap_set(pool->bitmap, pageno, count);
ptr = pool->vaddr + PAGE_SIZE * pageno;
*ret_page = pool->pages[pageno];
} else {
pr_err_once("ERROR: %u KiB atomic DMA coherent pool is too small!\n"
"Please increase it with coherent_pool= kernel parameter!\n",
(unsigned)pool->size / 1024);
*ret_page = phys_to_page(phys);
ptr = (void *)val;
}
spin_unlock_irqrestore(&pool->lock, flags);
return ptr;
}
static bool __in_atomic_pool(void *start, size_t size)
{
struct dma_pool *pool = &atomic_pool;
void *end = start + size;
void *pool_start = pool->vaddr;
void *pool_end = pool->vaddr + pool->size;
if (start < pool_start || start >= pool_end)
return false;
if (end <= pool_end)
return true;
WARN(1, "Wrong coherent size(%p-%p) from atomic pool(%p-%p)\n",
start, end - 1, pool_start, pool_end - 1);
return false;
return addr_in_gen_pool(atomic_pool, (unsigned long)start, size);
}
static int __free_from_pool(void *start, size_t size)
{
struct dma_pool *pool = &atomic_pool;
unsigned long pageno, count;
unsigned long flags;
if (!__in_atomic_pool(start, size))
return 0;
pageno = (start - pool->vaddr) >> PAGE_SHIFT;
count = size >> PAGE_SHIFT;
spin_lock_irqsave(&pool->lock, flags);
bitmap_clear(pool->bitmap, pageno, count);
spin_unlock_irqrestore(&pool->lock, flags);
gen_pool_free(atomic_pool, (unsigned long)start, size);
return 1;
}
@ -1271,29 +1196,8 @@ static void *
__iommu_alloc_remap(struct page **pages, size_t size, gfp_t gfp, pgprot_t prot,
const void *caller)
{
unsigned int i, nr_pages = PAGE_ALIGN(size) >> PAGE_SHIFT;
struct vm_struct *area;
unsigned long p;
area = get_vm_area_caller(size, VM_ARM_DMA_CONSISTENT | VM_USERMAP,
caller);
if (!area)
return NULL;
area->pages = pages;
area->nr_pages = nr_pages;
p = (unsigned long)area->addr;
for (i = 0; i < nr_pages; i++) {
phys_addr_t phys = __pfn_to_phys(page_to_pfn(pages[i]));
if (ioremap_page_range(p, p + PAGE_SIZE, phys, prot))
goto err;
p += PAGE_SIZE;
}
return area->addr;
err:
unmap_kernel_range((unsigned long)area->addr, size);
vunmap(area->addr);
return dma_common_pages_remap(pages, size,
VM_ARM_DMA_CONSISTENT | VM_USERMAP, prot, caller);
return NULL;
}
@ -1355,11 +1259,13 @@ static int __iommu_remove_mapping(struct device *dev, dma_addr_t iova, size_t si
static struct page **__atomic_get_pages(void *addr)
{
struct dma_pool *pool = &atomic_pool;
struct page **pages = pool->pages;
int offs = (addr - pool->vaddr) >> PAGE_SHIFT;
struct page *page;
phys_addr_t phys;
return pages + offs;
phys = gen_pool_virt_to_phys(atomic_pool, (unsigned long)addr);
page = phys_to_page(phys);
return (struct page **)page;
}
static struct page **__iommu_get_pages(void *cpu_addr, struct dma_attrs *attrs)
@ -1501,8 +1407,8 @@ void arm_iommu_free_attrs(struct device *dev, size_t size, void *cpu_addr,
}
if (!dma_get_attr(DMA_ATTR_NO_KERNEL_MAPPING, attrs)) {
unmap_kernel_range((unsigned long)cpu_addr, size);
vunmap(cpu_addr);
dma_common_free_remap(cpu_addr, size,
VM_ARM_DMA_CONSISTENT | VM_USERMAP);
}
__iommu_remove_mapping(dev, handle, size);

15
arch/arm/mm/flush.c
View File

@ -400,3 +400,18 @@ void __flush_anon_page(struct vm_area_struct *vma, struct page *page, unsigned l
*/
__cpuc_flush_dcache_area(page_address(page), PAGE_SIZE);
}
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
#ifdef CONFIG_HAVE_RCU_TABLE_FREE
void pmdp_splitting_flush(struct vm_area_struct *vma, unsigned long address,
pmd_t *pmdp)
{
pmd_t pmd = pmd_mksplitting(*pmdp);
VM_BUG_ON(address & ~PMD_MASK);
set_pmd_at(vma->vm_mm, address, pmdp, pmd);
/* dummy IPI to serialise against fast_gup */
kick_all_cpus_sync();
}
#endif /* CONFIG_HAVE_RCU_TABLE_FREE */
#endif /* CONFIG_TRANSPARENT_HUGEPAGE */

2
arch/arm/mm/init.c
View File

@ -322,7 +322,7 @@ void __init arm_memblock_init(const struct machine_desc *mdesc)
* reserve memory for DMA contigouos allocations,
* must come from DMA area inside low memory
*/
dma_contiguous_reserve(min(arm_dma_limit, arm_lowmem_limit));
dma_contiguous_reserve(arm_dma_limit);
arm_memblock_steal_permitted = false;
memblock_dump_all();

5
arch/arm64/Kconfig
View File

@ -18,6 +18,7 @@ config ARM64
select COMMON_CLK
select CPU_PM if (SUSPEND || CPU_IDLE)
select DCACHE_WORD_ACCESS
select GENERIC_ALLOCATOR
select GENERIC_CLOCKEVENTS
select GENERIC_CLOCKEVENTS_BROADCAST if SMP
select GENERIC_CPU_AUTOPROBE
@ -56,6 +57,7 @@ config ARM64
select HAVE_PERF_EVENTS
select HAVE_PERF_REGS
select HAVE_PERF_USER_STACK_DUMP
select HAVE_RCU_TABLE_FREE
select HAVE_SYSCALL_TRACEPOINTS
select IRQ_DOMAIN
select MODULES_USE_ELF_RELA
@ -109,6 +111,9 @@ config GENERIC_CALIBRATE_DELAY
config ZONE_DMA
def_bool y
config HAVE_GENERIC_RCU_GUP
def_bool y
config ARCH_DMA_ADDR_T_64BIT
def_bool y

21
arch/arm64/include/asm/pgtable.h
View File

@ -244,6 +244,16 @@ static inline void set_pte_at(struct mm_struct *mm, unsigned long addr,
#define __HAVE_ARCH_PTE_SPECIAL
static inline pte_t pud_pte(pud_t pud)
{
return __pte(pud_val(pud));
}
static inline pmd_t pud_pmd(pud_t pud)
{
return __pmd(pud_val(pud));
}
static inline pte_t pmd_pte(pmd_t pmd)
{
return __pte(pmd_val(pmd));
@ -261,7 +271,13 @@ static inline pmd_t pte_pmd(pte_t pte)
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
#define pmd_trans_huge(pmd) (pmd_val(pmd) && !(pmd_val(pmd) & PMD_TABLE_BIT))
#define pmd_trans_splitting(pmd) pte_special(pmd_pte(pmd))
#endif
#ifdef CONFIG_HAVE_RCU_TABLE_FREE
#define __HAVE_ARCH_PMDP_SPLITTING_FLUSH
struct vm_area_struct;
void pmdp_splitting_flush(struct vm_area_struct *vma, unsigned long address,
pmd_t *pmdp);
#endif /* CONFIG_HAVE_RCU_TABLE_FREE */
#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
#define pmd_young(pmd) pte_young(pmd_pte(pmd))
#define pmd_wrprotect(pmd) pte_pmd(pte_wrprotect(pmd_pte(pmd)))
@ -282,6 +298,7 @@ static inline pmd_t pte_pmd(pte_t pte)
#define mk_pmd(page,prot) pfn_pmd(page_to_pfn(page),prot)
#define pmd_page(pmd) pfn_to_page(__phys_to_pfn(pmd_val(pmd) & PHYS_MASK))
#define pud_write(pud) pte_write(pud_pte(pud))
#define pud_pfn(pud) (((pud_val(pud) & PUD_MASK) & PHYS_MASK) >> PAGE_SHIFT)
#define set_pmd_at(mm, addr, pmdp, pmd) set_pte_at(mm, addr, (pte_t *)pmdp, pmd_pte(pmd))
@ -383,6 +400,8 @@ static inline pmd_t *pmd_offset(pud_t *pud, unsigned long addr)
return (pmd_t *)pud_page_vaddr(*pud) + pmd_index(addr);
}
#define pud_page(pud) pmd_page(pud_pmd(pud))
#endif /* CONFIG_ARM64_PGTABLE_LEVELS > 2 */
#if CONFIG_ARM64_PGTABLE_LEVELS > 3

20
arch/arm64/include/asm/tlb.h
View File

@ -23,6 +23,20 @@
#include <asm-generic/tlb.h>
#include <linux/pagemap.h>
#include <linux/swap.h>
#ifdef CONFIG_HAVE_RCU_TABLE_FREE
#define tlb_remove_entry(tlb, entry) tlb_remove_table(tlb, entry)
static inline void __tlb_remove_table(void *_table)
{
free_page_and_swap_cache((struct page *)_table);
}
#else
#define tlb_remove_entry(tlb, entry) tlb_remove_page(tlb, entry)
#endif /* CONFIG_HAVE_RCU_TABLE_FREE */
/*
* There's three ways the TLB shootdown code is used:
* 1. Unmapping a range of vmas. See zap_page_range(), unmap_region().
@ -88,7 +102,7 @@ static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte,
{
pgtable_page_dtor(pte);
tlb_add_flush(tlb, addr);
tlb_remove_page(tlb, pte);
tlb_remove_entry(tlb, pte);
}
#if CONFIG_ARM64_PGTABLE_LEVELS > 2
@ -96,7 +110,7 @@ static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmdp,
unsigned long addr)
{
tlb_add_flush(tlb, addr);
tlb_remove_page(tlb, virt_to_page(pmdp));
tlb_remove_entry(tlb, virt_to_page(pmdp));
}
#endif
@ -105,7 +119,7 @@ static inline void __pud_free_tlb(struct mmu_gather *tlb, pud_t *pudp,
unsigned long addr)
{
tlb_add_flush(tlb, addr);
tlb_remove_page(tlb, virt_to_page(pudp));
tlb_remove_entry(tlb, virt_to_page(pudp));
}
#endif

164
arch/arm64/mm/dma-mapping.c
View File

@ -20,6 +20,7 @@
#include <linux/gfp.h>
#include <linux/export.h>
#include <linux/slab.h>
#include <linux/genalloc.h>
#include <linux/dma-mapping.h>
#include <linux/dma-contiguous.h>
#include <linux/vmalloc.h>
@ -38,6 +39,54 @@ static pgprot_t __get_dma_pgprot(struct dma_attrs *attrs, pgprot_t prot,
return prot;
}
static struct gen_pool *atomic_pool;
#define DEFAULT_DMA_COHERENT_POOL_SIZE SZ_256K
static size_t atomic_pool_size = DEFAULT_DMA_COHERENT_POOL_SIZE;
static int __init early_coherent_pool(char *p)
{
atomic_pool_size = memparse(p, &p);
return 0;
}
early_param("coherent_pool", early_coherent_pool);
static void *__alloc_from_pool(size_t size, struct page **ret_page)
{
unsigned long val;
void *ptr = NULL;
if (!atomic_pool) {
WARN(1, "coherent pool not initialised!\n");
return NULL;
}
val = gen_pool_alloc(atomic_pool, size);
if (val) {
phys_addr_t phys = gen_pool_virt_to_phys(atomic_pool, val);
*ret_page = phys_to_page(phys);
ptr = (void *)val;
}
return ptr;
}
static bool __in_atomic_pool(void *start, size_t size)
{
return addr_in_gen_pool(atomic_pool, (unsigned long)start, size);
}
static int __free_from_pool(void *start, size_t size)
{
if (!__in_atomic_pool(start, size))
return 0;
gen_pool_free(atomic_pool, (unsigned long)start, size);
return 1;
}
static void *__dma_alloc_coherent(struct device *dev, size_t size,
dma_addr_t *dma_handle, gfp_t flags,
struct dma_attrs *attrs)
@ -50,7 +99,7 @@ static void *__dma_alloc_coherent(struct device *dev, size_t size,
if (IS_ENABLED(CONFIG_ZONE_DMA) &&
dev->coherent_dma_mask <= DMA_BIT_MASK(32))
flags |= GFP_DMA;
if (IS_ENABLED(CONFIG_DMA_CMA)) {
if (IS_ENABLED(CONFIG_DMA_CMA) && (flags & __GFP_WAIT)) {
struct page *page;
size = PAGE_ALIGN(size);
@ -70,50 +119,54 @@ static void __dma_free_coherent(struct device *dev, size_t size,
void *vaddr, dma_addr_t dma_handle,
struct dma_attrs *attrs)
{
bool freed;
phys_addr_t paddr = dma_to_phys(dev, dma_handle);
if (dev == NULL) {
WARN_ONCE(1, "Use an actual device structure for DMA allocation\n");
return;
}
if (IS_ENABLED(CONFIG_DMA_CMA)) {
phys_addr_t paddr = dma_to_phys(dev, dma_handle);
dma_release_from_contiguous(dev,
freed = dma_release_from_contiguous(dev,
phys_to_page(paddr),
size >> PAGE_SHIFT);
} else {
if (!freed)
swiotlb_free_coherent(dev, size, vaddr, dma_handle);
}
}
static void *__dma_alloc_noncoherent(struct device *dev, size_t size,
dma_addr_t *dma_handle, gfp_t flags,
struct dma_attrs *attrs)
{
struct page *page, **map;
struct page *page;
void *ptr, *coherent_ptr;
int order, i;
size = PAGE_ALIGN(size);
order = get_order(size);
if (!(flags & __GFP_WAIT)) {
struct page *page = NULL;
void *addr = __alloc_from_pool(size, &page);
if (addr)
*dma_handle = phys_to_dma(dev, page_to_phys(page));
return addr;
}
ptr = __dma_alloc_coherent(dev, size, dma_handle, flags, attrs);
if (!ptr)
goto no_mem;
map = kmalloc(sizeof(struct page *) << order, flags & ~GFP_DMA);
if (!map)
goto no_map;
/* remove any dirty cache lines on the kernel alias */
__dma_flush_range(ptr, ptr + size);
/* create a coherent mapping */
page = virt_to_page(ptr);
for (i = 0; i < (size >> PAGE_SHIFT); i++)
map[i] = page + i;
coherent_ptr = vmap(map, size >> PAGE_SHIFT, VM_MAP,
__get_dma_pgprot(attrs, __pgprot(PROT_NORMAL_NC), false));
kfree(map);
coherent_ptr = dma_common_contiguous_remap(page, size, VM_USERMAP,
__get_dma_pgprot(attrs,
__pgprot(PROT_NORMAL_NC), false),
NULL);
if (!coherent_ptr)
goto no_map;
@ -132,6 +185,8 @@ static void __dma_free_noncoherent(struct device *dev, size_t size,
{
void *swiotlb_addr = phys_to_virt(dma_to_phys(dev, dma_handle));
if (__free_from_pool(vaddr, size))
return;
vunmap(vaddr);
__dma_free_coherent(dev, size, swiotlb_addr, dma_handle, attrs);
}
@ -307,6 +362,67 @@ EXPORT_SYMBOL(coherent_swiotlb_dma_ops);
extern int swiotlb_late_init_with_default_size(size_t default_size);
static int __init atomic_pool_init(void)
{
pgprot_t prot = __pgprot(PROT_NORMAL_NC);
unsigned long nr_pages = atomic_pool_size >> PAGE_SHIFT;
struct page *page;
void *addr;
unsigned int pool_size_order = get_order(atomic_pool_size);
if (dev_get_cma_area(NULL))
page = dma_alloc_from_contiguous(NULL, nr_pages,
pool_size_order);
else
page = alloc_pages(GFP_DMA, pool_size_order);
if (page) {
int ret;
void *page_addr = page_address(page);
memset(page_addr, 0, atomic_pool_size);
__dma_flush_range(page_addr, page_addr + atomic_pool_size);
atomic_pool = gen_pool_create(PAGE_SHIFT, -1);
if (!atomic_pool)
goto free_page;
addr = dma_common_contiguous_remap(page, atomic_pool_size,
VM_USERMAP, prot, atomic_pool_init);
if (!addr)
goto destroy_genpool;
ret = gen_pool_add_virt(atomic_pool, (unsigned long)addr,
page_to_phys(page),
atomic_pool_size, -1);
if (ret)
goto remove_mapping;
gen_pool_set_algo(atomic_pool,
gen_pool_first_fit_order_align,
(void *)PAGE_SHIFT);
pr_info("DMA: preallocated %zu KiB pool for atomic allocations\n",
atomic_pool_size / 1024);
return 0;
}
goto out;
remove_mapping:
dma_common_free_remap(addr, atomic_pool_size, VM_USERMAP);
destroy_genpool:
gen_pool_destroy(atomic_pool);
atomic_pool = NULL;
free_page:
if (!dma_release_from_contiguous(NULL, page, nr_pages))
__free_pages(page, pool_size_order);
out:
pr_err("DMA: failed to allocate %zu KiB pool for atomic coherent allocation\n",
atomic_pool_size / 1024);
return -ENOMEM;
}
static int __init swiotlb_late_init(void)
{
size_t swiotlb_size = min(SZ_64M, MAX_ORDER_NR_PAGES << PAGE_SHIFT);
@ -315,7 +431,17 @@ static int __init swiotlb_late_init(void)
return swiotlb_late_init_with_default_size(swiotlb_size);
}
arch_initcall(swiotlb_late_init);
static int __init arm64_dma_init(void)
{
int ret = 0;
ret |= swiotlb_late_init();
ret |= atomic_pool_init();
return ret;
}
arch_initcall(arm64_dma_init);
#define PREALLOC_DMA_DEBUG_ENTRIES 4096

16
arch/arm64/mm/flush.c
View File

@ -104,3 +104,19 @@ EXPORT_SYMBOL(flush_dcache_page);
*/
EXPORT_SYMBOL(flush_cache_all);
EXPORT_SYMBOL(flush_icache_range);
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
#ifdef CONFIG_HAVE_RCU_TABLE_FREE
void pmdp_splitting_flush(struct vm_area_struct *vma, unsigned long address,
pmd_t *pmdp)
{
pmd_t pmd = pmd_mksplitting(*pmdp);
VM_BUG_ON(address & ~PMD_MASK);
set_pmd_at(vma->vm_mm, address, pmdp, pmd);
/* dummy IPI to serialise against fast_gup */
kick_all_cpus_sync();
}
#endif /* CONFIG_HAVE_RCU_TABLE_FREE */
#endif /* CONFIG_TRANSPARENT_HUGEPAGE */

1
arch/cris/include/asm/Kbuild
View File

@ -15,6 +15,7 @@ generic-y += mcs_spinlock.h
generic-y += module.h
generic-y += preempt.h
generic-y += scatterlist.h
generic-y += sections.h
generic-y += trace_clock.h
generic-y += vga.h
generic-y += xor.h

7
arch/cris/include/asm/sections.h
View File

@ -1,7 +0,0 @@
#ifndef _CRIS_SECTIONS_H
#define _CRIS_SECTIONS_H
/* nothing to see, move along */
#include <asm-generic/sections.h>
#endif

16
arch/frv/include/asm/processor.h
View File

@ -34,22 +34,6 @@
/* Forward declaration, a strange C thing */
struct task_struct;
/*
* CPU type and hardware bug flags. Kept separately for each CPU.
*/
struct cpuinfo_frv {
#ifdef CONFIG_MMU
unsigned long *pgd_quick;
unsigned long *pte_quick;
unsigned long pgtable_cache_sz;
#endif
} __cacheline_aligned;
extern struct cpuinfo_frv __nongprelbss boot_cpu_data;
#define cpu_data (&boot_cpu_data)
#define current_cpu_data boot_cpu_data
/*
* Bus types
*/

8
arch/frv/kernel/irq-mb93091.c
View File

@ -107,25 +107,25 @@ static irqreturn_t fpga_interrupt(int irq, void *_mask)
static struct irqaction fpga_irq[4] = {
[0] = {
.handler = fpga_interrupt,
.flags = IRQF_DISABLED | IRQF_SHARED,
.flags = IRQF_SHARED,
.name = "fpga.0",
.dev_id = (void *) 0x0028UL,
},
[1] = {
.handler = fpga_interrupt,
.flags = IRQF_DISABLED | IRQF_SHARED,
.flags = IRQF_SHARED,
.name = "fpga.1",
.dev_id = (void *) 0x0050UL,
},
[2] = {
.handler = fpga_interrupt,
.flags = IRQF_DISABLED | IRQF_SHARED,
.flags = IRQF_SHARED,
.name = "fpga.2",
.dev_id = (void *) 0x1c00UL,
},
[3] = {
.handler = fpga_interrupt,
.flags = IRQF_DISABLED | IRQF_SHARED,
.flags = IRQF_SHARED,
.name = "fpga.3",
.dev_id = (void *) 0x6386UL,
}

1
arch/frv/kernel/irq-mb93093.c
View File

@ -105,7 +105,6 @@ static irqreturn_t fpga_interrupt(int irq, void *_mask)
static struct irqaction fpga_irq[1] = {
[0] = {
.handler = fpga_interrupt,
.flags = IRQF_DISABLED,
.name = "fpga.0",
.dev_id = (void *) 0x0700UL,
}

4
arch/frv/kernel/irq-mb93493.c
View File

@ -118,13 +118,13 @@ static irqreturn_t mb93493_interrupt(int irq, void *_piqsr)
static struct irqaction mb93493_irq[2] = {
[0] = {
.handler = mb93493_interrupt,
.flags = IRQF_DISABLED | IRQF_SHARED,
.flags = IRQF_SHARED,
.name = "mb93493.0",
.dev_id = (void *) __addr_MB93493_IQSR(0),
},
[1] = {
.handler = mb93493_interrupt,
.flags = IRQF_DISABLED | IRQF_SHARED,
.flags = IRQF_SHARED,
.name = "mb93493.1",
.dev_id = (void *) __addr_MB93493_IQSR(1),
}

2
arch/frv/kernel/setup.c
View File

@ -104,8 +104,6 @@ unsigned long __nongprelbss dma_coherent_mem_end;
unsigned long __initdata __sdram_old_base;
unsigned long __initdata num_mappedpages;
struct cpuinfo_frv __nongprelbss boot_cpu_data;
char __initdata command_line[COMMAND_LINE_SIZE];
char __initdata redboot_command_line[COMMAND_LINE_SIZE];

1
arch/frv/kernel/time.c
View File

@ -44,7 +44,6 @@ static irqreturn_t timer_interrupt(int irq, void *dummy);
static struct irqaction timer_irq = {
.handler = timer_interrupt,
.flags = IRQF_DISABLED,
.name = "timer",
};

1
arch/m32r/include/asm/Kbuild
View File

@ -8,4 +8,5 @@ generic-y += mcs_spinlock.h
generic-y += module.h
generic-y += preempt.h
generic-y += scatterlist.h
generic-y += sections.h
generic-y += trace_clock.h

7
arch/m32r/include/asm/sections.h
View File

@ -1,7 +0,0 @@
#ifndef _M32R_SECTIONS_H
#define _M32R_SECTIONS_H
/* nothing to see, move along */
#include <asm-generic/sections.h>
#endif /* _M32R_SECTIONS_H */

1
arch/m32r/kernel/time.c
View File

@ -134,7 +134,6 @@ static irqreturn_t timer_interrupt(int irq, void *dev_id)
static struct irqaction irq0 = {
.handler = timer_interrupt,
.flags = IRQF_DISABLED,
.name = "MFT2",
};

21
arch/m68k/kernel/sys_m68k.c
View File

@ -376,7 +376,6 @@ cache_flush_060 (unsigned long addr, int scope, int cache, unsigned long len)
asmlinkage int
sys_cacheflush (unsigned long addr, int scope, int cache, unsigned long len)
{
struct vm_area_struct *vma;
int ret = -EINVAL;
if (scope < FLUSH_SCOPE_LINE || scope > FLUSH_SCOPE_ALL ||
@ -389,17 +388,21 @@ sys_cacheflush (unsigned long addr, int scope, int cache, unsigned long len)
if (!capable(CAP_SYS_ADMIN))
goto out;
} else {
struct vm_area_struct *vma;
/* Check for overflow. */
if (addr + len < addr)
goto out;
/*
* Verify that the specified address region actually belongs
* to this process.
*/
vma = find_vma (current->mm, addr);
ret = -EINVAL;
/* Check for overflow. */
if (addr + len < addr)
goto out;
if (vma == NULL || addr < vma->vm_start || addr + len > vma->vm_end)
goto out;
down_read(&current->mm->mmap_sem);
vma = find_vma(current->mm, addr);
if (!vma || addr < vma->vm_start || addr + len > vma->vm_end)
goto out_unlock;
}
if (CPU_IS_020_OR_030) {
@ -429,7 +432,7 @@ sys_cacheflush (unsigned long addr, int scope, int cache, unsigned long len)
__asm__ __volatile__ ("movec %0, %%cacr" : : "r" (cacr));
}
ret = 0;
goto out;
goto out_unlock;
} else {
/*
* 040 or 060: don't blindly trust 'scope', someone could
@ -446,6 +449,8 @@ sys_cacheflush (unsigned long addr, int scope, int cache, unsigned long len)
ret = cache_flush_060 (addr, scope, cache, len);
}
}
out_unlock:
up_read(&current->mm->mmap_sem);
out:
return ret;
}

7
arch/mips/include/asm/suspend.h
View File

@ -1,7 +0,0 @@
#ifndef __ASM_SUSPEND_H
#define __ASM_SUSPEND_H
/* References to section boundaries */
extern const void __nosave_begin, __nosave_end;
#endif /* __ASM_SUSPEND_H */

2
arch/mips/power/cpu.c
View File

@ -7,7 +7,7 @@
* Author: Hu Hongbing <huhb@lemote.com>
* Wu Zhangjin <wuzhangjin@gmail.com>
*/
#include <asm/suspend.h>
#include <asm/sections.h>
#include <asm/fpu.h>
#include <asm/dsp.h>

1
arch/mn10300/include/asm/Kbuild
View File

@ -8,4 +8,5 @@ generic-y += irq_work.h
generic-y += mcs_spinlock.h
generic-y += preempt.h
generic-y += scatterlist.h
generic-y += sections.h
generic-y += trace_clock.h

1
arch/mn10300/include/asm/sections.h
View File

@ -1 +0,0 @@
#include <asm-generic/sections.h>

57
arch/powerpc/include/asm/pgtable.h
View File

@ -38,10 +38,9 @@ static inline int pte_none(pte_t pte) { return (pte_val(pte) & ~_PTE_NONE_MASK)
static inline pgprot_t pte_pgprot(pte_t pte) { return __pgprot(pte_val(pte) & PAGE_PROT_BITS); }
#ifdef CONFIG_NUMA_BALANCING
static inline int pte_present(pte_t pte)
{
return pte_val(pte) & (_PAGE_PRESENT | _PAGE_NUMA);
return pte_val(pte) & _PAGE_NUMA_MASK;
}
#define pte_present_nonuma pte_present_nonuma
@ -50,37 +49,6 @@ static inline int pte_present_nonuma(pte_t pte)
return pte_val(pte) & (_PAGE_PRESENT);
}
#define pte_numa pte_numa
static inline int pte_numa(pte_t pte)
{
return (pte_val(pte) &
(_PAGE_NUMA|_PAGE_PRESENT)) == _PAGE_NUMA;
}
#define pte_mknonnuma pte_mknonnuma
static inline pte_t pte_mknonnuma(pte_t pte)
{
pte_val(pte) &= ~_PAGE_NUMA;
pte_val(pte) |= _PAGE_PRESENT | _PAGE_ACCESSED;
return pte;
}
#define pte_mknuma pte_mknuma
static inline pte_t pte_mknuma(pte_t pte)
{
/*
* We should not set _PAGE_NUMA on non present ptes. Also clear the
* present bit so that hash_page will return 1 and we collect this
* as numa fault.
*/
if (pte_present(pte)) {
pte_val(pte) |= _PAGE_NUMA;
pte_val(pte) &= ~_PAGE_PRESENT;
} else
VM_BUG_ON(1);
return pte;
}
#define ptep_set_numa ptep_set_numa
static inline void ptep_set_numa(struct mm_struct *mm, unsigned long addr,
pte_t *ptep)
@ -92,12 +60,6 @@ static inline void ptep_set_numa(struct mm_struct *mm, unsigned long addr,
return;
}
#define pmd_numa pmd_numa
static inline int pmd_numa(pmd_t pmd)
{
return pte_numa(pmd_pte(pmd));
}
#define pmdp_set_numa pmdp_set_numa
static inline void pmdp_set_numa(struct mm_struct *mm, unsigned long addr,
pmd_t *pmdp)
@ -109,16 +71,21 @@ static inline void pmdp_set_numa(struct mm_struct *mm, unsigned long addr,
return;
}
#define pmd_mknonnuma pmd_mknonnuma
static inline pmd_t pmd_mknonnuma(pmd_t pmd)
/*
* Generic NUMA pte helpers expect pteval_t and pmdval_t types to exist
* which was inherited from x86. For the purposes of powerpc pte_basic_t and
* pmd_t are equivalent
*/
#define pteval_t pte_basic_t
#define pmdval_t pmd_t
static inline pteval_t ptenuma_flags(pte_t pte)
{
return pte_pmd(pte_mknonnuma(pmd_pte(pmd)));
return pte_val(pte) & _PAGE_NUMA_MASK;
}
#define pmd_mknuma pmd_mknuma
static inline pmd_t pmd_mknuma(pmd_t pmd)
static inline pmdval_t pmdnuma_flags(pmd_t pmd)
{
return pte_pmd(pte_mknuma(pmd_pte(pmd)));
return pmd_val(pmd) & _PAGE_NUMA_MASK;
}
# else

5
arch/powerpc/include/asm/pte-common.h
View File

@ -98,6 +98,11 @@ extern unsigned long bad_call_to_PMD_PAGE_SIZE(void);
_PAGE_USER | _PAGE_ACCESSED | \
_PAGE_RW | _PAGE_HWWRITE | _PAGE_DIRTY | _PAGE_EXEC)
#ifdef CONFIG_NUMA_BALANCING
/* Mask of bits that distinguish present and numa ptes */
#define _PAGE_NUMA_MASK (_PAGE_NUMA|_PAGE_PRESENT)
#endif
/*
* We define 2 sets of base prot bits, one for basic pages (ie,
* cacheable kernel and user pages) and one for non cacheable

4
arch/powerpc/kernel/suspend.c
View File

@ -9,9 +9,7 @@
#include <linux/mm.h>
#include <asm/page.h>
/* References to section boundaries */
extern const void __nosave_begin, __nosave_end;
#include <asm/sections.h>
/*
* pfn_is_nosave - check if given pfn is in the 'nosave' section

6
arch/s390/kernel/suspend.c
View File

@ -13,13 +13,9 @@
#include <asm/ipl.h>
#include <asm/cio.h>
#include <asm/pci.h>
#include <asm/sections.h>
#include "entry.h"
/*
* References to section boundaries
*/
extern const void __nosave_begin, __nosave_end;
/*
* The restore of the saved pages in an hibernation image will set
* the change and referenced bits in the storage key for each page.

1
arch/score/include/asm/Kbuild
View File

@ -10,6 +10,7 @@ generic-y += irq_work.h
generic-y += mcs_spinlock.h
generic-y += preempt.h
generic-y += scatterlist.h
generic-y += sections.h
generic-y += trace_clock.h
generic-y += xor.h
generic-y += serial.h

6
arch/score/include/asm/sections.h
View File

@ -1,6 +0,0 @@
#ifndef _ASM_SCORE_SECTIONS_H
#define _ASM_SCORE_SECTIONS_H
#include <asm-generic/sections.h>
#endif /* _ASM_SCORE_SECTIONS_H */

1
arch/sh/include/asm/sections.h
View File

@ -3,7 +3,6 @@
#include <asm-generic/sections.h>
extern long __nosave_begin, __nosave_end;
extern long __machvec_start, __machvec_end;
extern char __uncached_start, __uncached_end;
extern char __start_eh_frame[], __stop_eh_frame[];

4
arch/sparc/power/hibernate.c
View File

@ -9,11 +9,9 @@
#include <asm/hibernate.h>
#include <asm/visasm.h>
#include <asm/page.h>
#include <asm/sections.h>
#include <asm/tlb.h>
/* References to section boundaries */
extern const void __nosave_begin, __nosave_end;
struct saved_context saved_context;
/*

3
arch/unicore32/include/mach/pm.h
View File

@ -36,8 +36,5 @@ extern int puv3_pm_enter(suspend_state_t state);
/* Defined in hibernate_asm.S */
extern int restore_image(pgd_t *resume_pg_dir, struct pbe *restore_pblist);
/* References to section boundaries */
extern const void __nosave_begin, __nosave_end;
extern struct pbe *restore_pblist;
#endif

1
arch/unicore32/kernel/hibernate.c
View File

@ -18,6 +18,7 @@
#include <asm/page.h>
#include <asm/pgtable.h>
#include <asm/pgalloc.h>
#include <asm/sections.h>
#include <asm/suspend.h>
#include "mach/pm.h"

1
arch/x86/Kconfig
View File

@ -30,7 +30,6 @@ config X86
select HAVE_UNSTABLE_SCHED_CLOCK
select ARCH_SUPPORTS_NUMA_BALANCING if X86_64
select ARCH_SUPPORTS_INT128 if X86_64
select ARCH_WANTS_PROT_NUMA_PROT_NONE
select HAVE_IDE
select HAVE_OPROFILE
select HAVE_PCSPKR_PLATFORM

14
arch/x86/include/asm/pgtable_types.h
View File

@ -325,6 +325,20 @@ static inline pteval_t pte_flags(pte_t pte)
return native_pte_val(pte) & PTE_FLAGS_MASK;
}
#ifdef CONFIG_NUMA_BALANCING
/* Set of bits that distinguishes present, prot_none and numa ptes */
#define _PAGE_NUMA_MASK (_PAGE_NUMA|_PAGE_PROTNONE|_PAGE_PRESENT)
static inline pteval_t ptenuma_flags(pte_t pte)
{
return pte_flags(pte) & _PAGE_NUMA_MASK;
}
static inline pmdval_t pmdnuma_flags(pmd_t pmd)
{
return pmd_flags(pmd) & _PAGE_NUMA_MASK;
}
#endif /* CONFIG_NUMA_BALANCING */
#define pgprot_val(x) ((x).pgprot)
#define __pgprot(x) ((pgprot_t) { (x) } )

4
arch/x86/power/hibernate_32.c
View File

@ -13,13 +13,11 @@
#include <asm/page.h>
#include <asm/pgtable.h>
#include <asm/mmzone.h>
#include <asm/sections.h>
/* Defined in hibernate_asm_32.S */
extern int restore_image(void);
/* References to section boundaries */
extern const void __nosave_begin, __nosave_end;
/* Pointer to the temporary resume page tables */
pgd_t *resume_pg_dir;

4
arch/x86/power/hibernate_64.c
View File

@ -17,11 +17,9 @@
#include <asm/page.h>
#include <asm/pgtable.h>
#include <asm/mtrr.h>
#include <asm/sections.h>
#include <asm/suspend.h>
/* References to section boundaries */
extern __visible const void __nosave_begin, __nosave_end;
/* Defined in hibernate_asm_64.S */
extern asmlinkage __visible int restore_image(void);

3
drivers/base/Kconfig
View File

@ -252,6 +252,9 @@ config DMA_CMA
to allocate big physically-contiguous blocks of memory for use with
hardware components that do not support I/O map nor scatter-gather.
You can disable CMA by specifying "cma=0" on the kernel's command
line.
For more information see <include/linux/dma-contiguous.h>.
If unsure, say "n".

72
drivers/base/dma-mapping.c
View File

@ -10,6 +10,8 @@
#include <linux/dma-mapping.h>
#include <linux/export.h>
#include <linux/gfp.h>
#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <asm-generic/dma-coherent.h>
/*
@ -267,3 +269,73 @@ int dma_common_mmap(struct device *dev, struct vm_area_struct *vma,
return ret;
}
EXPORT_SYMBOL(dma_common_mmap);
#ifdef CONFIG_MMU
/*
* remaps an array of PAGE_SIZE pages into another vm_area
* Cannot be used in non-sleeping contexts
*/
void *dma_common_pages_remap(struct page **pages, size_t size,
unsigned long vm_flags, pgprot_t prot,
const void *caller)
{
struct vm_struct *area;
area = get_vm_area_caller(size, vm_flags, caller);
if (!area)
return NULL;
area->pages = pages;
if (map_vm_area(area, prot, pages)) {
vunmap(area->addr);
return NULL;
}
return area->addr;
}
/*
* remaps an allocated contiguous region into another vm_area.
* Cannot be used in non-sleeping contexts
*/
void *dma_common_contiguous_remap(struct page *page, size_t size,
unsigned long vm_flags,
pgprot_t prot, const void *caller)
{
int i;
struct page **pages;
void *ptr;
unsigned long pfn;
pages = kmalloc(sizeof(struct page *) << get_order(size), GFP_KERNEL);
if (!pages)
return NULL;
for (i = 0, pfn = page_to_pfn(page); i < (size >> PAGE_SHIFT); i++)
pages[i] = pfn_to_page(pfn + i);
ptr = dma_common_pages_remap(pages, size, vm_flags, prot, caller);
kfree(pages);
return ptr;
}
/*
* unmaps a range previously mapped by dma_common_*_remap
*/
void dma_common_free_remap(void *cpu_addr, size_t size, unsigned long vm_flags)
{
struct vm_struct *area = find_vm_area(cpu_addr);
if (!area || (area->flags & vm_flags) != vm_flags) {
WARN(1, "trying to free invalid coherent area: %p\n", cpu_addr);
return;
}
unmap_kernel_range((unsigned long)cpu_addr, size);
vunmap(cpu_addr);
}
#endif

42
drivers/base/memory.c
View File

@ -373,6 +373,45 @@ static ssize_t show_phys_device(struct device *dev,
return sprintf(buf, "%d\n", mem->phys_device);
}
#ifdef CONFIG_MEMORY_HOTREMOVE
static ssize_t show_valid_zones(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct memory_block *mem = to_memory_block(dev);
unsigned long start_pfn, end_pfn;
unsigned long nr_pages = PAGES_PER_SECTION * sections_per_block;
struct page *first_page;
struct zone *zone;
start_pfn = section_nr_to_pfn(mem->start_section_nr);
end_pfn = start_pfn + nr_pages;
first_page = pfn_to_page(start_pfn);
/* The block contains more than one zone can not be offlined. */
if (!test_pages_in_a_zone(start_pfn, end_pfn))
return sprintf(buf, "none\n");
zone = page_zone(first_page);
if (zone_idx(zone) == ZONE_MOVABLE - 1) {
/*The mem block is the last memoryblock of this zone.*/
if (end_pfn == zone_end_pfn(zone))
return sprintf(buf, "%s %s\n",
zone->name, (zone + 1)->name);
}
if (zone_idx(zone) == ZONE_MOVABLE) {
/*The mem block is the first memoryblock of ZONE_MOVABLE.*/
if (start_pfn == zone->zone_start_pfn)
return sprintf(buf, "%s %s\n",
zone->name, (zone - 1)->name);
}
return sprintf(buf, "%s\n", zone->name);
}
static DEVICE_ATTR(valid_zones, 0444, show_valid_zones, NULL);
#endif
static DEVICE_ATTR(phys_index, 0444, show_mem_start_phys_index, NULL);
static DEVICE_ATTR(state, 0644, show_mem_state, store_mem_state);
static DEVICE_ATTR(phys_device, 0444, show_phys_device, NULL);
@ -523,6 +562,9 @@ static struct attribute *memory_memblk_attrs[] = {
&dev_attr_state.attr,
&dev_attr_phys_device.attr,
&dev_attr_removable.attr,
#ifdef CONFIG_MEMORY_HOTREMOVE
&dev_attr_valid_zones.attr,
#endif
NULL
};

3
drivers/base/node.c
View File

@ -289,8 +289,6 @@ static int register_node(struct node *node, int num, struct node *parent)
device_create_file(&node->dev, &dev_attr_distance);
device_create_file(&node->dev, &dev_attr_vmstat);
scan_unevictable_register_node(node);
hugetlb_register_node(node);
compaction_register_node(node);
@ -314,7 +312,6 @@ void unregister_node(struct node *node)
device_remove_file(&node->dev, &dev_attr_distance);
device_remove_file(&node->dev, &dev_attr_vmstat);
scan_unevictable_unregister_node(node);
hugetlb_unregister_node(node); /* no-op, if memoryless node */
device_unregister(&node->dev);

106
drivers/block/zram/zram_drv.c
View File

@ -103,10 +103,10 @@ static ssize_t mem_used_total_show(struct device *dev,
down_read(&zram->init_lock);
if (init_done(zram))
val = zs_get_total_size_bytes(meta->mem_pool);
val = zs_get_total_pages(meta->mem_pool);
up_read(&zram->init_lock);
return scnprintf(buf, PAGE_SIZE, "%llu\n", val);
return scnprintf(buf, PAGE_SIZE, "%llu\n", val << PAGE_SHIFT);
}
static ssize_t max_comp_streams_show(struct device *dev,
@ -122,6 +122,72 @@ static ssize_t max_comp_streams_show(struct device *dev,
return scnprintf(buf, PAGE_SIZE, "%d\n", val);
}
static ssize_t mem_limit_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
u64 val;
struct zram *zram = dev_to_zram(dev);
down_read(&zram->init_lock);
val = zram->limit_pages;
up_read(&zram->init_lock);
return scnprintf(buf, PAGE_SIZE, "%llu\n", val << PAGE_SHIFT);
}
static ssize_t mem_limit_store(struct device *dev,
struct device_attribute *attr, const char *buf, size_t len)
{
u64 limit;
char *tmp;
struct zram *zram = dev_to_zram(dev);
limit = memparse(buf, &tmp);
if (buf == tmp) /* no chars parsed, invalid input */
return -EINVAL;
down_write(&zram->init_lock);
zram->limit_pages = PAGE_ALIGN(limit) >> PAGE_SHIFT;
up_write(&zram->init_lock);
return len;
}
static ssize_t mem_used_max_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
u64 val = 0;
struct zram *zram = dev_to_zram(dev);
down_read(&zram->init_lock);
if (init_done(zram))
val = atomic_long_read(&zram->stats.max_used_pages);
up_read(&zram->init_lock);
return scnprintf(buf, PAGE_SIZE, "%llu\n", val << PAGE_SHIFT);
}
static ssize_t mem_used_max_store(struct device *dev,
struct device_attribute *attr, const char *buf, size_t len)
{
int err;
unsigned long val;
struct zram *zram = dev_to_zram(dev);
struct zram_meta *meta = zram->meta;
err = kstrtoul(buf, 10, &val);
if (err || val != 0)
return -EINVAL;
down_read(&zram->init_lock);
if (init_done(zram))
atomic_long_set(&zram->stats.max_used_pages,
zs_get_total_pages(meta->mem_pool));
up_read(&zram->init_lock);
return len;
}
static ssize_t max_comp_streams_store(struct device *dev,
struct device_attribute *attr, const char *buf, size_t len)
{
@ -434,6 +500,21 @@ out_cleanup:
return ret;
}
static inline void update_used_max(struct zram *zram,
const unsigned long pages)
{
int old_max, cur_max;
old_max = atomic_long_read(&zram->stats.max_used_pages);
do {
cur_max = old_max;
if (pages > cur_max)
old_max = atomic_long_cmpxchg(
&zram->stats.max_used_pages, cur_max, pages);
} while (old_max != cur_max);
}
static int zram_bvec_write(struct zram *zram, struct bio_vec *bvec, u32 index,
int offset)
{
@ -445,6 +526,7 @@ static int zram_bvec_write(struct zram *zram, struct bio_vec *bvec, u32 index,
struct zram_meta *meta = zram->meta;
struct zcomp_strm *zstrm;
bool locked = false;
unsigned long alloced_pages;
page = bvec->bv_page;
if (is_partial_io(bvec)) {
@ -513,6 +595,16 @@ static int zram_bvec_write(struct zram *zram, struct bio_vec *bvec, u32 index,
ret = -ENOMEM;
goto out;
}
alloced_pages = zs_get_total_pages(meta->mem_pool);
if (zram->limit_pages && alloced_pages > zram->limit_pages) {
zs_free(meta->mem_pool, handle);
ret = -ENOMEM;
goto out;
}
update_used_max(zram, alloced_pages);
cmem = zs_map_object(meta->mem_pool, handle, ZS_MM_WO);
if ((clen == PAGE_SIZE) && !is_partial_io(bvec)) {
@ -606,6 +698,7 @@ static void zram_bio_discard(struct zram *zram, u32 index,
bit_spin_lock(ZRAM_ACCESS, &meta->table[index].value);
zram_free_page(zram, index);
bit_spin_unlock(ZRAM_ACCESS, &meta->table[index].value);
atomic64_inc(&zram->stats.notify_free);
index++;
n -= PAGE_SIZE;
}
@ -617,6 +710,9 @@ static void zram_reset_device(struct zram *zram, bool reset_capacity)
struct zram_meta *meta;
down_write(&zram->init_lock);
zram->limit_pages = 0;
if (!init_done(zram)) {
up_write(&zram->init_lock);
return;
@ -857,6 +953,10 @@ static DEVICE_ATTR(initstate, S_IRUGO, initstate_show, NULL);
static DEVICE_ATTR(reset, S_IWUSR, NULL, reset_store);
static DEVICE_ATTR(orig_data_size, S_IRUGO, orig_data_size_show, NULL);
static DEVICE_ATTR(mem_used_total, S_IRUGO, mem_used_total_show, NULL);
static DEVICE_ATTR(mem_limit, S_IRUGO | S_IWUSR, mem_limit_show,
mem_limit_store);
static DEVICE_ATTR(mem_used_max, S_IRUGO | S_IWUSR, mem_used_max_show,
mem_used_max_store);
static DEVICE_ATTR(max_comp_streams, S_IRUGO | S_IWUSR,
max_comp_streams_show, max_comp_streams_store);
static DEVICE_ATTR(comp_algorithm, S_IRUGO | S_IWUSR,
@ -885,6 +985,8 @@ static struct attribute *zram_disk_attrs[] = {
&dev_attr_orig_data_size.attr,
&dev_attr_compr_data_size.attr,
&dev_attr_mem_used_total.attr,
&dev_attr_mem_limit.attr,
&dev_attr_mem_used_max.attr,
&dev_attr_max_comp_streams.attr,
&dev_attr_comp_algorithm.attr,
NULL,

6
drivers/block/zram/zram_drv.h
View File

@ -90,6 +90,7 @@ struct zram_stats {
atomic64_t notify_free; /* no. of swap slot free notifications */
atomic64_t zero_pages; /* no. of zero filled pages */
atomic64_t pages_stored; /* no. of pages currently stored */
atomic_long_t max_used_pages; /* no. of maximum pages stored */
};
struct zram_meta {
@ -112,6 +113,11 @@ struct zram {
u64 disksize; /* bytes */
int max_comp_streams;
struct zram_stats stats;
/*
* the number of pages zram can consume for storing compressed data
*/
unsigned long limit_pages;
char compressor[10];
};
#endif

3
drivers/firmware/memmap.c
View File

@ -184,6 +184,9 @@ static int add_sysfs_fw_map_entry(struct firmware_map_entry *entry)
static int map_entries_nr;
static struct kset *mmap_kset;
if (entry->kobj.state_in_sysfs)
return -EEXIST;
if (!mmap_kset) {
mmap_kset = kset_create_and_add("memmap", NULL, firmware_kobj);
if (!mmap_kset)

1
drivers/virtio/Kconfig
View File

@ -25,6 +25,7 @@ config VIRTIO_PCI
config VIRTIO_BALLOON
tristate "Virtio balloon driver"
depends on VIRTIO
select MEMORY_BALLOON
---help---
This driver supports increasing and decreasing the amount
of memory within a KVM guest.

76
drivers/virtio/virtio_balloon.c
View File

@ -59,7 +59,7 @@ struct virtio_balloon
* Each page on this list adds VIRTIO_BALLOON_PAGES_PER_PAGE
* to num_pages above.
*/
struct balloon_dev_info *vb_dev_info;
struct balloon_dev_info vb_dev_info;
/* Synchronize access/update to this struct virtio_balloon elements */
struct mutex balloon_lock;
@ -127,7 +127,7 @@ static void set_page_pfns(u32 pfns[], struct page *page)
static void fill_balloon(struct virtio_balloon *vb, size_t num)
{
struct balloon_dev_info *vb_dev_info = vb->vb_dev_info;
struct balloon_dev_info *vb_dev_info = &vb->vb_dev_info;
/* We can only do one array worth at a time. */
num = min(num, ARRAY_SIZE(vb->pfns));
@ -163,15 +163,15 @@ static void release_pages_by_pfn(const u32 pfns[], unsigned int num)
/* Find pfns pointing at start of each page, get pages and free them. */
for (i = 0; i < num; i += VIRTIO_BALLOON_PAGES_PER_PAGE) {
struct page *page = balloon_pfn_to_page(pfns[i]);
balloon_page_free(page);
adjust_managed_page_count(page, 1);
put_page(page); /* balloon reference */
}
}
static void leak_balloon(struct virtio_balloon *vb, size_t num)
{
struct page *page;
struct balloon_dev_info *vb_dev_info = vb->vb_dev_info;
struct balloon_dev_info *vb_dev_info = &vb->vb_dev_info;
/* We can only do one array worth at a time. */
num = min(num, ARRAY_SIZE(vb->pfns));
@ -353,12 +353,11 @@ static int init_vqs(struct virtio_balloon *vb)
return 0;
}
static const struct address_space_operations virtio_balloon_aops;
#ifdef CONFIG_BALLOON_COMPACTION
/*
* virtballoon_migratepage - perform the balloon page migration on behalf of
* a compation thread. (called under page lock)
* @mapping: the page->mapping which will be assigned to the new migrated page.
* @vb_dev_info: the balloon device
* @newpage: page that will replace the isolated page after migration finishes.
* @page : the isolated (old) page that is about to be migrated to newpage.
* @mode : compaction mode -- not used for balloon page migration.
@ -373,17 +372,13 @@ static const struct address_space_operations virtio_balloon_aops;
* This function preforms the balloon page migration task.
* Called through balloon_mapping->a_ops->migratepage
*/
static int virtballoon_migratepage(struct address_space *mapping,
static int virtballoon_migratepage(struct balloon_dev_info *vb_dev_info,
struct page *newpage, struct page *page, enum migrate_mode mode)
{
struct balloon_dev_info *vb_dev_info = balloon_page_device(page);
struct virtio_balloon *vb;
struct virtio_balloon *vb = container_of(vb_dev_info,
struct virtio_balloon, vb_dev_info);
unsigned long flags;
BUG_ON(!vb_dev_info);
vb = vb_dev_info->balloon_device;
/*
* In order to avoid lock contention while migrating pages concurrently
* to leak_balloon() or fill_balloon() we just give up the balloon_lock
@ -395,21 +390,19 @@ static int virtballoon_migratepage(struct address_space *mapping,
if (!mutex_trylock(&vb->balloon_lock))
return -EAGAIN;
get_page(newpage); /* balloon reference */
/* balloon's page migration 1st step -- inflate "newpage" */
spin_lock_irqsave(&vb_dev_info->pages_lock, flags);
balloon_page_insert(newpage, mapping, &vb_dev_info->pages);
balloon_page_insert(vb_dev_info, newpage);
vb_dev_info->isolated_pages--;
__count_vm_event(BALLOON_MIGRATE);
spin_unlock_irqrestore(&vb_dev_info->pages_lock, flags);
vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
set_page_pfns(vb->pfns, newpage);
tell_host(vb, vb->inflate_vq);
/*
* balloon's page migration 2nd step -- deflate "page"
*
* It's safe to delete page->lru here because this page is at
* an isolated migration list, and this step is expected to happen here
*/
/* balloon's page migration 2nd step -- deflate "page" */
balloon_page_delete(page);
vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
set_page_pfns(vb->pfns, page);
@ -417,20 +410,15 @@ static int virtballoon_migratepage(struct address_space *mapping,
mutex_unlock(&vb->balloon_lock);
return MIGRATEPAGE_BALLOON_SUCCESS;
}
put_page(page); /* balloon reference */
/* define the balloon_mapping->a_ops callback to allow balloon page migration */
static const struct address_space_operations virtio_balloon_aops = {
.migratepage = virtballoon_migratepage,
};
return MIGRATEPAGE_SUCCESS;
}
#endif /* CONFIG_BALLOON_COMPACTION */
static int virtballoon_probe(struct virtio_device *vdev)
{
struct virtio_balloon *vb;
struct address_space *vb_mapping;
struct balloon_dev_info *vb_devinfo;
int err;
vdev->priv = vb = kmalloc(sizeof(*vb), GFP_KERNEL);
@ -446,30 +434,14 @@ static int virtballoon_probe(struct virtio_device *vdev)
vb->vdev = vdev;
vb->need_stats_update = 0;
vb_devinfo = balloon_devinfo_alloc(vb);
if (IS_ERR(vb_devinfo)) {
err = PTR_ERR(vb_devinfo);
goto out_free_vb;
}
vb_mapping = balloon_mapping_alloc(vb_devinfo,
(balloon_compaction_check()) ?
&virtio_balloon_aops : NULL);
if (IS_ERR(vb_mapping)) {
/*
* IS_ERR(vb_mapping) && PTR_ERR(vb_mapping) == -EOPNOTSUPP
* This means !CONFIG_BALLOON_COMPACTION, otherwise we get off.
*/
err = PTR_ERR(vb_mapping);
if (err != -EOPNOTSUPP)
goto out_free_vb_devinfo;
}
vb->vb_dev_info = vb_devinfo;
balloon_devinfo_init(&vb->vb_dev_info);
#ifdef CONFIG_BALLOON_COMPACTION
vb->vb_dev_info.migratepage = virtballoon_migratepage;
#endif
err = init_vqs(vb);
if (err)
goto out_free_vb_mapping;
goto out_free_vb;
vb->thread = kthread_run(balloon, vb, "vballoon");
if (IS_ERR(vb->thread)) {
@ -481,10 +453,6 @@ static int virtballoon_probe(struct virtio_device *vdev)
out_del_vqs:
vdev->config->del_vqs(vdev);
out_free_vb_mapping:
balloon_mapping_free(vb_mapping);
out_free_vb_devinfo:
balloon_devinfo_free(vb_devinfo);
out_free_vb:
kfree(vb);
out:
@ -510,8 +478,6 @@ static void virtballoon_remove(struct virtio_device *vdev)
kthread_stop(vb->thread);
remove_common(vb);
balloon_mapping_free(vb->vb_dev_info->mapping);
balloon_devinfo_free(vb->vb_dev_info);
kfree(vb);
}

7
fs/block_dev.c
View File

@ -304,6 +304,12 @@ static int blkdev_readpage(struct file * file, struct page * page)
return block_read_full_page(page, blkdev_get_block);
}
static int blkdev_readpages(struct file *file, struct address_space *mapping,
struct list_head *pages, unsigned nr_pages)
{
return mpage_readpages(mapping, pages, nr_pages, blkdev_get_block);
}
static int blkdev_write_begin(struct file *file, struct address_space *mapping,
loff_t pos, unsigned len, unsigned flags,
struct page **pagep, void **fsdata)
@ -1622,6 +1628,7 @@ static int blkdev_releasepage(struct page *page, gfp_t wait)
static const struct address_space_operations def_blk_aops = {
.readpage = blkdev_readpage,
.readpages = blkdev_readpages,
.writepage = blkdev_writepage,
.write_begin = blkdev_write_begin,
.write_end = blkdev_write_end,

28
fs/buffer.c
View File

@ -1253,7 +1253,7 @@ static struct buffer_head *__bread_slow(struct buffer_head *bh)
* a local interrupt disable for that.
*/
#define BH_LRU_SIZE 8
#define BH_LRU_SIZE 16
struct bh_lru {
struct buffer_head *bhs[BH_LRU_SIZE];
@ -2956,7 +2956,7 @@ static void end_bio_bh_io_sync(struct bio *bio, int err)
/*
* This allows us to do IO even on the odd last sectors
* of a device, even if the bh block size is some multiple
* of a device, even if the block size is some multiple
* of the physical sector size.
*
* We'll just truncate the bio to the size of the device,
@ -2966,10 +2966,11 @@ static void end_bio_bh_io_sync(struct bio *bio, int err)
* errors, this only handles the "we need to be able to
* do IO at the final sector" case.
*/
static void guard_bh_eod(int rw, struct bio *bio, struct buffer_head *bh)
void guard_bio_eod(int rw, struct bio *bio)
{
sector_t maxsector;
unsigned bytes;
struct bio_vec *bvec = &bio->bi_io_vec[bio->bi_vcnt - 1];
unsigned truncated_bytes;
maxsector = i_size_read(bio->bi_bdev->bd_inode) >> 9;
if (!maxsector)
@ -2984,23 +2985,20 @@ static void guard_bh_eod(int rw, struct bio *bio, struct buffer_head *bh)
return;
maxsector -= bio->bi_iter.bi_sector;
bytes = bio->bi_iter.bi_size;
if (likely((bytes >> 9) <= maxsector))
if (likely((bio->bi_iter.bi_size >> 9) <= maxsector))
return;
/* Uhhuh. We've got a bh that straddles the device size! */
bytes = maxsector << 9;
/* Uhhuh. We've got a bio that straddles the device size! */
truncated_bytes = bio->bi_iter.bi_size - (maxsector << 9);
/* Truncate the bio.. */
bio->bi_iter.bi_size = bytes;
bio->bi_io_vec[0].bv_len = bytes;
bio->bi_iter.bi_size -= truncated_bytes;
bvec->bv_len -= truncated_bytes;
/* ..and clear the end of the buffer for reads */
if ((rw & RW_MASK) == READ) {
void *kaddr = kmap_atomic(bh->b_page);
memset(kaddr + bh_offset(bh) + bytes, 0, bh->b_size - bytes);
kunmap_atomic(kaddr);
flush_dcache_page(bh->b_page);
zero_user(bvec->bv_page, bvec->bv_offset + bvec->bv_len,
truncated_bytes);
}
}
@ -3041,7 +3039,7 @@ int _submit_bh(int rw, struct buffer_head *bh, unsigned long bio_flags)
bio->bi_flags |= bio_flags;
/* Take care of bh's that straddle the end of the device */
guard_bh_eod(rw, bio, bh);
guard_bio_eod(rw, bio);
if (buffer_meta(bh))
rw |= REQ_META;

5
fs/internal.h
View File

@ -34,6 +34,11 @@ static inline int __sync_blockdev(struct block_device *bdev, int wait)
}
#endif
/*
* buffer.c
*/
extern void guard_bio_eod(int rw, struct bio *bio);
/*
* char_dev.c
*/

2
fs/mpage.c
View File

@ -28,6 +28,7 @@
#include <linux/backing-dev.h>
#include <linux/pagevec.h>
#include <linux/cleancache.h>
#include "internal.h"
/*
* I/O completion handler for multipage BIOs.
@ -57,6 +58,7 @@ static void mpage_end_io(struct bio *bio, int err)
static struct bio *mpage_bio_submit(int rw, struct bio *bio)
{
bio->bi_end_io = mpage_end_io;
guard_bio_eod(rw, bio);
submit_bio(rw, bio);
return NULL;
}

2
fs/notify/fanotify/fanotify_user.c
View File

@ -78,7 +78,7 @@ static int create_fd(struct fsnotify_group *group,
pr_debug("%s: group=%p event=%p\n", __func__, group, event);
client_fd = get_unused_fd();
client_fd = get_unused_fd_flags(group->fanotify_data.f_flags);
if (client_fd < 0)
return client_fd;

3
fs/notify/fsnotify.h
View File

@ -23,9 +23,6 @@ extern int fsnotify_add_vfsmount_mark(struct fsnotify_mark *mark,
struct fsnotify_group *group, struct vfsmount *mnt,
int allow_dups);
/* final kfree of a group */
extern void fsnotify_final_destroy_group(struct fsnotify_group *group);
/* vfsmount specific destruction of a mark */
extern void fsnotify_destroy_vfsmount_mark(struct fsnotify_mark *mark);
/* inode specific destruction of a mark */

2
fs/notify/group.c
View File

@ -31,7 +31,7 @@
/*
* Final freeing of a group
*/
void fsnotify_final_destroy_group(struct fsnotify_group *group)
static void fsnotify_final_destroy_group(struct fsnotify_group *group)
{
if (group->ops->free_group_priv)
group->ops->free_group_priv(group);

6
fs/notify/inotify/inotify_fsnotify.c
View File

@ -165,8 +165,10 @@ static void inotify_free_group_priv(struct fsnotify_group *group)
/* ideally the idr is empty and we won't hit the BUG in the callback */
idr_for_each(&group->inotify_data.idr, idr_callback, group);
idr_destroy(&group->inotify_data.idr);
atomic_dec(&group->inotify_data.user->inotify_devs);
free_uid(group->inotify_data.user);
if (group->inotify_data.user) {
atomic_dec(&group->inotify_data.user->inotify_devs);
free_uid(group->inotify_data.user);
}
}
static void inotify_free_event(struct fsnotify_event *fsn_event)

2
fs/ntfs/debug.c
View File

@ -112,7 +112,7 @@ void __ntfs_error(const char *function, const struct super_block *sb,
/* If 1, output debug messages, and if 0, don't. */
int debug_msgs = 0;
void __ntfs_debug (const char *file, int line, const char *function,
void __ntfs_debug(const char *file, int line, const char *function,
const char *fmt, ...)
{
struct va_format vaf;

5
fs/ntfs/file.c
View File

@ -1,7 +1,7 @@
/*
* file.c - NTFS kernel file operations. Part of the Linux-NTFS project.
*
* Copyright (c) 2001-2011 Anton Altaparmakov and Tuxera Inc.
* Copyright (c) 2001-2014 Anton Altaparmakov and Tuxera Inc.
*
* This program/include file is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License as published
@ -410,7 +410,8 @@ static inline int __ntfs_grab_cache_pages(struct address_space *mapping,
BUG_ON(!nr_pages);
err = nr = 0;
do {
pages[nr] = find_lock_page(mapping, index);
pages[nr] = find_get_page_flags(mapping, index, FGP_LOCK |
FGP_ACCESSED);
if (!pages[nr]) {
if (!*cached_page) {
*cached_page = page_cache_alloc(mapping);

2
fs/ntfs/super.c
View File

@ -3208,7 +3208,7 @@ static void __exit exit_ntfs_fs(void)
}
MODULE_AUTHOR("Anton Altaparmakov <anton@tuxera.com>");
MODULE_DESCRIPTION("NTFS 1.2/3.x driver - Copyright (c) 2001-2011 Anton Altaparmakov and Tuxera Inc.");
MODULE_DESCRIPTION("NTFS 1.2/3.x driver - Copyright (c) 2001-2014 Anton Altaparmakov and Tuxera Inc.");
MODULE_VERSION(NTFS_VERSION);
MODULE_LICENSE("GPL");
#ifdef DEBUG

15
fs/ocfs2/aops.c
View File

@ -1481,8 +1481,16 @@ static int ocfs2_write_begin_inline(struct address_space *mapping,
handle_t *handle;
struct ocfs2_dinode *di = (struct ocfs2_dinode *)wc->w_di_bh->b_data;
handle = ocfs2_start_trans(osb, OCFS2_INODE_UPDATE_CREDITS);
if (IS_ERR(handle)) {
ret = PTR_ERR(handle);
mlog_errno(ret);
goto out;
}
page = find_or_create_page(mapping, 0, GFP_NOFS);
if (!page) {
ocfs2_commit_trans(osb, handle);
ret = -ENOMEM;
mlog_errno(ret);
goto out;
@ -1494,13 +1502,6 @@ static int ocfs2_write_begin_inline(struct address_space *mapping,
wc->w_pages[0] = wc->w_target_page = page;
wc->w_num_pages = 1;
handle = ocfs2_start_trans(osb, OCFS2_INODE_UPDATE_CREDITS);
if (IS_ERR(handle)) {
ret = PTR_ERR(handle);
mlog_errno(ret);
goto out;
}
ret = ocfs2_journal_access_di(handle, INODE_CACHE(inode), wc->w_di_bh,
OCFS2_JOURNAL_ACCESS_WRITE);
if (ret) {

19
fs/ocfs2/cluster/heartbeat.c
View File

@ -2572,6 +2572,25 @@ int o2hb_check_node_heartbeating(u8 node_num)
}
EXPORT_SYMBOL_GPL(o2hb_check_node_heartbeating);
int o2hb_check_node_heartbeating_no_sem(u8 node_num)
{
unsigned long testing_map[BITS_TO_LONGS(O2NM_MAX_NODES)];
unsigned long flags;
spin_lock_irqsave(&o2hb_live_lock, flags);
o2hb_fill_node_map_from_callback(testing_map, sizeof(testing_map));
spin_unlock_irqrestore(&o2hb_live_lock, flags);
if (!test_bit(node_num, testing_map)) {
mlog(ML_HEARTBEAT,
"node (%u) does not have heartbeating enabled.\n",
node_num);
return 0;
}
return 1;
}
EXPORT_SYMBOL_GPL(o2hb_check_node_heartbeating_no_sem);
int o2hb_check_node_heartbeating_from_callback(u8 node_num)
{
unsigned long testing_map[BITS_TO_LONGS(O2NM_MAX_NODES)];

1
fs/ocfs2/cluster/heartbeat.h
View File

@ -80,6 +80,7 @@ void o2hb_fill_node_map(unsigned long *map,
void o2hb_exit(void);
int o2hb_init(void);
int o2hb_check_node_heartbeating(u8 node_num);
int o2hb_check_node_heartbeating_no_sem(u8 node_num);
int o2hb_check_node_heartbeating_from_callback(u8 node_num);
int o2hb_check_local_node_heartbeating(void);
void o2hb_stop_all_regions(void);

78
fs/ocfs2/cluster/netdebug.c
View File

@ -185,29 +185,13 @@ static const struct seq_operations nst_seq_ops = {
static int nst_fop_open(struct inode *inode, struct file *file)
{
struct o2net_send_tracking *dummy_nst;
struct seq_file *seq;
int ret;
dummy_nst = kmalloc(sizeof(struct o2net_send_tracking), GFP_KERNEL);
if (dummy_nst == NULL) {
ret = -ENOMEM;
goto out;
}
dummy_nst->st_task = NULL;
ret = seq_open(file, &nst_seq_ops);
if (ret)
goto out;
seq = file->private_data;
seq->private = dummy_nst;
dummy_nst = __seq_open_private(file, &nst_seq_ops, sizeof(*dummy_nst));
if (!dummy_nst)
return -ENOMEM;
o2net_debug_add_nst(dummy_nst);
dummy_nst = NULL;
out:
kfree(dummy_nst);
return ret;
return 0;
}
static int nst_fop_release(struct inode *inode, struct file *file)
@ -412,33 +396,27 @@ static const struct seq_operations sc_seq_ops = {
.show = sc_seq_show,
};
static int sc_common_open(struct file *file, struct o2net_sock_debug *sd)
static int sc_common_open(struct file *file, int ctxt)
{
struct o2net_sock_debug *sd;
struct o2net_sock_container *dummy_sc;
struct seq_file *seq;
int ret;
dummy_sc = kmalloc(sizeof(struct o2net_sock_container), GFP_KERNEL);
if (dummy_sc == NULL) {
ret = -ENOMEM;
goto out;
dummy_sc = kzalloc(sizeof(*dummy_sc), GFP_KERNEL);
if (!dummy_sc)
return -ENOMEM;
sd = __seq_open_private(file, &sc_seq_ops, sizeof(*sd));
if (!sd) {
kfree(dummy_sc);
return -ENOMEM;
}
dummy_sc->sc_page = NULL;
ret = seq_open(file, &sc_seq_ops);
if (ret)
goto out;
seq = file->private_data;
seq->private = sd;
sd->dbg_ctxt = ctxt;
sd->dbg_sock = dummy_sc;
o2net_debug_add_sc(dummy_sc);
dummy_sc = NULL;
out:
kfree(dummy_sc);
return ret;
return 0;
}
static int sc_fop_release(struct inode *inode, struct file *file)
@ -453,16 +431,7 @@ static int sc_fop_release(struct inode *inode, struct file *file)
static int stats_fop_open(struct inode *inode, struct file *file)
{
struct o2net_sock_debug *sd;
sd = kmalloc(sizeof(struct o2net_sock_debug), GFP_KERNEL);
if (sd == NULL)
return -ENOMEM;
sd->dbg_ctxt = SHOW_SOCK_STATS;
sd->dbg_sock = NULL;
return sc_common_open(file, sd);
return sc_common_open(file, SHOW_SOCK_STATS);
}
static const struct file_operations stats_seq_fops = {
@ -474,16 +443,7 @@ static const struct file_operations stats_seq_fops = {
static int sc_fop_open(struct inode *inode, struct file *file)
{
struct o2net_sock_debug *sd;
sd = kmalloc(sizeof(struct o2net_sock_debug), GFP_KERNEL);
if (sd == NULL)
return -ENOMEM;
sd->dbg_ctxt = SHOW_SOCK_CONTAINERS;
sd->dbg_sock = NULL;
return sc_common_open(file, sd);
return sc_common_open(file, SHOW_SOCK_CONTAINERS);
}
static const struct file_operations sc_seq_fops = {

43
fs/ocfs2/cluster/tcp.c
View File

@ -536,7 +536,7 @@ static void o2net_set_nn_state(struct o2net_node *nn,
if (nn->nn_persistent_error || nn->nn_sc_valid)
wake_up(&nn->nn_sc_wq);
if (!was_err && nn->nn_persistent_error) {
if (was_valid && !was_err && nn->nn_persistent_error) {
o2quo_conn_err(o2net_num_from_nn(nn));
queue_delayed_work(o2net_wq, &nn->nn_still_up,
msecs_to_jiffies(O2NET_QUORUM_DELAY_MS));
@ -1601,7 +1601,15 @@ static void o2net_start_connect(struct work_struct *work)
struct sockaddr_in myaddr = {0, }, remoteaddr = {0, };
int ret = 0, stop;
unsigned int timeout;
unsigned int noio_flag;
/*
* sock_create allocates the sock with GFP_KERNEL. We must set
* per-process flag PF_MEMALLOC_NOIO so that all allocations done
* by this process are done as if GFP_NOIO was specified. So we
* are not reentering filesystem while doing memory reclaim.
*/
noio_flag = memalloc_noio_save();
/* if we're greater we initiate tx, otherwise we accept */
if (o2nm_this_node() <= o2net_num_from_nn(nn))
goto out;
@ -1710,6 +1718,7 @@ out:
if (mynode)
o2nm_node_put(mynode);
memalloc_noio_restore(noio_flag);
return;
}
@ -1721,7 +1730,8 @@ static void o2net_connect_expired(struct work_struct *work)
spin_lock(&nn->nn_lock);
if (!nn->nn_sc_valid) {
printk(KERN_NOTICE "o2net: No connection established with "
"node %u after %u.%u seconds, giving up.\n",
"node %u after %u.%u seconds, check network and"
" cluster configuration.\n",
o2net_num_from_nn(nn),
o2net_idle_timeout() / 1000,
o2net_idle_timeout() % 1000);
@ -1835,6 +1845,15 @@ static int o2net_accept_one(struct socket *sock, int *more)
struct o2nm_node *local_node = NULL;
struct o2net_sock_container *sc = NULL;
struct o2net_node *nn;
unsigned int noio_flag;
/*
* sock_create_lite allocates the sock with GFP_KERNEL. We must set
* per-process flag PF_MEMALLOC_NOIO so that all allocations done
* by this process are done as if GFP_NOIO was specified. So we
* are not reentering filesystem while doing memory reclaim.
*/
noio_flag = memalloc_noio_save();
BUG_ON(sock == NULL);
*more = 0;
@ -1951,6 +1970,8 @@ out:
o2nm_node_put(local_node);
if (sc)
sc_put(sc);
memalloc_noio_restore(noio_flag);
return ret;
}
@ -2146,17 +2167,13 @@ int o2net_init(void)
o2quo_init();
if (o2net_debugfs_init())
return -ENOMEM;
goto out;
o2net_hand = kzalloc(sizeof(struct o2net_handshake), GFP_KERNEL);
o2net_keep_req = kzalloc(sizeof(struct o2net_msg), GFP_KERNEL);
o2net_keep_resp = kzalloc(sizeof(struct o2net_msg), GFP_KERNEL);
if (!o2net_hand || !o2net_keep_req || !o2net_keep_resp) {
kfree(o2net_hand);
kfree(o2net_keep_req);
kfree(o2net_keep_resp);
return -ENOMEM;
}
if (!o2net_hand || !o2net_keep_req || !o2net_keep_resp)
goto out;
o2net_hand->protocol_version = cpu_to_be64(O2NET_PROTOCOL_VERSION);
o2net_hand->connector_id = cpu_to_be64(1);
@ -2181,6 +2198,14 @@ int o2net_init(void)
}
return 0;
out:
kfree(o2net_hand);
kfree(o2net_keep_req);
kfree(o2net_keep_resp);
o2quo_exit();
return -ENOMEM;
}
void o2net_exit(void)

39
fs/ocfs2/dlm/dlmdebug.c
View File

@ -647,41 +647,30 @@ static const struct seq_operations debug_lockres_ops = {
static int debug_lockres_open(struct inode *inode, struct file *file)
{
struct dlm_ctxt *dlm = inode->i_private;
int ret = -ENOMEM;
struct seq_file *seq;
struct debug_lockres *dl = NULL;
struct debug_lockres *dl;
void *buf;
dl = kzalloc(sizeof(struct debug_lockres), GFP_KERNEL);
if (!dl) {
mlog_errno(ret);
buf = kmalloc(PAGE_SIZE, GFP_KERNEL);
if (!buf)
goto bail;
}
dl = __seq_open_private(file, &debug_lockres_ops, sizeof(*dl));
if (!dl)
goto bailfree;
dl->dl_len = PAGE_SIZE;
dl->dl_buf = kmalloc(dl->dl_len, GFP_KERNEL);
if (!dl->dl_buf) {
mlog_errno(ret);
goto bail;
}
ret = seq_open(file, &debug_lockres_ops);
if (ret) {
mlog_errno(ret);
goto bail;
}
seq = file->private_data;
seq->private = dl;
dl->dl_buf = buf;
dlm_grab(dlm);
dl->dl_ctxt = dlm;
return 0;
bailfree:
kfree(buf);
bail:
if (dl)
kfree(dl->dl_buf);
kfree(dl);
return ret;
mlog_errno(-ENOMEM);
return -ENOMEM;
}
static int debug_lockres_release(struct inode *inode, struct file *file)

44
fs/ocfs2/dlm/dlmdomain.c
View File

@ -839,7 +839,7 @@ static int dlm_query_join_handler(struct o2net_msg *msg, u32 len, void *data,
* to back off and try again. This gives heartbeat a chance
* to catch up.
*/
if (!o2hb_check_node_heartbeating(query->node_idx)) {
if (!o2hb_check_node_heartbeating_no_sem(query->node_idx)) {
mlog(0, "node %u is not in our live map yet\n",
query->node_idx);
@ -1975,24 +1975,22 @@ static struct dlm_ctxt *dlm_alloc_ctxt(const char *domain,
dlm = kzalloc(sizeof(*dlm), GFP_KERNEL);
if (!dlm) {
mlog_errno(-ENOMEM);
ret = -ENOMEM;
mlog_errno(ret);
goto leave;
}
dlm->name = kstrdup(domain, GFP_KERNEL);
if (dlm->name == NULL) {
mlog_errno(-ENOMEM);
kfree(dlm);
dlm = NULL;
ret = -ENOMEM;
mlog_errno(ret);
goto leave;
}
dlm->lockres_hash = (struct hlist_head **)dlm_alloc_pagevec(DLM_HASH_PAGES);
if (!dlm->lockres_hash) {
mlog_errno(-ENOMEM);
kfree(dlm->name);
kfree(dlm);
dlm = NULL;
ret = -ENOMEM;
mlog_errno(ret);
goto leave;
}
@ -2002,11 +2000,8 @@ static struct dlm_ctxt *dlm_alloc_ctxt(const char *domain,
dlm->master_hash = (struct hlist_head **)
dlm_alloc_pagevec(DLM_HASH_PAGES);
if (!dlm->master_hash) {
mlog_errno(-ENOMEM);
dlm_free_pagevec((void **)dlm->lockres_hash, DLM_HASH_PAGES);
kfree(dlm->name);
kfree(dlm);
dlm = NULL;
ret = -ENOMEM;
mlog_errno(ret);
goto leave;
}
@ -2017,14 +2012,8 @@ static struct dlm_ctxt *dlm_alloc_ctxt(const char *domain,
dlm->node_num = o2nm_this_node();
ret = dlm_create_debugfs_subroot(dlm);
if (ret < 0) {
dlm_free_pagevec((void **)dlm->master_hash, DLM_HASH_PAGES);
dlm_free_pagevec((void **)dlm->lockres_hash, DLM_HASH_PAGES);
kfree(dlm->name);
kfree(dlm);
dlm = NULL;
if (ret < 0)
goto leave;
}
spin_lock_init(&dlm->spinlock);
spin_lock_init(&dlm->master_lock);
@ -2085,6 +2074,19 @@ static struct dlm_ctxt *dlm_alloc_ctxt(const char *domain,
atomic_read(&dlm->dlm_refs.refcount));
leave:
if (ret < 0 && dlm) {
if (dlm->master_hash)
dlm_free_pagevec((void **)dlm->master_hash,
DLM_HASH_PAGES);
if (dlm->lockres_hash)
dlm_free_pagevec((void **)dlm->lockres_hash,
DLM_HASH_PAGES);
kfree(dlm->name);
kfree(dlm);
dlm = NULL;
}
return dlm;
}

3
fs/ocfs2/dlm/dlmmaster.c
View File

@ -625,9 +625,6 @@ struct dlm_lock_resource *dlm_new_lockres(struct dlm_ctxt *dlm,
return res;
error:
if (res && res->lockname.name)
kmem_cache_free(dlm_lockname_cache, (void *)res->lockname.name);
if (res)
kmem_cache_free(dlm_lockres_cache, res);
return NULL;

7
fs/ocfs2/dlm/dlmrecovery.c
View File

@ -1710,9 +1710,12 @@ int dlm_master_requery_handler(struct o2net_msg *msg, u32 len, void *data,
BUG();
} else
__dlm_lockres_grab_inflight_worker(dlm, res);
} else /* put.. incase we are not the master */
spin_unlock(&res->spinlock);
} else {
/* put.. incase we are not the master */
spin_unlock(&res->spinlock);
dlm_lockres_put(res);
spin_unlock(&res->spinlock);
}
}
spin_unlock(&dlm->spinlock);

23
fs/ocfs2/dlmglue.c
View File

@ -2892,37 +2892,24 @@ static int ocfs2_dlm_debug_release(struct inode *inode, struct file *file)
static int ocfs2_dlm_debug_open(struct inode *inode, struct file *file)
{
int ret;
struct ocfs2_dlm_seq_priv *priv;
struct seq_file *seq;
struct ocfs2_super *osb;
priv = kzalloc(sizeof(struct ocfs2_dlm_seq_priv), GFP_KERNEL);
priv = __seq_open_private(file, &ocfs2_dlm_seq_ops, sizeof(*priv));
if (!priv) {
ret = -ENOMEM;
mlog_errno(ret);
goto out;
mlog_errno(-ENOMEM);
return -ENOMEM;
}
osb = inode->i_private;
ocfs2_get_dlm_debug(osb->osb_dlm_debug);
priv->p_dlm_debug = osb->osb_dlm_debug;
INIT_LIST_HEAD(&priv->p_iter_res.l_debug_list);
ret = seq_open(file, &ocfs2_dlm_seq_ops);
if (ret) {
kfree(priv);
mlog_errno(ret);
goto out;
}
seq = file->private_data;
seq->private = priv;
ocfs2_add_lockres_tracking(&priv->p_iter_res,
priv->p_dlm_debug);
out:
return ret;
return 0;
}
static const struct file_operations ocfs2_dlm_debug_fops = {

47
fs/ocfs2/file.c
View File

@ -760,7 +760,7 @@ static int ocfs2_write_zero_page(struct inode *inode, u64 abs_from,
struct address_space *mapping = inode->i_mapping;
struct page *page;
unsigned long index = abs_from >> PAGE_CACHE_SHIFT;
handle_t *handle = NULL;
handle_t *handle;
int ret = 0;
unsigned zero_from, zero_to, block_start, block_end;
struct ocfs2_dinode *di = (struct ocfs2_dinode *)di_bh->b_data;
@ -769,11 +769,17 @@ static int ocfs2_write_zero_page(struct inode *inode, u64 abs_from,
BUG_ON(abs_to > (((u64)index + 1) << PAGE_CACHE_SHIFT));
BUG_ON(abs_from & (inode->i_blkbits - 1));
handle = ocfs2_zero_start_ordered_transaction(inode, di_bh);
if (IS_ERR(handle)) {
ret = PTR_ERR(handle);
goto out;
}
page = find_or_create_page(mapping, index, GFP_NOFS);
if (!page) {
ret = -ENOMEM;
mlog_errno(ret);
goto out;
goto out_commit_trans;
}
/* Get the offsets within the page that we want to zero */
@ -805,15 +811,6 @@ static int ocfs2_write_zero_page(struct inode *inode, u64 abs_from,
goto out_unlock;
}
if (!handle) {
handle = ocfs2_zero_start_ordered_transaction(inode,
di_bh);
if (IS_ERR(handle)) {
ret = PTR_ERR(handle);
handle = NULL;
break;
}
}
/* must not update i_size! */
ret = block_commit_write(page, block_start + 1,
@ -824,27 +821,29 @@ static int ocfs2_write_zero_page(struct inode *inode, u64 abs_from,
ret = 0;
}
/*
* fs-writeback will release the dirty pages without page lock
* whose offset are over inode size, the release happens at
* block_write_full_page().
*/
i_size_write(inode, abs_to);
inode->i_blocks = ocfs2_inode_sector_count(inode);
di->i_size = cpu_to_le64((u64)i_size_read(inode));
inode->i_mtime = inode->i_ctime = CURRENT_TIME;
di->i_mtime = di->i_ctime = cpu_to_le64(inode->i_mtime.tv_sec);
di->i_ctime_nsec = cpu_to_le32(inode->i_mtime.tv_nsec);
di->i_mtime_nsec = di->i_ctime_nsec;
if (handle) {
/*
* fs-writeback will release the dirty pages without page lock
* whose offset are over inode size, the release happens at
* block_write_full_page().
*/
i_size_write(inode, abs_to);
inode->i_blocks = ocfs2_inode_sector_count(inode);
di->i_size = cpu_to_le64((u64)i_size_read(inode));
inode->i_mtime = inode->i_ctime = CURRENT_TIME;
di->i_mtime = di->i_ctime = cpu_to_le64(inode->i_mtime.tv_sec);
di->i_ctime_nsec = cpu_to_le32(inode->i_mtime.tv_nsec);
di->i_mtime_nsec = di->i_ctime_nsec;
ocfs2_journal_dirty(handle, di_bh);
ocfs2_update_inode_fsync_trans(handle, inode, 1);
ocfs2_commit_trans(OCFS2_SB(inode->i_sb), handle);
}
out_unlock:
unlock_page(page);
page_cache_release(page);
out_commit_trans:
if (handle)
ocfs2_commit_trans(OCFS2_SB(inode->i_sb), handle);
out:
return ret;
}

2
fs/ocfs2/inode.h
View File

@ -162,7 +162,7 @@ static inline blkcnt_t ocfs2_inode_sector_count(struct inode *inode)
{
int c_to_s_bits = OCFS2_SB(inode->i_sb)->s_clustersize_bits - 9;
return (blkcnt_t)(OCFS2_I(inode)->ip_clusters << c_to_s_bits);
return (blkcnt_t)OCFS2_I(inode)->ip_clusters << c_to_s_bits;
}
/* Validate that a bh contains a valid inode */

2
fs/ocfs2/move_extents.c
View File

@ -404,7 +404,7 @@ static int ocfs2_find_victim_alloc_group(struct inode *inode,
* 'vict_blkno' was out of the valid range.
*/
if ((vict_blkno < le64_to_cpu(rec->c_blkno)) ||
(vict_blkno >= (le32_to_cpu(ac_dinode->id1.bitmap1.i_total) <<
(vict_blkno >= ((u64)le32_to_cpu(ac_dinode->id1.bitmap1.i_total) <<
bits_per_unit))) {
ret = -EINVAL;
goto out;

2
fs/ocfs2/stack_user.c
View File

@ -591,7 +591,7 @@ static int ocfs2_control_release(struct inode *inode, struct file *file)
*/
ocfs2_control_this_node = -1;
running_proto.pv_major = 0;
running_proto.pv_major = 0;
running_proto.pv_minor = 0;
}
out:

38
fs/proc/base.c
View File

@ -632,29 +632,35 @@ static const struct file_operations proc_single_file_operations = {
.release = single_release,
};
struct mm_struct *proc_mem_open(struct inode *inode, unsigned int mode)
{
struct task_struct *task = get_proc_task(inode);
struct mm_struct *mm = ERR_PTR(-ESRCH);
if (task) {
mm = mm_access(task, mode);
put_task_struct(task);
if (!IS_ERR_OR_NULL(mm)) {
/* ensure this mm_struct can't be freed */
atomic_inc(&mm->mm_count);
/* but do not pin its memory */
mmput(mm);
}
}
return mm;
}
static int __mem_open(struct inode *inode, struct file *file, unsigned int mode)
{
struct task_struct *task = get_proc_task(file_inode(file));
struct mm_struct *mm;
if (!task)
return -ESRCH;
mm = mm_access(task, mode);
put_task_struct(task);
struct mm_struct *mm = proc_mem_open(inode, mode);
if (IS_ERR(mm))
return PTR_ERR(mm);
if (mm) {
/* ensure this mm_struct can't be freed */
atomic_inc(&mm->mm_count);
/* but do not pin its memory */
mmput(mm);
}
file->private_data = mm;
return 0;
}

5
fs/proc/internal.h
View File

@ -268,8 +268,9 @@ extern int proc_remount(struct super_block *, int *, char *);
* task_[no]mmu.c
*/
struct proc_maps_private {
struct pid *pid;
struct inode *inode;
struct task_struct *task;
struct mm_struct *mm;
#ifdef CONFIG_MMU
struct vm_area_struct *tail_vma;
#endif
@ -278,6 +279,8 @@ struct proc_maps_private {
#endif
};
struct mm_struct *proc_mem_open(struct inode *inode, unsigned int mode);
extern const struct file_operations proc_pid_maps_operations;
extern const struct file_operations proc_tid_maps_operations;
extern const struct file_operations proc_pid_numa_maps_operations;

4
fs/proc/kcore.c
View File

@ -610,8 +610,10 @@ static void __init proc_kcore_text_init(void)
struct kcore_list kcore_modules;
static void __init add_modules_range(void)
{
kclist_add(&kcore_modules, (void *)MODULES_VADDR,
if (MODULES_VADDR != VMALLOC_START && MODULES_END != VMALLOC_END) {
kclist_add(&kcore_modules, (void *)MODULES_VADDR,
MODULES_END - MODULES_VADDR, KCORE_VMALLOC);
}
}
#else
static void __init add_modules_range(void)

3
fs/proc/page.c
View File

@ -133,6 +133,9 @@ u64 stable_page_flags(struct page *page)
if (PageBuddy(page))
u |= 1 << KPF_BUDDY;
if (PageBalloon(page))
u |= 1 << KPF_BALLOON;
u |= kpf_copy_bit(k, KPF_LOCKED, PG_locked);
u |= kpf_copy_bit(k, KPF_SLAB, PG_slab);

330
fs/proc/task_mmu.c
View File

@ -87,32 +87,14 @@ unsigned long task_statm(struct mm_struct *mm,
#ifdef CONFIG_NUMA
/*
* These functions are for numa_maps but called in generic **maps seq_file
* ->start(), ->stop() ops.
*
* numa_maps scans all vmas under mmap_sem and checks their mempolicy.
* Each mempolicy object is controlled by reference counting. The problem here
* is how to avoid accessing dead mempolicy object.
*
* Because we're holding mmap_sem while reading seq_file, it's safe to access
* each vma's mempolicy, no vma objects will never drop refs to mempolicy.
*
* A task's mempolicy (task->mempolicy) has different behavior. task->mempolicy
* is set and replaced under mmap_sem but unrefed and cleared under task_lock().
* So, without task_lock(), we cannot trust get_vma_policy() because we cannot
* gurantee the task never exits under us. But taking task_lock() around
* get_vma_plicy() causes lock order problem.
*
* To access task->mempolicy without lock, we hold a reference count of an
* object pointed by task->mempolicy and remember it. This will guarantee
* that task->mempolicy points to an alive object or NULL in numa_maps accesses.
* Save get_task_policy() for show_numa_map().
*/
static void hold_task_mempolicy(struct proc_maps_private *priv)
{
struct task_struct *task = priv->task;
task_lock(task);
priv->task_mempolicy = task->mempolicy;
priv->task_mempolicy = get_task_policy(task);
mpol_get(priv->task_mempolicy);
task_unlock(task);
}
@ -129,124 +111,154 @@ static void release_task_mempolicy(struct proc_maps_private *priv)
}
#endif
static void vma_stop(struct proc_maps_private *priv, struct vm_area_struct *vma)
static void vma_stop(struct proc_maps_private *priv)
{
if (vma && vma != priv->tail_vma) {
struct mm_struct *mm = vma->vm_mm;
release_task_mempolicy(priv);
up_read(&mm->mmap_sem);
mmput(mm);
}
struct mm_struct *mm = priv->mm;
release_task_mempolicy(priv);
up_read(&mm->mmap_sem);
mmput(mm);
}
static void *m_start(struct seq_file *m, loff_t *pos)
static struct vm_area_struct *
m_next_vma(struct proc_maps_private *priv, struct vm_area_struct *vma)
{
if (vma == priv->tail_vma)
return NULL;
return vma->vm_next ?: priv->tail_vma;
}
static void m_cache_vma(struct seq_file *m, struct vm_area_struct *vma)
{
if (m->count < m->size) /* vma is copied successfully */
m->version = m_next_vma(m->private, vma) ? vma->vm_start : -1UL;
}
static void *m_start(struct seq_file *m, loff_t *ppos)
{
struct proc_maps_private *priv = m->private;
unsigned long last_addr = m->version;
struct mm_struct *mm;
struct vm_area_struct *vma, *tail_vma = NULL;
loff_t l = *pos;
/* Clear the per syscall fields in priv */
priv->task = NULL;
priv->tail_vma = NULL;
/*
* We remember last_addr rather than next_addr to hit with
* vmacache most of the time. We have zero last_addr at
* the beginning and also after lseek. We will have -1 last_addr
* after the end of the vmas.
*/
struct vm_area_struct *vma;
unsigned int pos = *ppos;
/* See m_cache_vma(). Zero at the start or after lseek. */
if (last_addr == -1UL)
return NULL;
priv->task = get_pid_task(priv->pid, PIDTYPE_PID);
priv->task = get_proc_task(priv->inode);
if (!priv->task)
return ERR_PTR(-ESRCH);
mm = mm_access(priv->task, PTRACE_MODE_READ);
if (!mm || IS_ERR(mm))
return mm;
mm = priv->mm;
if (!mm || !atomic_inc_not_zero(&mm->mm_users))
return NULL;
down_read(&mm->mmap_sem);
tail_vma = get_gate_vma(priv->task->mm);
priv->tail_vma = tail_vma;
hold_task_mempolicy(priv);
/* Start with last addr hint */
vma = find_vma(mm, last_addr);
if (last_addr && vma) {
vma = vma->vm_next;
goto out;
priv->tail_vma = get_gate_vma(mm);
if (last_addr) {
vma = find_vma(mm, last_addr);
if (vma && (vma = m_next_vma(priv, vma)))
return vma;
}
/*
* Check the vma index is within the range and do
* sequential scan until m_index.
*/
vma = NULL;
if ((unsigned long)l < mm->map_count) {
vma = mm->mmap;
while (l-- && vma)
m->version = 0;
if (pos < mm->map_count) {
for (vma = mm->mmap; pos; pos--) {
m->version = vma->vm_start;
vma = vma->vm_next;
goto out;
}
return vma;
}
if (l != mm->map_count)
tail_vma = NULL; /* After gate vma */
/* we do not bother to update m->version in this case */
if (pos == mm->map_count && priv->tail_vma)
return priv->tail_vma;
out:
if (vma)
return vma;
release_task_mempolicy(priv);
/* End of vmas has been reached */
m->version = (tail_vma != NULL)? 0: -1UL;
up_read(&mm->mmap_sem);
mmput(mm);
return tail_vma;
vma_stop(priv);
return NULL;
}
static void *m_next(struct seq_file *m, void *v, loff_t *pos)
{
struct proc_maps_private *priv = m->private;
struct vm_area_struct *vma = v;
struct vm_area_struct *tail_vma = priv->tail_vma;
struct vm_area_struct *next;
(*pos)++;
if (vma && (vma != tail_vma) && vma->vm_next)
return vma->vm_next;
vma_stop(priv, vma);
return (vma != tail_vma)? tail_vma: NULL;
next = m_next_vma(priv, v);
if (!next)
vma_stop(priv);
return next;
}
static void m_stop(struct seq_file *m, void *v)
{
struct proc_maps_private *priv = m->private;
struct vm_area_struct *vma = v;
if (!IS_ERR(vma))
vma_stop(priv, vma);
if (priv->task)
if (!IS_ERR_OR_NULL(v))
vma_stop(priv);
if (priv->task) {
put_task_struct(priv->task);
priv->task = NULL;
}
}
static int proc_maps_open(struct inode *inode, struct file *file,
const struct seq_operations *ops, int psize)
{
struct proc_maps_private *priv = __seq_open_private(file, ops, psize);
if (!priv)
return -ENOMEM;
priv->inode = inode;
priv->mm = proc_mem_open(inode, PTRACE_MODE_READ);
if (IS_ERR(priv->mm)) {
int err = PTR_ERR(priv->mm);
seq_release_private(inode, file);
return err;
}
return 0;
}
static int proc_map_release(struct inode *inode, struct file *file)
{
struct seq_file *seq = file->private_data;
struct proc_maps_private *priv = seq->private;
if (priv->mm)
mmdrop(priv->mm);
return seq_release_private(inode, file);
}
static int do_maps_open(struct inode *inode, struct file *file,
const struct seq_operations *ops)
{
struct proc_maps_private *priv;
int ret = -ENOMEM;
priv = kzalloc(sizeof(*priv), GFP_KERNEL);
if (priv) {
priv->pid = proc_pid(inode);
ret = seq_open(file, ops);
if (!ret) {
struct seq_file *m = file->private_data;
m->private = priv;
} else {
kfree(priv);
}
return proc_maps_open(inode, file, ops,
sizeof(struct proc_maps_private));
}
static pid_t pid_of_stack(struct proc_maps_private *priv,
struct vm_area_struct *vma, bool is_pid)
{
struct inode *inode = priv->inode;
struct task_struct *task;
pid_t ret = 0;
rcu_read_lock();
task = pid_task(proc_pid(inode), PIDTYPE_PID);
if (task) {
task = task_of_stack(task, vma, is_pid);
if (task)
ret = task_pid_nr_ns(task, inode->i_sb->s_fs_info);
}
rcu_read_unlock();
return ret;
}
@ -256,7 +268,6 @@ show_map_vma(struct seq_file *m, struct vm_area_struct *vma, int is_pid)
struct mm_struct *mm = vma->vm_mm;
struct file *file = vma->vm_file;
struct proc_maps_private *priv = m->private;
struct task_struct *task = priv->task;
vm_flags_t flags = vma->vm_flags;
unsigned long ino = 0;
unsigned long long pgoff = 0;
@ -321,8 +332,7 @@ show_map_vma(struct seq_file *m, struct vm_area_struct *vma, int is_pid)
goto done;
}
tid = vm_is_stack(task, vma, is_pid);
tid = pid_of_stack(priv, vma, is_pid);
if (tid != 0) {
/*
* Thread stack in /proc/PID/task/TID/maps or
@ -349,15 +359,8 @@ done:
static int show_map(struct seq_file *m, void *v, int is_pid)
{
struct vm_area_struct *vma = v;
struct proc_maps_private *priv = m->private;
struct task_struct *task = priv->task;
show_map_vma(m, vma, is_pid);
if (m->count < m->size) /* vma is copied successfully */
m->version = (vma != get_gate_vma(task->mm))
? vma->vm_start : 0;
show_map_vma(m, v, is_pid);
m_cache_vma(m, v);
return 0;
}
@ -399,14 +402,14 @@ const struct file_operations proc_pid_maps_operations = {
.open = pid_maps_open,
.read = seq_read,
.llseek = seq_lseek,
.release = seq_release_private,
.release = proc_map_release,
};
const struct file_operations proc_tid_maps_operations = {
.open = tid_maps_open,
.read = seq_read,
.llseek = seq_lseek,
.release = seq_release_private,
.release = proc_map_release,
};
/*
@ -583,8 +586,6 @@ static void show_smap_vma_flags(struct seq_file *m, struct vm_area_struct *vma)
static int show_smap(struct seq_file *m, void *v, int is_pid)
{
struct proc_maps_private *priv = m->private;
struct task_struct *task = priv->task;
struct vm_area_struct *vma = v;
struct mem_size_stats mss;
struct mm_walk smaps_walk = {
@ -637,10 +638,7 @@ static int show_smap(struct seq_file *m, void *v, int is_pid)
mss.nonlinear >> 10);
show_smap_vma_flags(m, vma);
if (m->count < m->size) /* vma is copied successfully */
m->version = (vma != get_gate_vma(task->mm))
? vma->vm_start : 0;
m_cache_vma(m, vma);
return 0;
}
@ -682,14 +680,14 @@ const struct file_operations proc_pid_smaps_operations = {
.open = pid_smaps_open,
.read = seq_read,
.llseek = seq_lseek,
.release = seq_release_private,
.release = proc_map_release,
};
const struct file_operations proc_tid_smaps_operations = {
.open = tid_smaps_open,
.read = seq_read,
.llseek = seq_lseek,
.release = seq_release_private,
.release = proc_map_release,
};
/*
@ -1029,7 +1027,6 @@ static int pagemap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
spinlock_t *ptl;
pte_t *pte;
int err = 0;
pagemap_entry_t pme = make_pme(PM_NOT_PRESENT(pm->v2));
/* find the first VMA at or above 'addr' */
vma = find_vma(walk->mm, addr);
@ -1043,6 +1040,7 @@ static int pagemap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
for (; addr != end; addr += PAGE_SIZE) {
unsigned long offset;
pagemap_entry_t pme;
offset = (addr & ~PAGEMAP_WALK_MASK) >>
PAGE_SHIFT;
@ -1057,32 +1055,51 @@ static int pagemap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
if (pmd_trans_unstable(pmd))
return 0;
for (; addr != end; addr += PAGE_SIZE) {
int flags2;
/* check to see if we've left 'vma' behind
* and need a new, higher one */
if (vma && (addr >= vma->vm_end)) {
vma = find_vma(walk->mm, addr);
if (vma && (vma->vm_flags & VM_SOFTDIRTY))
flags2 = __PM_SOFT_DIRTY;
else
flags2 = 0;
pme = make_pme(PM_NOT_PRESENT(pm->v2) | PM_STATUS2(pm->v2, flags2));
while (1) {
/* End of address space hole, which we mark as non-present. */
unsigned long hole_end;
if (vma)
hole_end = min(end, vma->vm_start);
else
hole_end = end;
for (; addr < hole_end; addr += PAGE_SIZE) {
pagemap_entry_t pme = make_pme(PM_NOT_PRESENT(pm->v2));
err = add_to_pagemap(addr, &pme, pm);
if (err)
return err;
}
/* check that 'vma' actually covers this address,
* and that it isn't a huge page vma */
if (vma && (vma->vm_start <= addr) &&
!is_vm_hugetlb_page(vma)) {
if (!vma || vma->vm_start >= end)
break;
/*
* We can't possibly be in a hugetlb VMA. In general,
* for a mm_walk with a pmd_entry and a hugetlb_entry,
* the pmd_entry can only be called on addresses in a
* hugetlb if the walk starts in a non-hugetlb VMA and
* spans a hugepage VMA. Since pagemap_read walks are
* PMD-sized and PMD-aligned, this will never be true.
*/
BUG_ON(is_vm_hugetlb_page(vma));
/* Addresses in the VMA. */
for (; addr < min(end, vma->vm_end); addr += PAGE_SIZE) {
pagemap_entry_t pme;
pte = pte_offset_map(pmd, addr);
pte_to_pagemap_entry(&pme, pm, vma, addr, *pte);
/* unmap before userspace copy */
pte_unmap(pte);
err = add_to_pagemap(addr, &pme, pm);
if (err)
return err;
}
err = add_to_pagemap(addr, &pme, pm);
if (err)
return err;
if (addr == end)
break;
vma = find_vma(walk->mm, addr);
}
cond_resched();
@ -1415,7 +1432,6 @@ static int show_numa_map(struct seq_file *m, void *v, int is_pid)
struct vm_area_struct *vma = v;
struct numa_maps *md = &numa_priv->md;
struct file *file = vma->vm_file;
struct task_struct *task = proc_priv->task;
struct mm_struct *mm = vma->vm_mm;
struct mm_walk walk = {};
struct mempolicy *pol;
@ -1435,9 +1451,13 @@ static int show_numa_map(struct seq_file *m, void *v, int is_pid)
walk.private = md;
walk.mm = mm;
pol = get_vma_policy(task, vma, vma->vm_start);
mpol_to_str(buffer, sizeof(buffer), pol);
mpol_cond_put(pol);
pol = __get_vma_policy(vma, vma->vm_start);
if (pol) {
mpol_to_str(buffer, sizeof(buffer), pol);
mpol_cond_put(pol);
} else {
mpol_to_str(buffer, sizeof(buffer), proc_priv->task_mempolicy);
}
seq_printf(m, "%08lx %s", vma->vm_start, buffer);
@ -1447,7 +1467,7 @@ static int show_numa_map(struct seq_file *m, void *v, int is_pid)
} else if (vma->vm_start <= mm->brk && vma->vm_end >= mm->start_brk) {
seq_puts(m, " heap");
} else {
pid_t tid = vm_is_stack(task, vma, is_pid);
pid_t tid = pid_of_stack(proc_priv, vma, is_pid);
if (tid != 0) {
/*
* Thread stack in /proc/PID/task/TID/maps or
@ -1495,9 +1515,7 @@ static int show_numa_map(struct seq_file *m, void *v, int is_pid)
seq_printf(m, " N%d=%lu", nid, md->node[nid]);
out:
seq_putc(m, '\n');
if (m->count < m->size)
m->version = (vma != proc_priv->tail_vma) ? vma->vm_start : 0;
m_cache_vma(m, vma);
return 0;
}
@ -1528,20 +1546,8 @@ static const struct seq_operations proc_tid_numa_maps_op = {
static int numa_maps_open(struct inode *inode, struct file *file,
const struct seq_operations *ops)
{
struct numa_maps_private *priv;
int ret = -ENOMEM;
priv = kzalloc(sizeof(*priv), GFP_KERNEL);
if (priv) {
priv->proc_maps.pid = proc_pid(inode);
ret = seq_open(file, ops);
if (!ret) {
struct seq_file *m = file->private_data;
m->private = priv;
} else {
kfree(priv);
}
}
return ret;
return proc_maps_open(inode, file, ops,
sizeof(struct numa_maps_private));
}
static int pid_numa_maps_open(struct inode *inode, struct file *file)
@ -1558,13 +1564,13 @@ const struct file_operations proc_pid_numa_maps_operations = {
.open = pid_numa_maps_open,
.read = seq_read,
.llseek = seq_lseek,
.release = seq_release_private,
.release = proc_map_release,
};
const struct file_operations proc_tid_numa_maps_operations = {
.open = tid_numa_maps_open,
.read = seq_read,
.llseek = seq_lseek,
.release = seq_release_private,
.release = proc_map_release,
};
#endif /* CONFIG_NUMA */

86
fs/proc/task_nommu.c
View File

@ -123,6 +123,25 @@ unsigned long task_statm(struct mm_struct *mm,
return size;
}
static pid_t pid_of_stack(struct proc_maps_private *priv,
struct vm_area_struct *vma, bool is_pid)
{
struct inode *inode = priv->inode;
struct task_struct *task;
pid_t ret = 0;
rcu_read_lock();
task = pid_task(proc_pid(inode), PIDTYPE_PID);
if (task) {
task = task_of_stack(task, vma, is_pid);
if (task)
ret = task_pid_nr_ns(task, inode->i_sb->s_fs_info);
}
rcu_read_unlock();
return ret;
}
/*
* display a single VMA to a sequenced file
*/
@ -163,7 +182,7 @@ static int nommu_vma_show(struct seq_file *m, struct vm_area_struct *vma,
seq_pad(m, ' ');
seq_path(m, &file->f_path, "");
} else if (mm) {
pid_t tid = vm_is_stack(priv->task, vma, is_pid);
pid_t tid = pid_of_stack(priv, vma, is_pid);
if (tid != 0) {
seq_pad(m, ' ');
@ -212,22 +231,22 @@ static void *m_start(struct seq_file *m, loff_t *pos)
loff_t n = *pos;
/* pin the task and mm whilst we play with them */
priv->task = get_pid_task(priv->pid, PIDTYPE_PID);
priv->task = get_proc_task(priv->inode);
if (!priv->task)
return ERR_PTR(-ESRCH);
mm = mm_access(priv->task, PTRACE_MODE_READ);
if (!mm || IS_ERR(mm)) {
put_task_struct(priv->task);
priv->task = NULL;
return mm;
}
down_read(&mm->mmap_sem);
mm = priv->mm;
if (!mm || !atomic_inc_not_zero(&mm->mm_users))
return NULL;
down_read(&mm->mmap_sem);
/* start from the Nth VMA */
for (p = rb_first(&mm->mm_rb); p; p = rb_next(p))
if (n-- == 0)
return p;
up_read(&mm->mmap_sem);
mmput(mm);
return NULL;
}
@ -235,11 +254,13 @@ static void m_stop(struct seq_file *m, void *_vml)
{
struct proc_maps_private *priv = m->private;
if (!IS_ERR_OR_NULL(_vml)) {
up_read(&priv->mm->mmap_sem);
mmput(priv->mm);
}
if (priv->task) {
struct mm_struct *mm = priv->task->mm;
up_read(&mm->mmap_sem);
mmput(mm);
put_task_struct(priv->task);
priv->task = NULL;
}
}
@ -269,20 +290,33 @@ static int maps_open(struct inode *inode, struct file *file,
const struct seq_operations *ops)
{
struct proc_maps_private *priv;
int ret = -ENOMEM;
priv = kzalloc(sizeof(*priv), GFP_KERNEL);
if (priv) {
priv->pid = proc_pid(inode);
ret = seq_open(file, ops);
if (!ret) {
struct seq_file *m = file->private_data;
m->private = priv;
} else {
kfree(priv);
}
priv = __seq_open_private(file, ops, sizeof(*priv));
if (!priv)
return -ENOMEM;
priv->inode = inode;
priv->mm = proc_mem_open(inode, PTRACE_MODE_READ);
if (IS_ERR(priv->mm)) {
int err = PTR_ERR(priv->mm);
seq_release_private(inode, file);
return err;
}
return ret;
return 0;
}
static int map_release(struct inode *inode, struct file *file)
{
struct seq_file *seq = file->private_data;
struct proc_maps_private *priv = seq->private;
if (priv->mm)
mmdrop(priv->mm);
return seq_release_private(inode, file);
}
static int pid_maps_open(struct inode *inode, struct file *file)
@ -299,13 +333,13 @@ const struct file_operations proc_pid_maps_operations = {
.open = pid_maps_open,
.read = seq_read,
.llseek = seq_lseek,
.release = seq_release_private,
.release = map_release,
};
const struct file_operations proc_tid_maps_operations = {
.open = tid_maps_open,
.read = seq_read,
.llseek = seq_lseek,
.release = seq_release_private,
.release = map_release,
};

9
include/asm-generic/dma-mapping-common.h
View File

@ -179,6 +179,15 @@ dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg,
extern int dma_common_mmap(struct device *dev, struct vm_area_struct *vma,
void *cpu_addr, dma_addr_t dma_addr, size_t size);
void *dma_common_contiguous_remap(struct page *page, size_t size,
unsigned long vm_flags,
pgprot_t prot, const void *caller);
void *dma_common_pages_remap(struct page **pages, size_t size,
unsigned long vm_flags, pgprot_t prot,
const void *caller);
void dma_common_free_remap(void *cpu_addr, size_t size, unsigned long vm_flags);
/**
* dma_mmap_attrs - map a coherent DMA allocation into user space
* @dev: valid struct device pointer, or NULL for ISA and EISA-like devices

27
include/asm-generic/pgtable.h
View File

@ -664,11 +664,12 @@ static inline int pmd_trans_unstable(pmd_t *pmd)
}
#ifdef CONFIG_NUMA_BALANCING
#ifdef CONFIG_ARCH_USES_NUMA_PROT_NONE
/*
* _PAGE_NUMA works identical to _PAGE_PROTNONE (it's actually the
* same bit too). It's set only when _PAGE_PRESET is not set and it's
* never set if _PAGE_PRESENT is set.
* _PAGE_NUMA distinguishes between an unmapped page table entry, an entry that
* is protected for PROT_NONE and a NUMA hinting fault entry. If the
* architecture defines __PAGE_PROTNONE then it should take that into account
* but those that do not can rely on the fact that the NUMA hinting scanner
* skips inaccessible VMAs.
*
* pte/pmd_present() returns true if pte/pmd_numa returns true. Page
* fault triggers on those regions if pte/pmd_numa returns true
@ -677,16 +678,14 @@ static inline int pmd_trans_unstable(pmd_t *pmd)
#ifndef pte_numa
static inline int pte_numa(pte_t pte)
{
return (pte_flags(pte) &
(_PAGE_NUMA|_PAGE_PROTNONE|_PAGE_PRESENT)) == _PAGE_NUMA;
return ptenuma_flags(pte) == _PAGE_NUMA;
}
#endif
#ifndef pmd_numa
static inline int pmd_numa(pmd_t pmd)
{
return (pmd_flags(pmd) &
(_PAGE_NUMA|_PAGE_PROTNONE|_PAGE_PRESENT)) == _PAGE_NUMA;
return pmdnuma_flags(pmd) == _PAGE_NUMA;
}
#endif
@ -726,6 +725,8 @@ static inline pte_t pte_mknuma(pte_t pte)
{
pteval_t val = pte_val(pte);
VM_BUG_ON(!(val & _PAGE_PRESENT));
val &= ~_PAGE_PRESENT;
val |= _PAGE_NUMA;
@ -769,16 +770,6 @@ static inline void pmdp_set_numa(struct mm_struct *mm, unsigned long addr,
}
#endif
#else
extern int pte_numa(pte_t pte);
extern int pmd_numa(pmd_t pmd);
extern pte_t pte_mknonnuma(pte_t pte);
extern pmd_t pmd_mknonnuma(pmd_t pmd);
extern pte_t pte_mknuma(pte_t pte);
extern pmd_t pmd_mknuma(pmd_t pmd);
extern void ptep_set_numa(struct mm_struct *mm, unsigned long addr, pte_t *ptep);
extern void pmdp_set_numa(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp);
#endif /* CONFIG_ARCH_USES_NUMA_PROT_NONE */
#else
static inline int pmd_numa(pmd_t pmd)
{
return 0;

4
include/asm-generic/sections.h
View File

@ -3,6 +3,8 @@
/* References to section boundaries */
#include <linux/compiler.h>
/*
* Usage guidelines:
* _text, _data: architecture specific, don't use them in arch-independent code
@ -37,6 +39,8 @@ extern char __start_rodata[], __end_rodata[];
/* Start and end of .ctors section - used for constructor calls. */
extern char __ctors_start[], __ctors_end[];
extern __visible const void __nosave_begin, __nosave_end;
/* function descriptor handling (if any). Override
* in asm/sections.h */
#ifndef dereference_function_descriptor

171
include/linux/balloon_compaction.h
View File

@ -27,10 +27,13 @@
* counter raised only while it is under our special handling;
*
* iii. after the lockless scan step have selected a potential balloon page for
* isolation, re-test the page->mapping flags and the page ref counter
* isolation, re-test the PageBalloon mark and the PagePrivate flag
* under the proper page lock, to ensure isolating a valid balloon page
* (not yet isolated, nor under release procedure)
*
* iv. isolation or dequeueing procedure must clear PagePrivate flag under
* page lock together with removing page from balloon device page list.
*
* The functions provided by this interface are placed to help on coping with
* the aforementioned balloon page corner case, as well as to ensure the simple
* set of exposed rules are satisfied while we are dealing with balloon pages
@ -54,43 +57,22 @@
* balloon driver as a page book-keeper for its registered balloon devices.
*/
struct balloon_dev_info {
void *balloon_device; /* balloon device descriptor */
struct address_space *mapping; /* balloon special page->mapping */
unsigned long isolated_pages; /* # of isolated pages for migration */
spinlock_t pages_lock; /* Protection to pages list */
struct list_head pages; /* Pages enqueued & handled to Host */
int (*migratepage)(struct balloon_dev_info *, struct page *newpage,
struct page *page, enum migrate_mode mode);
};
extern struct page *balloon_page_enqueue(struct balloon_dev_info *b_dev_info);
extern struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info);
extern struct balloon_dev_info *balloon_devinfo_alloc(
void *balloon_dev_descriptor);
static inline void balloon_devinfo_free(struct balloon_dev_info *b_dev_info)
static inline void balloon_devinfo_init(struct balloon_dev_info *balloon)
{
kfree(b_dev_info);
}
/*
* balloon_page_free - release a balloon page back to the page free lists
* @page: ballooned page to be set free
*
* This function must be used to properly set free an isolated/dequeued balloon
* page at the end of a sucessful page migration, or at the balloon driver's
* page release procedure.
*/
static inline void balloon_page_free(struct page *page)
{
/*
* Balloon pages always get an extra refcount before being isolated
* and before being dequeued to help on sorting out fortuite colisions
* between a thread attempting to isolate and another thread attempting
* to release the very same balloon page.
*
* Before we handle the page back to Buddy, lets drop its extra refcnt.
*/
put_page(page);
__free_page(page);
balloon->isolated_pages = 0;
spin_lock_init(&balloon->pages_lock);
INIT_LIST_HEAD(&balloon->pages);
balloon->migratepage = NULL;
}
#ifdef CONFIG_BALLOON_COMPACTION
@ -98,107 +80,58 @@ extern bool balloon_page_isolate(struct page *page);
extern void balloon_page_putback(struct page *page);
extern int balloon_page_migrate(struct page *newpage,
struct page *page, enum migrate_mode mode);
extern struct address_space
*balloon_mapping_alloc(struct balloon_dev_info *b_dev_info,
const struct address_space_operations *a_ops);
static inline void balloon_mapping_free(struct address_space *balloon_mapping)
{
kfree(balloon_mapping);
}
/*
* page_flags_cleared - helper to perform balloon @page ->flags tests.
*
* As balloon pages are obtained from buddy and we do not play with page->flags
* at driver level (exception made when we get the page lock for compaction),
* we can safely identify a ballooned page by checking if the
* PAGE_FLAGS_CHECK_AT_PREP page->flags are all cleared. This approach also
* helps us skip ballooned pages that are locked for compaction or release, thus
* mitigating their racy check at balloon_page_movable()
*/
static inline bool page_flags_cleared(struct page *page)
{
return !(page->flags & PAGE_FLAGS_CHECK_AT_PREP);
}
/*
* __is_movable_balloon_page - helper to perform @page mapping->flags tests
* __is_movable_balloon_page - helper to perform @page PageBalloon tests
*/
static inline bool __is_movable_balloon_page(struct page *page)
{
struct address_space *mapping = page->mapping;
return mapping_balloon(mapping);
return PageBalloon(page);
}
/*
* balloon_page_movable - test page->mapping->flags to identify balloon pages
* that can be moved by compaction/migration.
*
* This function is used at core compaction's page isolation scheme, therefore
* most pages exposed to it are not enlisted as balloon pages and so, to avoid
* undesired side effects like racing against __free_pages(), we cannot afford
* holding the page locked while testing page->mapping->flags here.
* balloon_page_movable - test PageBalloon to identify balloon pages
* and PagePrivate to check that the page is not
* isolated and can be moved by compaction/migration.
*
* As we might return false positives in the case of a balloon page being just
* released under us, the page->mapping->flags need to be re-tested later,
* under the proper page lock, at the functions that will be coping with the
* balloon page case.
* released under us, this need to be re-tested later, under the page lock.
*/
static inline bool balloon_page_movable(struct page *page)
{
/*
* Before dereferencing and testing mapping->flags, let's make sure
* this is not a page that uses ->mapping in a different way
*/
if (page_flags_cleared(page) && !page_mapped(page) &&
page_count(page) == 1)
return __is_movable_balloon_page(page);
return false;
return PageBalloon(page) && PagePrivate(page);
}
/*
* isolated_balloon_page - identify an isolated balloon page on private
* compaction/migration page lists.
*
* After a compaction thread isolates a balloon page for migration, it raises
* the page refcount to prevent concurrent compaction threads from re-isolating
* the same page. For that reason putback_movable_pages(), or other routines
* that need to identify isolated balloon pages on private pagelists, cannot
* rely on balloon_page_movable() to accomplish the task.
*/
static inline bool isolated_balloon_page(struct page *page)
{
/* Already isolated balloon pages, by default, have a raised refcount */
if (page_flags_cleared(page) && !page_mapped(page) &&
page_count(page) >= 2)
return __is_movable_balloon_page(page);
return false;
return PageBalloon(page);
}
/*
* balloon_page_insert - insert a page into the balloon's page list and make
* the page->mapping assignment accordingly.
* the page->private assignment accordingly.
* @balloon : pointer to balloon device
* @page : page to be assigned as a 'balloon page'
* @mapping : allocated special 'balloon_mapping'
* @head : balloon's device page list head
*
* Caller must ensure the page is locked and the spin_lock protecting balloon
* pages list is held before inserting a page into the balloon device.
*/
static inline void balloon_page_insert(struct page *page,
struct address_space *mapping,
struct list_head *head)
static inline void balloon_page_insert(struct balloon_dev_info *balloon,
struct page *page)
{
page->mapping = mapping;
list_add(&page->lru, head);
__SetPageBalloon(page);
SetPagePrivate(page);
set_page_private(page, (unsigned long)balloon);
list_add(&page->lru, &balloon->pages);
}
/*
* balloon_page_delete - delete a page from balloon's page list and clear
* the page->mapping assignement accordingly.
* the page->private assignement accordingly.
* @page : page to be released from balloon's page list
*
* Caller must ensure the page is locked and the spin_lock protecting balloon
@ -206,8 +139,12 @@ static inline void balloon_page_insert(struct page *page,
*/
static inline void balloon_page_delete(struct page *page)
{
page->mapping = NULL;
list_del(&page->lru);
__ClearPageBalloon(page);
set_page_private(page, 0);
if (PagePrivate(page)) {
ClearPagePrivate(page);
list_del(&page->lru);
}
}
/*
@ -216,11 +153,7 @@ static inline void balloon_page_delete(struct page *page)
*/
static inline struct balloon_dev_info *balloon_page_device(struct page *page)
{
struct address_space *mapping = page->mapping;
if (likely(mapping))
return mapping->private_data;
return NULL;
return (struct balloon_dev_info *)page_private(page);
}
static inline gfp_t balloon_mapping_gfp_mask(void)
@ -228,36 +161,26 @@ static inline gfp_t balloon_mapping_gfp_mask(void)
return GFP_HIGHUSER_MOVABLE;
}
static inline bool balloon_compaction_check(void)
{
return true;
}
#else /* !CONFIG_BALLOON_COMPACTION */
static inline void *balloon_mapping_alloc(void *balloon_device,
const struct address_space_operations *a_ops)
static inline void balloon_page_insert(struct balloon_dev_info *balloon,
struct page *page)
{
return ERR_PTR(-EOPNOTSUPP);
}
static inline void balloon_mapping_free(struct address_space *balloon_mapping)
{
return;
}
static inline void balloon_page_insert(struct page *page,
struct address_space *mapping,
struct list_head *head)
{
list_add(&page->lru, head);
__SetPageBalloon(page);
list_add(&page->lru, &balloon->pages);
}
static inline void balloon_page_delete(struct page *page)
{
__ClearPageBalloon(page);
list_del(&page->lru);
}
static inline bool __is_movable_balloon_page(struct page *page)
{
return false;
}
static inline bool balloon_page_movable(struct page *page)
{
return false;
@ -289,9 +212,5 @@ static inline gfp_t balloon_mapping_gfp_mask(void)
return GFP_HIGHUSER;
}
static inline bool balloon_compaction_check(void)
{
return false;
}
#endif /* CONFIG_BALLOON_COMPACTION */
#endif /* _LINUX_BALLOON_COMPACTION_H */

2
include/linux/blkdev.h
View File

@ -1564,7 +1564,7 @@ static inline int blk_rq_map_integrity_sg(struct request_queue *q,
}
static inline struct blk_integrity *bdev_get_integrity(struct block_device *b)
{
return 0;
return NULL;
}
static inline struct blk_integrity *blk_get_integrity(struct gendisk *disk)
{

24
include/linux/compaction.h
View File

@ -2,14 +2,24 @@
#define _LINUX_COMPACTION_H
/* Return values for compact_zone() and try_to_compact_pages() */
/* compaction didn't start as it was deferred due to past failures */
#define COMPACT_DEFERRED 0
/* compaction didn't start as it was not possible or direct reclaim was more suitable */
#define COMPACT_SKIPPED 0
#define COMPACT_SKIPPED 1
/* compaction should continue to another pageblock */
#define COMPACT_CONTINUE 1
#define COMPACT_CONTINUE 2
/* direct compaction partially compacted a zone and there are suitable pages */
#define COMPACT_PARTIAL 2
#define COMPACT_PARTIAL 3
/* The full zone was compacted */
#define COMPACT_COMPLETE 3
#define COMPACT_COMPLETE 4
/* Used to signal whether compaction detected need_sched() or lock contention */
/* No contention detected */
#define COMPACT_CONTENDED_NONE 0
/* Either need_sched() was true or fatal signal pending */
#define COMPACT_CONTENDED_SCHED 1
/* Zone lock or lru_lock was contended in async compaction */
#define COMPACT_CONTENDED_LOCK 2
#ifdef CONFIG_COMPACTION
extern int sysctl_compact_memory;
@ -22,7 +32,8 @@ extern int sysctl_extfrag_handler(struct ctl_table *table, int write,
extern int fragmentation_index(struct zone *zone, unsigned int order);
extern unsigned long try_to_compact_pages(struct zonelist *zonelist,
int order, gfp_t gfp_mask, nodemask_t *mask,
enum migrate_mode mode, bool *contended);
enum migrate_mode mode, int *contended,
struct zone **candidate_zone);
extern void compact_pgdat(pg_data_t *pgdat, int order);
extern void reset_isolation_suitable(pg_data_t *pgdat);
extern unsigned long compaction_suitable(struct zone *zone, int order);
@ -91,7 +102,8 @@ static inline bool compaction_restarting(struct zone *zone, int order)
#else
static inline unsigned long try_to_compact_pages(struct zonelist *zonelist,
int order, gfp_t gfp_mask, nodemask_t *nodemask,
enum migrate_mode mode, bool *contended)
enum migrate_mode mode, int *contended,
struct zone **candidate_zone)
{
return COMPACT_CONTINUE;
}

7
include/linux/genalloc.h
View File

@ -110,6 +110,10 @@ extern void gen_pool_set_algo(struct gen_pool *pool, genpool_algo_t algo,
extern unsigned long gen_pool_first_fit(unsigned long *map, unsigned long size,
unsigned long start, unsigned int nr, void *data);
extern unsigned long gen_pool_first_fit_order_align(unsigned long *map,
unsigned long size, unsigned long start, unsigned int nr,
void *data);
extern unsigned long gen_pool_best_fit(unsigned long *map, unsigned long size,
unsigned long start, unsigned int nr, void *data);
@ -117,6 +121,9 @@ extern struct gen_pool *devm_gen_pool_create(struct device *dev,
int min_alloc_order, int nid);
extern struct gen_pool *dev_get_gen_pool(struct device *dev);
bool addr_in_gen_pool(struct gen_pool *pool, unsigned long start,
size_t size);
#ifdef CONFIG_OF
extern struct gen_pool *of_get_named_gen_pool(struct device_node *np,
const char *propname, int index);

2
include/linux/gfp.h
View File

@ -156,7 +156,7 @@ struct vm_area_struct;
#define GFP_DMA32 __GFP_DMA32
/* Convert GFP flags to their corresponding migrate type */
static inline int allocflags_to_migratetype(gfp_t gfp_flags)
static inline int gfpflags_to_migratetype(const gfp_t gfp_flags)
{
WARN_ON((gfp_flags & GFP_MOVABLE_MASK) == GFP_MOVABLE_MASK);

Some files were not shown because too many files have changed in this diff Show More