OpenCloudOS-Kernel/include/drm/drm_buddy.h

160 lines
4.1 KiB
C
Raw Normal View History

drm: move the buddy allocator from i915 into common drm Move the base i915 buddy allocator code into drm - Move i915_buddy.h to include/drm - Move i915_buddy.c to drm root folder - Rename "i915" string with "drm" string wherever applicable - Rename "I915" string with "DRM" string wherever applicable - Fix header file dependencies - Fix alignment issues - add Makefile support for drm buddy - export functions and write kerneldoc description - Remove i915 selftest config check condition as buddy selftest will be moved to drm selftest folder cleanup i915 buddy references in i915 driver module and replace with drm buddy v2: - include header file in alphabetical order(Thomas) - merged changes listed in the body section into a single patch to keep the build intact(Christian, Jani) v3: - make drm buddy a separate module(Thomas, Christian) v4: - Fix build error reported by kernel test robot <lkp@intel.com> - removed i915 buddy selftest from i915_mock_selftests.h to avoid build error - removed selftests/i915_buddy.c file as we create a new set of buddy test cases in drm/selftests folder v5: - Fix merge conflict issue v6: - replace drm_buddy_mm structure name as drm_buddy(Thomas, Christian) - replace drm_buddy_alloc() function name as drm_buddy_alloc_blocks() (Thomas) - replace drm_buddy_free() function name as drm_buddy_free_block() (Thomas) - export drm_buddy_free_block() function - fix multiple instances of KMEM_CACHE() entry v7: - fix warnings reported by kernel test robot <lkp@intel.com> - modify the license(Christian) v8: - fix warnings reported by kernel test robot <lkp@intel.com> Signed-off-by: Arunpravin <Arunpravin.PaneerSelvam@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220118104504.2349-1-Arunpravin.PaneerSelvam@amd.com Signed-off-by: Christian König <christian.koenig@amd.com>
2022-01-18 18:44:59 +08:00
/* SPDX-License-Identifier: MIT */
/*
* Copyright © 2021 Intel Corporation
*/
#ifndef __DRM_BUDDY_H__
#define __DRM_BUDDY_H__
#include <linux/bitops.h>
#include <linux/list.h>
#include <linux/slab.h>
#include <linux/sched.h>
#include <drm/drm_print.h>
#define range_overflows(start, size, max) ({ \
typeof(start) start__ = (start); \
typeof(size) size__ = (size); \
typeof(max) max__ = (max); \
(void)(&start__ == &size__); \
(void)(&start__ == &max__); \
start__ >= max__ || size__ > max__ - start__; \
})
drm: improve drm_buddy_alloc function - Make drm_buddy_alloc a single function to handle range allocation and non-range allocation demands - Implemented a new function alloc_range() which allocates the requested power-of-two block comply with range limitations - Moved order computation and memory alignment logic from i915 driver to drm buddy v2: merged below changes to keep the build unbroken - drm_buddy_alloc_range() becomes obsolete and may be removed - enable ttm range allocation (fpfn / lpfn) support in i915 driver - apply enhanced drm_buddy_alloc() function to i915 driver v3(Matthew Auld): - Fix alignment issues and remove unnecessary list_empty check - add more validation checks for input arguments - make alloc_range() block allocations as bottom-up - optimize order computation logic - replace uint64_t with u64, which is preferred in the kernel v4(Matthew Auld): - keep drm_buddy_alloc_range() function implementation for generic actual range allocations - keep alloc_range() implementation for end bias allocations v5(Matthew Auld): - modify drm_buddy_alloc() passing argument place->lpfn to lpfn as place->lpfn will currently always be zero for i915 v6(Matthew Auld): - fixup potential uaf - If we are unlucky and can't allocate enough memory when splitting blocks, where we temporarily end up with the given block and its buddy on the respective free list, then we need to ensure we delete both blocks, and no just the buddy, before potentially freeing them - fix warnings reported by kernel test robot <lkp@intel.com> v7(Matthew Auld): - revert fixup potential uaf - keep __alloc_range() add node to the list logic same as drm_buddy_alloc_blocks() by having a temporary list variable - at drm_buddy_alloc_blocks() keep i915 range_overflows macro and add a new check for end variable v8: - fix warnings reported by kernel test robot <lkp@intel.com> v9(Matthew Auld): - remove DRM_BUDDY_RANGE_ALLOCATION flag - remove unnecessary function description v10: - keep DRM_BUDDY_RANGE_ALLOCATION flag as removing the flag and replacing with (end < size) logic fails amdgpu driver load Signed-off-by: Arunpravin <Arunpravin.PaneerSelvam@amd.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Christian König <christian.koenig@amd.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220221164552.2434-1-Arunpravin.PaneerSelvam@amd.com
2022-02-22 00:45:48 +08:00
#define DRM_BUDDY_RANGE_ALLOCATION (1 << 0)
#define DRM_BUDDY_TOPDOWN_ALLOCATION (1 << 1)
drm: improve drm_buddy_alloc function - Make drm_buddy_alloc a single function to handle range allocation and non-range allocation demands - Implemented a new function alloc_range() which allocates the requested power-of-two block comply with range limitations - Moved order computation and memory alignment logic from i915 driver to drm buddy v2: merged below changes to keep the build unbroken - drm_buddy_alloc_range() becomes obsolete and may be removed - enable ttm range allocation (fpfn / lpfn) support in i915 driver - apply enhanced drm_buddy_alloc() function to i915 driver v3(Matthew Auld): - Fix alignment issues and remove unnecessary list_empty check - add more validation checks for input arguments - make alloc_range() block allocations as bottom-up - optimize order computation logic - replace uint64_t with u64, which is preferred in the kernel v4(Matthew Auld): - keep drm_buddy_alloc_range() function implementation for generic actual range allocations - keep alloc_range() implementation for end bias allocations v5(Matthew Auld): - modify drm_buddy_alloc() passing argument place->lpfn to lpfn as place->lpfn will currently always be zero for i915 v6(Matthew Auld): - fixup potential uaf - If we are unlucky and can't allocate enough memory when splitting blocks, where we temporarily end up with the given block and its buddy on the respective free list, then we need to ensure we delete both blocks, and no just the buddy, before potentially freeing them - fix warnings reported by kernel test robot <lkp@intel.com> v7(Matthew Auld): - revert fixup potential uaf - keep __alloc_range() add node to the list logic same as drm_buddy_alloc_blocks() by having a temporary list variable - at drm_buddy_alloc_blocks() keep i915 range_overflows macro and add a new check for end variable v8: - fix warnings reported by kernel test robot <lkp@intel.com> v9(Matthew Auld): - remove DRM_BUDDY_RANGE_ALLOCATION flag - remove unnecessary function description v10: - keep DRM_BUDDY_RANGE_ALLOCATION flag as removing the flag and replacing with (end < size) logic fails amdgpu driver load Signed-off-by: Arunpravin <Arunpravin.PaneerSelvam@amd.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Christian König <christian.koenig@amd.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220221164552.2434-1-Arunpravin.PaneerSelvam@amd.com
2022-02-22 00:45:48 +08:00
drm: move the buddy allocator from i915 into common drm Move the base i915 buddy allocator code into drm - Move i915_buddy.h to include/drm - Move i915_buddy.c to drm root folder - Rename "i915" string with "drm" string wherever applicable - Rename "I915" string with "DRM" string wherever applicable - Fix header file dependencies - Fix alignment issues - add Makefile support for drm buddy - export functions and write kerneldoc description - Remove i915 selftest config check condition as buddy selftest will be moved to drm selftest folder cleanup i915 buddy references in i915 driver module and replace with drm buddy v2: - include header file in alphabetical order(Thomas) - merged changes listed in the body section into a single patch to keep the build intact(Christian, Jani) v3: - make drm buddy a separate module(Thomas, Christian) v4: - Fix build error reported by kernel test robot <lkp@intel.com> - removed i915 buddy selftest from i915_mock_selftests.h to avoid build error - removed selftests/i915_buddy.c file as we create a new set of buddy test cases in drm/selftests folder v5: - Fix merge conflict issue v6: - replace drm_buddy_mm structure name as drm_buddy(Thomas, Christian) - replace drm_buddy_alloc() function name as drm_buddy_alloc_blocks() (Thomas) - replace drm_buddy_free() function name as drm_buddy_free_block() (Thomas) - export drm_buddy_free_block() function - fix multiple instances of KMEM_CACHE() entry v7: - fix warnings reported by kernel test robot <lkp@intel.com> - modify the license(Christian) v8: - fix warnings reported by kernel test robot <lkp@intel.com> Signed-off-by: Arunpravin <Arunpravin.PaneerSelvam@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220118104504.2349-1-Arunpravin.PaneerSelvam@amd.com Signed-off-by: Christian König <christian.koenig@amd.com>
2022-01-18 18:44:59 +08:00
struct drm_buddy_block {
#define DRM_BUDDY_HEADER_OFFSET GENMASK_ULL(63, 12)
#define DRM_BUDDY_HEADER_STATE GENMASK_ULL(11, 10)
#define DRM_BUDDY_ALLOCATED (1 << 10)
#define DRM_BUDDY_FREE (2 << 10)
#define DRM_BUDDY_SPLIT (3 << 10)
/* Free to be used, if needed in the future */
#define DRM_BUDDY_HEADER_UNUSED GENMASK_ULL(9, 6)
#define DRM_BUDDY_HEADER_ORDER GENMASK_ULL(5, 0)
u64 header;
struct drm_buddy_block *left;
struct drm_buddy_block *right;
struct drm_buddy_block *parent;
void *private; /* owned by creator */
/*
* While the block is allocated by the user through drm_buddy_alloc*,
* the user has ownership of the link, for example to maintain within
* a list, if so desired. As soon as the block is freed with
* drm_buddy_free* ownership is given back to the mm.
*/
struct list_head link;
struct list_head tmp_link;
};
/* Order-zero must be at least PAGE_SIZE */
#define DRM_BUDDY_MAX_ORDER (63 - PAGE_SHIFT)
/*
* Binary Buddy System.
*
* Locking should be handled by the user, a simple mutex around
* drm_buddy_alloc* and drm_buddy_free* should suffice.
*/
struct drm_buddy {
/* Maintain a free list for each order. */
struct list_head *free_list;
/*
* Maintain explicit binary tree(s) to track the allocation of the
* address space. This gives us a simple way of finding a buddy block
* and performing the potentially recursive merge step when freeing a
* block. Nodes are either allocated or free, in which case they will
* also exist on the respective free list.
*/
struct drm_buddy_block **roots;
/*
* Anything from here is public, and remains static for the lifetime of
* the mm. Everything above is considered do-not-touch.
*/
unsigned int n_roots;
unsigned int max_order;
/* Must be at least PAGE_SIZE */
u64 chunk_size;
u64 size;
u64 avail;
};
static inline u64
drm_buddy_block_offset(struct drm_buddy_block *block)
{
return block->header & DRM_BUDDY_HEADER_OFFSET;
}
static inline unsigned int
drm_buddy_block_order(struct drm_buddy_block *block)
{
return block->header & DRM_BUDDY_HEADER_ORDER;
}
static inline unsigned int
drm_buddy_block_state(struct drm_buddy_block *block)
{
return block->header & DRM_BUDDY_HEADER_STATE;
}
static inline bool
drm_buddy_block_is_allocated(struct drm_buddy_block *block)
{
return drm_buddy_block_state(block) == DRM_BUDDY_ALLOCATED;
}
static inline bool
drm_buddy_block_is_free(struct drm_buddy_block *block)
{
return drm_buddy_block_state(block) == DRM_BUDDY_FREE;
}
static inline bool
drm_buddy_block_is_split(struct drm_buddy_block *block)
{
return drm_buddy_block_state(block) == DRM_BUDDY_SPLIT;
}
static inline u64
drm_buddy_block_size(struct drm_buddy *mm,
struct drm_buddy_block *block)
{
return mm->chunk_size << drm_buddy_block_order(block);
}
int drm_buddy_init(struct drm_buddy *mm, u64 size, u64 chunk_size);
void drm_buddy_fini(struct drm_buddy *mm);
struct drm_buddy_block *
drm_get_buddy(struct drm_buddy_block *block);
drm: improve drm_buddy_alloc function - Make drm_buddy_alloc a single function to handle range allocation and non-range allocation demands - Implemented a new function alloc_range() which allocates the requested power-of-two block comply with range limitations - Moved order computation and memory alignment logic from i915 driver to drm buddy v2: merged below changes to keep the build unbroken - drm_buddy_alloc_range() becomes obsolete and may be removed - enable ttm range allocation (fpfn / lpfn) support in i915 driver - apply enhanced drm_buddy_alloc() function to i915 driver v3(Matthew Auld): - Fix alignment issues and remove unnecessary list_empty check - add more validation checks for input arguments - make alloc_range() block allocations as bottom-up - optimize order computation logic - replace uint64_t with u64, which is preferred in the kernel v4(Matthew Auld): - keep drm_buddy_alloc_range() function implementation for generic actual range allocations - keep alloc_range() implementation for end bias allocations v5(Matthew Auld): - modify drm_buddy_alloc() passing argument place->lpfn to lpfn as place->lpfn will currently always be zero for i915 v6(Matthew Auld): - fixup potential uaf - If we are unlucky and can't allocate enough memory when splitting blocks, where we temporarily end up with the given block and its buddy on the respective free list, then we need to ensure we delete both blocks, and no just the buddy, before potentially freeing them - fix warnings reported by kernel test robot <lkp@intel.com> v7(Matthew Auld): - revert fixup potential uaf - keep __alloc_range() add node to the list logic same as drm_buddy_alloc_blocks() by having a temporary list variable - at drm_buddy_alloc_blocks() keep i915 range_overflows macro and add a new check for end variable v8: - fix warnings reported by kernel test robot <lkp@intel.com> v9(Matthew Auld): - remove DRM_BUDDY_RANGE_ALLOCATION flag - remove unnecessary function description v10: - keep DRM_BUDDY_RANGE_ALLOCATION flag as removing the flag and replacing with (end < size) logic fails amdgpu driver load Signed-off-by: Arunpravin <Arunpravin.PaneerSelvam@amd.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Christian König <christian.koenig@amd.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220221164552.2434-1-Arunpravin.PaneerSelvam@amd.com
2022-02-22 00:45:48 +08:00
int drm_buddy_alloc_blocks(struct drm_buddy *mm,
u64 start, u64 end, u64 size,
u64 min_page_size,
struct list_head *blocks,
unsigned long flags);
drm: move the buddy allocator from i915 into common drm Move the base i915 buddy allocator code into drm - Move i915_buddy.h to include/drm - Move i915_buddy.c to drm root folder - Rename "i915" string with "drm" string wherever applicable - Rename "I915" string with "DRM" string wherever applicable - Fix header file dependencies - Fix alignment issues - add Makefile support for drm buddy - export functions and write kerneldoc description - Remove i915 selftest config check condition as buddy selftest will be moved to drm selftest folder cleanup i915 buddy references in i915 driver module and replace with drm buddy v2: - include header file in alphabetical order(Thomas) - merged changes listed in the body section into a single patch to keep the build intact(Christian, Jani) v3: - make drm buddy a separate module(Thomas, Christian) v4: - Fix build error reported by kernel test robot <lkp@intel.com> - removed i915 buddy selftest from i915_mock_selftests.h to avoid build error - removed selftests/i915_buddy.c file as we create a new set of buddy test cases in drm/selftests folder v5: - Fix merge conflict issue v6: - replace drm_buddy_mm structure name as drm_buddy(Thomas, Christian) - replace drm_buddy_alloc() function name as drm_buddy_alloc_blocks() (Thomas) - replace drm_buddy_free() function name as drm_buddy_free_block() (Thomas) - export drm_buddy_free_block() function - fix multiple instances of KMEM_CACHE() entry v7: - fix warnings reported by kernel test robot <lkp@intel.com> - modify the license(Christian) v8: - fix warnings reported by kernel test robot <lkp@intel.com> Signed-off-by: Arunpravin <Arunpravin.PaneerSelvam@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220118104504.2349-1-Arunpravin.PaneerSelvam@amd.com Signed-off-by: Christian König <christian.koenig@amd.com>
2022-01-18 18:44:59 +08:00
drm: implement a method to free unused pages On contiguous allocation, we round up the size to the *next* power of 2, implement a function to free the unused pages after the newly allocate block. v2(Matthew Auld): - replace function name 'drm_buddy_free_unused_pages' with drm_buddy_block_trim - replace input argument name 'actual_size' with 'new_size' - add more validation checks for input arguments - add overlaps check to avoid needless searching and splitting - merged the below patch to see the feature in action - add free unused pages support to i915 driver - lock drm_buddy_block_trim() function as it calls mark_free/mark_split are all globally visible v3(Matthew Auld): - remove trim method error handling as we address the failure case at drm_buddy_block_trim() function v4: - in case of trim, at __alloc_range() split_block failure path marks the block as free and removes it from the original list, potentially also freeing it, to overcome this problem, we turn the drm_buddy_block_trim() input node into a temporary node to prevent recursively freeing itself, but still retain the un-splitting/freeing of the other nodes(Matthew Auld) - modify the drm_buddy_block_trim() function return type v5(Matthew Auld): - revert drm_buddy_block_trim() function return type changes in v4 - modify drm_buddy_block_trim() passing argument n_pages to original_size as n_pages has already been rounded up to the next power-of-two and passing n_pages results noop v6: - fix warnings reported by kernel test robot <lkp@intel.com> v7: - modify drm_buddy_block_trim() function doc description - at drm_buddy_block_trim() handle non-allocated block as a serious programmer error - fix a typo Signed-off-by: Arunpravin <Arunpravin.PaneerSelvam@amd.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220221164552.2434-3-Arunpravin.PaneerSelvam@amd.com Signed-off-by: Christian König <christian.koenig@amd.com>
2022-02-22 00:45:50 +08:00
int drm_buddy_block_trim(struct drm_buddy *mm,
u64 new_size,
struct list_head *blocks);
drm: move the buddy allocator from i915 into common drm Move the base i915 buddy allocator code into drm - Move i915_buddy.h to include/drm - Move i915_buddy.c to drm root folder - Rename "i915" string with "drm" string wherever applicable - Rename "I915" string with "DRM" string wherever applicable - Fix header file dependencies - Fix alignment issues - add Makefile support for drm buddy - export functions and write kerneldoc description - Remove i915 selftest config check condition as buddy selftest will be moved to drm selftest folder cleanup i915 buddy references in i915 driver module and replace with drm buddy v2: - include header file in alphabetical order(Thomas) - merged changes listed in the body section into a single patch to keep the build intact(Christian, Jani) v3: - make drm buddy a separate module(Thomas, Christian) v4: - Fix build error reported by kernel test robot <lkp@intel.com> - removed i915 buddy selftest from i915_mock_selftests.h to avoid build error - removed selftests/i915_buddy.c file as we create a new set of buddy test cases in drm/selftests folder v5: - Fix merge conflict issue v6: - replace drm_buddy_mm structure name as drm_buddy(Thomas, Christian) - replace drm_buddy_alloc() function name as drm_buddy_alloc_blocks() (Thomas) - replace drm_buddy_free() function name as drm_buddy_free_block() (Thomas) - export drm_buddy_free_block() function - fix multiple instances of KMEM_CACHE() entry v7: - fix warnings reported by kernel test robot <lkp@intel.com> - modify the license(Christian) v8: - fix warnings reported by kernel test robot <lkp@intel.com> Signed-off-by: Arunpravin <Arunpravin.PaneerSelvam@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220118104504.2349-1-Arunpravin.PaneerSelvam@amd.com Signed-off-by: Christian König <christian.koenig@amd.com>
2022-01-18 18:44:59 +08:00
void drm_buddy_free_block(struct drm_buddy *mm, struct drm_buddy_block *block);
void drm_buddy_free_list(struct drm_buddy *mm, struct list_head *objects);
void drm_buddy_print(struct drm_buddy *mm, struct drm_printer *p);
void drm_buddy_block_print(struct drm_buddy *mm,
struct drm_buddy_block *block,
struct drm_printer *p);
#endif