It covers more than just ttm_bo_type_sg usage, like with say dma-buf,
since one other user is userptr in amdgpu, and in the future we might
have some more. Hence EXTERNAL is likely a more suitable name.
v2(Christian):
- Rename these to TTM_TT_FLAGS_*
- Fix up all the holes in the flag values
Suggested-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Christian König <christian.koenig@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210929132629.353541-1-matthew.auld@intel.com
Signed-off-by: Christian König <christian.koenig@amd.com>
__fls() on sparc64 return "int", but here it is expected as "unsigned
long" (x86). It will cause the build errors because the warning becomes
fatal while it is using sparc configuration. As suggested by Linus, it
can use min_t instead of min to force the type as "unsigned int".
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Huang Rui <ray.huang@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210907100302.3684453-1-ray.huang@amd.com
Switch back to using a spinlock again by moving the IOMMU unmap outside
of the locked region.
This avoids contention especially while freeing pages.
v2: Add a comment explaining why we need sync_shrinkers().
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210820120528.81114-3-christian.koenig@amd.com
The old implementation wasn't consistend on this.
But it looks like we depend on this so better bring it back.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reported-and-tested-by: Mike Galbraith <efault@gmx.de>
Fixes: d099fc8f54 ("drm/ttm: new TT backend allocation pool v3")
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210210160549.1462-1-christian.koenig@amd.com
This is just another feature which is only used by VMWGFX, so move
it into the driver instead.
I've tried to add the accounting sysfs file to the kobject of the drm
minor, but I'm not 100% sure if this works as expected.
v2: fix typo in KFD and avoid 64bit divide
v3: fix init order in VMWGFX
v4: use pdev sysfs reference instead of drm
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Zack Rusin <zackr@vmware.com> (v3)
Tested-by: Nirmoy Das <nirmoy.das@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210208133226.36955-2-christian.koenig@amd.com
Only initialize the DMA coherent pools if they are used.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/414957/
Useful for testing.
v2: add fs_reclaim_acquire()/_release()
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/414955/
Instead of printing this on the per device pool.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/414956/
The only flag we really need is __GFP_NOMEMALLOC, highmem depends on
dma32 and moveable/compound should never be set in the first place.
Signed-off-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/413812/
Link: https://patchwork.freedesktop.org/patch/413964/
Fixes: d099fc8f54 ("drm/ttm: new TT backend allocation pool v3")
Reported-by: Hans de Goede <hdegoede@redhat.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
set_pages_wb() might sleep and so we can't do this in an atomic context.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reported-by: Mikhail Gavrilov <mikhail.v.gavrilov@gmail.com>
Tested-by: Mikhail Gavrilov <mikhail.v.gavrilov@gmail.com>
Fixes: d099fc8f54 ("drm/ttm: new TT backend allocation pool v3")
Reviewed-by: Huang Rui <ray.huang@amd.com>
Link: https://patchwork.freedesktop.org/patch/413409/
check_unmap() is producing a warning about a missing map error check.
The return value from dma_map_page() should be checked for an error, not
the caller-provided dma_addr.
Fixes: d099fc8f54 ("drm/ttm: new TT backend allocation pool v3")
Signed-off-by: Jeremy Cline <jcline@redhat.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/413432/
Signed-off-by: Christian König <christian.koenig@amd.com>
Drivers are not supposed to use this directly any more.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Link: https://patchwork.freedesktop.org/patch/412156/
ttm_pool_type_count() is not used when debugfs is disabled:
drivers/gpu/drm/ttm/ttm_pool.c:243:21: error: unused function 'ttm_pool_type_count' [-Werror,-Wunused-function]
static unsigned int ttm_pool_type_count(struct ttm_pool_type *pt)
Move the definition into the #ifdef block.
Fixes: d099fc8f54 ("drm/ttm: new TT backend allocation pool v3")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Martin Peres <martin.peres@mupuf.org>
Reviewed-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/405695/
Signed-off-by: Christian König <christian.koenig@amd.com>
When we have mixed DMA32 and non DMA32 device in one system
it could otherwise happen that the DMA32 device gets pages
it can't work with.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Link: https://patchwork.freedesktop.org/patch/401317/
The pool parameter can be NULL if we free through the shrinker.
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Tested-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Reported-by: Andy Lavr <andy.lavr@gmail.com>
Tested-by: Andy Lavr <andy.lavr@gmail.com>
Link: https://patchwork.freedesktop.org/patch/399365/
During eviction we do want to trigger the OOM killer.
Only while doing new allocations we should try to avoid that and
return -ENOMEM to the application.
v2: rename the flag to gfp_retry_mayfail.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/398685/
This replaces the spaghetti code in the two existing page pools.
First of all depending on the allocation size it is between 3 (1GiB) and
5 (1MiB) times faster than the old implementation.
It makes better use of buddy pages to allow for larger physical contiguous
allocations which should result in better TLB utilization at least for
amdgpu.
Instead of a completely braindead approach of filling the pool with one
CPU while another one is trying to shrink it we only give back freed
pages.
This also results in much less locking contention and a trylock free MM
shrinker callback, so we can guarantee that pages are given back to the
system when needed.
Downside of this is that it takes longer for many small allocations until
the pool is filled up. We could address this, but I couldn't find an use
case where this actually matters. We also don't bother freeing large
chunks of pages any more since the CPU overhead in that path isn't really
that important.
The sysfs files are replaced with a single module parameter, allowing
users to override how many pages should be globally pooled in TTM. This
unfortunately breaks the UAPI slightly, but as far as we know nobody ever
depended on this.
Zeroing memory coming from the pool was handled inconsistently. The
alloc_pages() based pool was zeroing it, the dma_alloc_attr() based one
wasn't. For now the new implementation isn't zeroing pages from the pool
either and only sets the __GFP_ZERO flag when necessary.
The implementation has only 768 lines of code compared to the over 2600
of the old one, and also allows for saving quite a bunch of code in the
drivers since we don't need specialized handling there any more based on
kernel config.
Additional to all of that there was a neat bug with IOMMU, coherent DMA
mappings and huge pages which is now fixed in the new code as well.
v2: make ttm_pool_apply_caching static as reported by the kernel bot, add
some more checks
v3: fix some more checkpatch.pl warnings
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Madhav Chauhan <madhav.chauhan@amd.com>
Tested-by: Huang Rui <ray.huang@amd.com>
Link: https://patchwork.freedesktop.org/patch/397080/?series=83051&rev=1
Matthew Auld