This aids handling buffers moves with the scheduler.
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian K?nig <christian.koenig@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian K?nig <christian.koenig@amd.com>
Simply not used any more. Only keep 32bit atomic for fence sequence numbering.
v2: trivial rebase
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com> (v1)
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com> (v1)
Reviewed-by: Chunming Zhou <david1.zhou@amd.com> (v1)
Rename the function and update the related code with this modified function.
Add the new parameter of bool wait_all.
If wait_all is true, it will return when all fences are signaled or timeout.
If wait_all is false, it will return when any fence is signaled or timeout.
Signed-off-by: Junwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: Monk Liu <monk.liu@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Pagetables can be moved and therefore the page directory update can be necessary
for the current cs even if none of the the bo's are moved. In that scenario
there is no fence between the sdma0 and gfx ring, so we add one.
v2 (chk): rebased
Signed-off-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
It was just a wrapper for fence_wait anyway.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
The common kernel function does the same thing.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
scheduler fence is based on kernel fence framework.
v2: squash in Christian's build fix
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian K?nig <christian.koenig@amd.com>
Avoiding a couple of casts.
v2: rename c_entity to entity as well
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Cleanup the kernel context handling.
v2: rebased
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com> (v1)
We didn't initialized the mutex in the cloned bo list resulting in nice
warnings from lockdep. Also fixes error handling in this function.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
This way can avoid interrupt lost, and can process sched job exactly.
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
every sbumission should be able to get a fence.
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian K?nig <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <jammy.zhou@amd.com>
It is theoretically possible that a swapped out BO gets the
same GTT address, but different backing pages while being swapped in.
Instead just use another VA state to note updated areas.
Ported from not upstream yet radeon commit with the same name.
v2: fix some bugs in the original implementation found in the radeon code.
v3: squash in VCE/UVD fix
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
thus unnecessary wake up could be avoid between rings
v2:
move wait_queue_head to fence_drv from ring
Signed-off-by: monk.liu <monk.liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
origninal method will sleep/schedule at the granurarity of HZ/2 and
based on seq signal method, the new implement is based on kernel fance
interface, no unnecessary schedule at all
v2: replace logic of original amdgpu_fence_wait_any
Signed-off-by: monk.liu <monk.liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
if enabling scheduler, then the queued seq is assigned
when pushing job before emitting job.
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian K?nig <christian.koenig@amd.com>
This option can be used to specify the max number of submissions in the
active HW queue. The default value is 2 now.
Signed-off-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
This option can be used to specify the max job number in the job queue,
and it is 16 by default.
Signed-off-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
fence_process may be called from kthread, user thread and interrupt context.
it is possible to called concurrently, then will wake up fence queue multiple times.
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
use kernel context to submit command for vm
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Acked-by: Christian K?nig <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
user mode will still use pte ring as a normal ring.
if the prepare job generates another command(update pte) on its ring in scheduler,
then will kill scheduler which is going to waiting later job but pending running job.
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Acked-by: Christian K?nig <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
BO validation is moved to scheduler except usrptr which must be validated
in user process
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Acked-by: Christian K?nig <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Make the definitions common for all driver components
v2: fix kfd
Signed-off-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
We need to return the sequence number to userspace
even when we don't use user fences.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
This makes assigning VM IDs independent from the use of VM IDs.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
It's not used any more.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
This is a prerequisite for the GPU scheduler to make the order
of submission independent from the order of execution.
v2: properly implement the locking
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
This also requires some error handling from the callers of that function.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
The comment is misleading and incorrect, remove it.
Printing the id is completely meaningless and this practice
can cause a race conditions on command submission.
The flags and hangs fields are completely unused.
Give all fields a common indentation.
v2: remove fpriv reference and unused flags as well, fix debug message.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
CGS (Common Graphics Services) is an AMD cross component
abstraction layer to designed to better encapsulate
specific IP block drivers so different teams can effectively
work on differnet IP block drivers independently. It provides
a common interface for things like accessing registers,
allocating GPU memory, and registering interrupt sources.
The plan is to eventually move more and more IP drivers to
this interface. The first user is the ACP IP driver.
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Expose feature version to user space for RLC/MEC/MEC2 ucode as well
v2: fix coding style
Signed-off-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
If we fail during device init, record what state each
block is in so that we can tear down clearly.
Fixes various problems on device init failure.
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
For boards with bad VCE blocks, only configure the working
block.
v2: use the harvest info for pipe setup
v3: fix mask check as noted by Leo
v4: add dGPU support
Reviewed-by: Christian König <christian.koenig@amd.com> (v2)
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This patch adds an interface file between amdgpu and amdkfd. This
interface file is H/W agnostic, thus containing functions that
operate the same for any AMD APU/GPU H/W generation.
The functions in this interface mirror (some) of the functions in
radeon_kfd.c (the radeon<-->amdkfd interface file). The main functions
are:
- amdgpu_amdkfd_init - initialize the amdkfd module
- amdgpu_amdkfd_load_interface - load the H/W interface according to the
currently probed device
- amdgpu_amdkfd_device_probe - probe the device in amdkfd
- amdgpu_amdkfd_device_init - initialize the device in amdkfd
- amdgpu_amdkfd_interrupt - call the ISR of amdkfd
- amdgpu_amdkfd_suspend - suspend callback from amdgpu
- amdgpu_amdkfd_resume - resume callback from amdgpu
This patch also modifies the relevant amdgpu files, to use this new
interface.
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
And use common fence infrastructure for the wait.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Enable VCE dpm and powergating. VCE dpm dynamically scales the VCE clocks on
demand.
Signed-off-by: Sonny Jiang <sonny.jiang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
amdgpu submits both kernel and user fences, but just need one interrupt,
disable user fence interrupt and don't effect user fence.
v2: fix merge error
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_cs_find_mapping doesn't work without all buffers being validated,
so the TTM validation must be done first.
v2: only use amdgpu_cs_find_mapping for UVD/VCE VM emulation
Signed-off-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Track the type of vram on the board and provide a query for it.
User mode drivers and tools want this information for determining
bandwidth information and form informational purposes.
v2: fix build when CI support is not enabled
Signed-off-by: Ken Wang <Qingqing.Wang@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Provide this information to usermode drivers. We were previously
missing this info.
Signed-off-by: Ken Wang <Qingqing.Wang@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Add a query for the CE ram size. User mode drivers
will want to use this to determine how much size
of the cache on the CE.
Signed-off-by: Ken Wang <Qingqing.Wang@amd.com>
Reviewd-by: Jammy Zhou <Jammy.Zhou@amd.com>
bo_va may un-initialized, fix it.
Signed-off-by: monk.liu <monk.liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <jammy.zhou@amd.com>
Pass a ttm_placement pointer to amdgpu_bo_create_restricted
add min_offset to amdgpu_bo_pin_restricted. This makes it
easier to allocate memory with address restrictions. With
this patch we can also enable 2-ended allocation again.
v2: fix rebase conflicts
v3: memset placements before using
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
The structure is renamed and moved to amd_shared.h to make
the component independent. This makes it easier to add
new components in the future.
v2: fix include path
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Signed-off-by: yanyang1 <young.yang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
It's causing issues with VMID handling and comparing the
fence value two times actually doesn't make handling faster.
Port of radeon commit "d6d5c5b8364bcc4d52cddc68bcb0a330d2af20f3".
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Monk Liu <monk.liu@amd.com>
Firmware required BO size changes in terms of ASIC family
Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Properly protect the state and also handle submission failures.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Monk Liu <monk.liu@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Monk Liu <monk.liu@amd.com>
check the filp is not robust, and sometimes different contexts may
have same filp value.
v2: check both filp and ctx_id
Signed-off-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
It is required to support fence per context.
v2: add amdgpu_ctx_get/put
v3: improve get/put
v4: squash hlock fix
Signed-off-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
The existing locks were protecting the list, but not the elements.
v2: rename hlock to lock
Signed-off-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
This is the main header file for amdgpu.
v2: remove stable comments
Acked-by: Christian König <christian.koenig@amd.com>
Acked-by: Jammy Zhou <Jammy.Zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Chunming Zhou