2009-06-05 20:42:42 +08:00
|
|
|
/*
|
|
|
|
* Copyright 2008 Advanced Micro Devices, Inc.
|
|
|
|
* Copyright 2008 Red Hat Inc.
|
|
|
|
* Copyright 2009 Jerome Glisse.
|
|
|
|
*
|
|
|
|
* Permission is hereby granted, free of charge, to any person obtaining a
|
|
|
|
* copy of this software and associated documentation files (the "Software"),
|
|
|
|
* to deal in the Software without restriction, including without limitation
|
|
|
|
* the rights to use, copy, modify, merge, publish, distribute, sublicense,
|
|
|
|
* and/or sell copies of the Software, and to permit persons to whom the
|
|
|
|
* Software is furnished to do so, subject to the following conditions:
|
|
|
|
*
|
|
|
|
* The above copyright notice and this permission notice shall be included in
|
|
|
|
* all copies or substantial portions of the Software.
|
|
|
|
*
|
|
|
|
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
|
|
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
|
|
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
|
|
|
|
* THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
|
|
|
|
* OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
|
|
|
|
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
|
|
|
* OTHER DEALINGS IN THE SOFTWARE.
|
|
|
|
*
|
|
|
|
* Authors: Dave Airlie
|
|
|
|
* Alex Deucher
|
|
|
|
* Jerome Glisse
|
2012-05-09 21:34:58 +08:00
|
|
|
* Christian König
|
2009-06-05 20:42:42 +08:00
|
|
|
*/
|
|
|
|
#include <linux/seq_file.h>
|
include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.
percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.
http://userweb.kernel.org/~tj/misc/slabh-sweep.py
The script does the followings.
* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.
* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.
* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.
The conversion was done in the following steps.
1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.
2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.
3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.
4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.
5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.
6. percpu.h was updated not to include slab.h.
7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).
* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig
8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.
Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
2010-03-24 16:04:11 +08:00
|
|
|
#include <linux/slab.h>
|
2012-10-03 01:01:07 +08:00
|
|
|
#include <drm/drmP.h>
|
|
|
|
#include <drm/radeon_drm.h>
|
2009-06-05 20:42:42 +08:00
|
|
|
#include "radeon_reg.h"
|
|
|
|
#include "radeon.h"
|
|
|
|
#include "atom.h"
|
|
|
|
|
2012-05-09 21:34:58 +08:00
|
|
|
/*
|
2012-07-18 02:02:38 +08:00
|
|
|
* IB
|
|
|
|
* IBs (Indirect Buffers) and areas of GPU accessible memory where
|
|
|
|
* commands are stored. You can put a pointer to the IB in the
|
|
|
|
* command ring and the hw will fetch the commands from the IB
|
|
|
|
* and execute them. Generally userspace acceleration drivers
|
|
|
|
* produce command buffers which are send to the kernel and
|
|
|
|
* put in IBs for execution by the requested ring.
|
2012-05-09 21:34:58 +08:00
|
|
|
*/
|
2012-09-01 01:43:50 +08:00
|
|
|
static int radeon_debugfs_sa_init(struct radeon_device *rdev);
|
2009-06-05 20:42:42 +08:00
|
|
|
|
2012-07-18 02:02:38 +08:00
|
|
|
/**
|
|
|
|
* radeon_ib_get - request an IB (Indirect Buffer)
|
|
|
|
*
|
|
|
|
* @rdev: radeon_device pointer
|
|
|
|
* @ring: ring index the IB is associated with
|
|
|
|
* @ib: IB object returned
|
|
|
|
* @size: requested IB size
|
|
|
|
*
|
|
|
|
* Request an IB (all asics). IBs are allocated using the
|
|
|
|
* suballocator.
|
|
|
|
* Returns 0 on success, error on failure.
|
|
|
|
*/
|
2011-12-22 01:13:46 +08:00
|
|
|
int radeon_ib_get(struct radeon_device *rdev, int ring,
|
2012-08-07 00:57:44 +08:00
|
|
|
struct radeon_ib *ib, struct radeon_vm *vm,
|
|
|
|
unsigned size)
|
2009-06-05 20:42:42 +08:00
|
|
|
{
|
2012-05-10 22:46:43 +08:00
|
|
|
int i, r;
|
drm/radeon: introduce a sub allocator and convert ib pool to it v4
Somewhat specializaed sub-allocator designed to perform sub-allocation
for command buffer not only for current cs ioctl but for future command
submission ioctl as well. Patch also convert current ib pool to use
the sub allocator. Idea is that ib poll buffer can be share with other
command buffer submission not having 64K granularity.
v2 Harmonize pool handling and add suspend/resume callback to pin/unpin
sa bo (tested on rv280, rv370, r420, rv515, rv610, rv710, redwood, cayman,
rs480, rs690, rs880)
v3 Simplify allocator
v4 Fix radeon_ib_get error path to properly free fence
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
2011-11-16 00:48:34 +08:00
|
|
|
|
2012-05-09 21:35:02 +08:00
|
|
|
r = radeon_sa_bo_new(rdev, &rdev->ring_tmp_bo, &ib->sa_bo, size, 256, true);
|
2012-05-09 21:34:58 +08:00
|
|
|
if (r) {
|
|
|
|
dev_err(rdev->dev, "failed to get a new IB (%d)\n", r);
|
|
|
|
return r;
|
drm/radeon: introduce a sub allocator and convert ib pool to it v4
Somewhat specializaed sub-allocator designed to perform sub-allocation
for command buffer not only for current cs ioctl but for future command
submission ioctl as well. Patch also convert current ib pool to use
the sub allocator. Idea is that ib poll buffer can be share with other
command buffer submission not having 64K granularity.
v2 Harmonize pool handling and add suspend/resume callback to pin/unpin
sa bo (tested on rv280, rv370, r420, rv515, rv610, rv710, redwood, cayman,
rs480, rs690, rs880)
v3 Simplify allocator
v4 Fix radeon_ib_get error path to properly free fence
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
2011-11-16 00:48:34 +08:00
|
|
|
}
|
2012-05-09 21:34:58 +08:00
|
|
|
|
2012-05-10 22:46:43 +08:00
|
|
|
r = radeon_semaphore_create(rdev, &ib->semaphore);
|
|
|
|
if (r) {
|
|
|
|
return r;
|
|
|
|
}
|
|
|
|
|
2012-05-08 20:24:01 +08:00
|
|
|
ib->ring = ring;
|
|
|
|
ib->fence = NULL;
|
2012-05-09 21:35:02 +08:00
|
|
|
ib->ptr = radeon_sa_bo_cpu_addr(ib->sa_bo);
|
2012-08-07 00:57:44 +08:00
|
|
|
ib->vm = vm;
|
|
|
|
if (vm) {
|
2012-09-11 22:09:59 +08:00
|
|
|
/* ib pool is bound at RADEON_VA_IB_OFFSET in virtual address
|
|
|
|
* space and soffset is the offset inside the pool bo
|
2012-08-07 00:57:44 +08:00
|
|
|
*/
|
2012-09-11 22:09:59 +08:00
|
|
|
ib->gpu_addr = ib->sa_bo->soffset + RADEON_VA_IB_OFFSET;
|
2012-08-07 00:57:44 +08:00
|
|
|
} else {
|
|
|
|
ib->gpu_addr = radeon_sa_bo_gpu_addr(ib->sa_bo);
|
|
|
|
}
|
2012-05-09 21:35:02 +08:00
|
|
|
ib->is_const_ib = false;
|
2012-05-10 22:46:43 +08:00
|
|
|
for (i = 0; i < RADEON_NUM_RINGS; ++i)
|
|
|
|
ib->sync_to[i] = NULL;
|
2012-05-09 21:34:58 +08:00
|
|
|
|
|
|
|
return 0;
|
2009-06-05 20:42:42 +08:00
|
|
|
}
|
|
|
|
|
2012-07-18 02:02:38 +08:00
|
|
|
/**
|
|
|
|
* radeon_ib_free - free an IB (Indirect Buffer)
|
|
|
|
*
|
|
|
|
* @rdev: radeon_device pointer
|
|
|
|
* @ib: IB object to free
|
|
|
|
*
|
|
|
|
* Free an IB (all asics).
|
|
|
|
*/
|
2012-05-09 21:35:02 +08:00
|
|
|
void radeon_ib_free(struct radeon_device *rdev, struct radeon_ib *ib)
|
2009-06-05 20:42:42 +08:00
|
|
|
{
|
2012-05-10 22:46:43 +08:00
|
|
|
radeon_semaphore_free(rdev, &ib->semaphore, ib->fence);
|
2012-05-09 21:35:02 +08:00
|
|
|
radeon_sa_bo_free(rdev, &ib->sa_bo, ib->fence);
|
|
|
|
radeon_fence_unref(&ib->fence);
|
2009-06-05 20:42:42 +08:00
|
|
|
}
|
|
|
|
|
2013-02-02 00:32:42 +08:00
|
|
|
/**
|
|
|
|
* radeon_ib_sync_to - sync to fence before executing the IB
|
|
|
|
*
|
|
|
|
* @ib: IB object to add fence to
|
|
|
|
* @fence: fence to sync to
|
|
|
|
*
|
|
|
|
* Sync to the fence before executing the IB
|
|
|
|
*/
|
|
|
|
void radeon_ib_sync_to(struct radeon_ib *ib, struct radeon_fence *fence)
|
|
|
|
{
|
|
|
|
struct radeon_fence *other;
|
|
|
|
|
|
|
|
if (!fence)
|
|
|
|
return;
|
|
|
|
|
|
|
|
other = ib->sync_to[fence->ring];
|
|
|
|
ib->sync_to[fence->ring] = radeon_fence_later(fence, other);
|
|
|
|
}
|
|
|
|
|
2012-07-18 02:02:38 +08:00
|
|
|
/**
|
|
|
|
* radeon_ib_schedule - schedule an IB (Indirect Buffer) on the ring
|
|
|
|
*
|
|
|
|
* @rdev: radeon_device pointer
|
|
|
|
* @ib: IB object to schedule
|
|
|
|
* @const_ib: Const IB to schedule (SI only)
|
|
|
|
*
|
|
|
|
* Schedule an IB on the associated ring (all asics).
|
|
|
|
* Returns 0 on success, error on failure.
|
|
|
|
*
|
|
|
|
* On SI, there are two parallel engines fed from the primary ring,
|
|
|
|
* the CE (Constant Engine) and the DE (Drawing Engine). Since
|
|
|
|
* resource descriptors have moved to memory, the CE allows you to
|
|
|
|
* prime the caches while the DE is updating register state so that
|
|
|
|
* the resource descriptors will be already in cache when the draw is
|
|
|
|
* processed. To accomplish this, the userspace driver submits two
|
|
|
|
* IBs, one for the CE and one for the DE. If there is a CE IB (called
|
|
|
|
* a CONST_IB), it will be put on the ring prior to the DE IB. Prior
|
|
|
|
* to SI there was just a DE IB.
|
|
|
|
*/
|
2012-07-13 19:06:00 +08:00
|
|
|
int radeon_ib_schedule(struct radeon_device *rdev, struct radeon_ib *ib,
|
|
|
|
struct radeon_ib *const_ib)
|
2009-06-05 20:42:42 +08:00
|
|
|
{
|
2012-05-08 20:24:01 +08:00
|
|
|
struct radeon_ring *ring = &rdev->ring[ib->ring];
|
2012-05-10 22:46:43 +08:00
|
|
|
bool need_sync = false;
|
|
|
|
int i, r = 0;
|
2009-06-05 20:42:42 +08:00
|
|
|
|
2011-10-23 18:56:27 +08:00
|
|
|
if (!ib->length_dw || !ring->ready) {
|
2009-06-05 20:42:42 +08:00
|
|
|
/* TODO: Nothings in the ib we should report. */
|
2012-05-09 21:34:58 +08:00
|
|
|
dev_err(rdev->dev, "couldn't schedule ib\n");
|
2009-06-05 20:42:42 +08:00
|
|
|
return -EINVAL;
|
|
|
|
}
|
2009-09-15 09:12:56 +08:00
|
|
|
|
2009-06-29 16:29:13 +08:00
|
|
|
/* 64 dwords should be enough for fence too */
|
2012-05-10 22:46:43 +08:00
|
|
|
r = radeon_ring_lock(rdev, ring, 64 + RADEON_NUM_RINGS * 8);
|
2009-06-05 20:42:42 +08:00
|
|
|
if (r) {
|
2012-05-09 21:34:58 +08:00
|
|
|
dev_err(rdev->dev, "scheduling IB failed (%d).\n", r);
|
2009-06-05 20:42:42 +08:00
|
|
|
return r;
|
|
|
|
}
|
2012-05-10 22:46:43 +08:00
|
|
|
for (i = 0; i < RADEON_NUM_RINGS; ++i) {
|
|
|
|
struct radeon_fence *fence = ib->sync_to[i];
|
|
|
|
if (radeon_fence_need_sync(fence, ib->ring)) {
|
|
|
|
need_sync = true;
|
|
|
|
radeon_semaphore_sync_rings(rdev, ib->semaphore,
|
|
|
|
fence->ring, ib->ring);
|
|
|
|
radeon_fence_note_sync(fence, ib->ring);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
/* immediately free semaphore when we don't need to sync */
|
|
|
|
if (!need_sync) {
|
|
|
|
radeon_semaphore_free(rdev, &ib->semaphore, NULL);
|
|
|
|
}
|
2012-08-08 18:22:43 +08:00
|
|
|
/* if we can't remember our last VM flush then flush now! */
|
|
|
|
if (ib->vm && !ib->vm->last_flush) {
|
2012-10-03 02:43:38 +08:00
|
|
|
radeon_ring_vm_flush(rdev, ib->ring, ib->vm);
|
2012-08-08 18:22:43 +08:00
|
|
|
}
|
2012-07-13 19:06:00 +08:00
|
|
|
if (const_ib) {
|
|
|
|
radeon_ring_ib_execute(rdev, const_ib->ring, const_ib);
|
|
|
|
radeon_semaphore_free(rdev, &const_ib->semaphore, NULL);
|
|
|
|
}
|
2012-05-08 20:24:01 +08:00
|
|
|
radeon_ring_ib_execute(rdev, ib->ring, ib);
|
|
|
|
r = radeon_fence_emit(rdev, &ib->fence, ib->ring);
|
|
|
|
if (r) {
|
|
|
|
dev_err(rdev->dev, "failed to emit fence for new IB (%d)\n", r);
|
|
|
|
radeon_ring_unlock_undo(rdev, ring);
|
|
|
|
return r;
|
|
|
|
}
|
2012-07-13 19:06:00 +08:00
|
|
|
if (const_ib) {
|
|
|
|
const_ib->fence = radeon_fence_ref(ib->fence);
|
|
|
|
}
|
2012-08-08 18:22:43 +08:00
|
|
|
/* we just flushed the VM, remember that */
|
|
|
|
if (ib->vm && !ib->vm->last_flush) {
|
|
|
|
ib->vm->last_flush = radeon_fence_ref(ib->fence);
|
|
|
|
}
|
2011-10-23 18:56:27 +08:00
|
|
|
radeon_ring_unlock_commit(rdev, ring);
|
2009-06-05 20:42:42 +08:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2012-07-18 02:02:38 +08:00
|
|
|
/**
|
|
|
|
* radeon_ib_pool_init - Init the IB (Indirect Buffer) pool
|
|
|
|
*
|
|
|
|
* @rdev: radeon_device pointer
|
|
|
|
*
|
|
|
|
* Initialize the suballocator to manage a pool of memory
|
|
|
|
* for use as IBs (all asics).
|
|
|
|
* Returns 0 on success, error on failure.
|
|
|
|
*/
|
2009-06-05 20:42:42 +08:00
|
|
|
int radeon_ib_pool_init(struct radeon_device *rdev)
|
|
|
|
{
|
2012-05-09 21:34:58 +08:00
|
|
|
int r;
|
2009-06-05 20:42:42 +08:00
|
|
|
|
2012-05-09 21:34:58 +08:00
|
|
|
if (rdev->ib_pool_ready) {
|
2012-01-25 01:08:52 +08:00
|
|
|
return 0;
|
|
|
|
}
|
2012-05-09 21:34:58 +08:00
|
|
|
r = radeon_sa_bo_manager_init(rdev, &rdev->ring_tmp_bo,
|
2012-05-09 21:34:56 +08:00
|
|
|
RADEON_IB_POOL_SIZE*64*1024,
|
|
|
|
RADEON_GEM_DOMAIN_GTT);
|
|
|
|
if (r) {
|
|
|
|
return r;
|
|
|
|
}
|
2012-07-05 17:55:34 +08:00
|
|
|
|
|
|
|
r = radeon_sa_bo_manager_start(rdev, &rdev->ring_tmp_bo);
|
|
|
|
if (r) {
|
|
|
|
return r;
|
|
|
|
}
|
|
|
|
|
2012-05-09 21:34:58 +08:00
|
|
|
rdev->ib_pool_ready = true;
|
|
|
|
if (radeon_debugfs_sa_init(rdev)) {
|
|
|
|
dev_err(rdev->dev, "failed to register debugfs file for SA\n");
|
2009-06-05 20:42:42 +08:00
|
|
|
}
|
drm/radeon: introduce a sub allocator and convert ib pool to it v4
Somewhat specializaed sub-allocator designed to perform sub-allocation
for command buffer not only for current cs ioctl but for future command
submission ioctl as well. Patch also convert current ib pool to use
the sub allocator. Idea is that ib poll buffer can be share with other
command buffer submission not having 64K granularity.
v2 Harmonize pool handling and add suspend/resume callback to pin/unpin
sa bo (tested on rv280, rv370, r420, rv515, rv610, rv710, redwood, cayman,
rs480, rs690, rs880)
v3 Simplify allocator
v4 Fix radeon_ib_get error path to properly free fence
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
2011-11-16 00:48:34 +08:00
|
|
|
return 0;
|
2009-06-05 20:42:42 +08:00
|
|
|
}
|
|
|
|
|
2012-07-18 02:02:38 +08:00
|
|
|
/**
|
|
|
|
* radeon_ib_pool_fini - Free the IB (Indirect Buffer) pool
|
|
|
|
*
|
|
|
|
* @rdev: radeon_device pointer
|
|
|
|
*
|
|
|
|
* Tear down the suballocator managing the pool of memory
|
|
|
|
* for use as IBs (all asics).
|
|
|
|
*/
|
2009-06-05 20:42:42 +08:00
|
|
|
void radeon_ib_pool_fini(struct radeon_device *rdev)
|
|
|
|
{
|
2012-05-09 21:34:58 +08:00
|
|
|
if (rdev->ib_pool_ready) {
|
2012-07-05 17:55:34 +08:00
|
|
|
radeon_sa_bo_manager_suspend(rdev, &rdev->ring_tmp_bo);
|
2012-05-09 21:34:58 +08:00
|
|
|
radeon_sa_bo_manager_fini(rdev, &rdev->ring_tmp_bo);
|
|
|
|
rdev->ib_pool_ready = false;
|
2009-06-05 20:42:42 +08:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2012-07-18 02:02:38 +08:00
|
|
|
/**
|
|
|
|
* radeon_ib_ring_tests - test IBs on the rings
|
|
|
|
*
|
|
|
|
* @rdev: radeon_device pointer
|
|
|
|
*
|
|
|
|
* Test an IB (Indirect Buffer) on each ring.
|
|
|
|
* If the test fails, disable the ring.
|
|
|
|
* Returns 0 on success, error if the primary GFX ring
|
|
|
|
* IB test fails.
|
|
|
|
*/
|
2012-05-02 21:11:12 +08:00
|
|
|
int radeon_ib_ring_tests(struct radeon_device *rdev)
|
|
|
|
{
|
|
|
|
unsigned i;
|
|
|
|
int r;
|
|
|
|
|
|
|
|
for (i = 0; i < RADEON_NUM_RINGS; ++i) {
|
|
|
|
struct radeon_ring *ring = &rdev->ring[i];
|
|
|
|
|
|
|
|
if (!ring->ready)
|
|
|
|
continue;
|
|
|
|
|
|
|
|
r = radeon_ib_test(rdev, i, ring);
|
|
|
|
if (r) {
|
|
|
|
ring->ready = false;
|
|
|
|
|
|
|
|
if (i == RADEON_RING_TYPE_GFX_INDEX) {
|
|
|
|
/* oh, oh, that's really bad */
|
|
|
|
DRM_ERROR("radeon: failed testing IB on GFX ring (%d).\n", r);
|
|
|
|
rdev->accel_working = false;
|
|
|
|
return r;
|
|
|
|
|
|
|
|
} else {
|
|
|
|
/* still not good, but we can live with it */
|
|
|
|
DRM_ERROR("radeon: failed testing IB on ring %d (%d).\n", i, r);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2009-06-05 20:42:42 +08:00
|
|
|
/*
|
2012-07-18 02:02:38 +08:00
|
|
|
* Rings
|
|
|
|
* Most engines on the GPU are fed via ring buffers. Ring
|
|
|
|
* buffers are areas of GPU accessible memory that the host
|
|
|
|
* writes commands into and the GPU reads commands out of.
|
|
|
|
* There is a rptr (read pointer) that determines where the
|
|
|
|
* GPU is currently reading, and a wptr (write pointer)
|
|
|
|
* which determines where the host has written. When the
|
|
|
|
* pointers are equal, the ring is idle. When the host
|
|
|
|
* writes commands to the ring buffer, it increments the
|
|
|
|
* wptr. The GPU then starts fetching commands and executes
|
|
|
|
* them until the pointers are equal again.
|
2009-06-05 20:42:42 +08:00
|
|
|
*/
|
2012-09-01 01:43:50 +08:00
|
|
|
static int radeon_debugfs_ring_init(struct radeon_device *rdev, struct radeon_ring *ring);
|
2012-05-09 21:34:58 +08:00
|
|
|
|
2012-07-18 02:02:38 +08:00
|
|
|
/**
|
|
|
|
* radeon_ring_write - write a value to the ring
|
|
|
|
*
|
|
|
|
* @ring: radeon_ring structure holding ring information
|
|
|
|
* @v: dword (dw) value to write
|
|
|
|
*
|
|
|
|
* Write a value to the requested ring buffer (all asics).
|
|
|
|
*/
|
2012-05-09 21:34:58 +08:00
|
|
|
void radeon_ring_write(struct radeon_ring *ring, uint32_t v)
|
|
|
|
{
|
|
|
|
#if DRM_DEBUG_CODE
|
|
|
|
if (ring->count_dw <= 0) {
|
2012-10-16 01:16:22 +08:00
|
|
|
DRM_ERROR("radeon: writing more dwords to the ring than expected!\n");
|
2012-05-09 21:34:58 +08:00
|
|
|
}
|
|
|
|
#endif
|
|
|
|
ring->ring[ring->wptr++] = v;
|
|
|
|
ring->wptr &= ring->ptr_mask;
|
|
|
|
ring->count_dw--;
|
|
|
|
ring->ring_free_dw--;
|
|
|
|
}
|
|
|
|
|
2012-07-18 02:02:38 +08:00
|
|
|
/**
|
|
|
|
* radeon_ring_supports_scratch_reg - check if the ring supports
|
|
|
|
* writing to scratch registers
|
|
|
|
*
|
|
|
|
* @rdev: radeon_device pointer
|
|
|
|
* @ring: radeon_ring structure holding ring information
|
|
|
|
*
|
|
|
|
* Check if a specific ring supports writing to scratch registers (all asics).
|
|
|
|
* Returns true if the ring supports writing to scratch regs, false if not.
|
|
|
|
*/
|
2012-07-18 02:02:31 +08:00
|
|
|
bool radeon_ring_supports_scratch_reg(struct radeon_device *rdev,
|
|
|
|
struct radeon_ring *ring)
|
|
|
|
{
|
|
|
|
switch (ring->idx) {
|
|
|
|
case RADEON_RING_TYPE_GFX_INDEX:
|
|
|
|
case CAYMAN_RING_TYPE_CP1_INDEX:
|
|
|
|
case CAYMAN_RING_TYPE_CP2_INDEX:
|
|
|
|
return true;
|
|
|
|
default:
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2012-07-18 02:02:38 +08:00
|
|
|
/**
|
|
|
|
* radeon_ring_free_size - update the free size
|
|
|
|
*
|
|
|
|
* @rdev: radeon_device pointer
|
|
|
|
* @ring: radeon_ring structure holding ring information
|
|
|
|
*
|
|
|
|
* Update the free dw slots in the ring buffer (all asics).
|
|
|
|
*/
|
2011-10-23 18:56:27 +08:00
|
|
|
void radeon_ring_free_size(struct radeon_device *rdev, struct radeon_ring *ring)
|
2009-06-05 20:42:42 +08:00
|
|
|
{
|
2011-11-18 03:25:56 +08:00
|
|
|
u32 rptr;
|
|
|
|
|
2010-08-28 06:25:25 +08:00
|
|
|
if (rdev->wb.enabled)
|
2011-11-18 03:25:56 +08:00
|
|
|
rptr = le32_to_cpu(rdev->wb.wb[ring->rptr_offs/4]);
|
2011-10-13 18:48:45 +08:00
|
|
|
else
|
2011-11-18 03:25:56 +08:00
|
|
|
rptr = RREG32(ring->rptr_reg);
|
|
|
|
ring->rptr = (rptr & ring->ptr_reg_mask) >> ring->ptr_reg_shift;
|
2009-06-05 20:42:42 +08:00
|
|
|
/* This works because ring_size is a power of 2 */
|
2011-10-23 18:56:27 +08:00
|
|
|
ring->ring_free_dw = (ring->rptr + (ring->ring_size / 4));
|
|
|
|
ring->ring_free_dw -= ring->wptr;
|
|
|
|
ring->ring_free_dw &= ring->ptr_mask;
|
|
|
|
if (!ring->ring_free_dw) {
|
|
|
|
ring->ring_free_dw = ring->ring_size / 4;
|
2009-06-05 20:42:42 +08:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2012-07-18 02:02:38 +08:00
|
|
|
/**
|
|
|
|
* radeon_ring_alloc - allocate space on the ring buffer
|
|
|
|
*
|
|
|
|
* @rdev: radeon_device pointer
|
|
|
|
* @ring: radeon_ring structure holding ring information
|
|
|
|
* @ndw: number of dwords to allocate in the ring buffer
|
|
|
|
*
|
|
|
|
* Allocate @ndw dwords in the ring buffer (all asics).
|
|
|
|
* Returns 0 on success, error on failure.
|
|
|
|
*/
|
2011-10-23 18:56:27 +08:00
|
|
|
int radeon_ring_alloc(struct radeon_device *rdev, struct radeon_ring *ring, unsigned ndw)
|
2009-06-05 20:42:42 +08:00
|
|
|
{
|
|
|
|
int r;
|
|
|
|
|
2013-01-31 03:24:09 +08:00
|
|
|
/* make sure we aren't trying to allocate more space than there is on the ring */
|
|
|
|
if (ndw > (ring->ring_size / 4))
|
|
|
|
return -ENOMEM;
|
2009-06-05 20:42:42 +08:00
|
|
|
/* Align requested size with padding so unlock_commit can
|
|
|
|
* pad safely */
|
2011-10-23 18:56:27 +08:00
|
|
|
ndw = (ndw + ring->align_mask) & ~ring->align_mask;
|
|
|
|
while (ndw > (ring->ring_free_dw - 1)) {
|
|
|
|
radeon_ring_free_size(rdev, ring);
|
|
|
|
if (ndw < ring->ring_free_dw) {
|
2009-06-05 20:42:42 +08:00
|
|
|
break;
|
|
|
|
}
|
2012-07-18 02:02:30 +08:00
|
|
|
r = radeon_fence_wait_next_locked(rdev, ring->idx);
|
2010-05-01 03:24:17 +08:00
|
|
|
if (r)
|
2009-06-05 20:42:42 +08:00
|
|
|
return r;
|
|
|
|
}
|
2011-10-23 18:56:27 +08:00
|
|
|
ring->count_dw = ndw;
|
|
|
|
ring->wptr_old = ring->wptr;
|
2009-06-05 20:42:42 +08:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2012-07-18 02:02:38 +08:00
|
|
|
/**
|
|
|
|
* radeon_ring_lock - lock the ring and allocate space on it
|
|
|
|
*
|
|
|
|
* @rdev: radeon_device pointer
|
|
|
|
* @ring: radeon_ring structure holding ring information
|
|
|
|
* @ndw: number of dwords to allocate in the ring buffer
|
|
|
|
*
|
|
|
|
* Lock the ring and allocate @ndw dwords in the ring buffer
|
|
|
|
* (all asics).
|
|
|
|
* Returns 0 on success, error on failure.
|
|
|
|
*/
|
2011-10-23 18:56:27 +08:00
|
|
|
int radeon_ring_lock(struct radeon_device *rdev, struct radeon_ring *ring, unsigned ndw)
|
2010-05-01 03:24:17 +08:00
|
|
|
{
|
|
|
|
int r;
|
|
|
|
|
2012-05-09 21:34:45 +08:00
|
|
|
mutex_lock(&rdev->ring_lock);
|
2011-10-23 18:56:27 +08:00
|
|
|
r = radeon_ring_alloc(rdev, ring, ndw);
|
2010-05-01 03:24:17 +08:00
|
|
|
if (r) {
|
2012-05-09 21:34:45 +08:00
|
|
|
mutex_unlock(&rdev->ring_lock);
|
2010-05-01 03:24:17 +08:00
|
|
|
return r;
|
|
|
|
}
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2012-07-18 02:02:38 +08:00
|
|
|
/**
|
|
|
|
* radeon_ring_commit - tell the GPU to execute the new
|
|
|
|
* commands on the ring buffer
|
|
|
|
*
|
|
|
|
* @rdev: radeon_device pointer
|
|
|
|
* @ring: radeon_ring structure holding ring information
|
|
|
|
*
|
|
|
|
* Update the wptr (write pointer) to tell the GPU to
|
|
|
|
* execute new commands on the ring buffer (all asics).
|
|
|
|
*/
|
2011-10-23 18:56:27 +08:00
|
|
|
void radeon_ring_commit(struct radeon_device *rdev, struct radeon_ring *ring)
|
2009-06-05 20:42:42 +08:00
|
|
|
{
|
|
|
|
/* We pad to match fetch size */
|
2012-07-07 18:11:32 +08:00
|
|
|
while (ring->wptr & ring->align_mask) {
|
2011-11-18 03:25:56 +08:00
|
|
|
radeon_ring_write(ring, ring->nop);
|
2009-06-05 20:42:42 +08:00
|
|
|
}
|
|
|
|
DRM_MEMORYBARRIER();
|
2011-11-18 03:25:56 +08:00
|
|
|
WREG32(ring->wptr_reg, (ring->wptr << ring->ptr_reg_shift) & ring->ptr_reg_mask);
|
2011-10-23 18:56:27 +08:00
|
|
|
(void)RREG32(ring->wptr_reg);
|
2010-05-01 03:24:17 +08:00
|
|
|
}
|
|
|
|
|
2012-07-18 02:02:38 +08:00
|
|
|
/**
|
|
|
|
* radeon_ring_unlock_commit - tell the GPU to execute the new
|
|
|
|
* commands on the ring buffer and unlock it
|
|
|
|
*
|
|
|
|
* @rdev: radeon_device pointer
|
|
|
|
* @ring: radeon_ring structure holding ring information
|
|
|
|
*
|
|
|
|
* Call radeon_ring_commit() then unlock the ring (all asics).
|
|
|
|
*/
|
2011-10-23 18:56:27 +08:00
|
|
|
void radeon_ring_unlock_commit(struct radeon_device *rdev, struct radeon_ring *ring)
|
2010-05-01 03:24:17 +08:00
|
|
|
{
|
2011-10-23 18:56:27 +08:00
|
|
|
radeon_ring_commit(rdev, ring);
|
2012-05-09 21:34:45 +08:00
|
|
|
mutex_unlock(&rdev->ring_lock);
|
2009-06-05 20:42:42 +08:00
|
|
|
}
|
|
|
|
|
2012-07-18 02:02:38 +08:00
|
|
|
/**
|
|
|
|
* radeon_ring_undo - reset the wptr
|
|
|
|
*
|
|
|
|
* @ring: radeon_ring structure holding ring information
|
|
|
|
*
|
2012-11-21 05:31:06 +08:00
|
|
|
* Reset the driver's copy of the wptr (all asics).
|
2012-07-18 02:02:38 +08:00
|
|
|
*/
|
2012-05-09 21:34:45 +08:00
|
|
|
void radeon_ring_undo(struct radeon_ring *ring)
|
2009-06-05 20:42:42 +08:00
|
|
|
{
|
2011-10-23 18:56:27 +08:00
|
|
|
ring->wptr = ring->wptr_old;
|
2012-05-09 21:34:45 +08:00
|
|
|
}
|
|
|
|
|
2012-07-18 02:02:38 +08:00
|
|
|
/**
|
|
|
|
* radeon_ring_unlock_undo - reset the wptr and unlock the ring
|
|
|
|
*
|
|
|
|
* @ring: radeon_ring structure holding ring information
|
|
|
|
*
|
|
|
|
* Call radeon_ring_undo() then unlock the ring (all asics).
|
|
|
|
*/
|
2012-05-09 21:34:45 +08:00
|
|
|
void radeon_ring_unlock_undo(struct radeon_device *rdev, struct radeon_ring *ring)
|
|
|
|
{
|
|
|
|
radeon_ring_undo(ring);
|
|
|
|
mutex_unlock(&rdev->ring_lock);
|
2009-06-05 20:42:42 +08:00
|
|
|
}
|
|
|
|
|
2012-07-18 02:02:38 +08:00
|
|
|
/**
|
|
|
|
* radeon_ring_force_activity - add some nop packets to the ring
|
|
|
|
*
|
|
|
|
* @rdev: radeon_device pointer
|
|
|
|
* @ring: radeon_ring structure holding ring information
|
|
|
|
*
|
|
|
|
* Add some nop packets to the ring to force activity (all asics).
|
|
|
|
* Used for lockup detection to see if the rptr is advancing.
|
|
|
|
*/
|
2012-05-02 21:11:23 +08:00
|
|
|
void radeon_ring_force_activity(struct radeon_device *rdev, struct radeon_ring *ring)
|
|
|
|
{
|
|
|
|
int r;
|
|
|
|
|
|
|
|
radeon_ring_free_size(rdev, ring);
|
|
|
|
if (ring->rptr == ring->wptr) {
|
|
|
|
r = radeon_ring_alloc(rdev, ring, 1);
|
|
|
|
if (!r) {
|
|
|
|
radeon_ring_write(ring, ring->nop);
|
|
|
|
radeon_ring_commit(rdev, ring);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2012-07-18 02:02:38 +08:00
|
|
|
/**
|
2012-11-21 05:31:06 +08:00
|
|
|
* radeon_ring_lockup_update - update lockup variables
|
2012-07-18 02:02:38 +08:00
|
|
|
*
|
|
|
|
* @ring: radeon_ring structure holding ring information
|
|
|
|
*
|
|
|
|
* Update the last rptr value and timestamp (all asics).
|
|
|
|
*/
|
2012-05-02 21:11:20 +08:00
|
|
|
void radeon_ring_lockup_update(struct radeon_ring *ring)
|
|
|
|
{
|
|
|
|
ring->last_rptr = ring->rptr;
|
|
|
|
ring->last_activity = jiffies;
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* radeon_ring_test_lockup() - check if ring is lockedup by recording information
|
|
|
|
* @rdev: radeon device structure
|
|
|
|
* @ring: radeon_ring structure holding ring information
|
|
|
|
*
|
|
|
|
* We don't need to initialize the lockup tracking information as we will either
|
|
|
|
* have CP rptr to a different value of jiffies wrap around which will force
|
|
|
|
* initialization of the lockup tracking informations.
|
|
|
|
*
|
|
|
|
* A possible false positivie is if we get call after while and last_cp_rptr ==
|
|
|
|
* the current CP rptr, even if it's unlikely it might happen. To avoid this
|
|
|
|
* if the elapsed time since last call is bigger than 2 second than we return
|
|
|
|
* false and update the tracking information. Due to this the caller must call
|
|
|
|
* radeon_ring_test_lockup several time in less than 2sec for lockup to be reported
|
|
|
|
* the fencing code should be cautious about that.
|
|
|
|
*
|
|
|
|
* Caller should write to the ring to force CP to do something so we don't get
|
|
|
|
* false positive when CP is just gived nothing to do.
|
|
|
|
*
|
|
|
|
**/
|
|
|
|
bool radeon_ring_test_lockup(struct radeon_device *rdev, struct radeon_ring *ring)
|
|
|
|
{
|
|
|
|
unsigned long cjiffies, elapsed;
|
|
|
|
uint32_t rptr;
|
|
|
|
|
|
|
|
cjiffies = jiffies;
|
|
|
|
if (!time_after(cjiffies, ring->last_activity)) {
|
|
|
|
/* likely a wrap around */
|
|
|
|
radeon_ring_lockup_update(ring);
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
rptr = RREG32(ring->rptr_reg);
|
|
|
|
ring->rptr = (rptr & ring->ptr_reg_mask) >> ring->ptr_reg_shift;
|
|
|
|
if (ring->rptr != ring->last_rptr) {
|
|
|
|
/* CP is still working no lockup */
|
|
|
|
radeon_ring_lockup_update(ring);
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
elapsed = jiffies_to_msecs(cjiffies - ring->last_activity);
|
2012-05-02 21:11:21 +08:00
|
|
|
if (radeon_lockup_timeout && elapsed >= radeon_lockup_timeout) {
|
2012-05-02 21:11:20 +08:00
|
|
|
dev_err(rdev->dev, "GPU lockup CP stall for more than %lumsec\n", elapsed);
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
/* give a chance to the GPU ... */
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
2012-07-09 17:52:44 +08:00
|
|
|
/**
|
|
|
|
* radeon_ring_backup - Back up the content of a ring
|
|
|
|
*
|
|
|
|
* @rdev: radeon_device pointer
|
|
|
|
* @ring: the ring we want to back up
|
|
|
|
*
|
|
|
|
* Saves all unprocessed commits from a ring, returns the number of dwords saved.
|
|
|
|
*/
|
|
|
|
unsigned radeon_ring_backup(struct radeon_device *rdev, struct radeon_ring *ring,
|
|
|
|
uint32_t **data)
|
|
|
|
{
|
|
|
|
unsigned size, ptr, i;
|
|
|
|
|
|
|
|
/* just in case lock the ring */
|
|
|
|
mutex_lock(&rdev->ring_lock);
|
|
|
|
*data = NULL;
|
|
|
|
|
2012-07-18 02:02:31 +08:00
|
|
|
if (ring->ring_obj == NULL) {
|
2012-07-09 17:52:44 +08:00
|
|
|
mutex_unlock(&rdev->ring_lock);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* it doesn't make sense to save anything if all fences are signaled */
|
2012-07-18 02:02:30 +08:00
|
|
|
if (!radeon_fence_count_emitted(rdev, ring->idx)) {
|
2012-07-09 17:52:44 +08:00
|
|
|
mutex_unlock(&rdev->ring_lock);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* calculate the number of dw on the ring */
|
2012-07-18 02:02:31 +08:00
|
|
|
if (ring->rptr_save_reg)
|
|
|
|
ptr = RREG32(ring->rptr_save_reg);
|
|
|
|
else if (rdev->wb.enabled)
|
|
|
|
ptr = le32_to_cpu(*ring->next_rptr_cpu_addr);
|
|
|
|
else {
|
|
|
|
/* no way to read back the next rptr */
|
|
|
|
mutex_unlock(&rdev->ring_lock);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2012-07-09 17:52:44 +08:00
|
|
|
size = ring->wptr + (ring->ring_size / 4);
|
|
|
|
size -= ptr;
|
|
|
|
size &= ring->ptr_mask;
|
|
|
|
if (size == 0) {
|
|
|
|
mutex_unlock(&rdev->ring_lock);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* and then save the content of the ring */
|
2012-07-20 19:17:00 +08:00
|
|
|
*data = kmalloc_array(size, sizeof(uint32_t), GFP_KERNEL);
|
|
|
|
if (!*data) {
|
|
|
|
mutex_unlock(&rdev->ring_lock);
|
|
|
|
return 0;
|
|
|
|
}
|
2012-07-09 17:52:44 +08:00
|
|
|
for (i = 0; i < size; ++i) {
|
|
|
|
(*data)[i] = ring->ring[ptr++];
|
|
|
|
ptr &= ring->ptr_mask;
|
|
|
|
}
|
|
|
|
|
|
|
|
mutex_unlock(&rdev->ring_lock);
|
|
|
|
return size;
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* radeon_ring_restore - append saved commands to the ring again
|
|
|
|
*
|
|
|
|
* @rdev: radeon_device pointer
|
|
|
|
* @ring: ring to append commands to
|
|
|
|
* @size: number of dwords we want to write
|
|
|
|
* @data: saved commands
|
|
|
|
*
|
|
|
|
* Allocates space on the ring and restore the previously saved commands.
|
|
|
|
*/
|
|
|
|
int radeon_ring_restore(struct radeon_device *rdev, struct radeon_ring *ring,
|
|
|
|
unsigned size, uint32_t *data)
|
|
|
|
{
|
|
|
|
int i, r;
|
|
|
|
|
|
|
|
if (!size || !data)
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
/* restore the saved ring content */
|
|
|
|
r = radeon_ring_lock(rdev, ring, size);
|
|
|
|
if (r)
|
|
|
|
return r;
|
|
|
|
|
|
|
|
for (i = 0; i < size; ++i) {
|
|
|
|
radeon_ring_write(ring, data[i]);
|
|
|
|
}
|
|
|
|
|
|
|
|
radeon_ring_unlock_commit(rdev, ring);
|
|
|
|
kfree(data);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2012-07-18 02:02:38 +08:00
|
|
|
/**
|
|
|
|
* radeon_ring_init - init driver ring struct.
|
|
|
|
*
|
|
|
|
* @rdev: radeon_device pointer
|
|
|
|
* @ring: radeon_ring structure holding ring information
|
|
|
|
* @ring_size: size of the ring
|
|
|
|
* @rptr_offs: offset of the rptr writeback location in the WB buffer
|
|
|
|
* @rptr_reg: MMIO offset of the rptr register
|
|
|
|
* @wptr_reg: MMIO offset of the wptr register
|
|
|
|
* @ptr_reg_shift: bit offset of the rptr/wptr values
|
|
|
|
* @ptr_reg_mask: bit mask of the rptr/wptr values
|
|
|
|
* @nop: nop packet for this ring
|
|
|
|
*
|
|
|
|
* Initialize the driver information for the selected ring (all asics).
|
|
|
|
* Returns 0 on success, error on failure.
|
|
|
|
*/
|
2011-10-23 18:56:27 +08:00
|
|
|
int radeon_ring_init(struct radeon_device *rdev, struct radeon_ring *ring, unsigned ring_size,
|
2011-11-18 03:25:56 +08:00
|
|
|
unsigned rptr_offs, unsigned rptr_reg, unsigned wptr_reg,
|
|
|
|
u32 ptr_reg_shift, u32 ptr_reg_mask, u32 nop)
|
2009-06-05 20:42:42 +08:00
|
|
|
{
|
|
|
|
int r;
|
|
|
|
|
2011-10-23 18:56:27 +08:00
|
|
|
ring->ring_size = ring_size;
|
|
|
|
ring->rptr_offs = rptr_offs;
|
|
|
|
ring->rptr_reg = rptr_reg;
|
|
|
|
ring->wptr_reg = wptr_reg;
|
2011-11-18 03:25:56 +08:00
|
|
|
ring->ptr_reg_shift = ptr_reg_shift;
|
|
|
|
ring->ptr_reg_mask = ptr_reg_mask;
|
|
|
|
ring->nop = nop;
|
2009-06-05 20:42:42 +08:00
|
|
|
/* Allocate ring buffer */
|
2011-10-23 18:56:27 +08:00
|
|
|
if (ring->ring_obj == NULL) {
|
|
|
|
r = radeon_bo_create(rdev, ring->ring_size, PAGE_SIZE, true,
|
2012-05-11 06:33:13 +08:00
|
|
|
RADEON_GEM_DOMAIN_GTT,
|
|
|
|
NULL, &ring->ring_obj);
|
2009-06-05 20:42:42 +08:00
|
|
|
if (r) {
|
2009-11-20 21:29:23 +08:00
|
|
|
dev_err(rdev->dev, "(%d) ring create failed\n", r);
|
2009-06-05 20:42:42 +08:00
|
|
|
return r;
|
|
|
|
}
|
2011-10-23 18:56:27 +08:00
|
|
|
r = radeon_bo_reserve(ring->ring_obj, false);
|
2009-11-20 21:29:23 +08:00
|
|
|
if (unlikely(r != 0))
|
|
|
|
return r;
|
2011-10-23 18:56:27 +08:00
|
|
|
r = radeon_bo_pin(ring->ring_obj, RADEON_GEM_DOMAIN_GTT,
|
|
|
|
&ring->gpu_addr);
|
2009-06-05 20:42:42 +08:00
|
|
|
if (r) {
|
2011-10-23 18:56:27 +08:00
|
|
|
radeon_bo_unreserve(ring->ring_obj);
|
2009-11-20 21:29:23 +08:00
|
|
|
dev_err(rdev->dev, "(%d) ring pin failed\n", r);
|
2009-06-05 20:42:42 +08:00
|
|
|
return r;
|
|
|
|
}
|
2011-10-23 18:56:27 +08:00
|
|
|
r = radeon_bo_kmap(ring->ring_obj,
|
|
|
|
(void **)&ring->ring);
|
|
|
|
radeon_bo_unreserve(ring->ring_obj);
|
2009-06-05 20:42:42 +08:00
|
|
|
if (r) {
|
2009-11-20 21:29:23 +08:00
|
|
|
dev_err(rdev->dev, "(%d) ring map failed\n", r);
|
2009-06-05 20:42:42 +08:00
|
|
|
return r;
|
|
|
|
}
|
|
|
|
}
|
2011-10-23 18:56:27 +08:00
|
|
|
ring->ptr_mask = (ring->ring_size / 4) - 1;
|
|
|
|
ring->ring_free_dw = ring->ring_size / 4;
|
2012-07-18 02:02:31 +08:00
|
|
|
if (rdev->wb.enabled) {
|
|
|
|
u32 index = RADEON_WB_RING0_NEXT_RPTR + (ring->idx * 4);
|
|
|
|
ring->next_rptr_gpu_addr = rdev->wb.gpu_addr + index;
|
|
|
|
ring->next_rptr_cpu_addr = &rdev->wb.wb[index/4];
|
|
|
|
}
|
2012-05-02 21:11:11 +08:00
|
|
|
if (radeon_debugfs_ring_init(rdev, ring)) {
|
|
|
|
DRM_ERROR("Failed to register debugfs file for rings !\n");
|
|
|
|
}
|
2012-08-20 21:38:47 +08:00
|
|
|
radeon_ring_lockup_update(ring);
|
2009-06-05 20:42:42 +08:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2012-07-18 02:02:38 +08:00
|
|
|
/**
|
|
|
|
* radeon_ring_fini - tear down the driver ring struct.
|
|
|
|
*
|
|
|
|
* @rdev: radeon_device pointer
|
|
|
|
* @ring: radeon_ring structure holding ring information
|
|
|
|
*
|
|
|
|
* Tear down the driver information for the selected ring (all asics).
|
|
|
|
*/
|
2011-10-23 18:56:27 +08:00
|
|
|
void radeon_ring_fini(struct radeon_device *rdev, struct radeon_ring *ring)
|
2009-06-05 20:42:42 +08:00
|
|
|
{
|
2009-11-20 21:29:23 +08:00
|
|
|
int r;
|
2010-05-06 23:02:24 +08:00
|
|
|
struct radeon_bo *ring_obj;
|
2009-11-20 21:29:23 +08:00
|
|
|
|
2012-05-09 21:34:45 +08:00
|
|
|
mutex_lock(&rdev->ring_lock);
|
2011-10-23 18:56:27 +08:00
|
|
|
ring_obj = ring->ring_obj;
|
2012-05-09 21:34:45 +08:00
|
|
|
ring->ready = false;
|
2011-10-23 18:56:27 +08:00
|
|
|
ring->ring = NULL;
|
|
|
|
ring->ring_obj = NULL;
|
2012-05-09 21:34:45 +08:00
|
|
|
mutex_unlock(&rdev->ring_lock);
|
2010-05-06 23:02:24 +08:00
|
|
|
|
|
|
|
if (ring_obj) {
|
|
|
|
r = radeon_bo_reserve(ring_obj, false);
|
2009-11-20 21:29:23 +08:00
|
|
|
if (likely(r == 0)) {
|
2010-05-06 23:02:24 +08:00
|
|
|
radeon_bo_kunmap(ring_obj);
|
|
|
|
radeon_bo_unpin(ring_obj);
|
|
|
|
radeon_bo_unreserve(ring_obj);
|
2009-11-20 21:29:23 +08:00
|
|
|
}
|
2010-05-06 23:02:24 +08:00
|
|
|
radeon_bo_unref(&ring_obj);
|
2009-06-05 20:42:42 +08:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Debugfs info
|
|
|
|
*/
|
|
|
|
#if defined(CONFIG_DEBUG_FS)
|
2011-10-24 23:08:44 +08:00
|
|
|
|
|
|
|
static int radeon_debugfs_ring_info(struct seq_file *m, void *data)
|
|
|
|
{
|
|
|
|
struct drm_info_node *node = (struct drm_info_node *) m->private;
|
|
|
|
struct drm_device *dev = node->minor->dev;
|
|
|
|
struct radeon_device *rdev = dev->dev_private;
|
|
|
|
int ridx = *(int*)node->info_ent->data;
|
|
|
|
struct radeon_ring *ring = &rdev->ring[ridx];
|
|
|
|
unsigned count, i, j;
|
2013-01-03 06:30:34 +08:00
|
|
|
u32 tmp;
|
2011-10-24 23:08:44 +08:00
|
|
|
|
|
|
|
radeon_ring_free_size(rdev, ring);
|
|
|
|
count = (ring->ring_size / 4) - ring->ring_free_dw;
|
2013-01-03 06:30:34 +08:00
|
|
|
tmp = RREG32(ring->wptr_reg) >> ring->ptr_reg_shift;
|
|
|
|
seq_printf(m, "wptr(0x%04x): 0x%08x [%5d]\n", ring->wptr_reg, tmp, tmp);
|
|
|
|
tmp = RREG32(ring->rptr_reg) >> ring->ptr_reg_shift;
|
|
|
|
seq_printf(m, "rptr(0x%04x): 0x%08x [%5d]\n", ring->rptr_reg, tmp, tmp);
|
2012-07-06 22:22:55 +08:00
|
|
|
if (ring->rptr_save_reg) {
|
|
|
|
seq_printf(m, "rptr next(0x%04x): 0x%08x\n", ring->rptr_save_reg,
|
|
|
|
RREG32(ring->rptr_save_reg));
|
|
|
|
}
|
2013-01-03 06:30:34 +08:00
|
|
|
seq_printf(m, "driver's copy of the wptr: 0x%08x [%5d]\n", ring->wptr, ring->wptr);
|
|
|
|
seq_printf(m, "driver's copy of the rptr: 0x%08x [%5d]\n", ring->rptr, ring->rptr);
|
2013-01-12 04:19:43 +08:00
|
|
|
seq_printf(m, "last semaphore signal addr : 0x%016llx\n", ring->last_semaphore_signal_addr);
|
|
|
|
seq_printf(m, "last semaphore wait addr : 0x%016llx\n", ring->last_semaphore_wait_addr);
|
2011-10-24 23:08:44 +08:00
|
|
|
seq_printf(m, "%u free dwords in ring\n", ring->ring_free_dw);
|
|
|
|
seq_printf(m, "%u dwords in ring\n", count);
|
2013-01-03 06:30:34 +08:00
|
|
|
/* print 8 dw before current rptr as often it's the last executed
|
|
|
|
* packet that is the root issue
|
|
|
|
*/
|
|
|
|
i = (ring->rptr + ring->ptr_mask + 1 - 32) & ring->ptr_mask;
|
|
|
|
for (j = 0; j <= (count + 32); j++) {
|
|
|
|
seq_printf(m, "r[%5d]=0x%08x\n", i, ring->ring[i]);
|
2011-10-24 23:08:44 +08:00
|
|
|
i = (i + 1) & ring->ptr_mask;
|
|
|
|
}
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int radeon_ring_type_gfx_index = RADEON_RING_TYPE_GFX_INDEX;
|
|
|
|
static int cayman_ring_type_cp1_index = CAYMAN_RING_TYPE_CP1_INDEX;
|
|
|
|
static int cayman_ring_type_cp2_index = CAYMAN_RING_TYPE_CP2_INDEX;
|
2013-01-03 04:12:14 +08:00
|
|
|
static int radeon_ring_type_dma1_index = R600_RING_TYPE_DMA_INDEX;
|
|
|
|
static int radeon_ring_type_dma2_index = CAYMAN_RING_TYPE_DMA1_INDEX;
|
2011-10-24 23:08:44 +08:00
|
|
|
|
|
|
|
static struct drm_info_list radeon_debugfs_ring_info_list[] = {
|
|
|
|
{"radeon_ring_gfx", radeon_debugfs_ring_info, 0, &radeon_ring_type_gfx_index},
|
|
|
|
{"radeon_ring_cp1", radeon_debugfs_ring_info, 0, &cayman_ring_type_cp1_index},
|
|
|
|
{"radeon_ring_cp2", radeon_debugfs_ring_info, 0, &cayman_ring_type_cp2_index},
|
2013-01-03 04:12:14 +08:00
|
|
|
{"radeon_ring_dma1", radeon_debugfs_ring_info, 0, &radeon_ring_type_dma1_index},
|
|
|
|
{"radeon_ring_dma2", radeon_debugfs_ring_info, 0, &radeon_ring_type_dma2_index},
|
2011-10-24 23:08:44 +08:00
|
|
|
};
|
|
|
|
|
2012-05-09 21:34:51 +08:00
|
|
|
static int radeon_debugfs_sa_info(struct seq_file *m, void *data)
|
|
|
|
{
|
|
|
|
struct drm_info_node *node = (struct drm_info_node *) m->private;
|
|
|
|
struct drm_device *dev = node->minor->dev;
|
|
|
|
struct radeon_device *rdev = dev->dev_private;
|
|
|
|
|
2012-05-09 21:34:58 +08:00
|
|
|
radeon_sa_bo_dump_debug_info(&rdev->ring_tmp_bo, m);
|
2012-05-09 21:34:51 +08:00
|
|
|
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
static struct drm_info_list radeon_debugfs_sa_list[] = {
|
|
|
|
{"radeon_sa_info", &radeon_debugfs_sa_info, 0, NULL},
|
|
|
|
};
|
|
|
|
|
2009-06-05 20:42:42 +08:00
|
|
|
#endif
|
|
|
|
|
2012-09-01 01:43:50 +08:00
|
|
|
static int radeon_debugfs_ring_init(struct radeon_device *rdev, struct radeon_ring *ring)
|
2011-10-24 23:08:44 +08:00
|
|
|
{
|
|
|
|
#if defined(CONFIG_DEBUG_FS)
|
2012-05-02 21:11:11 +08:00
|
|
|
unsigned i;
|
|
|
|
for (i = 0; i < ARRAY_SIZE(radeon_debugfs_ring_info_list); ++i) {
|
|
|
|
struct drm_info_list *info = &radeon_debugfs_ring_info_list[i];
|
|
|
|
int ridx = *(int*)radeon_debugfs_ring_info_list[i].data;
|
|
|
|
unsigned r;
|
|
|
|
|
|
|
|
if (&rdev->ring[ridx] != ring)
|
|
|
|
continue;
|
|
|
|
|
|
|
|
r = radeon_debugfs_add_files(rdev, info, 1);
|
|
|
|
if (r)
|
|
|
|
return r;
|
|
|
|
}
|
2011-10-24 23:08:44 +08:00
|
|
|
#endif
|
2012-05-02 21:11:11 +08:00
|
|
|
return 0;
|
2011-10-24 23:08:44 +08:00
|
|
|
}
|
|
|
|
|
2012-09-01 01:43:50 +08:00
|
|
|
static int radeon_debugfs_sa_init(struct radeon_device *rdev)
|
2009-06-05 20:42:42 +08:00
|
|
|
{
|
|
|
|
#if defined(CONFIG_DEBUG_FS)
|
2012-05-09 21:34:58 +08:00
|
|
|
return radeon_debugfs_add_files(rdev, radeon_debugfs_sa_list, 1);
|
2009-06-05 20:42:42 +08:00
|
|
|
#else
|
|
|
|
return 0;
|
|
|
|
#endif
|
|
|
|
}
|