Provide helper function to compute the kernel memory size needed
for each buffer object. Move all the accounting inside ttm, simplifying
driver and avoiding code duplication accross them.
v2 fix accounting of ghost object, one would have thought that i
would have run into the issue since a longtime but it seems
ghost object are rare when you have plenty of vram ;)
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Move dma data to a superset ttm_dma_tt structure which herit
from ttm_tt. This allow driver that don't use dma functionalities
to not have to waste memory for it.
V2 Rebase on top of no memory account changes (where/when is my
delorean when i need it ?)
V3 Make sure page list is initialized empty
V4 typo/syntax fixes
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
In TTM world the pages for the graphic drivers are kept in three different
pools: write combined, uncached, and cached (write-back). When the pages
are used by the graphic driver the graphic adapter via its built in MMU
(or AGP) programs these pages in. The programming requires the virtual address
(from the graphic adapter perspective) and the physical address (either System RAM
or the memory on the card) which is obtained using the pci_map_* calls (which does the
virtual to physical - or bus address translation). During the graphic application's
"life" those pages can be shuffled around, swapped out to disk, moved from the
VRAM to System RAM or vice-versa. This all works with the existing TTM pool code
- except when we want to use the software IOTLB (SWIOTLB) code to "map" the physical
addresses to the graphic adapter MMU. We end up programming the bounce buffer's
physical address instead of the TTM pool memory's and get a non-worky driver.
There are two solutions:
1) using the DMA API to allocate pages that are screened by the DMA API, or
2) using the pci_sync_* calls to copy the pages from the bounce-buffer and back.
This patch fixes the issue by allocating pages using the DMA API. The second
is a viable option - but it has performance drawbacks and potential correctness
issues - think of the write cache page being bounced (SWIOTLB->TTM), the
WC is set on the TTM page and the copy from SWIOTLB not making it to the TTM
page until the page has been recycled in the pool (and used by another application).
The bounce buffer does not get activated often - only in cases where we have
a 32-bit capable card and we want to use a page that is allocated above the
4GB limit. The bounce buffer offers the solution of copying the contents
of that 4GB page to an location below 4GB and then back when the operation has been
completed (or vice-versa). This is done by using the 'pci_sync_*' calls.
Note: If you look carefully enough in the existing TTM page pool code you will
notice the GFP_DMA32 flag is used - which should guarantee that the provided page
is under 4GB. It certainly is the case, except this gets ignored in two cases:
- If user specifies 'swiotlb=force' which bounces _every_ page.
- If user is using a Xen's PV Linux guest (which uses the SWIOTLB and the
underlaying PFN's aren't necessarily under 4GB).
To not have this extra copying done the other option is to allocate the pages
using the DMA API so that there is not need to map the page and perform the
expensive 'pci_sync_*' calls.
This DMA API capable TTM pool requires for this the 'struct device' to
properly call the DMA API. It also has to track the virtual and bus address of
the page being handed out in case it ends up being swapped out or de-allocated -
to make sure it is de-allocated using the proper's 'struct device'.
Implementation wise the code keeps two lists: one that is attached to the
'struct device' (via the dev->dma_pools list) and a global one to be used when
the 'struct device' is unavailable (think shrinker code). The global list can
iterate over all of the 'struct device' and its associated dma_pool. The list
in dev->dma_pools can only iterate the device's dma_pool.
/[struct device_pool]\
/---------------------------------------------------| dev |
/ +-------| dma_pool |
/-----+------\ / \--------------------/
|struct device| /-->[struct dma_pool for WC]</ /[struct device_pool]\
| dma_pools +----+ /-| dev |
| ... | \--->[struct dma_pool for uncached]<-/--| dma_pool |
\-----+------/ / \--------------------/
\----------------------------------------------/
[Two pools associated with the device (WC and UC), and the parallel list
containing the 'struct dev' and 'struct dma_pool' entries]
The maximum amount of dma pools a device can have is six: write-combined,
uncached, and cached; then there are the DMA32 variants which are:
write-combined dma32, uncached dma32, and cached dma32.
Currently this code only gets activated when any variant of the SWIOTLB IOMMU
code is running (Intel without VT-d, AMD without GART, IBM Calgary and Xen PV
with PCI devices).
Tested-by: Michel Dänzer <michel@daenzer.net>
[v1: Using swiotlb_nr_tbl instead of swiotlb_enabled]
[v2: Major overhaul - added 'inuse_list' to seperate used from inuse and reorder
the order of lists to get better performance.]
[v3: Added comments/and some logic based on review, Added Jerome tag]
[v4: rebase on top of ttm_tt & ttm_backend merge]
[v5: rebase on top of ttm memory accounting overhaul]
[v6: New rebase on top of more memory accouting changes]
[v7: well rebase on top of no memory accounting changes]
[v8: make sure pages list is initialized empty]
[v9: calll ttm_mem_global_free_page in unpopulate for accurate accountg]
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Acked-by: Thomas Hellstrom <thellstrom@vmware.com>
Move the page allocation and freeing to driver callback and
provide ttm code helper function for those.
Most intrusive change, is the fact that we now only fully
populate an object this simplify some of code designed around
the page fault design.
V2 Rebase on top of memory accounting overhaul
V3 New rebase on top of more memory accouting changes
V4 Rebase on top of no memory account changes (where/when is my
delorean when i need it ?)
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
ttm_backend will only exist with a ttm_tt, and ttm_tt
will only be of interest when bound to a backend. Merge them
to avoid code and data duplication.
V2 Rebase on top of memory accounting overhaul
V3 Rebase on top of more memory accounting changes
V4 Rebase on top of no memory account changes (where/when is my
delorean when i need it ?)
V5 make sure ttm is unbound before destroying, change commit
message on suggestion from Tormod Volden
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Use the ttm_tt pages array for pages allocations, move the list
unwinding into the page allocation functions.
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
On failure we need to make sure the page we free has wb cache
attribute. Do this pas call the proper ttm page helper function.
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Split btw highmem and lowmem page was rendered useless by the
pool code. Remove it. Note further cleanup would change the
ttm page allocation helper to actualy take an array instead
of relying on list this could drasticly reduce the number of
function call in the common case of allocation whole buffer.
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
This was never use in none of the driver, properly using userspace
page for bo would need more code (vma interaction mostly). Removing
this dead code in preparation of ttm_tt & backend merge.
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
An unlikely race could case a bo to be returned reserved on an error path.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
* 'drm-core-next' of git://people.freedesktop.org/~airlied/linux: (290 commits)
Revert "drm/ttm: add a way to bo_wait for either the last read or last write"
Revert "drm/radeon/kms: add a new gem_wait ioctl with read/write flags"
vmwgfx: Don't pass unused arguments to do_dirty functions
vmwgfx: Emulate depth 32 framebuffers
drm/radeon: Lower the severity of the radeon lockup messages.
drm/i915/dp: Fix eDP on PCH DP on CPT/PPT
drm/i915/dp: Introduce is_cpu_edp()
drm/i915: use correct SPD type value
drm/i915: fix ILK+ infoframe support
drm/i915: add DP test request handling
drm/i915: read full receiver capability field during DP hot plug
drm/i915/dp: Remove eDP special cases from bandwidth checks
drm/i915/dp: Fix the math in intel_dp_link_required
drm/i915/panel: Always record the backlight level again (but cleverly)
i915: Move i915_read/write out of line
drm/i915: remove transcoder PLL mashing from mode_set per specs
drm/i915: if transcoder disable fails, say which
drm/i915: set watermarks for third pipe on IVB
drm/i915: export a CPT mode set verification function
drm/i915: fix transcoder PLL select masking
...
This reverts commit dfadbbdb57.
Further upstream discussion between Marek and Thomas decided this wasn't
fully baked and needed further work, so revert it before it hits mainline.
Signed-off-by: Dave Airlie <airlied@redhat.com>
There are a number of fixes in mainline required for code in -next,
also there was a few conflicts I'd rather resolve myself.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Conflicts:
drivers/gpu/drm/radeon/evergreen.c
drivers/gpu/drm/radeon/r600.c
drivers/gpu/drm/radeon/radeon_asic.h
Fixes an information leak to userspace, we were handing out un-zeroed pages
for any newly created TTM_PL_TT buffer.
Reported-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Tested-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Cc: stable@kernel.org
Signed-off-by: Dave Airlie <airlied@redhat.com>
Sometimes we want to know whether a buffer is busy and wait for it (bo_wait).
However, sometimes it would be more useful to be able to query whether
a buffer is busy and being either read or written, and wait until it's stopped
being either read or written. The point of this is to be able to avoid
unnecessary waiting, e.g. if a GPU has written something to a buffer and is now
reading that buffer, and a CPU wants to map that buffer for read, it needs to
only wait for the last write. If there were no write, there wouldn't be any
waiting needed.
This, or course, requires user space drivers to send read/write flags
with each relocation (like we have read/write domains in radeon, so we can
actually use those for something useful now).
Now how this patch works:
The read/write flags should passed to ttm_validate_buffer. TTM maintains
separate sync objects of the last read and write for each buffer, in addition
to the sync object of the last use of a buffer. ttm_bo_wait then operates
with one the sync objects.
Signed-off-by: Marek Olšák <maraeo@gmail.com>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
This was true for new TTM_PL_SYSTEM and new TTM_PL_TT cases, but wasn't
the case on TTM_PL_SYSTEM<->TTM_PL_TT moves, which causes trouble on some
paths as nouveau's move_notify() hook requires that the dma addresses be
valid at this point.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Nouveau makes the assumption that if a TTM is bound there will be a mm_node
around for it and the backwards ordering here resulted in a use-after-free
on some eviction paths.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
ttm_tt_destroy kfrees passed object, so we need to nullify
a reference to it.
Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Cc: stable@kernel.org
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
This allows us to move duplicated code in <asm/atomic.h>
(atomic_inc_not_zero() for now) to <linux/atomic.h>
Signed-off-by: Arun Sharma <asharma@fb.com>
Reviewed-by: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: David Miller <davem@davemloft.net>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Acked-by: Mike Frysinger <vapier@gentoo.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Soon tmpfs will stop supporting ->readpage and read_mapping_page(): once
"tmpfs: add shmem_read_mapping_page_gfp" has been applied, this patch can
be applied to ease the transition.
ttm_tt_swapin() and ttm_tt_swapout() use shmem_read_mapping_page() in
place of read_mapping_page(), since their swap_space has been created with
shmem_file_setup().
Signed-off-by: Hugh Dickins <hughd@google.com>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Thomas Hellstrom <thellstrom@vmware.com>
Cc: Dave Airlie <airlied@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
. and some comments to make it easier to understand.
Ackedby: Randy Dunlap <randy.dunlap@oracle.com>
[v2: Added some more updates from Randy Dunlap]
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Change each shrinker's API by consolidating the existing parameters into
shrink_control struct. This will simplify any further features added w/o
touching each file of shrinker.
[akpm@linux-foundation.org: fix build]
[akpm@linux-foundation.org: fix warning]
[kosaki.motohiro@jp.fujitsu.com: fix up new shrinker API]
[akpm@linux-foundation.org: fix xfs warning]
[akpm@linux-foundation.org: update gfs2]
Signed-off-by: Ying Han <yinghan@google.com>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
Acked-by: Pavel Emelyanov <xemul@openvz.org>
Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: Mel Gorman <mel@csn.ul.ie>
Acked-by: Rik van Riel <riel@redhat.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Hugh Dickins <hughd@google.com>
Cc: Dave Hansen <dave@linux.vnet.ibm.com>
Cc: Steven Whitehouse <swhiteho@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This reverts commit 69a07f0b11.
We've tracked a number of problems back to this, and Thomas
thinks we should redesign this for .40/41 anyways so I'm
happy to revert it.
Signed-off-by: Dave Airlie <airlied@redhat.com>
There's no need to pass kref_put() the address of a function (just the
function will do just fine) nor to cast its unused return to void.
Signed-off-by: Paul Bolle <pebolle@tiscali.nl>
Signed-off-by: Jiri Kosina <jkosina@suse.cz>
This reverts commit 5a893fc28f.
This causes a use after free in the ttm free alloc pages path,
when it tries to get the be after the be has been destroyed.
Signed-off-by: Dave Airlie <airlied@redhat.com>
* 'stable/ttm.pci-api.v5' of git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen:
ttm: Include the 'struct dev' when using the DMA API.
nouveau/ttm/PCIe: Use dma_addr if TTM has set it.
radeon/ttm/PCIe: Use dma_addr if TTM has set it.
ttm: Expand (*populate) to support an array of DMA addresses.
ttm: Utilize the DMA API for pages that have TTM_PAGE_FLAG_DMA32 set.
ttm: Introduce a placeholder for DMA (bus) addresses.
Nouveau doesn't have enough information at ttm_backend_func.bind() time
to implement things like tiled GART, or to keep a buffer at a constant
address in the GPU virtual address space no matter where in physical
memory it's placed.
To resolve this, nouveau will handle binding of all buffers to the GPU
itself from the move_notify() hook. This commit ensures it's called
for all buffer moves.
Talked to Dave about the impact on radeon, which uses move_notify, it
doesn't look like anything should break there.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Thomas Hellstrom <thomas@shipmail.org>
Signed-off-by: Dave Airlie <airlied@redhat.com>
This makes the accounting when using 'debug_dma_dump_mappings()'
and CONFIG_DMA_API_DEBUG=y be assigned to the correct device
instead of 'fallback'.
No functional change - just cosmetic.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
We pass in the array of ttm pages to be populated in the GART/MM
of the card (or AGP). Patch titled: "ttm: Utilize the DMA API for
pages that have TTM_PAGE_FLAG_DMA32 set." uses the DMA API to make
those pages have a proper DMA addresses (in the situation where
page_to_phys or virt_to_phys do not give use the DMA (bus) address).
Since we are using the DMA API on those pages, we should pass in the
DMA address to this function so it can save it in its proper fields
(later patches use it).
[v2: Added reviewed-by tag]
Reviewed-by: Thomas Hellstrom <thellstrom@shipmail.org>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Tested-by: Ian Campbell <ian.campbell@citrix.com>
For pages that have the TTM_PAGE_FLAG_DMA32 flag set we
use the DMA API. We save the bus address in our array which we
use to program the GART (see "radeon/ttm/PCIe: Use dma_addr if TTM
has set it." and "nouveau/ttm/PCIe: Use dma_addr if TTM has set it.").
The reason behind using the DMA API is that under Xen we would
end up programming the GART with the bounce buffer (SWIOTLB)
DMA address instead of the physical DMA address of the TTM page.
The reason being that alloc_page with GFP_DMA32 does not allocate
pages under the the 4GB mark when running under Xen hypervisor.
Under baremetal this means we do the DMA API call earlier instead
of when we program the GART.
For details please refer to:
https://lkml.org/lkml/2011/1/7/251
[v2: Fixed indentation, revised desc, added Reviewed-by]
Reviewed-by: Thomas Hellstrom <thomas@shipmail.org>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Tested-by: Ian Campbell <ian.campbell@citrix.com>
This is right now limited to only non-pool constructs.
[v2: Fixed indentation issues, add review-by tag]
Reviewed-by: Thomas Hellstrom <thomas@shipmail.org>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Tested-by: Ian Campbell <ian.campbell@citrix.com>
* 'drm-core-next' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6: (390 commits)
drm/radeon/kms: disable underscan by default
drm/radeon/kms: only enable hdmi features if the monitor supports audio
drm: Restore the old_fb upon modeset failure
drm/nouveau: fix hwmon device binding
radeon: consolidate asic-specific function decls for pre-r600
vga_switcheroo: comparing too few characters in strncmp()
drm/radeon/kms: add NI pci ids
drm/radeon/kms: don't enable pcie gen2 on NI yet
drm/radeon/kms: add radeon_asic struct for NI asics
drm/radeon/kms/ni: load default sclk/mclk/vddc at pm init
drm/radeon/kms: add ucode loader for NI
drm/radeon/kms: add support for DCE5 display LUTs
drm/radeon/kms: add ni_reg.h
drm/radeon/kms: add bo blit support for NI
drm/radeon/kms: always use writeback/events for fences on NI
drm/radeon/kms: adjust default clock/vddc tracking for pm on DCE5
drm/radeon/kms: add backend map workaround for barts
drm/radeon/kms: fill gpu init for NI asics
drm/radeon/kms: add disabled vbios accessor for NI asics
drm/radeon/kms: handle NI thermal controller
...
Make ttm_bo::ttm_bo_device_release call cancel_delayed_work_sync()
instead of calling cancel_delayed_work() followed by
flush_scheduled_work().
This is to prepare for the deprecation and removal of
flush_scheduled_work().
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc:: Thomas Hellstrom <thellstrom@vmware.com>
Cc:: Dave Airlie <airlied@redhat.com>
Drivers using their own implementation of io_mem_reserve/io_mem_free are
likely to store the tracking information for the map in mem.mm_node, so
it can't be freed while still mapped.
Signed-off-by: Ben Skeggs<bskeggs@redhat.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
This patch attempts to fix up shortcomings with the current calling
sequences.
1) There's a fastpath where no locking occurs and only io_mem_reserved is
called to obtain needed info for mapping. The fastpath is set per
memory type manager.
2) If the fastpath is disabled, io_mem_reserve and io_mem_free will be exactly
balanced and not called recursively for the same struct ttm_mem_reg.
3) Optionally the driver can choose to enable a per memory type manager LRU
eviction mechanism that, when io_mem_reserve returns -EAGAIN will attempt
to kill user-space mappings of memory in that manager to free up needed
resources
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Rather than having the driver supply the validation sequence, leave that
responsibility to TTM. This saves some confusion and a function argument.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Drastically reduce the number of spin lock / unlock operations by performing
unreserving and fencing under global locks.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Jerome Glisse <j.glisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
The bo lock used only to protect the bo sync object members, and since it
is a per bo lock, fencing a buffer list will see a lot of locks and unlocks.
Replace it with a per-device lock that protects the sync object members on
*all* bos. Reading and setting these members will always be very quick, so
the risc of heavy lock contention is microscopic. Note that waiting for
sync objects will always take place outside of this lock.
The bo device fence lock will eventually be replaced with a seqlock /
rcu mechanism so we can determine that a bo is idle under a
rcu / read seqlock.
However this change will allow us to batch fencing and unreserving of
buffers with a minimal amount of locking.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Jerome Glisse <j.glisse@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Add an aid for the driver to detect deadlocks on multi-bo reservations
Update documentation.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Jerome Glisse <j.glisse@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Avoid the ttm_bo_unreserve() spinlocks by calling
ttm_eu_backoff_reservation_locked under the lru spinlock.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Jerome Glisse <j.glisse@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Makes it possible to reserve a list of buffer objects with a single
spin lock / unlock if there is no contention.
Should improve cpu usage on SMP kernels.
v2: Initialize private list members on reserve and don't call
ttm_bo_list_ref_sub() with zero put_count.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
A process suspended waiting for a higher sequence or no sequence to unreserve,
a bo may be beaten to the reservation by a process with a lower sequence.
In that case the first process should give up trying to reserve and
return -EAGAIN. In order for that to happen, we must wake waiting processes
when we change sequence, so that they have a chance to detect the new
sequence.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Call destroy() on _all_ ttm_bo_init() failures, and make sure that
behavior is documented in the function description.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
This breaks vmwgfx non-root EGL clients and is a remnant from the
TTM user-space interface. This test should be done in the driver.
Replace the remaining placement test with a BUG_ON, since triggering
it is a driver bug.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
The sync object may disappear as soon as we release the bo::lock, so
take a reference on it while we use it.
One option would be to call sync_object_flush() before releasing the bo::lock,
but that would put an atomic requirement on that function.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
The driver (for example vmwgfx) may want to silently deal with the
error itself.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Since we're doing this outside of a spinlock to provide the necessary
barriers, add an explicit barrier.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Replace with BUG_ON(). These error messages remained from the time
when TTM was initialized from user-space. Nowadays hitting one of those
is really a kernel bug.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Searching for a free block in the range manager may in some situations be a
lenghty operation, and we want to avoid holding the global lru lock
during that time. Instead use a per-manager spinlock.
This leaves the global lru lock for quick lru list and swap list manipulation
only, including list manipulation associated with reserving buffer objects.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Remove an obsolete comment about mm nodes.
Document the new bo range manager interface.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
* 'drm-core-next' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6: (476 commits)
vmwgfx: Implement a proper GMR eviction mechanism
drm/radeon/kms: fix r6xx/7xx 1D tiling CS checker v2
drm/radeon/kms: properly compute group_size on 6xx/7xx
drm/radeon/kms: fix 2D tile height alignment in the r600 CS checker
drm/radeon/kms/evergreen: set the clear state to the blit state
drm/radeon/kms: don't poll dac load detect.
gpu: Add Intel GMA500(Poulsbo) Stub Driver
drm/radeon/kms: MC vram map needs to be >= pci aperture size
drm/radeon/kms: implement display watermark support for evergreen
drm/radeon/kms/evergreen: add some additional safe regs v2
drm/radeon/r600: fix tiling issues in CS checker.
drm/i915: Move gpu_write_list to per-ring
drm/i915: Invalidate the to-ring, flush the old-ring when updating domains
drm/i915/ringbuffer: Write the value passed in to the tail register
agp/intel: Restore valid PTE bit for Sandybridge after bdd3072
drm/i915: Fix flushing regression from 9af90d19f
drm/i915/sdvo: Remove unused encoding member
i915: enable AVI infoframe for intel_hdmi.c [v4]
drm/i915: Fix current fb blocking for page flip
drm/i915: IS_IRONLAKE is synonymous with gen == 5
...
Fix up conflicts in
- drivers/gpu/drm/i915/{i915_gem.c, i915/intel_overlay.c}: due to the
new simplified stack-based kmap_atomic() interface
- drivers/gpu/drm/vmwgfx/vmwgfx_drv.c: added .llseek entry due to BKL
removal cleanups.
Keep the current interface but ignore the KM_type and use a stack based
approach.
The advantage is that we get rid of crappy code like:
#define __KM_PTE \
(in_nmi() ? KM_NMI_PTE : \
in_irq() ? KM_IRQ_PTE : \
KM_PTE0)
and in general can stop worrying about what context we're in and what kmap
slots might be appropriate for that.
The downside is that FRV kmap_atomic() gets more expensive.
For now we use a CPP trick suggested by Andrew:
#define kmap_atomic(page, args...) __kmap_atomic(page)
to avoid having to touch all kmap_atomic() users in a single patch.
[ not compiled on:
- mn10300: the arch doesn't actually build with highmem to begin with ]
[akpm@linux-foundation.org: coding-style fixes]
[akpm@linux-foundation.org: fix up drivers/gpu/drm/i915/intel_overlay.c]
Acked-by: Rik van Riel <riel@redhat.com>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Acked-by: Chris Metcalf <cmetcalf@tilera.com>
Cc: David Howells <dhowells@redhat.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Russell King <rmk@arm.linux.org.uk>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: David Miller <davem@davemloft.net>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Dave Airlie <airlied@linux.ie>
Cc: Li Zefan <lizf@cn.fujitsu.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This commit replaces the ttm_bo_cleanup_ref function with two new functions.
One for the case where the bo is not yet on the delayed destroy list, and
one for the case where the bo was on the delayed destroy list, at least at
the time of call. This makes it possible to optimize the two cases somewhat.
It also enables the possibility to directly destroy buffers on the
delayed delete list when they are about to be evicted or swapped out.
Currently they were only evicted / swapped and destruction was left for the
delayed buffer destruction thread.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Function ttm_bo_wait_unreserved can be slightly simplified.
Signed-off-by: Jean Delvare <khali@linux-fr.org>
Cc: Thomas Hellstrom <thellstrom@vmware.com>
Cc: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
We need the unlocked variant for the new codepath introduced to fix the
race condition in master recently.
Signed-off-by: Dave Airlie <airlied@redhat.com>
[airlied - add fix for vmwgfx build]
* 'nouveau/for-airlied' of ../drm-nouveau-next: (93 commits)
drm/ttm: restructure to allow driver to plug in alternate memory manager
drm/ttm: introduce utility function to free an allocated memory node
drm/nouveau: fix thinkos in mem timing table recordlen check
drm/nouveau: parse voltage from perf 0x40 entires
drm/nouveau: don't use the default pll limits in table v2.1 on nv50+ cards
drm/nv50: Fix large 3D performance regression caused by the interchannel sync patches.
drm/nouveau: Synchronize buffer object moves in hardware.
drm/nouveau: Use semaphores to handle inter-channel sync in hardware.
drm/nouveau: Provide a means to have arbitrary work run on fence completion.
drm/nouveau: Minor refactoring/cleanup of the fence code.
drm/nouveau: Add a module option to force card POST.
drm/nv50: prevent (IB_PUT == IB_GET) for occurring unless idle
drm/nv0x-nv4x: Leave the 0x40 bit untouched when changing CRE_LCD.
drm/nv30-nv40: Fix postdivider mask when writing engine/memory PLLs.
drm/nouveau: Fix perf table parsing on BMP v5.25.
drm/nouveau: fix required mode bandwidth calculation for DP
drm/nouveau: fix typo in c2aa91afea5f7e7ae4530fabd37414a79c03328c
drm/nva3: split pm backend out from nv50
drm/nouveau: run perflvl and M table scripts on mem clock change
drm/nouveau: pass perflvl struct to clock_pre()
...
This fixes a race pointed out by Dave Airlie where we don't take a buffer
object about to be destroyed off the LRU lists properly. It also fixes a rare
case where a buffer object could be destroyed in the middle of an
accelerated eviction.
The patch also adds a utility function that can be used to prematurely
release GPU memory space usage of an object waiting to be destroyed.
For example during eviction or swapout.
The above mentioned commit didn't queue the buffer on the delayed destroy
list under some rare circumstances. It also didn't completely honor the
remove_all parameter.
Fixes:
https://bugzilla.redhat.com/show_bug.cgi?id=615505http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=591061
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Nouveau will need this on GeForce 8 and up to account for the GPU
reordering physical VRAM for some memory types.
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Acked-by: Thomas Hellström <thellstrom@vmware.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Existing core code/drivers call drm_mm_put_block on ttm_mem_reg.mm_node
directly. Future patches will modify TTM behaviour in such a way that
ttm_mem_reg.mm_node doesn't necessarily belong to drm_mm.
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Acked-by: Thomas Hellström <thellstrom@vmware.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Correct allocation flags type and function prototype for ANSI C compliance.
[airlied: whitespace fixed]
Signed-off-by: Daniel J Blueman <daniel.blueman@gmail.com>
Reviewed-by: Alex Deucher <alexdeucher@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
It makes sense for a BO to move after a process has requested
exclusive RW access on it (e.g. because the BO used to be located in
unmappable VRAM and we intercepted the CPU access from the fault
handler).
If we let the ghost object inherit cpu_writers from the original
object, ttm_bo_release_list() will raise a kernel BUG when the ghost
object is destroyed. This can be reproduced with the nouveau driver on
nv5x.
Reported-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Tested-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Dave Airlie <airlied@redhat.com>
I wrote this for the prime sharing work, but I also noticed other external
non-upstream drivers from a large company carrying a similiar patch, so I
may as well ship it in master.
Signed-off-by: Dave Airlie <airlied@redhat.com>
* 'drm-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6:
drm/radeon/kms: add quirk to make HP DV5000 laptop resume
drm/radeon/kms: fix RADEON_INFO_CRTC_FROM_ID info ioctl
Fix ttm_page_alloc.c build breakage
drm/radeon/kms: fix legacy LVDS dpms sequence
drm/radeon/kms: drop taking lock around crtc lookup.
The commit 1e8655f873
drm/ttm: Fix build on architectures without AGP
looks at TTM_HAS_AGP before it has been set in ttm_bo_driver.h
Move the conditional inclusion of <asm/agp.h> *after* we have included
ttm_bo_driver.h
Signed-of-by: Tony Luck <tony.luck@intel.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Add the shrinkers missed in the first conversion of the API in
commit 7f8275d0d6 ("mm: add context argument to
shrinker callback").
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Make inclusion of <asm/agp.h> conditional on TTM_HAS_AGP. The use
of the functions declared in it is already conditional.
Reported-by: Geert Stappers <stappers@stappers.nl>
Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
Tested-by: Geert Stappers <stappers@stappers.nl>
Signed-off-by: Dave Airlie <airlied@redhat.com>
* drm-intel-lru:
drm: implement helper functions for scanning lru list
drm_mm: extract check_free_mm_node
drm: sane naming for drm_mm.c
drm: kill dead code in drm_mm.c
drm: kill drm_mm_node->private
drm: use list_for_each_entry in drm_mm.c
Only ever assigned, never used.
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
[glisse: I will re-add if needed for range-restricted allocations]
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Repeated ttm_page_alloc_init/fini fails noisily because the pool
manager kobj isn't zeroed out between uses (we could do just that but
statically allocated kobjects are generally considered a bad thing).
Move it to kzalloc'ed memory.
Note that this patch drops the refcounting behavior of the pool
allocator init/fini functions: it would have led to a race condition
in its current form, and anyway it was never exploited.
This fixes a regression with reloading kms modules at runtime, since
page allocator was introduced.
Signed-off-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Non pooled page allocation should have GFP_USER set so allocation
can wait and reclaim page from other process (ie non atomic).
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
* 'drm-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6: (41 commits)
drm/radeon/kms: make sure display hw is disabled when suspending
drm/vmwgfx: Allow userspace to change default layout. Bump minor.
drm/vmwgfx: Fix framebuffer modesetting
drm/vmwgfx: Fix vga save / restore with display topology.
vgaarb: use MIT license
vgaarb: convert pr_devel() to pr_debug()
drm: fix typos in Linux DRM Developer's Guide
drm/radeon/kms/pm: voltage fixes
drm/radeon/kms/pm: radeon_set_power_state fixes
drm/radeon/kms/pm: patch default power state with default clocks/voltages on r6xx+
drm/radeon/kms/pm: enable SetVoltage on r7xx/evergreen
drm/radeon/kms/pm: add support for SetVoltage cmd table (V2)
drm/radeon/kms/evergreen: add initial CS parser
drm/kms: disable/enable poll around switcheroo on/off
drm/nouveau: fixup confusion over which handle the DSM is hanging off.
drm/nouveau: attempt to get bios from ACPI v3
drm/nv50: cast IGP memory location to u64 before shifting
drm/ttm: Fix ttm_page_alloc.c
drm/ttm: Fix cached TTM page allocation.
drm/vmwgfx: Remove some leftover debug messages.
...
Fix a number of typos misspellings and checkpatch.pl warnings.
Replace "[ttm] " with TTM_PFX
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
This patch fixes a regression introduced with the pool page allocator
in the event that there are no highmem pages (for example x86_64),
in which case cached page allocation would fail.
Tested with the vmwgfx driver on a 64-bit vm.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
* 'drm-for-2.6.35' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6: (207 commits)
drm/radeon/kms/pm/r600: select the mid clock mode for single head low profile
drm/radeon: fix power supply kconfig interaction.
drm/radeon/kms: record object that have been list reserved
drm/radeon: AGP memory is only I/O if the aperture can be mapped by the CPU.
drm/radeon/kms: don't default display priority to high on rs4xx
drm/edid: fix typo in 1600x1200@75 mode
drm/nouveau: fix i2c-related init table handlers
drm/nouveau: support init table i2c device identifier 0x81
drm/nouveau: ensure we've parsed i2c table entry for INIT_*I2C* handlers
drm/nouveau: display error message for any failed init table opcode
drm/nouveau: fix init table handlers to return proper error codes
drm/nv50: support fractional feedback divider on newer chips
drm/nv50: fix monitor detection on certain chipsets
drm/nv50: store full dcb i2c entry from vbios
drm/nv50: fix suspend/resume with DP outputs
drm/nv50: output calculated crtc pll when debugging on
drm/nouveau: dump pll limits entries when debugging is on
drm/nouveau: bios parser fixes for eDP boards
drm/nouveau: fix a nouveau_bo dereference after it's been destroyed
drm/nv40: remove some completed ctxprog TODOs
...
We want to be able to prevent the delayed workqueue from changing state
while we're reclocking, so add an API to block and unblock it.
Signed-off-by: Matthew Garrett <mjg@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
It's unused and buggy in its current form, since it can place a bo
in the reserved state without removing it from lru lists.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
If the memory is not iomem we should not try to
ioremap it. Should fix :
https://bugs.freedesktop.org/show_bug.cgi?id=27822
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Tested-by: Rafał Miłecki <zajec5@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
* drm-ttm-unmappable:
drm/radeon/kms: enable use of unmappable VRAM V2
drm/ttm: remove io_ field from TTM V6
drm/vmwgfx: add support for new TTM fault callback V5
drm/nouveau/kms: add support for new TTM fault callback V5
drm/radeon/kms: add support for new fault callback V7
drm/ttm: ttm_fault callback to allow driver to handle bo placement V6
drm/ttm: split no_wait argument in 2 GPU or reserve wait
Conflicts:
drivers/gpu/drm/nouveau/nouveau_bo.c
All TTM driver have been converted to new io_mem_reserve/free
interface which allow driver to choose and return proper io
base, offset to core TTM for ioremapping if necessary. This
patch remove what is now deadcode.
V2 adapt to match with change in first patch of the patchset
V3 update after io_mem_reserve/io_mem_free callback balancing
V4 adjust to minor cleanup
V5 remove the needs ioremap flag
V6 keep the ioremapping facility in TTM
[airlied- squashed driver removals in here also]
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
On fault the driver is given the opportunity to perform any operation
it sees fit in order to place the buffer into a CPU visible area of
memory. This patch doesn't break TTM users, nouveau, vmwgfx and radeon
should keep working properly. Future patch will take advantage of this
infrastructure and remove the old path from TTM once driver are
converted.
V2 return VM_FAULT_NOPAGE if callback return -EBUSY or -ERESTARTSYS
V3 balance io_mem_reserve and io_mem_free call, fault_reserve_notify
is responsible to perform any necessary task for mapping to succeed
V4 minor cleanup, atomic_t -> bool as member is protected by reserve
mecanism from concurent access
V5 the callback is now responsible for iomapping the bo and providing
a virtual address this simplify TTM and will allow to get rid of
TTM_MEMTYPE_FLAG_NEEDS_IOREMAP
V6 use the bus addr data to decide to ioremap or this isn't needed
but we don't necesarily need to ioremap in the callback but still
allow driver to use static mapping
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
* drm-ttm-pool:
drm/ttm: using kmalloc/kfree requires including slab.h
drm/ttm: include linux/seq_file.h for seq_printf
drm/ttm: Add sysfs interface to control pool allocator.
drm/ttm: Use set_pages_array_wc instead of set_memory_wc.
arch/x86: Add array variants for setting memory to wc caching.
drm/nouveau: Add ttm page pool debugfs file.
drm/radeon/kms: Add ttm page pool debugfs file.
drm/ttm: Add debugfs output entry to pool allocator.
drm/ttm: add pool wc/uc page allocator V3
There is case where we want to be able to wait only for the
GPU while not waiting for other buffer to be unreserved. This
patch split the no_wait argument all the way down in the whole
ttm path so that upper level can decide on what to wait on or
not.
[airlied: squashed these 4 for bisectability reasons.]
drm/radeon/kms: update to TTM no_wait splitted argument
drm/nouveau: update to TTM no_wait splitted argument
drm/vmwgfx: update to TTM no_wait splitted argument
[vmwgfx patch: Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>]
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Acked-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Fixes
drivers/gpu/drm/ttm/ttm_page_alloc.c: In function 'ttm_page_alloc_debugfs':
drivers/gpu/drm/ttm/ttm_page_alloc.c:829: error: implicit declaration of
function 'seq_printf'
Signed-off-by: Matt Turner <mattst88@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Sysfs interface allows user to configure pool allocator functionality and
change limits for the size of pool.
Signed-off-by: Pauli Nieminen <suokkos@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Using single call to set multiple pages to wc reduces number of expensive cache
flushes.
Signed-off-by: Pauli Nieminen <suokkos@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
ttm_page_alloc_debugfs can be registered to output the state
of pools.
Debugfs file will output number of pages freed from the pool,
number of pages in pool now and the lowes number of pages in
pool since previous shrink.
Signed-off-by: Pauli Nieminen <suokkos@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
On AGP system we might allocate/free routinely uncached or wc memory,
changing page from cached (wb) to uc or wc is very expensive and involves
a lot of flushing. To improve performance this allocator use a pool
of uc,wc pages.
Pools are protected with spinlocks to allow multiple threads to allocate pages
simultanously. Expensive operations are done outside of spinlock to maximize
concurrency.
Pools are linked lists of pages that were recently freed. mm shrink callback
allows kernel to claim back pages when they are required for something else.
Fixes:
* set_pages_array_wb handles highmem pages so we don't have to remove them
from pool.
* Add count parameter to ttm_put_pages to avoid looping in free code.
* Change looping from _safe to normal in pool fill error path.
* Initialize sum variable and make the loop prettier in get_num_unused_pages.
* Moved pages_freed reseting inside the loop in ttm_page_pool_free.
* Add warning comment about spinlock context in ttm_page_pool_free.
Based on Jerome Glisse's and Dave Airlie's pool allocator.
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Pauli Nieminen <suokkos@gmail.com>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.
percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.
http://userweb.kernel.org/~tj/misc/slabh-sweep.py
The script does the followings.
* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.
* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.
* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.
The conversion was done in the following steps.
1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.
2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.
3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.
4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.
5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.
6. percpu.h was updated not to include slab.h.
7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).
* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig
8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.
Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
Replace sequential calls to kobject_init() and kobject_add() with the
combo wrapper kobject_init_and_add(), which provides the same
semantics.
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Now that the drm core can do this, lets just use it, split the code out
so TTM doesn't have to drag all of drmP.h in.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Constify struct sysfs_ops.
This is part of the ops structure constification
effort started by Arjan van de Ven et al.
Benefits of this constification:
* prevents modification of data that is shared
(referenced) by many other structure instances
at runtime
* detects/prevents accidental (but not intentional)
modification attempts on archs that enforce
read-only kernel data at runtime
* potentially better optimized code as the compiler
can assume that the const data cannot be changed
* the compiler/linker move const data into .rodata
and therefore exclude them from false sharing
Signed-off-by: Emese Revfy <re.emese@gmail.com>
Acked-by: David Teigland <teigland@redhat.com>
Acked-by: Matt Domsch <Matt_Domsch@dell.com>
Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com>
Acked-by: Hans J. Koch <hjk@linutronix.de>
Acked-by: Pekka Enberg <penberg@cs.helsinki.fi>
Acked-by: Jens Axboe <jens.axboe@oracle.com>
Acked-by: Stephen Hemminger <shemminger@vyatta.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
- Without this change I get a general protection fault.
- Also use PTR_ERR where applicable.
Signed-off-by: Maarten Maathuis <madman2003@gmail.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Acked-by: Thomas Hellstrom <thellstrom@vmware.com>
Cc: stable@kernel.org
Signed-off-by: Dave Airlie <airlied@redhat.com>
http://bugzilla.kernel.org/show_bug.cgi?id=15328
This fixes a serious regression on AGP/non-PAT systems, where
pages were ending up in the wrong state and slowing down the
whole system.
[airlied: taken this from the bug as the other option is to revert
the change which caused it].
Tested-by: John W. Linville (in bug).
Signed-off-by: Dave Airlie <airlied@redhat.com>
If the buffer object was already in the requested memory type, but
outside of the requested range it was never moved into the requested range.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
* 'drm-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6:
drm/radeon/kms: Fix oops after radeon_cs_parser_init() failure.
drm/radeon/kms: move radeon KMS on/off switch out of staging.
drm/radeon/kms: Bailout of blit if error happen & protect with mutex V3
drm/vmwgfx: Don't send bad flags to the host
drm/vmwgfx: Request SVGA version 2 and bail if not found
drm/vmwgfx: Correctly detect 3D
drm/ttm: remove unnecessary save_flags and ttm_flag_masked in ttm_bo_util.c
drm/kms: Remove incorrect comment in struct drm_mode_modeinfo
drm/ttm: remove padding from ttm_ref_object on 64bit builds
drm/radeon/kms: release agp on error.
drm/kms/radeon/agp: Move the check of the aper_size after drm_acp_acquire and drm_agp_info
drm/kms/radeon/agp: Fix warning, format ‘%d’ expects type ‘int’, but argument 4 has type ‘size_t’
drm/ttm: Avoid conflicting reserve_memtype during ttm_tt_set_page_caching.
drm/kms/radeon: pick digitial encoders smarter. (v3)
drm/radeon/kms: use active device to pick connector for encoder
drm/radeon/kms: fix incorrect logic in DP vs eDP connector checking.
Re-order structure ttm_ref_object to remove 8 bytes of alignment padding
on 64 bit builds, so shrinking its size from 72 to 64 bytes allowing it
to fit into a smaller slab.
Signed-off-by: Richard Kennedy <richard@rsk.demon.co.uk>
Acked-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Fixes errors like:
> reserve_ram_pages_type failed 0x15b7a000-0x15b7b000, track 0x8, req 0x10
when a BO is moved between WC and UC areas.
Reported-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Francisco Jerez <currojerez@riseup.net>
Acked-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
* 'drm-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6: (95 commits)
drm/radeon/kms: preface warning printk with driver name
drm/radeon/kms: drop unnecessary printks.
drm: fix regression in fb blank handling
drm/radeon/kms: make hibernate work on IGPs
drm/vmwgfx: Optimize memory footprint for DMA buffers.
drm/ttm: Allow system memory as a busy placement.
drm/ttm: Fix race condition in ttm_bo_delayed_delete (v3, final)
drm/nv50: prevent switching off SOR when in use for DVI-over-DP
drm/nv50: fail auxch transaction if reply count not what we expect
drm/nouveau: fix failure path if userspace specifies no valid memtypes
drm/nouveau: report LVDS as disconnected if lid closed
drm/radeon/kms: fix legacy get_engine/memory clock
drm/radeon/kms/atom: atom parser fixes
drm/radeon/kms: clean up atombios pll code
drm/radeon/kms: clean up pll struct
drm/radeon/kms/atom: fix crtc lock ordering
drm/radeon: r6xx/r7xx possible security issue, system ram access
drm/radeon/kms: r600/r700 don't test ib if ib initialization fails
drm/radeon/kms: Forbid creation of framebuffer with no valid GEM object
drm/radeon/kms: r600 handle irq vector ring overflow
...
This is needed to fix a vmwgfx memory usage bug.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Resending this with Thomas Hellstrom's signoff for merging into 2.6.33
ttm_bo_delayed_delete has a race condition, because after we do:
kref_put(&nentry->list_kref, ttm_bo_release_list);
we are not holding the list lock and not holding any reference to
objects, and thus every bo in the list can be removed and freed at
this point.
However, we then use the next pointer we stored, which is not guaranteed
to be valid.
This was apparently the cause of some Nouveau oopses I experienced.
This patch rewrites the function so that it keeps the reference to nentry
until nentry itself is freed and we already got a reference to nentry->next.
v2 updated by me according to Thomas Hellstrom's feedback.
v3 proposed by Thomas Hellstrom. Commit comment updated by me.
Both updates fixed minor efficiency/style issues only and all three versions
should be correct.
Signed-off-by: Luca Barbieri <luca@luca-barbieri.com>
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
This is a convention that the vmwgfx driver has come to rely on.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
This is needed for a bugfix in the vmwgfx driver.
Drivers may have GPU bindings on buffers that core TTM is not aware of,
and TTM may view those buffers as ordinary system memory buffers.
Add a notifier to such drivers when TTM is about to move the buffer
contents out to swappable memory. The driver must then release any
private GPU bindings on those buffers.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
* 'drm-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6:
drm/radeon/kms: fix r100->r500 CS checker for compressed textures. (v2)
drm/radeon/kms: allow for texture tiling
drm/radeon/kms: init pm on all chipsets
drm/radeon/kms: HDMI support for R600 KMS
drm/radeon/kms: make sure mc is initialized before mapping blit bo
drm/radeon/kms: Return to userspace on ERESTARTSYS
drm/radeon/gem: don't leak a gem object if reserve fails on get tiling (v2)
drm/radeon/kms: don't report allocate failure on ERESTARTSYS
drm/radeon/kms: Check if bo we got from ttm are radeon object or not
drm/radeon/kms: If no placement is supplied fallback to system
drm/ttm: Fix memory type manager debug information printing
drm/ttm: Fix printk format & compute bo->mem.size at bo initialization
drm/ttm: Fix potential ttm_mem_evict_first races.
drm/ttm: Delayed delete fixes.
drm/ttm: fix two bugs in new placement routines.
drm/ttm: fix incorrect logic in ttm_bo_io path
drm/nouveau: remove use of -ERESTART
nouveau: Fix endianness with new context program loader
drm/nouveau: fix build with CONFIG_AGP=n
drm/nouveau: fix ch7006 build
System memory type doesn't have a drm_mm manager associated to
it. This patch avoid trying to call drm_mm_debug on unitialized
drm_mm when printing debug info on the system memory manager.
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
1) The function was previously called with a potentially empty
LRU list which would have lead to an OOPS or servere corruption.
2) In rare cases, after reservation has succeeded, another process may
already have evicted it or even pinned it. We must revalidate the
buffer status after releasing the lru lock.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
1) Remove from lru before reserving so we avoid competing with
evicting processes.
2) Avoid calling kref_put() on bo::list_kref while spinlocked.
3) Additional refcounting bug-checking.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
a) the loops were going to <= not <, leading to illegal memory access
b) the busy placement checks were using the placement arrays not the
busy placement ones.
Acked-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
This path isn't used by radeon yet, but future drivers will want it,
so fix it right.
Reported-by: Luca Barbieri <luca@luca-barbieri.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
* 'drm-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6:
drm/ttm: export some functions useful to drivers using ttm
drm/radeon/kms/avivo: fix typo in new_pll module description
drm/radeon/kms: Convert radeon to new ttm_bo_init
drm/ttm: Convert ttm_buffer_object_init to use ttm_placement
* 'drm-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6: (189 commits)
drm/radeon/kms: fix warning about cur_placement being uninitialised.
drm/ttm: Print debug information on memory manager when eviction fails
drm: Add memory manager debug function
drm/radeon/kms: restore surface registers on resume.
drm/radeon/kms/r600/r700: fallback gracefully on ucode failure
drm/ttm: Initialize eviction placement in case the driver callback doesn't
drm/radeon/kms: cleanup structure and module if initialization fails
drm/radeon/kms: actualy set the eviction placements we choose
drm/radeon/kms: Fix NULL ptr dereference
drm/radeon/kms/avivo: add support for new pll selection algo
drm/radeon/kms/avivo: fix some bugs in the display bandwidth setup
drm/radeon/kms: fix return value from fence function.
drm/radeon: Remove tests for -ERESTART from the TTM code.
drm/ttm: Have the TTM code return -ERESTARTSYS instead of -ERESTART.
drm/radeon/kms: Convert radeon to new TTM validation API (V2)
drm/ttm: Rework validation & memory space allocation (V3)
drm: Add search/get functions to get a block in a specific range
drm/radeon/kms: fix avivo tiling regression since radeon object rework
drm/i915: Remove a debugging printk from hangcheck
drm/radeon/kms: make sure i2c id matches
...
These are functions required by nouveau which will be merged later.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Convert ttm_buffer_object_init to use struct ttm_placement and
rename to ttm_bo_init for consistency with function naming. This
allow to give more complex placement at buffer creation. For
instance you ask to allocate bo into vram first but if there is
not enough vram you can give system as a second possible
placement. It also allow to create buffer in a specific range.
Also rename ttm_buffer_object_validate to ttm_bo_validate.
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
This add helper function to print information on eviction placements
and memory manager status when eviction fails to allocate memory
space.
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
This would allow to catch driver callback error of not properly
setting the eviction placement structure.
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@linux.ie>
Return -ERESTARTSYS instead of -ERESTART when interrupted by a signal.
The -ERESTARTSYS is converted to an -EINTR by the kernel signal layer
before returned to user-space.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
This change allow driver to pass sorted memory placement,
from most prefered placement to least prefered placement.
In order to avoid long function prototype a structure is
used to gather memory placement informations such as range
restriction (if you need a buffer to be in given range).
Range restriction is determined by fpfn & lpfn which are
the first page and last page number btw which allocation
can happen. If those fields are set to 0 ttm will assume
buffer can be put anywhere in the address space (thus it
avoids putting a burden on the driver to always properly
set those fields).
This patch also factor few functions like evicting first
entry of lru list or getting a memory space. This avoid
code duplication.
V2: Change API to use placement flags and array instead
of packing placement order into a quadword.
V3: Make sure we set the appropriate mem.placement flag
when validating or allocation memory space.
[Pending Thomas Hellstrom further review but okay
from preliminary review so far].
Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
None of the in-tree drivers use user objects yet so this wasn't hitting
us.
Stanse found unreachable code in ttm_bo_add_ttm:
http://decibel.fi.muni.cz/~xslaby/stanse/error.cgi?db=32&id=714#l238
Reported-by: Jiri Slaby <jirislaby@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Utilities to reserve, unreserve and fence a list of TTM
buffer objects in a deadlock-safe manner.
Used by the vmwgfx driver.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
This is intended to be used by ttm-aware drivers to
1) Block clients to inactive masters when
they try to validate buffers for GPU use.
2) Optionally block clients to the current master when
there is thrashing due to GPU memory shortage.
Used by the vmwgfx driver.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Add objects needed for user-space to maintain reference counts on ttm objects.
This is used by the vmwgfx driver which allows user-space to maintain
map-counts on dma buffers, lock-counts on the ttm lock and ref-counts on
gpu surfaces, gpu contexts and dma buffer.
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
I moved the allocation until after the check for (si->totalhigh == 0).
Signed-off-by: Dan Carpenter <error27@gmail.com>
Acked-By: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
* 'drm-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6: (52 commits)
drm/kms: Init the CRTC info fields for modes forced from the command line.
drm/radeon/r600: CS parser updates
drm/radeon/kms: add debugfs for power management for AtomBIOS devices
drm/radeon/kms: initial mode validation support
drm/radeon/kms/atom/dce3: call transmitter init on mode set
drm/radeon/kms: store detailed connector info
drm/radeon/kms/atom/dce3: fix up usPixelClock calculation for Transmitter tables
drm/radeon/kms/r600: fix rs880 support v2
drm/radeon/kms/r700: fix some typos in chip init
drm/radeon/kms: remove some misleading debugging output
drm/radeon/kms: stop putting VRAM at 0 in MC space on r600s.
drm/radeon/kms: disable D1VGA and D2VGA if enabled
drm/radeon/kms: Don't RMW CP_RB_CNTL
drm/radeon/kms: fix coherency issues on AGP cards.
drm/radeon/kms: fix rc410 suspend/resume.
drm/radeon/kms: add quirk for hp dc5750
drm/radeon/kms/atom: fix potential oops in spread spectrum code
drm/kms: typo fix
drm/radeon/kms/atom: Make card_info per device
drm/radeon/kms/atom: Fix DVO support
...
When we are evicting from VRAM->RAM we allocate the ttm object,
but we don't set the caching policy on it before blitting into it.
This means on AGP we end up blitting into cached pages, and
the CPU later flushes out on top of them. This was mostly seen as
font corruption.
The other question is why we don't evict VRAM->GTT in a lot of cases,
this would save us some cache transitions since a lot of objects
that are evicted from VRAM will probably end up being pulled back in
a few operations later, and evicting them to system memory involves
2 unnecessary cache transitions.
Signed-off-by: Dave Airlie <airlied@redhat.com>
conflict in radeon since new init path merged with vga arb code.
Conflicts:
drivers/gpu/drm/radeon/radeon.h
drivers/gpu/drm/radeon/radeon_asic.h
drivers/gpu/drm/radeon/radeon_device.c
Jerome Glisse