Commit Graph

631 Commits

Author SHA1 Message Date
Christian König 56b3d20413 drm/ttm: make the device list mutex static
This way it can protect the whole BO global state.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-11-05 14:21:19 -05:00
Christian König 27eb1fa913 drm/ttm: use a static ttm_mem_global instance
As the name says we only need one global instance of ttm_mem_global.

Drop all the driver initialization and just use a single exported
instance which is initialized during BO global initialization.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-11-05 14:21:18 -05:00
Thomas Zimmermann 105f20706f drm/ttm: Provide ttm_bo_global_{init/release}() for struct ttm_bo_global
So far, struct ttm_bo_global_ref was the only way of initializing a struct
ttm_bo_global. Providing separate initializer and release functions for
struct ttm_bo_global gives drivers the option of implementing their own
init and release callbacks for drm_global_references of type
DRM_GLOBAL_TTM_BO.

The original functions for initializing and releasing via struct
ttm_bo_global_ref are wrappers around the new interfaces.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-11-05 14:20:52 -05:00
Thomas Zimmermann e55a5c9b5f drm/ttm: Rename ttm_bo_global_{init,release}() to ttm_bo_global_ref_{,}()
The functions ttm_bo_global_init() and ttm_bo_global_release() do not
receive an argument of type struct ttm_bo_global. Both take a struct
drm_global_reference that contains points to a struct ttm_bo_global_ref.
Renaming them reflects this.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-11-05 14:20:52 -05:00
Christian König ca05359f1e dma-buf: allow reserving more than one shared fence slot
Let's support simultaneous submissions to multiple engines.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Link: https://patchwork.kernel.org/patch/10626149/
2018-10-25 13:45:07 +02:00
Christian König df2fc43d09 list: introduce list_bulk_move_tail helper
Move all entries between @first and including @last before @head.

This is useful for LRU lists where a whole block of entries should be
moved to the end of the list.

Used as a band aid in TTM, but better placed in the common list headers.

Acked-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-10-10 15:20:54 -05:00
Thomas Hellstrom 24dc64c1ba drm/ttm: Export ttm_bo_get_unless_zero()
Export ttm_bo_get_unless_zero() to be used when looking up buffer
objects that are removed from the lookup structure in the destructor.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Sinclair Yeh <syeh@vmware.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
2018-09-28 08:55:10 +02:00
Thomas Hellstrom 0b8762e997 drm/ttm, drm/vmwgfx: Move the lock- and object functionality to the vmwgfx driver
No other driver is using this functionality so move it out of TTM and
into the vmwgfx driver. Update includes and remove exports.
Also annotate to remove false static analyzer lock balance warnings.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
2018-09-27 15:19:20 +02:00
Christian König 03651735fb drm/ttm: once more fix ttm_bo_bulk_move_lru_tail
While cutting the lists we sometimes accidentally added a list_head from
the stack to the LRUs, effectively corrupting the list.

Remove the list cutting and use explicit list manipulation instead.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-and-Tested: Huang Rui <ray.huang@amd.com>
Tested-by: Mike Lothian <mike@fireburn.co.uk>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-09-13 15:14:11 -05:00
Christian König 972a21f946 drm/ttm: fix ttm_bo_bulk_move_helper
Staring at the function for six hours, just to essentially move one line
of code. The problem was that the first list_cut_position call could result
in list2 pointing to la-la-land.

Signed-off-by: Christian König <christian.koenig@amd.com>
Tested-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-09-02 10:16:32 -05:00
Michel Dänzer 25eef4214a drm/ttm: Initialize local lists in ttm_bo_bulk_move_helper
The first parameter of list_cut_position() must point to an initialized
list.

Noticed thanks to KASAN pointing out something's fishy here.

Fixes: "drm/ttm: add bulk move function on LRU"
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-08-29 12:35:59 -05:00
Huang Rui 7748e2dcda drm/ttm: add bulk move function on LRU
This function allow us to bulk move a group of BOs to the tail of their LRU.
The positions of group of BOs are stored on the (first, last) bulk_move_pos
structure.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Huang Rui <ray.huang@amd.com>
Tested-by: Mike Lothian <mike@fireburn.co.uk>
Tested-by: Dieter Nützel <Dieter@nuetzel-hh.de>
Acked-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-08-27 11:11:21 -05:00
Christian König 9a2779528e drm/ttm: revise ttm_bo_move_to_lru_tail to support bulk moves
When move a BO to the end of LRU, it need remember the BO positions.
Make sure all moved bo in between "first" and "last". And they will be bulk
moving together.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Huang Rui <ray.huang@amd.com>
Tested-by: Mike Lothian <mike@fireburn.co.uk>
Tested-by: Dieter Nützel <Dieter@nuetzel-hh.de>
Acked-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-08-27 11:11:21 -05:00
Huang Rui 02b29caf18 drm/ttm: remove dead codes
These codes are not used.

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-08-27 11:11:02 -05:00
Huang Rui df36b2fb83 drm/ttm: clean up non-x86 definitions on ttm_tt
All non-x86 definitions are moved to ttm_set_memory header, so remove it from
ttm_tt.c.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-08-01 17:23:56 -05:00
Huang Rui fe710322b8 drm/ttm: fix missed conversion of set_pages_array_uc
This patch fixed the error when do not configure CONFIG_X86, otherwise, below
error will be encountered.

All errors (new ones prefixed by >>):

   drivers/gpu/drm/ttm/ttm_page_alloc_dma.c: In function 'ttm_set_pages_caching':
>> drivers/gpu/drm/ttm/ttm_page_alloc_dma.c:272:7: error: implicit declaration of function 'set_pages_array_uc'; did you mean
+'ttm_set_pages_array_uc'? [-Werror=implicit-function-declaration]
      r = set_pages_array_uc(pages, cpages);
          ^~~~~~~~~~~~~~~~~~
          ttm_set_pages_array_uc
   cc1: some warnings being treated as errors

Reported-by: kbuild test robot <lkp@intel.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-08-01 17:22:20 -05:00
Bas Nieuwenhuizen 610b399f1f drm/ttm: Merge hugepage attr changes in ttm_dma_page_put. (v2)
Every set_pages_array_wb call resulted in cross-core
interrupts and TLB flushes. Merge more of them for
less overhead.

This reduces the time needed to free a 1.6 GiB GTT WC
buffer as part of Vulkan CTS from  ~2 sec to < 0.25 sec.
(Allocation still takes more than 2 sec though)

(v2): use set_pages_wb instead of set_memory_wb.

Signed-off-by: Bas Nieuwenhuizen <basni@chromium.org>
Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-07-27 15:00:22 -05:00
Huang Rui d55f9b8742 drm/ttm: clean up non-x86 definitions on ttm_page_alloc
All non-x86 definitions are moved to ttm_set_memory header, so remove it from
ttm_page_alloc.c.

Suggested-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Bas Nieuwenhuizen <basni@chromium.org>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-07-27 15:00:14 -05:00
Huang Rui c7bb1e57e2 drm/ttm: clean up non-x86 definitions on ttm_page_alloc_dma
All non-x86 definitions are moved to ttm_set_memory header, so remove it from
ttm_page_alloc_dma.c.

Suggested-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Bas Nieuwenhuizen <basni@chromium.org>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-07-27 15:00:08 -05:00
Thomas Zimmermann f44907593d drm/ttm: Replace ttm_bo_unref() with ttm_bo_put()
A call to ttm_bo_unref() clears the supplied pointer to NULL, while
ttm_bo_put() does not. None of the converted call sites requires the
pointer to become NULL, so the respective assign operations has been
left out from the patch.

Signed-off-by: Thomas Zimmermann <contact@tzimmermann.org>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-07-10 14:18:28 -05:00
Thomas Zimmermann 8129fdad38 drm/ttm: Replace ttm_bo_reference() with ttm_bo_get()
Signed-off-by: Thomas Zimmermann <contact@tzimmermann.org>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-07-10 14:18:22 -05:00
Thomas Zimmermann 89c815ef07 drm/ttm: Introduce ttm_bo_get() and ttm_bo_put() for ref counting
The TTM buffer-object interface provides ttm_bo_reference() and
ttm_bo_unref() for managing reference counts. Replacing them with
ttm_bo_get() and ttm_bo_put() aligns the API with conventions used
throughout the Linux kernel.

The implementation of ttm_bo_unref() clears the supplied pointer
to NULL. This leads to workarounds where the caller saves the
pointer's value before de-referencing the BO. ttm_bo_put() does
not clear the supplied pointer.

Signed-off-by: Thomas Zimmermann <contact@tzimmermann.org>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-07-10 14:18:15 -05:00
Gustavo A. R. Silva 31e1c59796 drm/ttm: use swap macro in ttm_bo_handle_move_mem
Make use of the swap macro and remove unnecessary variable *tmp_mem*.
This makes the code easier to read and maintain. Also, reduces the
stack usage.

This code was detected with the help of Coccinelle.

Signed-off-by: Gustavo A. R. Silva <gustavo@embeddedor.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-07-10 14:18:08 -05:00
Dave Airlie 565c17b5f0 Merge branch 'drm-next-4.19' of git://people.freedesktop.org/~agd5f/linux into drm-next
First feature request for 4.19.  Highlights:
- Add initial amdgpu documentation
- Add initial GPU scheduler documention
- GPU scheduler fixes for dying processes
- Add support for the JPEG engine on VCN
- Switch CI to use powerplay by default
- EDC support for CZ
- More powerplay cleanups
- Misc DC fixes

Signed-off-by: Dave Airlie <airlied@redhat.com>

Link: https://patchwork.freedesktop.org/patch/msgid/20180621161138.3008-1-alexander.deucher@amd.com
2018-06-22 13:19:05 +10:00
Souptick Joarder 4daa4fba3a gpu: drm: ttm: Adding new return type vm_fault_t
Use new return type vm_fault_t for fault handler. For
now, this is just documenting that the function returns
a VM_FAULT value rather than an errno. Once all instances
are converted, vm_fault_t will become a distinct type.

Ref-> commit 1c8f422059 ("mm: change return type to vm_fault_t")

Previously vm_insert_{mixed,pfn} returns err which driver
mapped into VM_FAULT_* type. The new function
vmf_insert_{mixed,pfn} will replace this inefficiency by
returning VM_FAULT_* type.

Signed-off-by: Souptick Joarder <jrdr.linux@gmail.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-06-19 13:17:38 -05:00
Kees Cook 6da2ec5605 treewide: kmalloc() -> kmalloc_array()
The kmalloc() function has a 2-factor argument form, kmalloc_array(). This
patch replaces cases of:

        kmalloc(a * b, gfp)

with:
        kmalloc_array(a * b, gfp)

as well as handling cases of:

        kmalloc(a * b * c, gfp)

with:

        kmalloc(array3_size(a, b, c), gfp)

as it's slightly less ugly than:

        kmalloc_array(array_size(a, b), c, gfp)

This does, however, attempt to ignore constant size factors like:

        kmalloc(4 * 1024, gfp)

though any constants defined via macros get caught up in the conversion.

Any factors with a sizeof() of "unsigned char", "char", and "u8" were
dropped, since they're redundant.

The tools/ directory was manually excluded, since it has its own
implementation of kmalloc().

The Coccinelle script used for this was:

// Fix redundant parens around sizeof().
@@
type TYPE;
expression THING, E;
@@

(
  kmalloc(
-	(sizeof(TYPE)) * E
+	sizeof(TYPE) * E
  , ...)
|
  kmalloc(
-	(sizeof(THING)) * E
+	sizeof(THING) * E
  , ...)
)

// Drop single-byte sizes and redundant parens.
@@
expression COUNT;
typedef u8;
typedef __u8;
@@

(
  kmalloc(
-	sizeof(u8) * (COUNT)
+	COUNT
  , ...)
|
  kmalloc(
-	sizeof(__u8) * (COUNT)
+	COUNT
  , ...)
|
  kmalloc(
-	sizeof(char) * (COUNT)
+	COUNT
  , ...)
|
  kmalloc(
-	sizeof(unsigned char) * (COUNT)
+	COUNT
  , ...)
|
  kmalloc(
-	sizeof(u8) * COUNT
+	COUNT
  , ...)
|
  kmalloc(
-	sizeof(__u8) * COUNT
+	COUNT
  , ...)
|
  kmalloc(
-	sizeof(char) * COUNT
+	COUNT
  , ...)
|
  kmalloc(
-	sizeof(unsigned char) * COUNT
+	COUNT
  , ...)
)

// 2-factor product with sizeof(type/expression) and identifier or constant.
@@
type TYPE;
expression THING;
identifier COUNT_ID;
constant COUNT_CONST;
@@

(
- kmalloc
+ kmalloc_array
  (
-	sizeof(TYPE) * (COUNT_ID)
+	COUNT_ID, sizeof(TYPE)
  , ...)
|
- kmalloc
+ kmalloc_array
  (
-	sizeof(TYPE) * COUNT_ID
+	COUNT_ID, sizeof(TYPE)
  , ...)
|
- kmalloc
+ kmalloc_array
  (
-	sizeof(TYPE) * (COUNT_CONST)
+	COUNT_CONST, sizeof(TYPE)
  , ...)
|
- kmalloc
+ kmalloc_array
  (
-	sizeof(TYPE) * COUNT_CONST
+	COUNT_CONST, sizeof(TYPE)
  , ...)
|
- kmalloc
+ kmalloc_array
  (
-	sizeof(THING) * (COUNT_ID)
+	COUNT_ID, sizeof(THING)
  , ...)
|
- kmalloc
+ kmalloc_array
  (
-	sizeof(THING) * COUNT_ID
+	COUNT_ID, sizeof(THING)
  , ...)
|
- kmalloc
+ kmalloc_array
  (
-	sizeof(THING) * (COUNT_CONST)
+	COUNT_CONST, sizeof(THING)
  , ...)
|
- kmalloc
+ kmalloc_array
  (
-	sizeof(THING) * COUNT_CONST
+	COUNT_CONST, sizeof(THING)
  , ...)
)

// 2-factor product, only identifiers.
@@
identifier SIZE, COUNT;
@@

- kmalloc
+ kmalloc_array
  (
-	SIZE * COUNT
+	COUNT, SIZE
  , ...)

// 3-factor product with 1 sizeof(type) or sizeof(expression), with
// redundant parens removed.
@@
expression THING;
identifier STRIDE, COUNT;
type TYPE;
@@

(
  kmalloc(
-	sizeof(TYPE) * (COUNT) * (STRIDE)
+	array3_size(COUNT, STRIDE, sizeof(TYPE))
  , ...)
|
  kmalloc(
-	sizeof(TYPE) * (COUNT) * STRIDE
+	array3_size(COUNT, STRIDE, sizeof(TYPE))
  , ...)
|
  kmalloc(
-	sizeof(TYPE) * COUNT * (STRIDE)
+	array3_size(COUNT, STRIDE, sizeof(TYPE))
  , ...)
|
  kmalloc(
-	sizeof(TYPE) * COUNT * STRIDE
+	array3_size(COUNT, STRIDE, sizeof(TYPE))
  , ...)
|
  kmalloc(
-	sizeof(THING) * (COUNT) * (STRIDE)
+	array3_size(COUNT, STRIDE, sizeof(THING))
  , ...)
|
  kmalloc(
-	sizeof(THING) * (COUNT) * STRIDE
+	array3_size(COUNT, STRIDE, sizeof(THING))
  , ...)
|
  kmalloc(
-	sizeof(THING) * COUNT * (STRIDE)
+	array3_size(COUNT, STRIDE, sizeof(THING))
  , ...)
|
  kmalloc(
-	sizeof(THING) * COUNT * STRIDE
+	array3_size(COUNT, STRIDE, sizeof(THING))
  , ...)
)

// 3-factor product with 2 sizeof(variable), with redundant parens removed.
@@
expression THING1, THING2;
identifier COUNT;
type TYPE1, TYPE2;
@@

(
  kmalloc(
-	sizeof(TYPE1) * sizeof(TYPE2) * COUNT
+	array3_size(COUNT, sizeof(TYPE1), sizeof(TYPE2))
  , ...)
|
  kmalloc(
-	sizeof(TYPE1) * sizeof(THING2) * (COUNT)
+	array3_size(COUNT, sizeof(TYPE1), sizeof(TYPE2))
  , ...)
|
  kmalloc(
-	sizeof(THING1) * sizeof(THING2) * COUNT
+	array3_size(COUNT, sizeof(THING1), sizeof(THING2))
  , ...)
|
  kmalloc(
-	sizeof(THING1) * sizeof(THING2) * (COUNT)
+	array3_size(COUNT, sizeof(THING1), sizeof(THING2))
  , ...)
|
  kmalloc(
-	sizeof(TYPE1) * sizeof(THING2) * COUNT
+	array3_size(COUNT, sizeof(TYPE1), sizeof(THING2))
  , ...)
|
  kmalloc(
-	sizeof(TYPE1) * sizeof(THING2) * (COUNT)
+	array3_size(COUNT, sizeof(TYPE1), sizeof(THING2))
  , ...)
)

// 3-factor product, only identifiers, with redundant parens removed.
@@
identifier STRIDE, SIZE, COUNT;
@@

(
  kmalloc(
-	(COUNT) * STRIDE * SIZE
+	array3_size(COUNT, STRIDE, SIZE)
  , ...)
|
  kmalloc(
-	COUNT * (STRIDE) * SIZE
+	array3_size(COUNT, STRIDE, SIZE)
  , ...)
|
  kmalloc(
-	COUNT * STRIDE * (SIZE)
+	array3_size(COUNT, STRIDE, SIZE)
  , ...)
|
  kmalloc(
-	(COUNT) * (STRIDE) * SIZE
+	array3_size(COUNT, STRIDE, SIZE)
  , ...)
|
  kmalloc(
-	COUNT * (STRIDE) * (SIZE)
+	array3_size(COUNT, STRIDE, SIZE)
  , ...)
|
  kmalloc(
-	(COUNT) * STRIDE * (SIZE)
+	array3_size(COUNT, STRIDE, SIZE)
  , ...)
|
  kmalloc(
-	(COUNT) * (STRIDE) * (SIZE)
+	array3_size(COUNT, STRIDE, SIZE)
  , ...)
|
  kmalloc(
-	COUNT * STRIDE * SIZE
+	array3_size(COUNT, STRIDE, SIZE)
  , ...)
)

// Any remaining multi-factor products, first at least 3-factor products,
// when they're not all constants...
@@
expression E1, E2, E3;
constant C1, C2, C3;
@@

(
  kmalloc(C1 * C2 * C3, ...)
|
  kmalloc(
-	(E1) * E2 * E3
+	array3_size(E1, E2, E3)
  , ...)
|
  kmalloc(
-	(E1) * (E2) * E3
+	array3_size(E1, E2, E3)
  , ...)
|
  kmalloc(
-	(E1) * (E2) * (E3)
+	array3_size(E1, E2, E3)
  , ...)
|
  kmalloc(
-	E1 * E2 * E3
+	array3_size(E1, E2, E3)
  , ...)
)

// And then all remaining 2 factors products when they're not all constants,
// keeping sizeof() as the second factor argument.
@@
expression THING, E1, E2;
type TYPE;
constant C1, C2, C3;
@@

(
  kmalloc(sizeof(THING) * C2, ...)
|
  kmalloc(sizeof(TYPE) * C2, ...)
|
  kmalloc(C1 * C2 * C3, ...)
|
  kmalloc(C1 * C2, ...)
|
- kmalloc
+ kmalloc_array
  (
-	sizeof(TYPE) * (E2)
+	E2, sizeof(TYPE)
  , ...)
|
- kmalloc
+ kmalloc_array
  (
-	sizeof(TYPE) * E2
+	E2, sizeof(TYPE)
  , ...)
|
- kmalloc
+ kmalloc_array
  (
-	sizeof(THING) * (E2)
+	E2, sizeof(THING)
  , ...)
|
- kmalloc
+ kmalloc_array
  (
-	sizeof(THING) * E2
+	E2, sizeof(THING)
  , ...)
|
- kmalloc
+ kmalloc_array
  (
-	(E1) * E2
+	E1, E2
  , ...)
|
- kmalloc
+ kmalloc_array
  (
-	(E1) * (E2)
+	E1, E2
  , ...)
|
- kmalloc
+ kmalloc_array
  (
-	E1 * E2
+	E1, E2
  , ...)
)

Signed-off-by: Kees Cook <keescook@chromium.org>
2018-06-12 16:19:22 -07:00
Dirk Hohndel 1297bf2e91 Add SPDX idenitifier and clarify license
This is dual licensed under GPL-2.0 or MIT.

Signed-off-by: Dirk Hohndel (VMware) <dirk@hohndel.org>
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-05-15 13:44:27 -05:00
Junwei Zhang 967c650d49 drm/ttm: remove priority hard code when initializing ttm bo
Then priority could be set before initialization.
By default, it requires to kzalloc ttm bo. In fact, we always do so.

Signed-off-by: Junwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: David Zhou <david1.zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-05-15 13:44:23 -05:00
Michel Dänzer 00862edba1 drm/ttm: Use GFP_TRANSHUGE_LIGHT for allocating huge pages
GFP_TRANSHUGE tries very hard to allocate huge pages, which can result
in long delays with high memory pressure. I have observed firefox
freezing for up to around a minute due to this while restic was taking
a full system backup.

Since we don't really need huge pages, use GFP_TRANSHUGE_LIGHT |
__GFP_NORETRY instead, in order to fail quickly when there are no huge
pages available.

Set __GFP_KSWAPD_RECLAIM as well, in order for huge pages to be freed
up in the background if necessary.

With these changes, I'm no longer seeing freezes during a restic backup.

Cc: stable@vger.kernel.org
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-05-15 13:44:17 -05:00
Christian König 5452cf44d6 drm/ttm: keep a reference to transfer pipelined BOs
Make sure the transfered BO is never destroy before the transfer BO.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-05-15 13:43:11 -05:00
Thomas Hellstrom 9c11fcf1a7 drm/ttm: Export the ttm_k[un]map_atomic_prot API.
It will be used by vmwgfx cpu blit.

Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Brian Paul <brianp@vmware.com>
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
2018-03-22 11:10:06 +01:00
Thomas Hellstrom 403c1826a4 drm/ttm: Clean up kmap_atomic_prot selection code
Use helpers to perform the kmap_atomic_prot() functionality to
a) Avoid in-function ifdefs that violate the kernel coding policy,
b) Facilitate exporting the functionality.

This commit should not change any functionality.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
2018-03-22 11:09:37 +01:00
Christian König 536bbeba9b drm/ttm: move initializing ttm->sg into ttm_tt_init_fields
Better to set this with all other fields as well.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-03-14 14:38:28 -05:00
Christian König dde5da2379 drm/ttm: add bo as parameter to the ttm_tt_create callback
Instead of calculating the size in bytes just to recalculate the number
of pages from it pass the BO directly to the function.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-03-14 14:38:27 -05:00
Christian König 5d95109815 drm/ttm: add ttm_bo_pipeline_gutting
Allows us to gut a BO of it's backing store when the driver says that it
isn't needed any more.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-03-14 14:38:27 -05:00
Christian König 75a57669cb drm/ttm: add ttm_sg_tt_init
This allows drivers to only allocate dma addresses, but not a page
array.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-03-14 14:38:24 -05:00
Christian König 81f5ec0255 drm/ttm: move ttm_tt defines into ttm_tt.h
Let's stop mangling everything in a single header and create one header
per object instead.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Roger He <Hongbo.He@amd.com>
Acked-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-03-14 14:38:24 -05:00
Christian König 45a9d154f6 drm/ttm: cleanup ttm_tt_create
Cleanup ttm_tt_create a bit.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-26 23:09:46 -05:00
Christian König 97b7e1b8b5 drm/ttm: move ttm_tt_create into ttm_tt.c v2
Rename ttm_bo_add_ttm to ttm_tt_create and move it into ttm_tt.c.

v2: separate the cleanup.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-26 23:09:46 -05:00
Roger He ec3fe391bd drm/ttm: check if free mem space is under the lower limit
the free mem space and the lower limit both include two parts:
system memory and swap space.

For the OOM triggered by TTM, that is the case as below:
first swap space is full of swapped out pages and soon
system memory also is filled up with ttm pages. and then
any memory allocation request will run into OOM.

to cover two cases:
a. if no swap disk at all or free swap space is under swap mem
   limit but available system mem is bigger than sys mem limit,
   allow TTM allocation;

b. if the available system mem is less than sys mem limit but
   free swap space is bigger than swap mem limit, allow TTM
   allocation.

v2: merge two memory limit(swap and system) into one
v3: keep original behavior except ttm_opt_ctx->flags with
    TTM_OPT_FLAG_FORCE_ALLOC
v4: always set force_alloc as tx->flags & TTM_OPT_FLAG_FORCE_ALLOC
v5: add an attribute for lower_mem_limit
v6: set lower_mem_limit as 0 to keep original behavior

Signed-off-by: Roger He <Hongbo.He@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-26 23:09:46 -05:00
Christian König 724daa4fd6 drm/ttm: drop persistent_swap_storage from ttm_bo_init and co
Never used as parameter, the only driver actually using this is nouveau
and there it is initialized after the BO is initialized.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-26 23:09:45 -05:00
Christian König 231cdafc75 drm/ttm: drop ttm->dummy_read_page
Only used by the AGP backend and there it can be easily accessed using
ttm->bdev->glob.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-26 23:09:45 -05:00
Christian König 3231a7696e drm/ttm: drop ttm->glob
The pointer is available as ttm->bdev->glob as well.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-26 23:09:45 -05:00
Christian König 3839263362 drm/ttm: drop bo->glob
The pointer is available as bo->bdev->glob as well.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-26 23:09:44 -05:00
Christian König e44fcf71f4 drm/ttm: add default implementations for ttm_tt_(un)populate
Use ttm_pool_populate/ttm_pool_unpopulate if the driver doesn't provide
a function.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-26 23:09:41 -05:00
Roger He 40d5250dbb drm/ttm: set TTM_OPT_FLAG_FORCE_ALLOC in ttm_bo_force_list_clean
Because ttm_bo_force_list_clean() is only called on two occasions:
1. By ttm_bo_evict_mm() during suspend.
2. By ttm_bo_clean_mm() when the driver unloads.
On both cases we absolutely don't want any memory allocation failure.

Signed-off-by: Roger He <Hongbo.He@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-26 23:09:34 -05:00
Roger He aa7662b67b drm/ttm: add bit flag TTM_OPT_FLAG_FORCE_ALLOC
set TTM_OPT_FLAG_FORCE_ALLOC when we are servicing for page
fault routine.

for ttm_mem_global_reserve if in page fault routine, allow the gtt
pages reservation always. because page fault routing already grabbed
system memory and the allowance of this exception is harmless.
Otherwise, it will trigger OOM killer.

will be used later.

v2: set the FORCE_ALLOC always
v3: minor refine

Signed-off-by: Roger He <Hongbo.He@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-26 23:09:34 -05:00
Roger He d330fca115 drm/ttm: use bit flag to replace allow_reserved_eviction in ttm_operation_ctx
for saving memory and more bit flag can be used in future

Signed-off-by: Roger He <Hongbo.He@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-26 23:09:34 -05:00
Christian König ec92937056 drm/ttm: set page mapping during allocation
To aid debugging set the page mapping during allocation instead of
during VM faults.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-26 23:09:33 -05:00
Christian König 25893a14c9 drm/ttm: add ttm_tt_populate wrapper
Stop calling the driver callback directly.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-19 14:19:52 -05:00
Tom St Denis 7ca34ddc78 drm/ttm: Simplify ttm_dma_page_put()
Remove redundant store of return code.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-19 14:19:02 -05:00
Tom St Denis 42bdbb6e0f drm/ttm: Fix coding style in ttm_dma_pool_alloc_new_pages()
Add missing {} braces.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-19 14:19:01 -05:00
Tom St Denis ddde985c41 drm/ttm: Fix coding style in ttm_tt_swapout()
Add missing {} braces.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-19 14:19:01 -05:00
Tom St Denis 07d48da4f4 drm/ttm: Simplify ttm_eu_reserve_buffers()
Hoist the comparison of the ret to -EDEADLK above
the two code paths to simplify the function.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-19 14:19:00 -05:00
Tom St Denis de8dfb8e34 drm/ttm: Remove unncessary retval from ttm_bo_vm_fault()
The dual ret/retval was more complex than need be.  Now
we drop the retval variable and assign the appropriate VM
codes to ret instead.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-19 14:18:59 -05:00
Tom St Denis 449f797a94 drm/ttm: Fix coding style in ttm_bo_move_memcpy()
Add missing {} braces.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-19 14:18:59 -05:00
Tom St Denis add3d95d73 drm/ttm: Simplify ttm_dma_find_pool() (v2)
Flip the logic of the comparison and remove
the redudant variable for the pool address.

(v2): Remove {} bracing.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-19 14:18:58 -05:00
Tom St Denis c68edaa0c1 drm/ttm: Fix coding style in ttm_pool_store()
Correct missing {} style.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-19 14:18:58 -05:00
Tom St Denis 5b4262d7a2 drm/ttm: Change ttm_tt page allocations to return errors
Explicitly return errors in ttm_tt_alloc_page_directory() and
ttm_dma_tt_alloc_page_directory() instead of relying on
further logic to detect errors.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-19 14:18:57 -05:00
Tom St Denis 420457acfb drm/ttm: Add a default BO destructor to simplify code (v2)
(v2): Remove stray ; noticed by Felix

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-19 14:18:56 -05:00
Tom St Denis 43c7c41b25 drm/ttm: Fix coding style in ttm_bo.c
Correct indentation and {} brace style.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-19 14:18:56 -05:00
Roger He 4d869f2598 drm/ttm: add the missed global memory count update
when ttm_mem_global_alloc_page fails, we should call
ttm_mem_global_free_page to update memory count for
the ttm pages which already run ttm_mem_global_alloc_page
successfully

Signed-off-by: Roger He <Hongbo.He@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-19 14:17:59 -05:00
Andrey Grodzovsky cb5f1a52ca drm/ttm: Allow page allocations w/o triggering OOM..
This to allow drivers to choose to avoid OOM invocation and handle
page allocation failures instead.

v2:
Remove extra new lines.

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-02-19 14:17:58 -05:00
Tom St Denis 95244db2d3 drm/ttm: Fix 'buf' pointer update in ttm_bo_vm_access_kmap() (v2)
The buf pointer was not being incremented inside the loop
meaning the same block of data would be read or written
repeatedly.

(v2) Change 'buf' pointer to uint8_t* type

Cc: stable@vger.kernel.org
Fixes: 09ac4fcb3f ("drm/ttm: Implement vm_operations_struct.access v2")

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-01-29 23:09:31 -05:00
Roger He a86170290c drm/ttm: fix missing parameter change for ttm_bo_cleanup_refs
Missed in the patche:
dc94777 drm/ttm: enable swapout for reserved BOs during allocation.
don't unreserve the BO if it is not reserved by itself.

Signed-off-by: Roger He <Hongbo.He@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-01-29 22:53:12 -05:00
Dave Airlie 22bc72c807 Merge branch 'drm-next-4.16' of git://people.freedesktop.org/~agd5f/linux into drm-next
A few more fixes for 4.16, nothing major.

A few more fixes for 4.16.  This is on top of the pull request from
last week.  Most notable change here is a fix to the link order for
the now separate from amdgpu GPU scheduler to fix crashes when the
modules are build into the kernel rather than as modules.

* 'drm-next-4.16' of git://people.freedesktop.org/~agd5f/linux:
  drm: fix gpu scheduler link order
  drm/amd/display: Demote error print to debug print when ATOM impl missing
  drm/amdgpu: Avoid leaking PM domain on driver unbind (v2)
  drm/amd/amdgpu: Add Polaris version check
  drm/amdgpu: Reenable manual GPU reset from sysfs
  drm/amdgpu: disable MMHUB power gating on raven
  drm/ttm: Don't unreserve swapped BOs that were previously reserved
  drm/ttm: Don't add swapped BOs to swap-LRU list
  drm/amdgpu: only check for ECC on Vega10
  drm/amd/powerplay: Fix smu_table_entry.handle type
  drm/ttm: add VADDR_FLAG_UPDATED_COUNT to correctly update dma_page global count
  drm/radeon: fill in rb backend map on evergreen/ni.
  drm/amdgpu/gfx9: fix ngg enablement to clear gds reserved memory (v2)
  drm/ttm: only free pages rather than update global memory count together
  drm/amdgpu: fix CPU based VM updates
  drm/amdgpu: fix typo in amdgpu_vce_validate_bo
  drm/amdgpu: fix amdgpu_vm_pasid_fault_credit
  drm/ttm: check the return value of register_shrinker
  drm/radeon: fix sparse warning: Should it be static?
2018-01-25 11:40:54 +10:00
Felix Kuehling ad76c65ec3 drm/ttm: Don't unreserve swapped BOs that were previously reserved
If ttm_bo_swapout doesn't own the lock, don't release it. Someone
else probably depends on it still being locked.

Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-01-18 16:14:32 -05:00
Felix Kuehling fd5002d6a3 drm/ttm: Don't add swapped BOs to swap-LRU list
A BO that's already swapped would be added back to the swap-LRU list
for example if its validation failed under high memory pressure. This
could later lead to swapping it out again and leaking previous swap
storage.

This commit adds a condition to prevent that from happening.

v2: Check page_flags instead of swap_storage

Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org
2018-01-18 16:14:03 -05:00
Roger He 8f2112f84c drm/ttm: add VADDR_FLAG_UPDATED_COUNT to correctly update dma_page global count
add this for correctly updating global mem count in ttm_mem_zone.
before that when ttm_mem_global_alloc_page fails, we would update all
dma_page's global mem count in ttm_dma->pages_list. but actually here
we should not update for the last dma_page.

v2: only the update of last dma_page is not right
v3: use lower bits of dma_page vaddr

Signed-off-by: Roger He <Hongbo.He@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-01-18 16:07:26 -05:00
Dave Airlie 4a6cc7a44e Linux 4.15-rc8
-----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJaW+iVAAoJEHm+PkMAQRiGCDsIAJALNpX7odTx/8y+yCSWbpBH
 E57iwr4rmnI6tXJY6gqBUWTYnjAcf4b8IsHGCO6q3WIE3l/kt+m3eA21a32mF2Db
 /bfPGTOWu5LoOnFqzgH2kiFuC3Y474toxpld2YtkQWYxi5W7SUtIHi/jGgkUprth
 g15yPfwYgotJd/gpmPfBDMPlYDYvLlnPYbTG6ZWdMbg39m2RF2m0BdQ6aBFLHvbJ
 IN0tjCM6hrLFBP0+6Zn60pevUW9/AFYotZn2ankNTk5QVCQm14rgQIP+Pfoa5WpE
 I25r0DbkG2jKJCq+tlgIJjxHKD37GEDMc4T8/5Y8CNNeT9Q8si9EWvznjaAPazw=
 =o5gx
 -----END PGP SIGNATURE-----

BackMerge tag 'v4.15-rc8' into drm-next

Linux 4.15-rc8

Daniel requested this for so the intel CI won't fall over on drm-next
so often.
2018-01-18 09:32:15 +10:00
Roger He 9483ce7c6e drm/ttm: only free pages rather than update global memory count together
if ttm_get_pages or ttm_mem_global_alloc_page fail, should not update
global memory count.

Signed-off-by: Roger He <Hongbo.He@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Tested-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Acked-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-01-16 15:35:27 -05:00
Roger He e2721595e4 drm/ttm: check the return value of register_shrinker
This fixes the build warning:
"ignoring return value of 'register_shrinker', declared with
attribute warn_unused_result [-Wunused-result]"

Signed-off-by: Roger He <Hongbo.He@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-01-16 15:35:26 -05:00
Christian König 40b4667bab drm/ttm: specify DMA_ATTR_NO_WARN for huge page pools
Suppress warning messages when allocating huge pages fails since we can
always fall back to normal pages.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-01-10 15:44:51 -05:00
Tan Xiaojun e83bf4adf5 drm/ttm: remove ttm_bo_default_io_mem_pfn
No one will use this function except ttm_bo_io_mem_pfn() now, so move
the calculation of ttm_bo_default_io_mem_pfn() into ttm_bo_io_mem_pfn()
and do some cleanup.

Signed-off-by: Tan Xiaojun <tanxiaojun@huawei.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-01-10 15:44:51 -05:00
Xiongwei Song 19d859a720 drm/ttm: check the return value of kzalloc
In the function ttm_page_alloc_init, kzalloc call is made for variable
_manager, we need to check its return value, it may return NULL.

Signed-off-by: Xiongwei Song <sxwjean@gmail.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2018-01-02 14:54:05 -05:00
Roger He dc947770cf drm/ttm: enable swapout for reserved BOs during allocation
if the bo shares same reservation object then not lock it again
at swapout time to make it possible to swap out.

v2: refine the commmit message

Reviewed-by: Thomas Hellström <thellstrom@vmware.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chuming Zhou <david1.zhou@amd.com>
Signed-off-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-28 09:48:20 -05:00
Roger He d5769ba315 drm/ttm: add new function to check if bo is allowable to evict or swapout
extract a function as ttm_bo_evict_swapout_allowable since eviction and
swapout can share same logic.

v2: modify commit message and add description in the code

Reviewed-by: Thomas Hellström <thellstrom@vmware.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chuming Zhou <david1.zhou@amd.com>
Signed-off-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-28 09:48:20 -05:00
Roger He 993baf1556 drm/ttm: use an operation ctx for ttm_tt_bind
forward the operation context to ttm_tt_bind as well,
and the ultimate goal is swapout enablement for reserved BOs.

v2: use common term rather than amd specific

Reviewed-by: Thomas Hellström <thellstrom@vmware.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chuming Zhou <david1.zhou@amd.com>
Signed-off-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-28 09:48:20 -05:00
Roger He d0cef9fa44 drm/ttm: use an operation ctx for ttm_tt_populate in ttm_bo_driver (v2)
forward the operation context to ttm_tt_populate as well,
and the ultimate goal is swapout enablement for reserved BOs.

v2: squash in fix for vboxvideo

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-28 09:48:19 -05:00
Roger He 9de2fb99eb drm/ttm: use an operation ctx for ttm_mem_global_alloc_page
forward the operation context to ttm_mem_global_alloc_page as well,
and the ultimate goal is swapout enablement for reserved BOs.

Here reserved BOs refer to all the BOs which share same reservation object

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-27 11:38:55 -05:00
Roger He 279c01f6ef drm/ttm: use an operation ctx for ttm_mem_global_alloc
forward the operation context to ttm_mem_global_alloc as well, and the
ultimate goal is swapout enablement for reserved BOs

Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-27 11:38:54 -05:00
Roger He a6c26af8a4 drm/ttm: call ttm_bo_swapout directly when ttm shrink
remove the extra indirection because we have only one implementation anyway

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-27 11:38:11 -05:00
Tan Xiaojun c67fa6edc8 drm/ttm: add ttm_bo_io_mem_pfn to check io_mem_pfn
The io_mem_pfn field was added in commit ea642c3216 ("drm/ttm: add
io_mem_pfn callback") and is called unconditionally. However, not all
drivers were updated to set it.

Use the ttm_bo_default_io_mem_pfn function if a driver did not set its
own. And add new function ttm_bo_io_mem_pfn() as wrapper.

Signed-off-by: Michal Srb <msrb@suse.com>
Signed-off-by: Tan Xiaojun <tanxiaojun@huawei.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-27 11:34:37 -05:00
Christian König 6a83a55303 drm/ttm: drop the spin in delayed delete if the trylock doesn't work
Thomas actually noticed that, but I didn't realized what he meant until
now.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-27 11:34:25 -05:00
Christian König 78f99c6d5c drm/ttm: cleanup some old defines
Use pr_debug instead of TTM_DEBUG, fix the lockdep assert and remove the
unused constant.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Roger He <Hongbo.He@amd.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-18 11:53:01 -05:00
Christian König 2b7e35fbf8 drm/ttm: cleanup some more resv->lock uses
Use the reservation wrapper for this.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Roger He <Hongbo.He@amd.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-18 11:52:54 -05:00
Christian König 827ed2b06b drm/ttm: use try_lock in ttm_bo_delayed_delete again
We only need to wait for the contended lock when the reservation object is
shared or when we want to remove everything. A trylock should be sufficient
in all other cases.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Roger He <Hongbo.He@amd.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-18 11:52:47 -05:00
Roger He 3e98d829ad drm/ttm: use an ttm operation ctx for ttm_bo_move_xxx
include ttm_bo_move_memcpy and ttm_bo_move_ttm

Signed-off-by: Roger He <Hongbo.He@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-15 17:14:25 -05:00
Roger He 4985c9fc5f drm/ttm: enable eviction for Per-VM-BO
allow eviction of BOs reserved by the caller when they are
not part of the current working set.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-15 17:13:01 -05:00
Roger He 5cc636846d drm/ttm: init locked again to prevent incorrect unlock
Signed-off-by: Roger He <Hongbo.He@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-14 10:23:02 -05:00
Monk Liu 13d3fc69a0 drm/ttm: max_cpages is in unit of native page
fix calculation.

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-12 15:39:45 -05:00
Monk Liu a782fc8cc6 drm/ttm: fix incorrect calculate on shrink_pages
shrink_pages is in unit of Order after ttm_page_pool_free,
but it is used by nr_free in next round so need change
it into native page unit

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Roger He <Hongbo.He@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-12 15:39:20 -05:00
Monk Liu a8d25a8629 drm/ttm: max_cpages is in unit of native page
fix calculation.

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-12 14:48:52 -05:00
Monk Liu 0aaa59f525 drm/ttm: fix incorrect calculate on shrink_pages
shrink_pages is in unit of Order after ttm_page_pool_free,
but it is used by nr_free in next round so need change
it into native page unit

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Roger He <Hongbo.He@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-12 14:46:25 -05:00
Christian König ae937fe196 drm/ttm: swap consecutive allocated pooled pages v4
When we detect consecutive allocation of pages swap them to avoid
accidentally freeing them as huge page.

v2: use swap
v3: check if it's really the first allocated page
v4: don't touch the loop variable

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Roger He <Hongbo.He@amd.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Tested-by: Dieter Nützel <Dieter@nuetzel-hh.de>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-06 12:48:32 -05:00
Christian König fdb1a2236b drm/ttm: swap consecutive allocated cached pages v3
When we detect consecutive allocation of pages swap them to avoid
accidentally freeing them as huge page.

v2: use swap
v3: check if it's really the first allocated page

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Roger He <Hongbo.He@amd.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-06 12:48:32 -05:00
Joe Perches cf7d1bdf2b drm/ttm: Use a static string instead of an array of char *
Make the object a bit smaller by using a simple string instead
of a format string and array of char *.

$ size drivers/gpu/drm/ttm/ttm_page_alloc_dma.o*
   text	   data	    bss	    dec	    hex	filename
   8820	    216	   4136	  13172	   3374	drivers/gpu/drm/ttm/ttm_page_alloc_dma.o.defconfig.new
   8910	    216	   4136	  13262	   33ce	drivers/gpu/drm/ttm/ttm_page_alloc_dma.o.defconfig.old
  25383	   5044	   4384	  34811	   87fb	drivers/gpu/drm/ttm/ttm_page_alloc_dma.o.allyesconfig.new
  25797	   5428	   4384	  35609	   8b19	drivers/gpu/drm/ttm/ttm_page_alloc_dma.o.allyesconfig.old

Miscellanea:

o The h array had more entries than were emitted, all are now removed

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Joe Perches <joe@perches.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-06 12:48:27 -05:00
Roger He 1bfcbad18f drm/ttm: roundup the shrink request to prevent skip huge pool
e.g. shrink reqeust is less than 512, the logic will skip huge pool

Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-06 12:48:15 -05:00
Roger He 444f8ef3f8 drm/ttm: add page order support in ttm_pages_put
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-06 12:48:15 -05:00
Roger He 154683dd39 drm/ttm: add set_pages_wb for handling page order more than zero
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2017-12-06 12:48:14 -05:00