Go to file
Israel Rukshin de0ae958de RDMA/mlx5: Improve PI handover performance
In some loads, there is performance degradation when using KLM mkey
instead of MTT mkey. This is because KLM descriptor access is via
indirection that might require more HW resources and cycles.
Using KLM descriptor is not necessary when there are no gaps at the
data/metadata sg lists. As an optimization, use MTT mkey whenever it
is possible. For that matter, allocate internal MTT mkey and choose the
effective pi_mr for in transaction according to the required mapping
scheme.

The setup of the tested benchmark (using iSER ULP):
 - 2 servers with 24 cores (1 initiator and 1 target)
 - ConnectX-4/ConnectX-5 adapters
 - 24 target sessions with 1 LUN each
 - ramdisk backstore
 - PI active

Performance results running fio (24 jobs, 128 iodepth) using
write_generate=1 and read_verify=1 (w/w.o/baseline):

bs      IOPS(read)                IOPS(write)
----    ----------                ----------
512   1262.4K/1243.3K/1147.1K    1732.1K/1725.1K/1423.8K
4k    570902/571233/457874       773982/743293/642080
32k   72086/72388/71933          96164/71789/93249

Using write_generate=0 and read_verify=0 (w/w.o patch):
bs      IOPS(read)                IOPS(write)
----    ----------                ----------
512   1600.1K/1572.1K/1393.3K    1830.3K/1823.5K/1557.2K
4k    937272/921992/762934       815304/753772/646071
32k   77369/75052/72058          97435/73180/94612

Signed-off-by: Israel Rukshin <israelr@mellanox.com>
Reviewed-by: Max Gurtovoy <maxg@mellanox.com>
Suggested-by: Max Gurtovoy <maxg@mellanox.com>
Suggested-by: Idan Burstein <idanb@mellanox.com>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2019-06-24 11:49:28 -03:00
Documentation rdma: Remove nes 2019-06-13 09:59:49 -04:00
LICENSES LICENSES: Rename other to deprecated 2019-05-03 06:34:32 -06:00
arch Merge branch 'akpm' (patches from Andrew) 2019-05-19 12:15:32 -07:00
block for-5.2/block-post-20190516 2019-05-16 19:08:15 -07:00
certs kexec, KEYS: Make use of platform keyring for signature verify 2019-02-04 17:34:07 -05:00
crypto Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next 2019-05-07 22:03:58 -07:00
drivers RDMA/mlx5: Improve PI handover performance 2019-06-24 11:49:28 -03:00
fs This pull request contains the following fixes for UBIFS: 2019-05-19 15:22:03 -07:00
include RDMA/mlx5: Remove unused IB_WR_REG_SIG_MR code 2019-06-24 11:49:28 -03:00
init initramfs: don't free a non-existent initrd 2019-05-18 15:52:26 -07:00
ipc ipc: do cyclic id allocation for the ipc object. 2019-05-14 19:52:52 -07:00
kernel Merge branch 'akpm' (patches from Andrew) 2019-05-19 12:15:32 -07:00
lib slab: remove /proc/slab_allocators 2019-05-16 15:51:55 -07:00
mm Merge branch 'akpm' (patches from Andrew) 2019-05-19 12:15:32 -07:00
net Merge remote-tracking branch 'mlx5-next/mlx5-next' into HEAD 2019-06-18 22:44:36 -04:00
samples samples: guard sub-directories with CONFIG options 2019-05-18 11:29:01 +09:00
scripts kconfig: use 'else ifneq' for Makefile to improve readability 2019-05-19 09:34:35 +09:00
security Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2019-05-13 15:15:00 -07:00
sound sound fixes for 5.2-rc1 2019-05-17 13:57:54 -07:00
tools Merge branch 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2019-05-19 11:20:22 -07:00
usr user/Makefile: Fix typo and capitalization in comment section 2018-12-11 00:18:03 +09:00
virt * ARM: support for SVE and Pointer Authentication in guests, PMU improvements 2019-05-17 10:33:30 -07:00
.clang-format Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2019-04-17 11:26:25 -07:00
.cocciconfig
.get_maintainer.ignore Opt out of scripts/get_maintainer.pl 2019-05-16 10:53:40 -07:00
.gitattributes .gitattributes: set git diff driver for C source code files 2016-10-07 18:46:30 -07:00
.gitignore .gitignore: exclude .get_maintainer.ignore and .gitattributes 2019-05-18 11:49:54 +09:00
.mailmap A reasonably busy cycle for docs, including: 2019-05-08 12:42:50 -07:00
COPYING COPYING: use the new text with points to the license files 2018-03-23 12:41:45 -06:00
CREDITS Char/Misc driver patches for 5.1-rc1 2019-03-06 14:18:59 -08:00
Kbuild Kbuild updates for v5.1 2019-03-10 17:48:21 -07:00
Kconfig kconfig: move the "Executable file formats" menu to fs/Kconfig.binfmt 2018-08-02 08:06:55 +09:00
MAINTAINERS rdma: Remove nes 2019-06-13 09:59:49 -04:00
Makefile Linux 5.2-rc1 2019-05-19 15:47:09 -07:00
README Drop all 00-INDEX files from Documentation/ 2018-09-09 15:08:58 -06:00

README

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.