Commit Graph

349 Commits

Author SHA1 Message Date
Frank Li e75d5ae8ab NTB: epf: Allow more flexibility in the memory BAR map method
Support the below BAR configuration methods for epf NTB.

BAR 0: config and scratchpad
BAR 2: doorbell
BAR 4: memory map windows

Set difference BAR number information into struct ntb_epf_data. So difference
VID/PID can choose different BAR configurations. There are difference
BAR map method between epf NTB and epf vNTB Endpoint function.

Signed-off-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2022-08-09 17:54:03 -04:00
Dave Jiang a914fc529f ntb: intel: add GNR support for Intel PCIe gen5 NTB
Add Intel Granite Rapids NTB PCI device ID and related enabling.
Expectation is same hardware interface as Saphire Rapids Xeon platforms.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Allen Hubbe <allenbh@gmail.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2022-08-09 11:54:41 -04:00
Dan Carpenter 45e1058b77 NTB: ntb_tool: uninitialized heap data in tool_fn_write()
The call to:

	ret = simple_write_to_buffer(buf, size, offp, ubuf, size);

will return success if it is able to write even one byte to "buf".
The value of "*offp" controls which byte.  This could result in
reading uninitialized data when we do the sscanf() on the next line.

This code is not really desigined to handle partial writes where
*offp is non-zero and the "buf" is preserved and re-used between writes.
Just ban partial writes and replace the simple_write_to_buffer() with
copy_from_user().

Fixes: 578b881ba9 ("NTB: Add tool test client")
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2022-08-09 11:54:41 -04:00
Justin Stitt a44252d5c3 ntb: idt: fix clang -Wformat warnings
When building with Clang we encounter these warnings:
| drivers/ntb/hw/idt/ntb_hw_idt.c:2409:28: error: format specifies type
| 'unsigned char' but the argument has type 'int' [-Werror,-Wformat]
| "\t%hhu-%hhu.\t", idx + cnt - 1);
-
| drivers/ntb/hw/idt/ntb_hw_idt.c:2438:29: error: format specifies type
| 'unsigned char' but the argument has type 'int' [-Werror,-Wformat]
| "\t%hhu-%hhu.\t", idx + cnt - 1);
-
| drivers/ntb/hw/idt/ntb_hw_idt.c:2484:15: error: format specifies type
| 'unsigned char' but the argument has type 'int' [-Werror,-Wformat], src);

For the first two warnings the format specifier used is `%hhu` which
describes a u8. Both `idx` and `cnt` are u8 as well. However, the
expression as a whole is promoted to an int as you cannot get
smaller-than-int from addition. Therefore, to fix the warning, use the
promoted-to-type's format specifier -- in this case `%d`.

example:
``
uint8_t a = 4, b = 7;
int size = sizeof(a + b - 1);
printf("%d\n", size);
// output: 4
```

For the last warning, src is of type `int` while the format specifier
describes a u8. The fix here is just to use the proper specifier `%d`.

See more:
(https://wiki.sei.cmu.edu/confluence/display/c/INT02-C.+Understand+integer+conversion+rules)
"Integer types smaller than int are promoted when an operation is
performed on them. If all values of the original type can be represented
as an int, the value of the smaller type is converted to an int;
otherwise, it is converted to an unsigned int."

Link: https://github.com/ClangBuiltLinux/linux/issues/378
Signed-off-by: Justin Stitt <justinstitt@google.com>
Acked-by: Serge Semin <fancer.lancer@gmail.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2022-08-09 11:54:40 -04:00
Dave Jiang d5081bf5dc ntb: intel: fix port config status offset for SPR
The field offset for port configuration status on SPR has been changed to
bit 14 from ICX where it resides at bit 12. By chance link status detection
continued to work on SPR. This is due to bit 12 being a configuration bit
which is in sync with the status bit. Fix this by checking for a SPR device
and checking correct status bit.

Fixes: 26bfe3d0b2 ("ntb: intel: Add Icelake (gen4) support for Intel NTB")
Tested-by: Jerry Dai <jerry.dai@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2022-01-28 10:19:16 -05:00
Gustavo A. R. Silva 305325688f NTB/msi: Use struct_size() helper in devm_kzalloc()
Make use of the struct_size() helper instead of an open-coded version,
in order to avoid any potential type mistakes or integer overflows that,
in the worst scenario, could lead to heap overflows.

Also, address the following sparse warnings:
drivers/ntb/msi.c:46:23: warning: using sizeof on a flexible structure

Link: https://github.com/KSPP/linux/issues/174
Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2022-01-23 16:15:15 -05:00
Linus Torvalds 96000bc956 New AMD PCI ID for NTB, and a number of bug fixes for ntb_hw_switchtec
for Linux v5.17
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEoE9b9c3U2JxX98mqbmZLrHqL0iMFAmHd/AYACgkQbmZLrHqL
 0iMyRA/+KnlvCJMxDdeZ7dcK2s86BuvWESfkRbJYAMBNX6HiuH3SHll/UNF8UvLU
 CIl12cqALKwu618J37gbF9HHuVu+VVk1Dl9ib+mdoqGL/kPqX0SqtyP4lCyKg6HK
 POnOgjZWd0YR9tNjBukRx5JnLH1KtjaLVLWPxTmb0WgwQjcNqXWtRIWjHovLRThJ
 938P02JQQv4TUcFuazXmdgiF0vUm+KPLpBP4urgPSSTPmdKLHvQY5ScJr+Do7OS5
 51mxPkgF1JBnvJiuCWdZOXFpoA/o8UG3REVoGPMzIBStc+1vnFWu1eo80DIfXbz/
 zYPYj+Foq4/ZsAJU5j4FHzVs9SkP8VKClENa8BNDoUiSEyYbDMFbmw4D+VOV0Z/n
 CmQiOQqZh5cshQL7V0v+6pNYYyQBGK2JpCC8EvTWIlYWohmYvu/WLCpVwk4PCjBX
 caM0l9f99dTdzgPcRMC2CnKiqrAutIcCoaefyqVNIyMLiI0SyoYaBcbDeXNmujLV
 w5JGjaeCiv8AJYo6/XvqCrizYC5ETwNy+Rt4nyXH5gTGSGFH0hJ+Ip/8DPVQ1e/K
 DEEVZACh6uPP/ySSjZZEGagqI7YetSu3TEU1+8n1j4IA/ugP7XBwG1ZZQ9rEMcfw
 kG+VJwOil8VeRaL+osXsfU8xxRNIB6zuBajT0FG0eCpkAAIZWqw=
 =TMHM
 -----END PGP SIGNATURE-----

Merge tag 'ntb-5.17' of git://github.com/jonmason/ntb

Pull NTB updates from Jon Mason:
 "New AMD PCI ID for NTB, and a number of bug fixes for ntb_hw_switchtec
  for Linux v5.17"

* tag 'ntb-5.17' of git://github.com/jonmason/ntb:
  ntb_hw_switchtec: Fix a minor issue in config_req_id_table()
  ntb_hw_switchtec: Remove code for disabling ID protection
  ntb_hw_switchtec: Update the way of getting VEP instance ID
  ntb_hw_switchtec: AND with the part_map for a valid tpart_vec
  ntb_hw_switchtec: Fix bug with more than 32 partitions
  ntb_hw_switchtec: Fix pff ioread to read into mmio_part_cfg_all
  ntb_hw_switchtec: fix the spelling of "its"
  NTB/msi: Fix ntbm_msi_request_threaded_irq() kernel-doc comment
  ntb_hw_amd: Add NTB PCI ID for new gen CPU
2022-01-17 08:14:18 +02:00
Kelvin Cao 8cd778650a ntb_hw_switchtec: Fix a minor issue in config_req_id_table()
The req_id_table_size field is 16-bit wide, use ioread16() to read the
value.

Signed-off-by: Kelvin Cao <kelvin.cao@microchip.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2022-01-11 15:38:59 -05:00
Kelvin Cao 1d3cfc2835 ntb_hw_switchtec: Remove code for disabling ID protection
ID protection is a firmware setting for NT window access control. With
it enabled, only the posted requests with requester IDs in the requester
ID table will be allowed to access the NT windows. Otherwise all posted
requests are allowed. Normally user will configure it statically via the
Switchtec config file, and it will take effect when the firmware boots
up. The driver can also toggle the ID protection setting dynamically,
which will overwrite the static setting in the Switchtec config file as
a side effect.

Currently, the driver disables the ID protection. However, it's not
necessary to disable the ID protection at the driver level as the driver
has already configured the proper requester IDs in the requester ID
table to allow the corresponding posted requests to hit the NT windows.
Remove the code that disables the ID protection to make the static
setting prevail.

Note: ID protection is not applicable to non-posted requests.

Signed-off-by: Kelvin Cao <kelvin.cao@microchip.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2022-01-11 15:38:59 -05:00
Kelvin Cao 2f58265e16 ntb_hw_switchtec: Update the way of getting VEP instance ID
Gen4 firmware adds DMA VEP and NVMe VEP support in VEP (virtual EP)
instance ID register in addtion to management EP. Update the way of
getting management VEP instance ID.

Signed-off-by: Kelvin Cao <kelvin.cao@microchip.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2022-01-11 15:38:59 -05:00
Jeremy Pallotta 857e239c3e ntb_hw_switchtec: AND with the part_map for a valid tpart_vec
Some firmware versions return 1 in the target partition vector for
undefined partitions. AND with the part_map to give a valid tpart_vec.

Signed-off-by: Jeremy Pallotta <jmpallotta@gmail.com>
Signed-off-by: Kelvin Cao <kelvin.cao@microchip.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2022-01-11 15:38:59 -05:00
Wesley Sheng 7ff351c86b ntb_hw_switchtec: Fix bug with more than 32 partitions
Switchtec could support as mush as 48 partitions, but ffs & fls are
for 32 bit argument, in case of partition index larger than 31, the
current code could not parse the peer partition index correctly.
Change to the 64 bit version __ffs64 & fls64 accordingly to fix this
bug.

Fixes: 3df54c870f ("ntb_hw_switchtec: Allow using Switchtec NTB in multi-partition setups")
Signed-off-by: Wesley Sheng <wesley.sheng@microchip.com>
Signed-off-by: Kelvin Cao <kelvin.cao@microchip.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2022-01-11 15:38:59 -05:00
Jeremy Pallotta 32c3d375b0 ntb_hw_switchtec: Fix pff ioread to read into mmio_part_cfg_all
Array mmio_part_cfg_all holds the partition configuration of all
partitions, with partition number as index. Fix this by reading into
mmio_part_cfg_all for pff.

Fixes: 0ee28f26f3 ("NTB: switchtec_ntb: Add link management")
Signed-off-by: Jeremy Pallotta <jmpallotta@gmail.com>
Signed-off-by: Kelvin Cao <kelvin.cao@microchip.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2022-01-11 15:38:59 -05:00
Randy Dunlap 78c5335b1a ntb_hw_switchtec: fix the spelling of "its"
Use the possessive "its" instead of the contraction "it's" (it is)
in user messages.

Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Cc: Kurt Schwemmer <kurt.schwemmer@microsemi.com>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2022-01-11 15:38:59 -05:00
Yang Li e70dc09426 NTB/msi: Fix ntbm_msi_request_threaded_irq() kernel-doc comment
Add the description of @msi_desc and change the @devname to @name
in ntbm_msi_request_threaded_irq() kernel-doc comment to remove
some warnings found by running scripts/kernel-doc, which is caused
by using 'make W=1'.
drivers/ntb/msi.c:285: warning: Function parameter or member 'name' not
described in 'ntbm_msi_request_threaded_irq'
drivers/ntb/msi.c:285: warning: Function parameter or member 'msi_desc'
not described in 'ntbm_msi_request_threaded_irq'
drivers/ntb/msi.c:285: warning: Excess function parameter 'devname'
description in 'ntbm_msi_request_threaded_irq'

Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Signed-off-by: Yang Li <yang.lee@linux.alibaba.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2022-01-11 15:38:59 -05:00
Sanjay R Mehta 0d5924ec4b ntb_hw_amd: Add NTB PCI ID for new gen CPU
Add NTB support for new generation of processor

Signed-off-by: Sanjay R Mehta <sanju.mehta@amd.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2022-01-11 15:38:59 -05:00
Thomas Gleixner 68e3183580 NTB/msi: Convert to msi_on_each_desc()
Replace the about to vanish iterators, make use of the filtering and take
the descriptor lock around the iteration.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Link: https://lore.kernel.org/r/20211206210748.683004012@linutronix.de
2021-12-16 22:22:19 +01:00
Linus Torvalds 1735715e0f Bug fixes and clean-ups for Linux v5.15
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEoE9b9c3U2JxX98mqbmZLrHqL0iMFAmE1QcoACgkQbmZLrHqL
 0iNwVA/9F+hiD9kfIj9nN0ljc4Y1JIzq5H2nL7VC0VBpAr1RxnDxlKIVBr2jbm6F
 YSk8MBmLhVtVIc2SJ8+oOzRAuZ1lChj1BWHy+OY00Q4TQmZ1zAJzqfMwm8lYSZgd
 y5THLlz2MHmH6HcqmvOCMoQoWVcxY99dvKkFzkl/f/pF8OoLv7eo9QeFNxkwySF0
 DVvZFZavFjrST3S+zYkA4VifhOGdeUbWmpIOvlv0e89Y3WG9XMJXy2E818ke/qRb
 tXD/xH83j0pB0vjCB2N4NnDxO1uOU/a/lgLSbX2F0eLhLMJ8byyiB/2vbUgHOLxl
 F8t3Fe+EITZIeuJ+zWJJPJ/t/qBVAKk8jLUVfSqmJXivHJU6HM0dZyNqEbaX00y/
 9FN+uRZM7PXb6U5Gb4lehCsuMAA+SV0BpTXcLBWvL59k2Ne3DtIDh8YF3EpfWhHh
 H7KRGqKCPCdpjP43JxNwuRonnKKUO4ovQ9aMOnYFBh7/Fdc4Awgpt9gLB893/ivn
 o5h4+PS39KJLkV1/a3w/kn6LLdsBArAfFVyboUxSgbCyfW3rphazvTu34B9Fm1Py
 AIsnW8OtW/V9+n8olOkQ2NdiB+v3FXOBVmjOyk5j/Vr5ZoEJyNocW3llCxbe5V93
 DLvhSS/DWmiJVfGLz92bq/2QFqGKf3bWI8C/auMOKuHjqDewOR8=
 =1kqW
 -----END PGP SIGNATURE-----

Merge tag 'ntb-5.15' of git://github.com/jonmason/ntb

Pull NTB updates from Jon Mason:
 "Bug fixes and clean-ups for Linux v5.15"

* tag 'ntb-5.15' of git://github.com/jonmason/ntb:
  NTB: switch from 'pci_' to 'dma_' API
  ntb: ntb_pingpong: remove redundant initialization of variables msg_data and spad_data
  NTB: perf: Fix an error code in perf_setup_inbuf()
  NTB: Fix an error code in ntb_msit_probe()
  ntb: intel: remove invalid email address in header comment
2021-09-07 13:05:02 -07:00
Christophe JAILLET 38de3afffb NTB: switch from 'pci_' to 'dma_' API
The wrappers in include/linux/pci-dma-compat.h should go away.

The patch has been generated with the coccinelle script below.

It has been compile tested.

@@
@@
-    PCI_DMA_BIDIRECTIONAL
+    DMA_BIDIRECTIONAL

@@
@@
-    PCI_DMA_TODEVICE
+    DMA_TO_DEVICE

@@
@@
-    PCI_DMA_FROMDEVICE
+    DMA_FROM_DEVICE

@@
@@
-    PCI_DMA_NONE
+    DMA_NONE

@@
expression e1, e2, e3;
@@
-    pci_alloc_consistent(e1, e2, e3)
+    dma_alloc_coherent(&e1->dev, e2, e3, GFP_)

@@
expression e1, e2, e3;
@@
-    pci_zalloc_consistent(e1, e2, e3)
+    dma_alloc_coherent(&e1->dev, e2, e3, GFP_)

@@
expression e1, e2, e3, e4;
@@
-    pci_free_consistent(e1, e2, e3, e4)
+    dma_free_coherent(&e1->dev, e2, e3, e4)

@@
expression e1, e2, e3, e4;
@@
-    pci_map_single(e1, e2, e3, e4)
+    dma_map_single(&e1->dev, e2, e3, e4)

@@
expression e1, e2, e3, e4;
@@
-    pci_unmap_single(e1, e2, e3, e4)
+    dma_unmap_single(&e1->dev, e2, e3, e4)

@@
expression e1, e2, e3, e4, e5;
@@
-    pci_map_page(e1, e2, e3, e4, e5)
+    dma_map_page(&e1->dev, e2, e3, e4, e5)

@@
expression e1, e2, e3, e4;
@@
-    pci_unmap_page(e1, e2, e3, e4)
+    dma_unmap_page(&e1->dev, e2, e3, e4)

@@
expression e1, e2, e3, e4;
@@
-    pci_map_sg(e1, e2, e3, e4)
+    dma_map_sg(&e1->dev, e2, e3, e4)

@@
expression e1, e2, e3, e4;
@@
-    pci_unmap_sg(e1, e2, e3, e4)
+    dma_unmap_sg(&e1->dev, e2, e3, e4)

@@
expression e1, e2, e3, e4;
@@
-    pci_dma_sync_single_for_cpu(e1, e2, e3, e4)
+    dma_sync_single_for_cpu(&e1->dev, e2, e3, e4)

@@
expression e1, e2, e3, e4;
@@
-    pci_dma_sync_single_for_device(e1, e2, e3, e4)
+    dma_sync_single_for_device(&e1->dev, e2, e3, e4)

@@
expression e1, e2, e3, e4;
@@
-    pci_dma_sync_sg_for_cpu(e1, e2, e3, e4)
+    dma_sync_sg_for_cpu(&e1->dev, e2, e3, e4)

@@
expression e1, e2, e3, e4;
@@
-    pci_dma_sync_sg_for_device(e1, e2, e3, e4)
+    dma_sync_sg_for_device(&e1->dev, e2, e3, e4)

@@
expression e1, e2;
@@
-    pci_dma_mapping_error(e1, e2)
+    dma_mapping_error(&e1->dev, e2)

@@
expression e1, e2;
@@
-    pci_set_dma_mask(e1, e2)
+    dma_set_mask(&e1->dev, e2)

@@
expression e1, e2;
@@
-    pci_set_consistent_dma_mask(e1, e2)
+    dma_set_coherent_mask(&e1->dev, e2)

Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Acked-by: Serge Semin <fancer.lancer@gmail.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2021-09-05 18:08:14 -04:00
Colin Ian King e631548027 ntb: ntb_pingpong: remove redundant initialization of variables msg_data and spad_data
The variables msg_data and spad_data are being initialized with values
that are never read, they are being updated later on. The initializations
are redundant and can be removed.

Addresses-Coverity: ("Unused value")
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Acked-by: Serge Semin <fancer.lancer@gmail.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2021-09-05 18:04:46 -04:00
Yang Li 0097ae5f7a NTB: perf: Fix an error code in perf_setup_inbuf()
When the function IS_ALIGNED() returns false, the value of ret is 0.
So, we set ret to -EINVAL to indicate this error.

Clean up smatch warning:
drivers/ntb/test/ntb_perf.c:602 perf_setup_inbuf() warn: missing error
code 'ret'.

Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Signed-off-by: Yang Li <yang.lee@linux.alibaba.com>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2021-09-04 18:36:04 -04:00
Yang Li 319f83ac98 NTB: Fix an error code in ntb_msit_probe()
When the value of nm->isr_ctx is false, the value of ret is 0.
So, we set ret to -ENOMEM to indicate this error.

Clean up smatch warning:
drivers/ntb/test/ntb_msi_test.c:373 ntb_msit_probe() warn: missing
error code 'ret'.

Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Signed-off-by: Yang Li <yang.lee@linux.alibaba.com>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2021-09-04 18:36:04 -04:00
Dave Jiang f3b6b10fcc ntb: intel: remove invalid email address in header comment
Remove Jon's old email address.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2021-09-04 18:36:04 -04:00
Uwe Kleine-König fc7a6209d5 bus: Make remove callback return void
The driver core ignores the return value of this callback because there
is only little it can do when a device disappears.

This is the final bit of a long lasting cleanup quest where several
buses were converted to also return void from their remove callback.
Additionally some resource leaks were fixed that were caused by drivers
returning an error code in the expectation that the driver won't go
away.

With struct bus_type::remove returning void it's prevented that newly
implemented buses return an ignored error code and so don't anticipate
wrong expectations for driver authors.

Reviewed-by: Tom Rix <trix@redhat.com> (For fpga)
Reviewed-by: Mathieu Poirier <mathieu.poirier@linaro.org>
Reviewed-by: Cornelia Huck <cohuck@redhat.com> (For drivers/s390 and drivers/vfio)
Acked-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk> (For ARM, Amba and related parts)
Acked-by: Mark Brown <broonie@kernel.org>
Acked-by: Chen-Yu Tsai <wens@csie.org> (for sunxi-rsb)
Acked-by: Pali Rohár <pali@kernel.org>
Acked-by: Mauro Carvalho Chehab <mchehab@kernel.org> (for media)
Acked-by: Hans de Goede <hdegoede@redhat.com> (For drivers/platform)
Acked-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Acked-By: Vinod Koul <vkoul@kernel.org>
Acked-by: Juergen Gross <jgross@suse.com> (For xen)
Acked-by: Lee Jones <lee.jones@linaro.org> (For mfd)
Acked-by: Johannes Thumshirn <jth@kernel.org> (For mcb)
Acked-by: Johan Hovold <johan@kernel.org>
Acked-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org> (For slimbus)
Acked-by: Kirti Wankhede <kwankhede@nvidia.com> (For vfio)
Acked-by: Maximilian Luz <luzmaximilian@gmail.com>
Acked-by: Heikki Krogerus <heikki.krogerus@linux.intel.com> (For ulpi and typec)
Acked-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> (For ipack)
Acked-by: Geoff Levand <geoff@infradead.org> (For ps3)
Acked-by: Yehezkel Bernat <YehezkelShB@gmail.com> (For thunderbolt)
Acked-by: Alexander Shishkin <alexander.shishkin@linux.intel.com> (For intel_th)
Acked-by: Dominik Brodowski <linux@dominikbrodowski.net> (For pcmcia)
Acked-by: Rafael J. Wysocki <rafael@kernel.org> (For ACPI)
Acked-by: Bjorn Andersson <bjorn.andersson@linaro.org> (rpmsg and apr)
Acked-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> (For intel-ish-hid)
Acked-by: Dan Williams <dan.j.williams@intel.com> (For CXL, DAX, and NVDIMM)
Acked-by: William Breathitt Gray <vilhelm.gray@gmail.com> (For isa)
Acked-by: Stefan Richter <stefanr@s5r6.in-berlin.de> (For firewire)
Acked-by: Benjamin Tissoires <benjamin.tissoires@redhat.com> (For hid)
Acked-by: Thorsten Scherer <t.scherer@eckelmann.de> (For siox)
Acked-by: Sven Van Asbroeck <TheSven73@gmail.com> (For anybuss)
Acked-by: Ulf Hansson <ulf.hansson@linaro.org> (For MMC)
Acked-by: Wolfram Sang <wsa@kernel.org> # for I2C
Acked-by: Sudeep Holla <sudeep.holla@arm.com>
Acked-by: Geert Uytterhoeven <geert@linux-m68k.org>
Acked-by: Dmitry Torokhov <dmitry.torokhov@gmail.com>
Acked-by: Finn Thain <fthain@linux-m68k.org>
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Link: https://lore.kernel.org/r/20210713193522.1770306-6-u.kleine-koenig@pengutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-07-21 11:53:42 +02:00
Kishon Vijay Abraham I 812ce2f8d1 NTB: Add support for EPF PCI Non-Transparent Bridge
Add support for EPF PCI Non-Transparent Bridge (NTB) devices.  This driver
is platform independent and may be used by any platform that has multiple
PCI endpoint instances configured using the pci-epf-ntb driver.  The driver
connnects to the standard NTB subsystem interface. The EPF NTB device has a
configurable number of memory windows (max 4), a configurable number of
doorbells (max 32), and a configurable number of scratch-pad registers.

Link: https://lore.kernel.org/r/20210201195809.7342-16-kishon@ti.com
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
2021-02-23 14:12:53 -06:00
Linus Torvalds 52cd5f9c22 Big fix for IDT NTB and Intel NTB LTR management support
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEoE9b9c3U2JxX98mqbmZLrHqL0iMFAl/oloUACgkQbmZLrHqL
 0iOnHBAAlQmxy/OBQSeOVT4mZh954zUwUl8CTpOLraMgNh/aUGK48MxGgDNQ0k77
 WFtoKKGqdeAAyVyQmnirtWi811tbCt+wl07jorOuO79AXvx11IQEI2qA2udoexlO
 xrI/7UukVWWeOvRuP6Nbi2iJvzkuJ7h9hgyHqloBj63PNh5PSJb1u8T+48yyVvvM
 LftGsIW6FOc1Dl6ZHBnezd6mNjqsBJMyggkD2BR/QOwEAJI7mWI4ihU6fZSzSoAv
 o69V/SVMAiDzUWsFlzvOIPfNgQ4pw7HbIyS80sj4oFGL5meiuH7L7RrtMLVOKvIm
 fYmhqt+1F36NiRGIPibPjD9tgt1jCXFfh/R4ZuLldJ/vjVZxP4Bqoyhvbntum8o5
 quKq5zO/Ou1b/9f9uBzJ31/EnOqVg3nNx/i09t5KH1Knp0kfLMTPgEtdyRZbm2+V
 oQ+iCUiO5FTbWZhW+/CgM59HRSM3LtXCRateMEcSkQxEa6smCKAL4BuV9tIRN93g
 MotqKfSmvOovQC/tixxAI2SxwmdovtssrELxcvbsiqjlh3PAmp1IhA9q/yPW2g4/
 vzK+2cYLWDovdERCGPo4i+Eb838nufEXhf0OEQowkwM66V86sdCRUJLFPUJdw7l5
 3XgNWC086TXpKSP9URnRUnRPhDecdwmVotWxfXBewiNYZyY1AQE=
 =dBeS
 -----END PGP SIGNATURE-----

Merge tag 'ntb-5.11' of git://github.com/jonmason/ntb

Pull NTB fixes from Jon Mason:
 "Bug fix for IDT NTB and Intel NTB LTR management support"

* tag 'ntb-5.11' of git://github.com/jonmason/ntb:
  ntb: intel: add Intel NTB LTR vendor support for gen4 NTB
  ntb: idt: fix error check in ntb_hw_idt.c
2020-12-27 09:22:55 -08:00
Thomas Gleixner 1110918e43 NTB/msi: Use irq_has_action()
Use the proper core function.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
Link: https://lore.kernel.org/r/20201210194044.255887860@linutronix.de
2020-12-15 16:19:32 +01:00
Dave Jiang 75b6f6487c ntb: intel: add Intel NTB LTR vendor support for gen4 NTB
Intel NTB device has custom LTR management that is not compliant with the
PCIe standard. Add support to set LTR status triggered by link status
change.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2020-12-06 18:18:03 -05:00
Wang Qing 91b8246de8 ntb: idt: fix error check in ntb_hw_idt.c
idt_create_dev never return NULL and fix smatch warning.

Signed-off-by: Wang Qing <wangqing@vivo.com>
Acked-by: Serge Semin <fancer.lancer@gmail.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2020-12-06 18:14:44 -05:00
Linus Torvalds e5acf0ed8a Bug fixes for v5.10
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEoE9b9c3U2JxX98mqbmZLrHqL0iMFAl+Vfl8ACgkQbmZLrHqL
 0iP4Zw//cXq6UWPQwpZoe0S2IiKUTelDStCNxohPUgnJqJW9aLxAV4aL4jEonvkV
 5Z13OWtjusnI3DZuWjK8M5ECSPncCqNEez4o04BRr/uzIvHi0qx27JFk7ZMPMXG+
 AsdHHzoiqawa7zG/SJHUZ8ge4AjcR8pfZMCx+l2WK4zeLbCzt+5Rm3MZvJps8xJ/
 k47AB2JwJxNp3OdoFVXWYMahhSI3z1JoGg6ty6PYRfOMjB4/HPFxnF6q5FMdKo21
 p4oxr10rzc1u6DjiWyleKPJCC7x2/K+0eCenm83xL5n40mGC9Ps6H9lHydierLy+
 kv/SrZNxcTfwxW7jn6gZ8yPThTqwRqjd05NQ488jnmV/7RqehRPDBVzsl7QOX/Rd
 gb+XFBwCN20nIkOM4QKz4hHJ6zrOVOv1APVzIUkbzo87RXo64do1gqY6Q3UJvFgr
 adsdPtFQ48qELmfS+NB9LCZ5KezzWFeQb6MJeXv0m+jwDVOe3ybMiukJP0cszR3g
 2vLu6aEkLXYZfiOB0ueV9GziopJV5u248oYtcxAJBAX9WpMD1gjS7wRllxsCRhm1
 YtyK6if678Ce1T5KNCgdGdM6C9nFKI+h4GIoOErd63lwweUOMglAMlBVU29kGIV/
 pLDXgfJcQMbZf/mMC7Z0AxmE6FVuZPH9tQk0Jz03SBRzz1YIANg=
 =YIEm
 -----END PGP SIGNATURE-----

Merge tag 'ntb-5.10' of git://github.com/jonmason/ntb

Pull NTB fixes from Jon Mason.

* tag 'ntb-5.10' of git://github.com/jonmason/ntb:
  NTB: Use struct_size() helper in devm_kzalloc()
  ntb: intel: Fix memleak in intel_ntb_pci_probe
  NTB: hw: amd: fix an issue about leak system resources
2020-10-25 11:12:31 -07:00
Gustavo A. R. Silva b8e2c8bbdf NTB: Use struct_size() helper in devm_kzalloc()
Make use of the struct_size() helper instead of an open-coded version
in order to avoid any potential type mistakes. Also, remove unnecessary
variable _struct_size_.

This code was detected with the help of Coccinelle and, audited and
fixed manually.

Addresses-KSPP-ID: https://github.com/KSPP/linux/issues/83
Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2020-08-24 10:58:06 -04:00
Dinghao Liu dbb8df5c2d ntb: intel: Fix memleak in intel_ntb_pci_probe
The default error branch of a series of pdev_is_gen calls
should free ndev just like what we've done in these calls.

Fixes: 26bfe3d0b2 ("ntb: intel: Add Icelake (gen4) support for Intel NTB")
Signed-off-by: Dinghao Liu <dinghao.liu@zju.edu.cn>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2020-08-24 10:44:47 -04:00
Kaige Li 44a0a3c179 NTB: hw: amd: fix an issue about leak system resources
The related system resources were not released when pci_set_dma_mask(),
pci_set_consistent_dma_mask(), or pci_iomap() return error in the
amd_ntb_init_pci() function. Add pci_release_regions() to fix it.

Fixes: a1b3695820 ("NTB: Add support for AMD PCI-Express Non-Transparent Bridge")
Signed-off-by: Kaige Li <likaige@loongson.cn>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2020-08-23 22:36:32 -04:00
Gustavo A. R. Silva df561f6688 treewide: Use fallthrough pseudo-keyword
Replace the existing /* fall through */ comments and its variants with
the new pseudo-keyword macro fallthrough[1]. Also, remove unnecessary
fall-through markings when it is the case.

[1] https://www.kernel.org/doc/html/v5.7/process/deprecated.html?highlight=fallthrough#implicit-switch-case-fall-through

Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
2020-08-23 17:36:59 -05:00
Krzysztof Kozlowski 58184e95e9 ntb: intel: constify ioreadX() iomem argument (as in generic implementation)
The ioreadX() helpers have inconsistent interface.  On some architectures
void *__iomem address argument is a pointer to const, on some not.

Implementations of ioreadX() do not modify the memory under the address so
they can be converted to a "const" version for const-safety and
consistency among architectures.

Signed-off-by: Krzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Cc: Allen Hubbe <allenbh@gmail.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: Helge Deller <deller@gmx.de>
Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
Cc: Jakub Kicinski <kuba@kernel.org>
Cc: "James E.J. Bottomley" <James.Bottomley@HansenPartnership.com>
Cc: Jason Wang <jasowang@redhat.com>
Cc: Jon Mason <jdmason@kudzu.us>
Cc: Kalle Valo <kvalo@codeaurora.org>
Cc: Matt Turner <mattst88@gmail.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Rich Felker <dalias@libc.org>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Link: http://lkml.kernel.org/r/20200709072837.5869-4-krzk@kernel.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-08-14 19:56:57 -07:00
Logan Gunthorpe 34d8673a01 NTB: perf: Fix race condition when run with ntb_test
When running ntb_test, the script tries to run the ntb_perf test
immediately after probing the modules. Since adding multi-port support,
this fails seeing the new initialization procedure in ntb_perf
can not complete instantly.

To fix this we add a completion which is waited on when a test is
started. In this way, run can be written any time after the module is
loaded and it will wait for the initialization to complete instead of
sending an error.

Fixes: 5648e56d03 ("NTB: ntb_perf: Add full multi-port NTB API support")
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Acked-by: Allen Hubbe <allenbh@gmail.com>
Tested-by: Alexander Fomichev <fomichev.ru@gmail.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2020-06-05 20:02:09 -04:00
Logan Gunthorpe b54369a248 NTB: perf: Fix support for hardware that doesn't have port numbers
Legacy drivers do not have port numbers (but is reliably only two ports)
and was broken by the recent commit that added mult-port support to
ntb_perf. This is especially important to support the cross link
topology which is perfectly symmetric and cannot assign unique port
numbers easily.

Hardware that returns zero for both the local port and the peer should
just always use gidx=0 for the only peer.

Fixes: 5648e56d03 ("NTB: ntb_perf: Add full multi-port NTB API support")
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Acked-by: Allen Hubbe <allenbh@gmail.com>
Tested-by: Alexander Fomichev <fomichev.ru@gmail.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2020-06-05 20:02:09 -04:00
Logan Gunthorpe a9c4211ac9 NTB: perf: Don't require one more memory window than number of peers
ntb_perf should not require more than one memory window per peer. This
was probably an off-by-one error.

Fixes: 5648e56d03 ("NTB: ntb_perf: Add full multi-port NTB API support")
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Acked-by: Allen Hubbe <allenbh@gmail.com>
Tested-by: Alexander Fomichev <fomichev.ru@gmail.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2020-06-05 20:02:08 -04:00
Logan Gunthorpe ca93c45755 NTB: ntb_pingpong: Choose doorbells based on port number
This commit fixes pingpong support for existing drivers that do not
implement ntb_default_port_number() and ntb_default_peer_port_number().
This is required for hardware (like the crosslink topology of
switchtec) which cannot assign reasonable port numbers to each port due
to its perfect symmetry.

Instead of picking the doorbell to use based on the the index of the
peer, we use the peer's port number. This is a bit clearer and easier
to understand.

Fixes: c7aeb0afdc ("NTB: ntb_pp: Add full multi-port NTB API support")
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Acked-by: Allen Hubbe <allenbh@gmail.com>
Tested-by: Alexander Fomichev <fomichev.ru@gmail.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2020-06-05 20:02:08 -04:00
Logan Gunthorpe fc8b086d9d NTB: Fix the default port and peer numbers for legacy drivers
When the commit adding ntb_default_port_number() and
ntb_default_peer_port_number()  entered the kernel there was no
users of it so it was impossible to tell what the API needed.

When a user finally landed a year later (ntb_pingpong) there were
more NTB topologies were created and no consideration was considered
to how other drivers had changed.

Now that there is a user it can be fixed to provide a sensible default
for the legacy drivers that do not implement ntb_{peer_}port_number().
Seeing ntb_pingpong doesn't check error codes returning EINVAL was also
not sensible.

Patches for ntb_pingpong and ntb_perf follow (which are broken
otherwise) to support hardware that doesn't have port numbers. This is
important not only to not break support with existing drivers but for
the cross link topology which, due to its perfect symmetry, cannot
assign unique port numbers to each side.

Fixes: 1e5301196a ("NTB: Add indexed ports NTB API")
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Acked-by: Allen Hubbe <allenbh@gmail.com>
Tested-by: Alexander Fomichev <fomichev.ru@gmail.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2020-06-05 20:02:08 -04:00
Logan Gunthorpe 40da7d9a93 NTB: Revert the change to use the NTB device dev for DMA allocations
Commit 417cf39cfe ("NTB: Set dma mask and dma coherent mask to NTB
devices") started using the NTB device for DMA allocations which was
turns out was wrong. If the IOMMU is enabled, such alloctanions will
always fail with messages such as:

  DMAR: Allocating domain for 0000:02:00.1 failed

This is because the IOMMU has not setup the device for such use.

Change the tools back to using the PCI device for allocations seeing
it doesn't make sense to add an IOMMU group for the non-physical NTB
device. Also remove the code that sets the DMA mask as it no longer
makes sense to do this.

Fixes: 7f46c8b3a5 ("NTB: ntb_tool: Add full multi-port NTB API support")
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Tested-by: Alexander Fomichev <fomichev.ru@gmail.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2020-06-05 20:02:08 -04:00
Logan Gunthorpe 912e12813d NTB: ntb_tool: reading the link file should not end in a NULL byte
When running ntb_test this warning is issued:

./ntb_test.sh: line 200: warning: command substitution: ignored null
byte in input

This is caused by the kernel returning one more byte than is necessary
when reading the link file.

Reduce the number of bytes read back to 2 as it was before the
commit that regressed this.

Fixes: 7f46c8b3a5 ("NTB: ntb_tool: Add full multi-port NTB API support")
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Acked-by: Allen Hubbe <allenbh@gmail.com>
Tested-by: Alexander Fomichev <fomichev.ru@gmail.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2020-06-05 20:02:08 -04:00
Sanjay R Mehta 9cb8bfdf52 ntb_perf: avoid false dma unmap of destination address
The DMA map and unmap of destination address is already being
done in perf_init_test() and perf_clear_test() functions.
Hence avoiding it by making necessary changes in perf_copy_chunk()
function.

Signed-off-by: Sanjay R Mehta <sanju.mehta@amd.com>
Signed-off-by: Arindam Nath <arindam.nath@amd.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2020-06-05 20:02:08 -04:00
Sanjay R Mehta d769966577 ntb_perf: increase sleep time from one milli sec to one sec
After trying to send commands for a maximum of MSG_TRIES
re-tries, link-up fails due to short sleep time(1ms) between
re-tries. Hence increasing the sleep time to one second providing
sufficient time for perf link-up.

Signed-off-by: Sanjay R Mehta <sanju.mehta@amd.com>
Signed-off-by: Arindam Nath <arindam.nath@amd.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2020-06-05 20:02:08 -04:00
Sanjay R Mehta 433efe7206 ntb_tool: pass correct struct device to dma_alloc_coherent
Currently, ntb->dev is passed to dma_alloc_coherent
and dma_free_coherent calls. The returned dma_addr_t
is the CPU physical address. This works fine as long
as IOMMU is disabled. But when IOMMU is enabled, we
need to make sure that IOVA is returned for dma_addr_t.
So the correct way to achieve this is by changing the
first parameter of dma_alloc_coherent() as ntb->pdev->dev
instead.

Fixes: 5648e56d03 ("NTB: ntb_perf: Add full multi-port NTB API support")
Signed-off-by: Sanjay R Mehta <sanju.mehta@amd.com>
Signed-off-by: Arindam Nath <arindam.nath@amd.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2020-06-05 20:02:08 -04:00
Sanjay R Mehta 98f4e14026 ntb_perf: pass correct struct device to dma_alloc_coherent
Currently, ntb->dev is passed to dma_alloc_coherent
and dma_free_coherent calls. The returned dma_addr_t
is the CPU physical address. This works fine as long
as IOMMU is disabled. But when IOMMU is enabled, we
need to make sure that IOVA is returned for dma_addr_t.
So the correct way to achieve this is by changing the
first parameter of dma_alloc_coherent() as ntb->pdev->dev
instead.

Fixes: 5648e56d03 ("NTB: ntb_perf: Add full multi-port NTB API support")
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Sanjay R Mehta <sanju.mehta@amd.com>
Signed-off-by: Arindam Nath <arindam.nath@amd.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2020-06-05 20:02:08 -04:00
Logan Gunthorpe f80fe8944e ntb: hw: remove the code that sets the DMA mask
This patch removes the code that sets the DMA mask as it no longer
makes sense to do this.

Fixes: 7f46c8b3a5 ("NTB: ntb_tool: Add full multi-port NTB API support")
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Tested-by: Alexander Fomichev <fomichev.ru@gmail.com>
Signed-off-by: Sanjay R Mehta <sanju.mehta@amd.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2020-06-05 20:02:08 -04:00
Dave Jiang 893733c58d ntb: intel: fix static declaration
intel_ntb4_link_disable() missing static declaration.

Reported-by: kbuild test robot <lkp@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2020-06-05 20:02:08 -04:00
Dave Jiang 134a86545c ntb: intel: add hw workaround for NTB BAR alignment
Add NTB_HWERR_BAR_ALIGN hw errata flag to work around issue where the
aligment for the XLAT base must be BAR size aligned rather than 4k page
aligned. On ICX platform, the XLAT base can be 4k page size aligned
rather than BAR size aligned unlike the previous gen Intel NTB. However,
a silicon errata prevented this from working as expected and a workaround
is introduced to resolve the issue.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2020-06-05 20:02:00 -04:00
Dave Jiang 26bfe3d0b2 ntb: intel: Add Icelake (gen4) support for Intel NTB
Adding 4th generation Intel NTB support bits. There are a lot of common
parts that the gen4 NTB has with gen3 NTB on Skylake. The commonalities are
reused in gen4 Icelake NTB.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Jon Mason <jdmason@kudzu.us>
2020-04-20 10:16:23 -04:00