New NPIV_ENQUIRY_CHANNEL and NPIV_SETUP_CHANNEL management datagrams (MADs)
were defined in a previous patchset. If the client advertises a desire to
use channels and the partner VIOS is channel capable then the client must
proceed with channel enquiry to determine the maximum number of channels
the VIOS is capable of providing, and registering SubCRQs via channel setup
with the VIOS immediately following NPIV Login. This handshaking should not
be performed for subsequent NPIV Logins unless the CRQ connection has been
reset.
Implement these two new MADs and issue them following a successful NPIV
login where the VIOS has set the SUPPORT_CHANNELS capability bit in the
NPIV Login response.
Link: https://lore.kernel.org/r/20210114203148.246656-13-tyreld@linux.ibm.com
Reviewed-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Allocate a set of Sub-CRQs in advance. During channel setup the client and
VIOS negotiate the number of queues the VIOS supports and the number that
the client desires to request. Its possible that the final channel
resources allocated is less than requested, but the client is still
responsible for sending handles for every queue it is hoping for.
Also, provide deallocation cleanup routines.
Link: https://lore.kernel.org/r/20210114203148.246656-8-tyreld@linux.ibm.com
Reviewed-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Subordinate Command Response Queues (Sub CRQ) are used in conjunction with
the primary CRQ when more than one queue is needed by the virtual I/O
adapter. Recent phyp firmware versions support Sub CRQ's with ibmvfc
adapters. This feature is a prerequisite for supporting multiple hardware
backed submission queues in the vfc adapter.
The Sub CRQ command element differs from the standard CRQ in that it is
32bytes long as opposed to 16bytes for the latter. Despite this extra
16bytes the ibmvfc protocol will use the original CRQ command element
mapped to the first 16bytes of the Sub CRQ element initially.
Add definitions for the Sub CRQ command element and queue.
Link: https://lore.kernel.org/r/20210114203148.246656-7-tyreld@linux.ibm.com
Reviewed-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Introduce several new vhost fields for managing MQ state of the adapter as
well as initial defaults for MQ enablement.
Link: https://lore.kernel.org/r/20210114203148.246656-2-tyreld@linux.ibm.com
Reviewed-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Drain the command queue and place all commands on a completion list.
Perform command completion on that list outside the host/queue locks.
Further, move purged command compeletions outside the host_lock as well.
Link: https://lore.kernel.org/r/20210106201835.1053593-5-tyreld@linux.ibm.com
Reviewed-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Define per-queue locks for protecting queue state and event pool sent/free
lists. The evt list lock is initially redundant but it allows the driver to
be modified in the follow-up patches to relax the queue locking around
submissions and completions.
Link: https://lore.kernel.org/r/20210106201835.1053593-4-tyreld@linux.ibm.com
Reviewed-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
There is currently a single command event pool per host. In anticipation of
providing multiple queues add a per-queue event pool definition and
reimplement the existing CRQ to use its queue defined event pool for
command submission and completion.
Link: https://lore.kernel.org/r/20210106201835.1053593-3-tyreld@linux.ibm.com
Reviewed-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
The primary and async CRQs are nearly identical outside of the format and
length of each message entry in the dma mapped page that represents the
queue data. These queues can be represented with a generic queue structure
that uses a union to differentiate between message format of the mapped
page.
This structure will further be leveraged in a followup patcheset that
introduces Sub-CRQs.
Link: https://lore.kernel.org/r/20210106201835.1053593-2-tyreld@linux.ibm.com
Reviewed-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Introduce a target_wwpn field to several MADs. Its possible that a SCSI ID
of a target can change due to some fabric changes. The WWPN of the SCSI
target provides a better way to identify the target. Also, add flags for
receiving MAD versioning information and advertising client support for
targetWWPN with the VIOS. This latter capability flag will be required for
future clients capable of requesting multiple hardware queues from the host
adapter.
Link: https://lore.kernel.org/r/20201118011104.296999-3-tyreld@linux.ibm.com
Acked-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
When a canister on a FS9100, or similar storage, running in NPIV mode, is
rebooted, its WWPNs will fail over to another canister. When this occurs,
we see a WWPN going away from the fabric at one N-Port ID, and, a short
time later, the same WWPN appears at a different N-Port ID. When the
canister is fully operational again, the WWPNs fail back to the original
canister. If there is any I/O outstanding to the target when this occurs,
it will result in the implicit logout the ibmvfc driver issues before
removing the rport to fail. When the WWPN then shows up at a different
N-Port ID, and we issue a PLOGI to it, the VIOS will see that it still has
a login for this WWPN at the old N-Port ID, which results in the VIOS
simulating a link down / link up sequence to the client, in order to get
the VIOS and client LPAR in sync.
The patch below improves the way we handle this scenario so as to avoid the
link bounce, which affects all targets under the virtual host adapter. The
change is to utilize the Move Login MAD, which will work even when I/O is
outstanding to the target. The change only alters the target state machine
for the case where the implicit logout fails prior to deleting the rport.
If this implicit logout fails, we defer deleting the ibmvfc_target object
after calling fc_remote_port_delete. This enables us to later retry the
implicit logout after terminate_rport_io occurs, or to issue the Move Login
request if a WWPN shows up at a new N-Port ID prior to this occurring.
This has been tested by IBM's storage interoperability team on a FS9100,
forcing the failover to occur. With debug tracing enabled in the ibmvfc
driver, we confirmed the move login was sent in this scenario and confirmed
the link bounce no longer occurred.
[mkp: fix checkpatch warnings]
Link: https://lore.kernel.org/r/1599859706-8505-1-git-send-email-brking@linux.vnet.ibm.com
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
VIOS partitions with SLI-4 enabled Emulex adapters will be capable of
driving I/O in parallel through mulitple work queues or channels, and with
new hypervisor firmware that supports multiple interrupt sources an ibmvfc
NPIV single initiator can be modified to exploit end-to-end channelization
in a PowerVM environment.
VIOS hosts will also be able to expose fabric perfromance impact
notifications (FPIN) via a new asynchronous event to ibmvfc clients that
advertise support via IBMVFC_CAN_HANDLE_FPIN in their capabilities flag
during NPIV_LOGIN.
This patch introduces three new Management Datagrams (MADs) for
channelization support negotiation as well as the FPIN asynchronous
event and FPIN status flags. Follow up work is required to plumb the
ibmvfc client driver to use these new interfaces.
Link: https://lore.kernel.org/r/20200904232936.840193-2-tyreld@linux.ibm.com
Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Update ibmvfc.h structs to use the preferred __packed and __aligned()
attribute macros defined in include/linux/compiler_attributes.h in place of
__attribute__().
Link: https://lore.kernel.org/r/20200904232936.840193-1-tyreld@linux.ibm.com
Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
When an SVC node goes down as part of a node reboot, its WWPNs are moved to
the remaining node. When the node is back online, its WWPNs are moved
back. The result is that the WWPN moves from one NPort_ID to another, then
back again. The ibmvfc driver was forcing the old port to be removed, but
not sending an implicit logout. When the WWPN showed up at the new
location, the PLOGI failed as there was already a login established for the
old scsi id. The patch below fixes this by ensuring we always send an
implicit logout for any scsi id associated with an rport prior to calling
fc_remote_port_delete.
Link: https://lore.kernel.org/r/1582767943-16611-1-git-send-email-brking@linux.vnet.ibm.com
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Based on 1 normalized pattern(s):
this program is free software you can redistribute it and or modify
it under the terms of the gnu general public license as published by
the free software foundation either version 2 of the license or at
your option any later version this program is distributed in the
hope that it will be useful but without any warranty without even
the implied warranty of merchantability or fitness for a particular
purpose see the gnu general public license for more details you
should have received a copy of the gnu general public license along
with this program if not write to the free software foundation inc
59 temple place suite 330 boston ma 02111 1307 usa
extracted by the scancode license scanner the SPDX license identifier
GPL-2.0-or-later
has been chosen to replace the boilerplate/reference in 1334 file(s).
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Allison Randal <allison@lohutok.net>
Reviewed-by: Richard Fontana <rfontana@redhat.com>
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190527070033.113240726@linutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
No change to functionality. Simply make transport event messages a little
clearer, and rework CRQ format enums such that we have separate enums for
INIT messages and XPORT events.
[mkp: typo]
Signed-off-by: Tyrel Datwyler <tyreld@linux.vnet.ibm.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
The fcp_rsp_info structure as defined in the FC spec has an initial 3
bytes reserved field. The ibmvfc driver mistakenly defined this field as
4 bytes resulting in the rsp_code field being defined in what should be
the start of the second reserved field and thus always being reported as
zero by the driver.
Ideally, we should wire ibmvfc up with libfc for the sake of code
deduplication, and ease of maintaining standardized structures in a
single place. However, for now simply fixup the definition in ibmvfc for
backporting to distros on older kernels. Wiring up with libfc will be
done in a followup patch.
Cc: <stable@vger.kernel.org>
Reported-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Tyrel Datwyler <tyreld@linux.vnet.ibm.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
The ibmvfc driver currently doesn't support FC Class 3 Error Recovery.
However, it is simply a matter of informing the VIOS that the payload
expects to use sequence level error recovery via a bit flag in the
ibmvfc_cmd structure.
This patch adds a module parameter to enable error recovery support at
boot time. When enabled the RETRY service parameter bit is set during
PRLI, and ibmvfc_cmd->flags includes the IBMVFC_CLASS_3_ERR bit.
Signed-off-by: Tyrel Datwyler <tyreld@linux.vnet.ibm.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
This driver is a pick up of the old IBM VIO scsi Target Driver
that was started by Nick and Fujita 2-4 years ago.
http://comments.gmane.org/gmane.linux.scsi/90119
The driver provides a virtual SCSI device on IBM Power Servers.
This patch contains the fifth version for an initial merge of the
tcm ibmvscsis driver. More information on this driver and config
can be found:
https://github.com/powervm/ibmvscsis/wiki/Configurationhttp://www.linux-iscsi.org/wiki/IBM_vSCSI
(Drop extra libsrp review breakage + Fix kconfig typo - nab)
Signed-off-by: Steven Royer <seroyer@linux.vnet.ibm.com>
Signed-off-by: Tyrel Datwyler <tyreld@linux.vnet.ibm.com>
Signed-off-by: Michael Cyr <mikecyr@linux.vnet.ibm.com>
Signed-off-by: Bryant G. Ly <bryantly@linux.vnet.ibm.com>
Cc: FUJITA Tomonori <tomof@acm.org>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
Added big endian annotations to relevant data structure fields, and necessary
byte swappings to support little endian builds.
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: Tyrel Datwyler <tyreld@linux.vnet.ibm.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Acked-by: Robert Jennings <rcj@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
Adds support for a new VIOS feature that allows ibmvfc to
optimize terminate_rport_io by telling the VIOS the target
is no longer accessible on the fabric and that it should
not send an ABTS out on the fabric to the device.
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Acked-by: Robert Jennings <rcj@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
By changing field ordering we can avoid a couple of memory holes in
the tables that use the ibmvfc_async_desc structure.
Signed-off-by: Robert Jennings <rcj@linux.vnet.ibm.com>
Acked-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
By default, ibmvfc does not log any async events in order
to avoid flooding the log with them. Improve on this by
logging by default events that are not likely to flood the
log, such as link up/down. Having these events in the log
will improve the ability to debug issues with ibmvfc.
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
The ibmvfc driver was incorrectly obtaining a scsi_target pointer
from an fc_rport. The way it is coded ensures that ibmvfc's
terminate_rport_io handler does absolutely nothing. Fix this up
to iterate through affected devices differently, sending cancel
and abort task set as appropriate. Without this patch,
fast_io_fail_tmo is broken for ibmvfc.
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Commit 43c8da907c introduced a race
condition which can occur when adding/deleting rports. There are
two possible threads now that can be deleting rports in the ibmvfc
driver, which can result in list_del being called twice, resulting
in an oops. This patch adds a new state to the ibmvfc_target struct
to indicate the target has been removed from the list and is in
the process of being deleted.
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
This fixes a softlockup seen on resume. During resume, the CRQ
must be reenabled. However, the H_ENABLE_CRQ hcall used to do
this may return H_BUSY or H_LONG_BUSY. When this happens, the
caller is expected to retry later. Normally the H_ENABLE_CRQ
succeeds relatively soon. However, we have seen cases where
this can take long enough to see softlockup warnings.
This patch changes a simple loop, which was causing the
softlockup, to a loop at task level which sleeps between
retries rather than simply spinning.
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
If a command times out resulting in EH getting invoked, we wait for the
aborted commands to come back after sending the abort. Shorten
the amount of time we wait for these responses, to ensure we don't
get stuck in EH for several minutes.
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Fixes a problem seen where sending a PRLI to a target
resulted in it sending a LOGO. This caused the ibmvfc
driver to go back through discovery again, which caused
another PRLI attempt, which caused another LOGO. Fix this
behavior by ignoring LOGO if we haven't even logged into
the target yet.
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
The virtual I/O server controlling the NPIV adapter associated with
a virtual fibre channel adapter can send a HALT event to the client.
When this occurs, the client can no longer send commands until a RESUME
is received. By adding support for flush on halt, we will get all of
our outstanding commands flushed back before the Virtual I/O server
enters the halt state, eliminating potential command timeouts for
outstanding commands which might occur if we did not support this feature.
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
This patch adds support for a new command supported by the Virtual I/O
Server, NPIV Logout. The command will abort all outstanding commands
and log out of the fabric. Currently, the only way to do this is
by breaking the CRQ, which can take a fairly long time when lots of
commands are outstanding. The NPIV Logout commands provides a mechanism
to accomplish virtually the same function, but is much faster.
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
The ibmvfc driver currently logs errors during discovery for several
transient fabric errors, which generally get retried. If retries
do not work, we see multiple errors in the log. If retries do work,
we see errors in the log which may be confusing since the retry worked.
This patch enhances the discovery time error logging to only log errors
for command failures during discovery if all allowed retries have been
used up. The existing behavior of logging all failures can be restored
by setting the hosts log_level to a value of 3 or greater.
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
Bump driver version to 1.0.5.
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
The ibmvfc driver currently breaks the CRQ and essentially
resets the entire virtual FC adapter, killing all outstanding
ops to all attached targets, if an ADISC times out during target
discover/rediscovery. This patch adds some code to cancel the
ADISC if it times out, which prevents a single ADISC timeout from
affecting the other devices attached to the fabric.
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
This patch fixes a problem of possible dropped interrupts. Currently,
the ibmvfc driver has a race condition where after ibmvfc_interrupt
gets run, the platform code clears the interrupt. This can result in
lost interrupts and, in worst case scenarios, result in command
timeouts. Fix this by implementing a tasklet similar to what the
ibmvscsi driver does so that interrupt processing is no longer done in
the actual interrupt handler, which eliminates the race.
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
During cancel testing it has been shown that 15 seconds is not
nearly long enough for the VIOS to respond to a cancel under
loaded situations. Increasing this timeout to 60 seconds allows
time for the VIOS to cancel the outstanding commands and prevents
us from escalating to a full host reset, which can take much longer.
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
This is a powerpc specific driver.
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
While doing various error injection testing, such as cable
pulls and target moves, some issues were observed in handling
these events. This patch improves the way these events are handled
by increasing the delay waiting for the fabric to settle and also
changes the behavior of Link Up to break the CRQ to ensure everything
gets cleaned up properly on the VIOS.
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
Adds a delay prior to retrying a failed NPIV login. This fixes
a scenario if the backing fibre channel adapter is getting reset
due to an EEH event, NPIV login will fail. Currently, ibmvfc
retries three times very quickly, resets the CRQ and tries one
more time. If the adapter is getting reset due to EEH, this isn't
enough time. This adds a delay prior to retrying a failed NPIV
login and also increments the number of retries.
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
The virtual fibre channel stack can return a failure response for a command
indicating the port login has been invalidated without sending the client
an async event. Add code to handle this response and initiate a PLOGI.
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
The CRQs used by the ibmvfc driver are read and written by both
the client and the server. Therefore, we need to mark them volatile
so that we do not cache their contents when handling an interrupt.
This fixes a problem which can surface as occasional command timeouts.
No commands were actually timing out, but due to accessing cached data
for the CRQ in the interrupt handler, the interrupt was not processing
all command completions as it should.
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
If the ibmvfc driver is in discovery attempting to log into a target
and it encounters an error, the command may get retried one or more
times, depending on the error received. If the retries are
unsuccessful such that the discovery thread gives up on discovery to
that target, the target ends up in a state where, if SCSI core had
previously known about the device, the host will get unblocked but the
host will not be logged into the target, causing any commands sent to
the target to fail. This patch fixes this so that if this occurs, the
target is deleted such that the normal dev_loss processing can occur
instead.
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com>
Tyrel Datwyler