It is unavoidable that zfcp_scsi_queuecommand() has to finish requests
with DID_IMM_RETRY (like fc_remote_port_chkready()) during the time
window when zfcp detected an unavailable rport but
fc_remote_port_delete(), which is asynchronous via
zfcp_scsi_schedule_rport_block(), has not yet blocked the rport.
However, for the case when the rport becomes available again, we should
prevent unblocking the rport too early. In contrast to other FCP LLDDs,
zfcp has to open each LUN with the FCP channel hardware before it can
send I/O to a LUN. So if a port already has LUNs attached and we
unblock the rport just after port recovery, recoveries of LUNs behind
this port can still be pending which in turn force
zfcp_scsi_queuecommand() to unnecessarily finish requests with
DID_IMM_RETRY.
This also opens a time window with unblocked rport (until the followup
LUN reopen recovery has finished). If a scsi_cmnd timeout occurs during
this time window fc_timed_out() cannot work as desired and such command
would indeed time out and trigger scsi_eh. This prevents a clean and
timely path failover. This should not happen if the path issue can be
recovered on FC transport layer such as path issues involving RSCNs.
Fix this by only calling zfcp_scsi_schedule_rport_register(), to
asynchronously trigger fc_remote_port_add(), after all LUN recoveries as
children of the rport have finished and no new recoveries of equal or
higher order were triggered meanwhile. Finished intentionally includes
any recovery result no matter if successful or failed (still unblock
rport so other successful LUNs work). For simplicity, we check after
each finished LUN recovery if there is another LUN recovery pending on
the same port and then do nothing. We handle the special case of a
successful recovery of a port without LUN children the same way without
changing this case's semantics.
For debugging we introduce 2 new trace records written if the rport
unblock attempt was aborted due to still unfinished or freshly triggered
recovery. The records are only written above the default trace level.
Benjamin noticed the important special case of new recovery that can be
triggered between having given up the erp_lock and before calling
zfcp_erp_action_cleanup() within zfcp_erp_strategy(). We must avoid the
following sequence:
ERP thread rport_work other context
------------------------- -------------- --------------------------------
port is unblocked, rport still blocked,
due to pending/running ERP action,
so ((port->status & ...UNBLOCK) != 0)
and (port->rport == NULL)
unlock ERP
zfcp_erp_action_cleanup()
case ZFCP_ERP_ACTION_REOPEN_LUN:
zfcp_erp_try_rport_unblock()
((status & ...UNBLOCK) != 0) [OLD!]
zfcp_erp_port_reopen()
lock ERP
zfcp_erp_port_block()
port->status clear ...UNBLOCK
unlock ERP
zfcp_scsi_schedule_rport_block()
port->rport_task = RPORT_DEL
queue_work(rport_work)
zfcp_scsi_rport_work()
(port->rport_task != RPORT_ADD)
port->rport_task = RPORT_NONE
zfcp_scsi_rport_block()
if (!port->rport) return
zfcp_scsi_schedule_rport_register()
port->rport_task = RPORT_ADD
queue_work(rport_work)
zfcp_scsi_rport_work()
(port->rport_task == RPORT_ADD)
port->rport_task = RPORT_NONE
zfcp_scsi_rport_register()
(port->rport == NULL)
rport = fc_remote_port_add()
port->rport = rport;
Now the rport was erroneously unblocked while the zfcp_port is blocked.
This is another situation we want to avoid due to scsi_eh
potential. This state would at least remain until the new recovery from
the other context finished successfully, or potentially forever if it
failed. In order to close this race, we take the erp_lock inside
zfcp_erp_try_rport_unblock() when checking the status of zfcp_port or
LUN. With that, the possible corresponding rport state sequences would
be: (unblock[ERP thread],block[other context]) if the ERP thread gets
erp_lock first and still sees ((port->status & ...UNBLOCK) != 0),
(block[other context],NOP[ERP thread]) if the ERP thread gets erp_lock
after the other context has already cleard ...UNBLOCK from port->status.
Since checking fields of struct erp_action is unsafe because they could
have been overwritten (re-used for new recovery) meanwhile, we only
check status of zfcp_port and LUN since these are only changed under
erp_lock elsewhere. Regarding the check of the proper status flags (port
or port_forced are similar to the shown adapter recovery):
[zfcp_erp_adapter_shutdown()]
zfcp_erp_adapter_reopen()
zfcp_erp_adapter_block()
* clear UNBLOCK ---------------------------------------+
zfcp_scsi_schedule_rports_block() |
write_lock_irqsave(&adapter->erp_lock, flags);-------+ |
zfcp_erp_action_enqueue() | |
zfcp_erp_setup_act() | |
* set ERP_INUSE -----------------------------------|--|--+
write_unlock_irqrestore(&adapter->erp_lock, flags);--+ | |
.context-switch. | |
zfcp_erp_thread() | |
zfcp_erp_strategy() | |
write_lock_irqsave(&adapter->erp_lock, flags);------+ | |
... | | |
zfcp_erp_strategy_check_target() | | |
zfcp_erp_strategy_check_adapter() | | |
zfcp_erp_adapter_unblock() | | |
* set UNBLOCK -----------------------------------|--+ |
zfcp_erp_action_dequeue() | |
* clear ERP_INUSE ---------------------------------|-----+
... |
write_unlock_irqrestore(&adapter->erp_lock, flags);-+
Hence, we should check for both UNBLOCK and ERP_INUSE because they are
interleaved. Also we need to explicitly check ERP_FAILED for the link
down case which currently does not clear the UNBLOCK flag in
zfcp_fsf_link_down_info_eval().
Signed-off-by: Steffen Maier <maier@linux.vnet.ibm.com>
Fixes: 8830271c48 ("[SCSI] zfcp: Dont fail SCSI commands when transitioning to blocked fc_rport")
Fixes: a2fa0aede0 ("[SCSI] zfcp: Block FC transport rports early on errors")
Fixes: 5f852be9e1 ("[SCSI] zfcp: Fix deadlock between zfcp ERP and SCSI")
Fixes: 338151e066 ("[SCSI] zfcp: make use of fc_remote_port_delete when target port is unavailable")
Fixes: 3859f6a248 ("[PATCH] zfcp: add rports to enable scsi_add_device to work again")
Cc: <stable@vger.kernel.org> #2.6.32+
Reviewed-by: Benjamin Block <bblock@linux.vnet.ibm.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
When SCSI EH invokes zFCP's callbacks for eh_device_reset_handler() and
eh_target_reset_handler(), it expects us to relent the ownership over
the given scsi_cmnd and all other scsi_cmnds within the same scope - LUN
or target - when returning with SUCCESS from the callback ('release'
them). SCSI EH can then reuse those commands.
We did not follow this rule to release commands upon SUCCESS; and if
later a reply arrived for one of those supposed to be released commands,
we would still make use of the scsi_cmnd in our ingress tasklet. This
will at least result in undefined behavior or a kernel panic because of
a wrong kernel pointer dereference.
To fix this, we NULLify all pointers to scsi_cmnds (struct zfcp_fsf_req
*)->data in the matching scope if a TMF was successful. This is done
under the locks (struct zfcp_adapter *)->abort_lock and (struct
zfcp_reqlist *)->lock to prevent the requests from being removed from
the request-hashtable, and the ingress tasklet from making use of the
scsi_cmnd-pointer in zfcp_fsf_fcp_cmnd_handler().
For cases where a reply arrives during SCSI EH, but before we get a
chance to NULLify the pointer - but before we return from the callback
-, we assume that the code is protected from races via the CAS operation
in blk_complete_request() that is called in scsi_done().
The following stacktrace shows an example for a crash resulting from the
previous behavior:
Unable to handle kernel pointer dereference at virtual kernel address fffffee17a672000
Oops: 0038 [#1] SMP
CPU: 2 PID: 0 Comm: swapper/2 Not tainted
task: 00000003f7ff5be0 ti: 00000003f3d38000 task.ti: 00000003f3d38000
Krnl PSW : 0404d00180000000 00000000001156b0 (smp_vcpu_scheduled+0x18/0x40)
R:0 T:1 IO:0 EX:0 Key:0 M:1 W:0 P:0 AS:3 CC:1 PM:0 EA:3
Krnl GPRS: 000000200000007e 0000000000000000 fffffee17a671fd8 0000000300000015
ffffffff80000000 00000000005dfde8 07000003f7f80e00 000000004fa4e800
000000036ce8d8f8 000000036ce8d9c0 00000003ece8fe00 ffffffff969c9e93
00000003fffffffd 000000036ce8da10 00000000003bf134 00000003f3b07918
Krnl Code: 00000000001156a2: a7190000 lghi %r1,0
00000000001156a6: a7380015 lhi %r3,21
#00000000001156aa: e32050000008 ag %r2,0(%r5)
>00000000001156b0: 482022b0 lh %r2,688(%r2)
00000000001156b4: ae123000 sigp %r1,%r2,0(%r3)
00000000001156b8: b2220020 ipm %r2
00000000001156bc: 8820001c srl %r2,28
00000000001156c0: c02700000001 xilf %r2,1
Call Trace:
([<0000000000000000>] 0x0)
[<000003ff807bdb8e>] zfcp_fsf_fcp_cmnd_handler+0x3de/0x490 [zfcp]
[<000003ff807be30a>] zfcp_fsf_req_complete+0x252/0x800 [zfcp]
[<000003ff807c0a48>] zfcp_fsf_reqid_check+0xe8/0x190 [zfcp]
[<000003ff807c194e>] zfcp_qdio_int_resp+0x66/0x188 [zfcp]
[<000003ff80440c64>] qdio_kick_handler+0xdc/0x310 [qdio]
[<000003ff804463d0>] __tiqdio_inbound_processing+0xf8/0xcd8 [qdio]
[<0000000000141fd4>] tasklet_action+0x9c/0x170
[<0000000000141550>] __do_softirq+0xe8/0x258
[<000000000010ce0a>] do_softirq+0xba/0xc0
[<000000000014187c>] irq_exit+0xc4/0xe8
[<000000000046b526>] do_IRQ+0x146/0x1d8
[<00000000005d6a3c>] io_return+0x0/0x8
[<00000000005d6422>] vtime_stop_cpu+0x4a/0xa0
([<0000000000000000>] 0x0)
[<0000000000103d8a>] arch_cpu_idle+0xa2/0xb0
[<0000000000197f94>] cpu_startup_entry+0x13c/0x1f8
[<0000000000114782>] smp_start_secondary+0xda/0xe8
[<00000000005d6efe>] restart_int_handler+0x56/0x6c
[<0000000000000000>] 0x0
Last Breaking-Event-Address:
[<00000000003bf12e>] arch_spin_lock_wait+0x56/0xb0
Suggested-by: Steffen Maier <maier@linux.vnet.ibm.com>
Signed-off-by: Benjamin Block <bblock@linux.vnet.ibm.com>
Fixes: ea127f9754 ("[PATCH] s390 (7/7): zfcp host adapter.") (tglx/history.git)
Cc: <stable@vger.kernel.org> #2.6.32+
Signed-off-by: Steffen Maier <maier@linux.vnet.ibm.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
On a successful end of reopen port forced,
zfcp_erp_strategy_followup_success() re-uses the port erp_action
and the subsequent zfcp_erp_action_cleanup() now
sees ZFCP_ERP_SUCCEEDED with
erp_action->action==ZFCP_ERP_ACTION_REOPEN_PORT
instead of ZFCP_ERP_ACTION_REOPEN_PORT_FORCED
but must not perform zfcp_scsi_schedule_rport_register().
We can detect this because the fresh port reopen erp_action
is in its very first step ZFCP_ERP_STEP_UNINITIALIZED.
Otherwise this opens a time window with unblocked rport
(until the followup port reopen recovery would block it again).
If a scsi_cmnd timeout occurs during this time window
fc_timed_out() cannot work as desired and such command
would indeed time out and trigger scsi_eh. This prevents
a clean and timely path failover.
This should not happen if the path issue can be recovered
on FC transport layer such as path issues involving RSCNs.
Also, unnecessary and repeated DID_IMM_RETRY for pending and
undesired new requests occur because internally zfcp still
has its zfcp_port blocked.
As follow-on errors with scsi_eh, it can cause,
in the worst case, permanently lost paths due to one of:
sd <scsidev>: [<scsidisk>] Medium access timeout failure. Offlining disk!
sd <scsidev>: Device offlined - not ready after error recovery
For fix validation and to aid future debugging with other recoveries
we now also trace (un)blocking of rports.
Signed-off-by: Steffen Maier <maier@linux.vnet.ibm.com>
Fixes: 5767620c38 ("[SCSI] zfcp: Do not unblock rport from REOPEN_PORT_FORCED")
Fixes: a2fa0aede0 ("[SCSI] zfcp: Block FC transport rports early on errors")
Fixes: 5f852be9e1 ("[SCSI] zfcp: Fix deadlock between zfcp ERP and SCSI")
Fixes: 338151e066 ("[SCSI] zfcp: make use of fc_remote_port_delete when target port is unavailable")
Fixes: 3859f6a248 ("[PATCH] zfcp: add rports to enable scsi_add_device to work again")
Cc: <stable@vger.kernel.org> #2.6.32+
Reviewed-by: Benjamin Block <bblock@linux.vnet.ibm.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
'0' is now used as the default cmd_per_lun value,
so there's no need to explicitly set it to '1' in the
host template.
Signed-off-by: Hannes Reinecke <hare@suse.de>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: James Bottomley <JBottomley@Odin.com>
Drop the now unused reason argument from the ->change_queue_depth method.
Also add a return value to scsi_adjust_queue_depth, and rename it to
scsi_change_queue_depth now that it can be used as the default
->change_queue_depth implementation.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Mike Christie <michaelc@cs.wisc.edu>
Reviewed-by: Hannes Reinecke <hare@suse.de>
All drivers use the implementation for ramping the queue up and down, so
instead of overloading the change_queue_depth method call the
implementation diretly if the driver opts into it by setting the
track_queue_depth flag in the host template.
Note that a few drivers validated the new queue depth in their
change_queue_depth method, but as we never go over the queue depth
set during slave_configure or the sysfs file this isn't nessecary
and can safely be removed.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Mike Christie <michaelc@cs.wisc.edu>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Reviewed-by: Venkatesh Srinivas <venkateshs@google.com>
Remove the tagged argument from scsi_adjust_queue_depth, and just let it
handle the queue depth. For most drivers those two are fairly separate,
given that most modern drivers don't care about the SCSI "tagged" status
of a command at all, and many old drivers allow queuing of multiple
untagged commands in the driver.
Instead we start out with the ->simple_tags flag set before calling
->slave_configure, which is how all drivers actually looking at
->simple_tags except for one worke anyway. The one other case looks
broken, but I've kept the behavior as-is for now.
Except for that we only change ->simple_tags from the ->change_queue_type,
and when rejecting a tag message in a single driver, so keeping this
churn out of scsi_adjust_queue_depth is a clear win.
Now that the usage of scsi_adjust_queue_depth is more obvious we can
also remove all the trivial instances in ->slave_alloc or ->slave_configure
that just set it to the cmd_per_lun default.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Mike Christie <michaelc@cs.wisc.edu>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com>
Commit 86a9668a8d
"[SCSI] zfcp: support for hardware data router"
reduced the initial block queue limits in the scsi_host_template to the
absolute minimum and adjusted them later on. However, the adjustment was
too late for the BSG devices of Scsi_Host and fc_host.
Therefore, ioctl(..., SG_IO, ...) with request or response size > 4kB to a
BSG device of an fc_host or a Scsi_Host fails with EINVAL. As a result,
users of such ioctl such as HBA_SendCTPassThru() in libzfcphbaapi return
with error HBA_STATUS_ERROR.
Initialize the block queue limits in zfcp_scsi_host_template to the
greatest common denominator (GCD).
While we cannot exploit the slightly enlarged maximum request size with
data router, this should be neglectible. Doing so also avoids running into
trouble after live guest relocation (LGR) / migration from a data router
FCP device to an FCP device that does not support data router. In that
case, zfcp would figure out the new limits on adapter recovery, but the
fc_host and Scsi_Host (plus in fact all sdevs) still exist with the old and
now too large queue limits.
It should also OK, not to use half the size as in the DIX case, because
fc_host and Scsi_Host do not transport FCP requests including SCSI commands
using protection data.
Signed-off-by: Steffen Maier <maier@linux.vnet.ibm.com>
Reviewed-by: Martin Peschke <mpeschke@linux.vnet.ibm.com>
Cc: <stable@vger.kernel.org> #3.2+
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
Remove the file name from the comment at top of many files. In most
cases the file name was wrong anyway, so it's rather pointless.
Also unify the IBM copyright statement. We did have a lot of sightly
different statements and wanted to change them one after another
whenever a file gets touched. However that never happened. Instead
people start to take the old/"wrong" statements to use as a template
for new files.
So unify all of them in one go.
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
zfcp_scsi_slave_destroy erroneously always tried to finish its task
even if the corresponding previous zfcp_scsi_slave_alloc returned
early. This can lead to kernel page faults on accessing uninitialized
fields of struct zfcp_scsi_dev in zfcp_erp_lun_shutdown_wait. Take the
port field of the struct to determine if slave_alloc returned early.
This zfcp bug is exposed by 4e6c82b (in turn fixing f7c9c6b to be
compatible with 21208ae) which can call slave_destroy for a
corresponding previous slave_alloc that did not finish.
This patch is based on James Bottomley's fix suggestion in
http://www.spinics.net/lists/linux-scsi/msg55449.html.
Signed-off-by: Steffen Maier <maier@linux.vnet.ibm.com>
Cc: <stable@kernel.org> #2.6.38+
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
Fix several compile errors on s390 caused by splitting module.h.
Some include additions [e.g. qdio_setup.c, zfcp_qdio.c] are in
anticipation of pending changes queued for s390 that increase
the modular use footprint.
[PG: added additional obvious changes since Heiko's original patch]
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
FICON Express8S supports hardware data router, which requires an
adapted qdio request format.
This part 2/2 exploits the functionality in zfcp.
Signed-off-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Steffen Maier <maier@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
DIF/DIX support for zfcp is no longer experimental,
and config option is no longer necessary.
Return error from queuecommand for unsupported data directions.
Signed-off-by: Steffen Maier <maier@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
This allows us to move duplicated code in <asm/atomic.h>
(atomic_inc_not_zero() for now) to <linux/atomic.h>
Signed-off-by: Arun Sharma <asharma@fb.com>
Reviewed-by: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: David Miller <davem@davemloft.net>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Acked-by: Mike Frysinger <vapier@gentoo.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Query the FC symbolic port name for reporting in the fc_host sysfs and
enable the symbolic_name attribute in the fc_host sysfs. When running
in NPIV mode, extend the symbolic port name with the devno and the
hostname. This allows better identification of Linux systems for SAN
and storage administrators.
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: Steffen Maier <maier@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
The SCSI host and transport templates are the only members left in the
global zfcp_data struct. Move them out of zfcp_data and remove the
now unused zfcp_data struct. Also update the names of the register and
unregister functions to use the zfcp_scsi prefix.
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: Steffen Maier <maier@linux.vnet.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
The zfcpdump tool requires a method to attach exactly one LUN. The
easiest way to achieve this is to add a new zfcp module parameter.
When allow_lun_scan is set to "false", zfcp only accepts LUNs that
have been configured through the unit_add sysfs interface.
Reviewed-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
This patch is the final cleanup of the redesign from the zfcp tracing.
Structures and elements which were used by multiple areas of the
former debug tracing are now changed to the new scheme.
Signed-off-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
This patch is the continuation to redesign the zfcp tracing to a more
straight-forward and easy to extend scheme.
This patch deals with all trace records of the zfcp SCSI area.
Signed-off-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Interrupting the connection to the FCP channel while I/O requests are
being issued can lead to this deadlock. scsi_dispatch_cmd already
holds the host_lock while the recovery trigger tries to acquire the
host_lock again when iterating through the scsi_devices.
INFO: lockdep is turned off.
BUG: spinlock lockup on CPU#1, blast/9660, 0000000078f38878
CPU: 1 Not tainted 2.6.35.7SWEN2 #2
Process blast (pid: 9660, task: 0000000071f75940, ksp: 0000000074393ac0)
0000000074393640 00000000743935c0 0000000000000002 0000000000000000
0000000074393660 00000000743935d8 00000000743935d8 00000000005590c2
0000000000000000 0000000078f38878 0000000026ede800 0000000078f38878
000000000000000d 040000000000000c 0000000074393628 0000000000000000
0000000000000000 0000000000100b2a 00000000743935c0 0000000074393600
Call Trace:
([<0000000000100a32>] show_trace+0xee/0x144)
[<00000000003be202>] do_raw_spin_lock+0x112/0x178
[<000000000055d408>] _raw_spin_lock_irqsave+0x90/0xb0
[<00000000003f1514>] __scsi_iterate_devices+0x38/0xbc
[<00000000004849b0>] zfcp_erp_clear_adapter_status+0xd0/0x16c
[<000000000048587a>] zfcp_erp_adapter_reopen+0x3a/0xb4
[<0000000000489812>] zfcp_fsf_req_send+0x166/0x180
[<000000000048c8d6>] zfcp_fsf_fcp_cmnd+0x272/0x408
[<000000000048f864>] zfcp_scsi_queuecommand+0x11c/0x1e0
[<00000000003f1f2a>] scsi_dispatch_cmd+0x1d6/0x324
[<00000000003f9910>] scsi_request_fn+0x42c/0x56c
[<00000000003828ae>] __blk_run_queue+0x86/0x140
[<000000000037f742>] elv_insert+0x11a/0x208
[<000000000038104c>] blk_insert_cloned_request+0x84/0xe4
[<000003c0032b7c64>] dm_dispatch_request+0x6c/0x94 [dm_mod]
[<000003c0032b7d5c>] map_request+0xd0/0x100 [dm_mod]
[<000003c0032b9a78>] dm_request_fn+0xec/0x1bc [dm_mod]
[<0000000000382c0e>] generic_unplug_device+0x5a/0x6c
[<000003c0032b7f98>] dm_unplug_all+0x74/0x9c [dm_mod]
[<00000000001d1272>] sync_page+0x76/0x9c
[<00000000001d12ba>] sync_page_killable+0x22/0x60
[<000000000055a768>] __wait_on_bit_lock+0xc0/0x124
[<00000000001d1140>] __lock_page_killable+0x78/0x84
[<00000000001d351c>] generic_file_aio_read+0x5a4/0x7e8
[<0000000000228ec0>] do_sync_read+0xc8/0x12c
[<0000000000229edc>] vfs_read+0xac/0x1ac
[<000000000022a0d8>] SyS_read+0x58/0xa8
[<00000000001146de>] sysc_noemu+0x10/0x16
[<00000200000493c4>] 0x200000493c4
INFO: lockdep is turned off.
Call zfcp_fsf_fcp_cmnd without the host_lock and disable the
interrupts when acquiring the req_q_lock. According to the patch
description in "[PATCH] Eliminate error handler overload of the SCSI
serial number", the serial_number is not used, so simply drop the
queuecommand wrapper function and run zfcp_scsi_queuecommand without
holding the host_lock.
Reviewed-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Move the mid-layer's ->queuecommand() invocation from being locked
with the host lock to being unlocked to facilitate speeding up the
critical path for drivers who don't need this lock taken anyway.
The patch below presents a simple SCSI host lock push-down as an
equivalent transformation. No locking or other behavior should change
with this patch. All existing bugs and locking orders are preserved.
Additionally, add one parameter to queuecommand,
struct Scsi_Host *
and remove one parameter from queuecommand,
void (*done)(struct scsi_cmnd *)
Scsi_Host* is a convenient pointer that most host drivers need anyway,
and 'done' is redundant to struct scsi_cmnd->scsi_done.
Minimal code disturbance was attempted with this change. Most drivers
needed only two one-line modifications for their host lock push-down.
Signed-off-by: Jeff Garzik <jgarzik@redhat.com>
Acked-by: James Bottomley <James.Bottomley@suse.de>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Replace the zfcp_modify_<xxx>_status functions and its accompanying wrappers
with dedicated status modifier functions. This eases code readability and
maintenance.
Signed-off-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Enable the LUN scanning mechanism in the SCSI midlayer:
- Do not set the disable_target_scan bit in the FC transport class.
- Set max_lun to 0xFFFFFFFF to allow the midlayer scan to include
the two-level hierachical LUNs (like 0x40XX40XX00000000, but in SCSI
midlayer LUN format).
- Set max_id to a high value to allow triggering the SCSI device
rescan from sysfs.
When running in NPIV mode, zfcp accepts all LUNs in slave_attach. When
running in non-NPIV mode, the list of zfcp_unit structs determines
which SCSI devices are allowed on the current system.
Reviewed-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
This is the large change to switch from using the data in
zfcp_unit to zfcp_scsi_dev. Keeping everything working requires doing
the switch in one piece. To ensure that no code keeps using the data
in zfcp_unit, this patch also removes the data from zfcp_unit that is
now being replaced with zfcp_scsi_dev.
For zfcp, the scsi_device together with zfcp_scsi_dev exist from the
call of slave_alloc to the call of slave_destroy. The data in
zfcp_scsi_dev is initialized in zfcp_scsi_slave_alloc and the LUN is
opened; the final shutdown for the LUN is run from slave_destroy.
Where the scsi_device or zfcp_scsi_dev is needed, the pointer to the
scsi_device is passed as function argument and inside the function
converted to the pointer to zfcp_scsi_dev; this avoids back and forth
conversion betweeen scsi_device and zfcp_scsi_dev.
While changing the function arguments from zfcp_unit to scsi_device,
the functions names are renamed form "unit" to "lun". This is to have
a seperation between zfcp_scsi_dev/LUN and the zfcp_unit; only code
referring to the remaining configuration information in zfcp_unit
struct uses "unit".
Reviewed-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Move the code for managing zfcp_unit devices to the new file
zfcp_unit.c. This is in preparation for the change that zfcp_unit will
only track the LUNs configured via unit_add, other data will be moved
from zfcp_unit to the new struct zfcp_scsi_dev.
Reviewed-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
When sending DIX integrity segments with an I/O request, the
restriction for the maximum number of segments is still the same for
the zfcp hardware. Report the new sg_prot_tablesize for the SCSI host,
so that the number of integrity segments plus the number of data
segments is not larger than the hardware limit. This results in using
half of the hardware segments for integrity data and the other half
for regular data.
Reviewed-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: Jens Axboe <axboe@carl.home.kernel.dk>
Introduce support for DIF/DIX in zfcp: Report the capabilities for the
Scsi_host, map the protection data when issuing I/O requests and
handle the new error codes. Also add the fsf data_direction field to
the hba trace, it is useful information for debugging in that area.
This is an EXPERIMENTAL feature for now.
Signed-off-by: Felix Beck <felix.beck@de.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
A lot of functions require the amount of SBALs as one of their
parameter which is most times invariable. Therefore remove this
parameter and set the SBAL value explicitly if a non standard value is
required. In addition the warning message "oversized data" is
replaced with a BUG_ON() statement assuring the limits defined and
requested by zfcp.
Signed-off-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Configuring a LUN in zfcp, also creates a SCSI device. For
consistency, it makes sense to remove the SCSI device when the LUN is
deconfigured. Replace the flush_work with the call to
scsi_remove_device: scsi_remove_device also takes the scan_mutex that
synchronizes itself with any long running device discovery.
Reviewed-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
When running in non-NPIV mode, the port_reopen in terminate_rport_io
might succeed even though the remote port is not available. If the
same port connection is held open from another operating system, the
reopen is only a virtual operation and might not hit the SAN. Fix this
by changing the call to forced_reopen that forces a logout/login
operation in the SAN.
Reviewed-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
When the successful return of an adisc is the final step to set the
port online, the registration of SCSI devices might be omitted. SCSI
devices that have been removed before (due to a short dev_loss_tmo
setting) might not be attached again.
The problem is that the registration of SCSI devices is done only
after erp has finished. The correct place would be after the call to
fc_remote_port_add to mimick the scan in the FC transport class.
Change the registration of SCSI devices to be triggered after the
fc_remote_port_add call. For the initial inquiry command to succeed,
the unit must also be open. If the unit reopen is still pending, the
inquiry command to the LUN will be deferred with DID_IMM_RETRY, so
there is no harm from this approach.
Reviewed-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Move the code accessing the qdio sbales and zfcp_qdio_req struct to
the zfcp_qdio files and provide helper functions for accessing the
qdio related parts.
Reviewed-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Instead of dealing with large segments in the scatter-gather lists in
zfcp_qdio.c, report the limits to the upper layers. With these limits
in place, the code for mapping large data blocks to multiple sbales
can be removed.
Reviewed-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
The return code FAST_IO_FAIL from fc_block_scsi_eh indicates that the
pending I/O requests have been terminated as a result of the
fast_io_fail_tmo. Pass this return code back to the scsi eh to stop
the scsi eh in this case.
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.
percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.
http://userweb.kernel.org/~tj/misc/slabh-sweep.py
The script does the followings.
* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.
* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.
* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.
The conversion was done in the following steps.
1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.
2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.
3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.
4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.
5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.
6. percpu.h was updated not to include slab.h.
7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).
* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig
8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.
Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
Move the decision which trace tag and trace level to use for the scsi
result trace to zfcp_dbf.h. zfcp_dbf_scsi_result is already an inline
function, so move the trace code there, simplifying the response
handling in zfcp_fsf.c.
Reviewed-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Kernel code uses dev as short name for the struct device. Rename the
sysfs_device in zfcp_unit and zfcp_port to match this convention.
Reviewed-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
The smatch tool from http://repo.or.cz/w/smatch.git warns about this:
drivers/s390/scsi/zfcp_scsi.c +64 zfcp_scsi_command_fail(5) warn: variable dereferenced before check 'scpnt->device'
drivers/s390/scsi/zfcp_scsi.c +64 zfcp_scsi_command_fail(5) warn: variable dereferenced before check 'scpnt->device->host'
drivers/s390/scsi/zfcp_scsi.c +93 zfcp_scsi_queuecommand(23) warn: variable dereferenced before check 'unit'
Fix the first two warnings by removing the checks for scpnt->device
and -> host: As long as the SCSI command exists, there is also a
scsi_device and a Scsi_Host.
Fix the last warning by removing the BUG_ON checks in
zfcp_scsi_queuecommand, they are leftovers from previous paranoia
about wrong pointers between data structures.
Reviewed-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Move the code for tracking FSF requests to new file to have this code
in one place. The functions for adding and removing requests on the
I/O path are already inline. The alloc and free functions are only
called once, so it does not hurt to inline them and add them to the
same file.
Reviewed-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Introduce a zfcp callback for timeouts triggered from FC BSG. With
zfcp, the underlying hardware cannot abort CT or ELS requests, so
there is nothing to do when the block layer timeout expires. To avoid
interference with the block layer timeout, simply indicate that the
block layer timer should be reset. The timer running in the hardware
for the pending CT or ELS request will return the request when it
expires.
Signed-off-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
The flag ZFCP_STATUS_FSFREQ_TMFUNCNOTSUPP is never set and hence can
be removed. This is a leftover from the time when zfcp had to decide
whether the target supports a "logical unit reset" or not. Nowadays,
the SCSI midlayer calls the eh_device_reset_handler or the
eh_target_reset_handler and zfcp simply maps this to a "logical unit
reset" or a "target reset".
Reviewed-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Enable the display of supported and active fc4s for zfcp in the FC
transport class. zfcp only supports FCP, so simply hard-code this
information. The zfcp hbaapi already has this information hardcoded,
but this would allow to switch from the coding in the zfcp hbaapi to
the common FC transport attributes in the future.
Reviewed-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
In case the SCSI error recovery starts because of a SCSI command
timeout, but then something else triggers the rport to be deleted, the
SCSI error recovery will run to the end and set the SCSI device
offline. To prevent this, call the FC transport function
fc_block_scsi_eh which waits until the rport leaves the BLOCKED state.
This guarantees that communication is possible if the rport is ONLINE,
or the SCSI devices will be removed if the rport state switches to
NOT_PRESENT.
Reviewed-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Remove some redundancies in FC related code and trace:
- drop redundant data from SAN trace (local s_id that only changes
during link down, ls_code that is already part of payload, d_id in
ct response trace that is always the same as in ct request trace)
- use one common fsf struct to hold zfcp data for ct and els requests
- leverage common fsf struct for FC passthrough job data, allocate it
with dd_bsg_data for passthrough requests and unify common code for
ct and els passthrough request
- simplify callback handling in zfcp_fc
Reviewed-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Use common data structures for FCP CMND, FCP RSP and related
definitions and remove zfcp private definitions. Split the FCP CMND
setup and FCP RSP evaluation code in seperate functions. Use inline
functions to not negatively impact the I/O path.
Reviewed-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
If an error occurs that triggers the call to fc_remote_port_delete,
ideally this call would happen before any I/O is passed back to the
SCSI midlayer through scsi_done. The SCSI midlayer will retry the
commands and fc_remote_port_chkready will return the correct status
code. But with the delay between calling scsi_done in softirq context
and the call to fc_remote_port_delete from the workqueue, there is a
window where zfcp returns DID_ERROR. This leads to SCSI error recovery
which then leads to offline SCSI devices since all recovery actions
will fail with the rport now being blocked.
In this window, zfcp has to return DID_IMM_RETRY just as the FC
transport class would do in fc_remote_port_chkready for the blocked
fc_rport. As soon as the fc_rport is BLOCKED, fc_remote_port_chkready
will do the right thing.
Additionally, there are two more cases to catch in zfcp_scsi_queuecommand:
- After the port has been opened, the unit has to be opened. During
this period I/O has to be retried. This can also be handled with
DID_IMM_RETRY.
- If the access to the unit fails, but the port is good, then
this single unit cannot be accessed and I/O to this unit has to fail
without involving the FC transport class.
Reviewed-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Replace the local reference counting by already available mechanisms
offered by kref. Where possible existing device structures were used,
including the same functionality.
Signed-off-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
The global config_lock was used to protect the configuration organized
in independent lists. It is not necessary to have a lock on driver
level for this purpose. This patch replaces the global config_lock
with a set of local list locks.
Signed-off-by: Swen Schillig <swen@vnet.ibm.com>
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Adapt the change_queue_depth callback in zfcp for the new reason
parameter. Simply pass each call back to the SCSI midlayer, there are
no resource adjustments necessary for zfcp.
Signed-off-by: Christof Schmitt <christof.schmitt@de.ibm.com>
Removes check for (depth <= default_depth) in case of
SCSI_QDEPTH_RAMP_UP call back, not needed after added
max_queue_depth per sdev.
Signed-off-by: Vasu Dev <vasu.dev@intel.com>
Signed-off-by: James Bottomley <James.Bottomley@suse.de>
Steffen Maier