PA Layer issues a LINERESET to the PHY at the recovery step in the Power
Mode change operation. If it happens during auto or manual hibern8 enter,
even if hibern8 enter succeeds, UFS power mode shall be set to PWM-G1 mode
and kept in that mode after exit from hibern8, leading to bad performance.
Handle the LINERESET in the eh_work by restoring power mode to HS mode
after all pending reqs and tasks are cleared from doorbell.
Link: https://lore.kernel.org/r/1598321228-21093-3-git-send-email-cang@codeaurora.org
Signed-off-by: Can Guo <cang@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Add the crypto registers and structs defined in v2.1 of the JEDEC UFSHCI
specification in preparation to add support for inline encryption to UFS.
Link: https://lore.kernel.org/r/20200706200414.2027450-2-satyat@google.com
Reviewed-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Stanley Chu <stanley.chu@mediatek.com>
Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com>
Signed-off-by: Satya Tangirala <satyat@google.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Add SPDX GPL-2.0 to UFS driver files that specified the GPL version 2
license, remove the full boilerplate text.
Link: https://lore.kernel.org/r/20200605200520.20831-2-huobean@gmail.com
Reviewed-by: Tomas Winkler <tomas.winkler@intel.com>
Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com>
Signed-off-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Return IRQ_HANDLED only if the irq is really handled, this will help in
catching spurious interrupts that go unhandled.
Link: https://lore.kernel.org/r/1573798172-20534-6-git-send-email-cang@codeaurora.org
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
Signed-off-by: Can Guo <cang@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Currently auto-hibernate is activated if host supports auto-hibern8
capability. However error-handling is not implemented, which makes the
feature somewhat risky.
If either "Hibernate Enter" or "Hibernate Exit" fail during auto-hibernate
flow, the corresponding interrupt "UIC_HIBERNATE_ENTER" or
"UIC_HIBERNATE_EXIT" shall be raised according to UFS specification.
This patch adds auto-hibernate error-handling:
- Monitor "Hibernate Enter" and "Hibernate Exit" interrupts after
auto-hibernate feature is activated.
- If a failure happens, trigger error-handling just like
"manual-hibernate" failure and apply the same recovery flow: schedule
UFS error handler in ufshcd_check_errors(), and then do host reset and
restore in UFS error handler.
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com>
Reviewed-by: Avri Altman <Avri.Altman@wdc.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Remove the pointless task_req_upiu and task_rsp_upiu indirections,
which are __le32 arrays always cast to given structures and just add
the members directly. Also clean up variables names in use in the
callers a bit to make the code more readable.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
UFS host controllers may support an autonomous power management feature
called the Auto-Hibernate Idle Timer. The timer is set to the number of
microseconds of idle time before the UFS host controller will autonomously
put the link into Hibernate state. That will save power at the expense of
increased latency. Any access to the host controller interface registers
will automatically put the link out of Hibernate state. So once configured,
the feature is transparent to the driver.
Expose the Auto-Hibernate Idle Timer value via SysFS to allow users to
choose between power efficiency or lower latency. Set a default value of
150 ms.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Acked-by: Stanislav Nijnikov <stanislav.nijnikov@wdc.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
These would be used in the future in some specific drivers.
Signed-off-by: Kiwoong Kim <kwmad.kim@samsung.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Since we have converted all the user of UFS_BIT() macro with the actual
bit position, let remove unused UFS_BIT()macro.
Signed-off-by: Alim Akhtar <alim.akhtar@samsung.com>
Reviewed-by: Bart Van Assche <bart.vanassche@wdc.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
HOST_ERROR_INDICATOR and DEVICE_ERROR_INDICATOR are not used
anywhere. Also as per JESD223C specification, bit[7:4] are reserved.
Lets remove these #defines.
Signed-off-by: Alim Akhtar <alim.akhtar@samsung.com>
Reviewed-by: Bart Van Assche <bart.vanassche@wdc.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Currently UFS HCI uses UFS_BIT() macro to get various bit position for
the hardware registers status bits. Which makes code longer instead of
shorter. This macro does not improve code readability as well. Lets
re-write these macro definition with the actual bit position.
Suggested-by: Bart Van Assche <Bart.VanAssche@wdc.com>
Signed-off-by: Alim Akhtar <alim.akhtar@samsung.com>
Reviewed-by: Bart Van Assche <bart.vanassche@wdc.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
It could be just cmp 0xe instead of >>1 and cmp 0x7, with readable code.
Signed-off-by: Tomohiro Kusumi <tkusumi@tuxera.com>
Reviewed-by: Subhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Add macros for register bits that can be found in JESD223C (v2.1).
Not all registers are defined in ufshci.h (i.e. some are unused whether
macros are defined or undefined), but all the bits for those registers
that are already defined should appear here.
Signed-off-by: Tomohiro Kusumi <tkusumi@tuxera.com>
Reviewed-by: Subhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Inserts driver dumps for UFS Host Controller registers, Transfer Requests
and Task Management Requests.
The dumps will occur on driver initialization failure, ufshcd_abort() and
on error handling path.
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
The UFS HCI v2.1 includes a few additional registers. This change
updates the HCI register, the UFS version register content and the
Interrupt Status register.
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
The PHY_ADAPTER_ERROR status register indicates PHY lane errors
reported by the M-PHY layer. In some occasions the controller
can recover from such errors. When the error is not recoverable,
a stuck DB error will occur. Since the stuck DB error is spotted
separately, no action other than clearing the register is necessary.
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Add link status to ufshci.
Signed-off-by: Joao Pinto <jpinto@synopsys.com>
Acked-by: Tomas Winkler <tomas.winkler@intel.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Add UFS 2.0 support to the UFS core driver.
Signed-off-by: Joao Pinto <jpinto@synopsys.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Tomas Winkler <tomas.winkler@intel.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Optimal values of local UniPro parameters like PA_Hibern8Time &
PA_TActivate can help reduce the hibern8 exit latency. If both host and
device supports UniPro ver1.6 or later, these parameters will be
automatically tuned during link startup itself. But if either host or
device doesn't support UniPro ver 1.6 or later, we have to manually
tune them. But to keep manual tuning logic simple, we will only do
manual tuning if local unipro version doesn't support ver1.6 or later.
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Some vendor's UFS device sends back to back NACs for the DL data frames
causing the host controller to raise the DFES error status. Sometimes
such UFS devices send back to back NAC without waiting for new
retransmitted DL frame from the host and in such cases it might be
possible the Host UniPro goes into bad state without raising the DFES
error interrupt. If this happens then all the pending commands would
timeout only after respective SW command (which is generally too
large).
This change workarounds such device behaviour like this:
- As soon as SW sees the DL NAC error, it would schedule the error
handler
- Error handler would sleep for 50ms to see if there any fatal errors
raised by UFS controller.
- If there are fatal errors then SW does normal error recovery.
- If there are no fatal errors then SW sends the NOP command to
device to check if link is alive.
- If NOP command times out, SW does normal error recovery
- If NOP command succeed, skip the error handling.
If DL NAC error is seen multiple times with some vendor's UFS devices
then enable this quirk to initiate quick error recovery and also
silence related error logs to reduce spamming of kernel logs.
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Some host controller hardware controllers may not advertise correct
version in UFS HCI VER register. To workaround this, add new quirk
and call the host controller hardware vendor specific callback to
get the correct UFS HCI version register value.
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
Reviewed-by: Akinobu Mita <akinobu.mita@gmail.com>
Signed-off-by: James Bottomley <JBottomley@Odin.com>
LCC (Line Control Command) are being used for communication between
UFS host and UFS device.
New commercial UFS devices don't have the issues with LCC processing
but UFS host controller might still have the issue with LCC processing,
hence, added a routine to disable TX LCC on the device.
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
Reviewed-by: Akinobu Mita <akinobu.mita@gmail.com>
Signed-off-by: James Bottomley <JBottomley@Odin.com>
This patch adds support for UFS device and UniPro link power management
during runtime/system PM.
Main idea is to define multiple UFS low power levels based on UFS device
and UFS link power states. This would allow any specific platform or pci
driver to choose the best suited low power level during runtime and
system suspend based on their power goals.
bkops handlig:
To put the UFS device in sleep state when bkops is disabled, first query
the bkops status from the device and enable bkops on device only if
device needs time to perform the bkops.
START_STOP handling:
Before sending START_STOP_UNIT to the device well-known logical unit
(w-lun) to make sure that the device w-lun unit attention condition is
cleared.
Write protection:
UFS device specification allows LUs to be write protected, either
permanently or power on write protected. If any LU is power on write
protected and if the card is power cycled (by powering off VCCQ and/or
VCC rails), LU's write protect status would be lost. So this means those
LUs can be written now. To ensures that UFS device is power cycled only
if the power on protect is not set for any of the LUs, check if power on
write protect is set and if device is in sleep/power-off state & link in
inactive state (Hibern8 or OFF state).
If none of the Logical Units on UFS device is power on write protected
then all UFS device power rails (VCC, VCCQ & VCCQ2) can be turned off if
UFS device is in power-off state and UFS link is in OFF state. But current
implementation would disable all device power rails even if UFS link is
not in OFF state.
Low power mode:
If UFS link is in OFF state then UFS host controller can be power collapsed
to avoid leakage current from it. Note that if UFS host controller is power
collapsed, full UFS reinitialization will be required on resume to
re-establish the link between host and device.
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: Christoph Hellwig <hch@lst.de>
The data byte count field of PRDT indicates the length of data block
which is a segment of data transfer for SCSI commands. The value of
this field shall have Dword granularity and the the maximum of length
is 256KB.
This adjusts dma pad mask and max segment size to the above-mentioned
PRDT limitations.
Signed-off-by: Akinobu Mita <mita@fixstars.com>
Reviewed-by: Subhash Jadavani <subhashj@codeaurora.org>
Tested-by: Dolev Raviv <draviv@codeaurora.org>
Acked-by: Vinayak Holikatti <vinholikatti@gmail.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Fix many warnings with incorrect endian assumptions
which makes the code unportable to new architectures.
The UFS specification defines the byte order as big-endian
for UPIU structure and little-endian for the host controller
transfer/task management descriptors.
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
Acked-by: Vinayak Holikatti <vinholikatti@gmail.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Setting PA_PWRMode using DME_SET triggers the power mode
change. And then the result will be given by the HCS.UPMCRS.
This operation should be done atomically.
Signed-off-by: Seungwon Jeon <tgih.jun@samsung.com>
Reviewed-by: Subhash Jadavani <subhashj@codeaurora.org>
Tested-by: Yaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: Santosh Y <santoshsy@gmail.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
Implements to support GET and SET operations of the DME.
These operations are used to configure the behavior of
the UNIPRO. Along with basic operation, {Peer/AttrSetType}
can be mixed.
Signed-off-by: Seungwon Jeon <tgih.jun@samsung.com>
Reviewed-by: Subhash Jadavani <subhashj@codeaurora.org>
Tested-by: Yaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: Santosh Y <santoshsy@gmail.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
IACTH(Interrupt aggregation counter threshold) value is allowed
up to 0x1F and current setting value is the maximum.
This value is related with NUTRS(max:0x20) of HCI's capability.
Considering HCI controller doesn't support the maximum, IACTH
setting should be adjusted with possible value.
For that, existing 'ufshcd_config_int_aggr' is split into two part
[reset, configure].
Signed-off-by: Seungwon Jeon <tgih.jun@samsung.com>
Reviewed-by: Subhash Jadavani <subhashj@codeaurora.org>
Tested-by: Yaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: Santosh Y <santoshsy@gmail.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
Allow UFS device to complete its initialization and accept
SCSI commands by setting fDeviceInit flag. The device may take
time for this operation and hence the host should poll until
fDeviceInit flag is toggled to zero. This step is mandated by
UFS device specification for device initialization completion.
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: Santosh Y <santoshsy@gmail.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
It makes interrupt setting more flexible especially
for disabling. And wrong bit mask is fixed for ver 1.0.
[17:16] is added for mask.
Signed-off-by: Seungwon Jeon <tgih.jun@samsung.com>
Tested-by: Maya Erez <merez@codeaurora.org>
Signed-off-by: Santosh Y <santoshsy@gmail.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
This patch segregates the PCI specific code in ufshcd.c to make it
ready for splitting into core ufs driver and PCI glue driver. Also
copyright header modification to remove extra warranty disclaim.
Reviewed-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Namjae Jeon <linkinjeon@gmail.com>
Reviewed-by: Subhash Jadavani <subhashj@codeaurora.org>
Tested-by: Maya Erez <merez@codeaurora.org>
Signed-off-by: Vinayak Holikatti <vinholikatti@gmail.com>
Signed-off-by: Santosh Yaraganavi <santoshsy@gmail.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
This patch adds support for Universal Flash Storage(UFS)
host controllers. The UFS host controller driver
includes host controller initialization method.
The Initialization process involves following steps:
- Initiate UFS Host Controller initialization process by writing
to Host controller enable register
- Configure UFS Host controller registers with host memory space
datastructure offsets.
- Unipro link startup procedure
- Check for connected device
- Configure UFS host controller to process requests
- Enable required interrupts
- Configure interrupt aggregation
[jejb: fix warnings in 32 bit compile]
Signed-off-by: Santosh Yaraganavi <santoshsy@gmail.com>
Signed-off-by: Vinayak Holikatti <vinholikatti@gmail.com>
Reviewed-by: Arnd Bergmann <arnd@linaro.org>
Reviewed-by: Vishak G <vishak.g@samsung.com>
Reviewed-by: Girish K S <girish.shivananjappa@linaro.org>
Reviewed-by: Namjae Jeon <linkinjeon@gmail.com>
Signed-off-by: James Bottomley <JBottomley@Parallels.com>
Can Guo