We have two knobs to control flush for write booster,
fWriteBoosterBufferFlushDuringHibernate and fWriteBoosterBufferFlushEn.
Some vendors use only fWriteBoosterBufferFlushDuringHibernate because this
can reportedly cover most scenarios. Also, there have been some reports
that flush by fWriteBoosterBufferFlushEn could lead to increased power
consumption thanks to unexpected internal operations. Consequently, we need
a way to enable or disable fWriteBoosterEn operations. Add quirk to bypass
manual flush.
Link: https://lore.kernel.org/r/ffdb0eda30515809f0ad9ee936b26917ee9b4593.1598319701.git.kwmad.kim@samsung.com
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Kiwoong Kim <kwmad.kim@samsung.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Error recovery can be invoked from multiple code paths, including hibern8
enter/exit (from ufshcd_link_recovery), ufshcd_eh_host_reset_handler() and
eh_work scheduled from IRQ context. Ultimately, these paths are all trying
to invoke ufshcd_reset_and_restore() in either a synchronous or
asynchronous manner. This causes problems:
- If link recovery happens during ungate work, ufshcd_hold() would be
called recursively. Although commit 53c12d0ef6 ("scsi: ufs: fix error
recovery after the hibern8 exit failure") fixed a deadlock due to
recursive calls of ufshcd_hold() by adding a check of eh_in_progress
into ufshcd_hold, this check allows eh_work to run in parallel while
link recovery is running.
- Similar concurrency can also happen when error recovery is invoked from
ufshcd_eh_host_reset_handler and ufshcd_link_recovery.
- Concurrency can even happen between eh_works. eh_work, currently queued
on system_wq, is allowed to have multiple instances running in parallel,
but we don't have proper protection for that.
If any of above concurrency scenarios happen, error recovery would fail and
lead ufs device and host into bad states. To fix the concurrency problem,
this change queues eh_work on a single threaded workqueue and removes link
recovery calls from the hibern8 enter/exit path. In addition, make use of
eh_work in eh_host_reset_handler instead of calling
ufshcd_reset_and_restore. This unifies the UFS error recovery mechanism.
According to the UFSHCI JEDEC spec, hibern8 enter/exit error occurs when
the link is broken. This essentially applies to any power mode change
operations (since they all use PACP_PWR cmds in UniPro layer). So, if a
power mode change operation (including AH8 enter/exit) fails, mark link
state as UIC_LINK_BROKEN_STATE and schedule the eh_work. In this case,
error handler needs to do a full reset and restore to recover the link back
to active. Before the link state is recovered to active,
ufshcd_uic_pwr_ctrl simply returns -ENOLINK to avoid more errors.
Link: https://lore.kernel.org/r/1596975355-39813-6-git-send-email-cang@codeaurora.org
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Can Guo <cang@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Information about the last interrupt status and timestamp is helpful when
debugging system stability issues (IRQ starvation, for instance). Add this
information to ufshcd_print_host_state() output.
In addition, UFS device information such as model name and firmware version
also comes in handy during debugging. This is printed as well.
Link: https://lore.kernel.org/r/1596975355-39813-5-git-send-email-cang@codeaurora.org
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Reviewed-by: Hongwu Su <hongwus@codeaurora.org>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Reviewed-by: Stanley Chu <stanley.chu@mediatek.com>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Can Guo <cang@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
On Snapdragon SoCs, the Linux kernel isn't permitted to directly access the
standard UFS crypto configuration registers. Instead, programming and
evicting keys must be done through vendor-specific SMC calls.
To support this hardware, add a ->program_key() method to 'struct
ufs_hba_variant_ops'. This allows overriding the UFS standard key
programming / eviction procedure.
Link: https://lore.kernel.org/r/20200710072013.177481-5-ebiggers@kernel.org
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Wire up ufshcd.c with the UFS Crypto API, the block layer inline encryption
additions and the keyslot manager.
Many existing inline crypto devices require some additional behaviour not
specified in the UFSHCI v2.1 specification - as such the vendor specific
drivers will need to be updated where necessary to make it possible to use
those devices. Some of these changes have already been proposed upstream,
such as for the Qualcomm 845 SoC at
https://lkml.kernel.org/linux-scsi/20200501045111.665881-1-ebiggers@kernel.org/
and for ufs-mediatek at
https://lkml.kernel.org/linux-scsi/20200304022101.14165-1-stanley.chu@mediatek.com/
This patch has been tested on the db845c, sm8150-mtp and sm8250-mtp
(which have Qualcomm chipsets) and on some mediatek chipsets using these
aforementioned vendor specific driver updates.
Link: https://lore.kernel.org/r/20200706200414.2027450-4-satyat@google.com
Reviewed-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Stanley Chu <stanley.chu@mediatek.com>
Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com>
Signed-off-by: Satya Tangirala <satyat@google.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Introduce functions to manipulate UFS inline encryption hardware in line
with the JEDEC UFSHCI v2.1 specification and to work with the block keyslot
manager.
The UFS crypto API will assume by default that a vendor driver doesn't
support UFS crypto, even if the hardware advertises the capability, because
a lot of hardware requires some special handling that's not specified in
the aforementioned JEDEC spec. Each vendor driver must explicitly set
hba->caps |= UFSHCD_CAP_CRYPTO before ufshcd_hba_init_crypto_capabilities()
is called to opt-in to UFS crypto support.
Link: https://lore.kernel.org/r/20200706200414.2027450-3-satyat@google.com
Reviewed-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Stanley Chu <stanley.chu@mediatek.com>
Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com>
Signed-off-by: Satya Tangirala <satyat@google.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Add the crypto registers and structs defined in v2.1 of the JEDEC UFSHCI
specification in preparation to add support for inline encryption to UFS.
Link: https://lore.kernel.org/r/20200706200414.2027450-2-satyat@google.com
Reviewed-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Stanley Chu <stanley.chu@mediatek.com>
Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com>
Signed-off-by: Satya Tangirala <satyat@google.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
The UFS load calculation is based on "total_time" and "busy_time" in a
devfreq window. However, the source of time is different for both
parameters: "busy_time" is assigned from "jiffies" thus has different
accuracy from "total_time" which is assigned from ktime_get().
In addition, the time of window boundary is not exactly the same as the
starting busy time in this window if UFS is actually busy in the beginning
of the window. A similar accuracy error may also happen for the end of busy
time in current window.
To guarantee the precision of load calculation, we need to
1. Align time accuracy of both devfreq_dev_status.total_time and
devfreq_dev_status.busy_time. For example, use "ktime_get()" directly.
2. Align the following timelines:
- The beginning time of devfreq windows
- The beginning of busy time in a new window
- The end of busy time in the current window
Link: https://lore.kernel.org/r/20200611101043.6379-1-stanley.chu@mediatek.com
Fixes: a3cd5ec55f ("scsi: ufs: add load based scaling of UFS gear")
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Remove unused fields "cmd_active" and "result" in struct ufs_command.
Link: https://lore.kernel.org/r/20200615072235.23042-2-stanley.chu@mediatek.com
Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Add SPDX GPL-2.0 to UFS driver files that specified the GPL version 2
license, remove the full boilerplate text.
Link: https://lore.kernel.org/r/20200605200520.20831-2-huobean@gmail.com
Reviewed-by: Tomas Winkler <tomas.winkler@intel.com>
Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com>
Signed-off-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
At UFS initialization stage, to get the length of the descriptor,
ufshcd_read_desc_length() was being called 6 times. Instead, we will
capture the descriptor size the first time we'll read it.
Delete unnecessary redundant code, remove ufshcd_read_desc_length(),
ufshcd_init_desc_sizes(), and boost UFS initialization.
Link: https://lore.kernel.org/r/20200603091959.27618-5-huobean@gmail.com
Acked-by: Avri Altman <avri.altman@wdc.com>
Reviewed-by: Bart van Assche <bvanassche@acm.org>
Reviewed-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Some controller like Exynos determines if FATAL ERROR (0x7) in OCS field in
UTRD occurs for values other than GOOD (0x0) in STATUS field in response
upiu as well as errors that a host controller can't cover. This patch is
to prevent from reporting command results in those cases.
Link: https://lore.kernel.org/r/20200528011658.71590-6-alim.akhtar@samsung.com
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Kiwoong Kim <kwmad.kim@samsung.com>
Signed-off-by: Alim Akhtar <alim.akhtar@samsung.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Some UFS host controllers like Exynos uses granularities of PRDT length and
offset as bytes, whereas others use actual segment count.
Link: https://lore.kernel.org/r/20200528011658.71590-5-alim.akhtar@samsung.com
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Kiwoong Kim <kwmad.kim@samsung.com>
Signed-off-by: Alim Akhtar <alim.akhtar@samsung.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Some host controllers don't support host controller enable via HCE.
Link: https://lore.kernel.org/r/20200528011658.71590-4-alim.akhtar@samsung.com
Reviewed-by: Can Guo <cang@codeaurora.org>
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Seungwon Jeon <essuuj@gmail.com>
Signed-off-by: Alim Akhtar <alim.akhtar@samsung.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Some host controllers support interrupt aggregation but don't allow
resetting counter and timer in software.
Link: https://lore.kernel.org/r/20200528011658.71590-3-alim.akhtar@samsung.com
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Seungwon Jeon <essuuj@gmail.com>
Signed-off-by: Alim Akhtar <alim.akhtar@samsung.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
With the correct behavior, setting the bit to '0' indicates clear and '1'
indicates no change. If host controller handles this the other way around,
UFSHCI_QUIRK_BROKEN_REQ_LIST_CLR can be used.
Link: https://lore.kernel.org/r/20200528011658.71590-2-alim.akhtar@samsung.com
Reviewed-by: Can Guo <cang@codeaurora.org>
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Seungwon Jeon <essuuj@gmail.com>
Signed-off-by: Alim Akhtar <alim.akhtar@samsung.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Currently UFS host driver promises VCC supply if UFS device needs to do
WriteBooster flush during runtime suspend.
However the UFS specification mentions:
"While the flushing operation is in progress, the device is in Active power
mode."
Therefore UFS host driver needs to promise more: Keep UFS device as "Active
power mode", otherwise UFS device shall not do any flush if device enters
Sleep or PowerDown power mode. Similarly, the same promises shall be
applied if device needs urgent BKOP during runtime suspend.
Fix this by not changing device power mode if WriteBooster flush or urgent
BKOP is required in ufshcd_suspend().
Now, if device finishes its job but is not resumed for a very long time,
system will have unnecessary power drain because VCC is still supplied. A
method to re-check the threshold of keeping VCC supply is required to fix
the power drain. However, the threshold re-check needs to re-activate the
link first because the decision depends on the latest device status.
Also introduce a delayed work to force runtime resume after a certain delay
during runtime suspend. This makes threshold re-check happen natually in
the entry of the next runtime-suspend. The device can continue its
WriteBooster flush or urgent BKOP jobs soon after resumed if device has no
upcoming requests and link enters hibern8 state either by Auto-Hibern8 or
hibern8 during clk-gating scheme. This solution not only prevents power
drain but also makes as much use of time as possible for device's
background jobs.
Link: https://lore.kernel.org/r/20200522083212.4008-5-stanley.chu@mediatek.com
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
For WriteBooster feature related attributes, the index used by query shall
be LUN ID if LU Dedicated buffer mode is enabled.
Link: https://lore.kernel.org/r/20200522083212.4008-4-stanley.chu@mediatek.com
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
The ufshcd_wait_for_register() function either sleeps or spins until the
specified register has reached the desired value. Busy-waiting is not only
considered a bad practice but also has a bad impact on energy
consumption. Always sleep instead of spinning by making sure that all
ufshcd_wait_for_register() calls happen from a context where it is allowed
to sleep. The only function call that has to be moved is the
ufshcd_hba_stop() call in ufshcd_host_reset_and_restore().
Link: https://lore.kernel.org/r/20200507222750.19113-1-bvanassche@acm.org
Cc: Can Guo <cang@codeaurora.org>
Cc: Avri Altman <avri.altman@wdc.com>
Cc: Bean Huo <beanhuo@micron.com>
Cc: Alim Akhtar <alim.akhtar@samsung.com>
Cc: Asutosh Das <asutoshd@codeaurora.org>
Tested-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Stanley Chu <stanley.chu@mediatek.com>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Allow flush threshold for WriteBooster to be customizable by vendors. To
achieve this, make the value a variable in struct ufs_hba_variant_params.
Also introduce UFS_WB_BUF_REMAIN_PERCENT() macro to provide a more flexible
way to specify WriteBooster available buffer values.
Link: https://lore.kernel.org/r/20200509093716.21010-4-stanley.chu@mediatek.com
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
The UFS driver is growing more and more customizable parameters. Collect
them in one place.
Link: https://lore.kernel.org/r/20200509093716.21010-2-stanley.chu@mediatek.com
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
According to UFS specification, there are two WriteBooster mode of
operations: "LU dedicated buffer" mode and "shared buffer" mode. In the
"LU dedicated buffer" mode, the WriteBooster Buffer is dedicated to a
logical unit.
If the device supports the "LU dedicated buffer" mode, this mode is
configured by setting bWriteBoosterBufferType to 00h. The logical unit
WriteBooster Buffer size is configured by setting the
dLUNumWriteBoosterBufferAllocUnits field of the related Unit
Descriptor. Only a value greater than zero enables the WriteBooster feature
in the logical unit.
Modify ufshcd_wb_probe() as above description to support LU Dedicated
buffer mode.
Note that according to UFS 3.1 specification, the valid value of
bDeviceMaxWriteBoosterLUs parameter in Geometry Descriptor is 1, which
means at most one LUN can have WriteBooster buffer in "LU dedicated buffer
mode". Therefore this patch supports only one LUN with WriteBooster
enabled. All WriteBooster related sysfs nodes are specifically mapped to
the LUN with WriteBooster enabled in LU Dedicated buffer mode.
Link: https://lore.kernel.org/r/20200508080115.24233-7-stanley.chu@mediatek.com
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
For preparation of LU Dedicated buffer mode support on WriteBooster
feature, "index" parameter shall be added and allowed to be specified by
callers.
Link: https://lore.kernel.org/r/20200508080115.24233-6-stanley.chu@mediatek.com
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Reviewed-by: Can Guo <cang@codeaurora.org>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Export ufs_fixup_device_setup() to allow vendors to re-use it for fixing
device quriks on specified UFS hosts.
Link: https://lore.kernel.org/r/20200508080115.24233-4-stanley.chu@mediatek.com
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Some UFS deivces may have required device quirks or have non-standard
features which are enabled only on specified UFS hosts or for special
customers.
To not "pollute" common device quirk list, i.e. ufs_fixups table, for those
devices mentioned above, introduce "fixup_dev_quirks" vops to allow vendors
to fix or modify device quirks accordingly.
Link: https://lore.kernel.org/r/20200508080115.24233-3-stanley.chu@mediatek.com
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
The write performance of TLC NAND is considerably lower than SLC NAND.
Using SLC NAND as a WriteBooster Buffer enables the write request to be
processed with lower latency and improves the overall write performance.
Adds support for shared-buffer mode WriteBooster.
WriteBooster enable: SW enables it when clocks are scaled up, thus it's
enabled only in high load conditions.
WriteBooster disable: SW will disable the feature, when clocks are scaled
down. Thus writes would go as normal writes.
To keep the endurance of the WriteBooster Buffer at a maximum, this
load-based toggling is adopted.
Link: https://lore.kernel.org/r/2871444d9083b0e9323ef6d8ff1b544b7784adc9.1587591527.git.asutoshd@codeaurora.org
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Currently ufshcd driver assumes that bInitPowerMode parameter is not
changed by any vendors thus device power mode can be set as "Active" during
initialization.
According to UFS JEDEC specification, device power mode shall be "Active"
after HW Reset is triggered if the bInitPowerMode parameter in Device
Descriptor is default value.
By above description, we can set device power mode as "Active" after device
reset is triggered by vendor's callback. With this change, the link startup
performance can be improved in some cases by not setting link_startup_again
as true in ufshcd_link_startup().
Link: https://lore.kernel.org/r/20200327095835.10293-1-stanley.chu@mediatek.com
Reviewed-by: Can Guo <cang@codeaurora.org>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Export ufshcd_link_recovery to allow vendors to recover failed link in
vendor's callbacks.
Link: https://lore.kernel.org/r/20200327095329.10083-2-stanley.chu@mediatek.com
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
We were setting bActiveICCLevel attribute for UFS device only once but the
type of this attribute has changed from persistent to volatile since UFS
device specification v2.1. This attribute is set to the default value after
power cycle or hardware reset event. It isn't safe to rely on prefetched
data (only used for bActiveICCLevel attribute now). Hence this change
removes the code related to data prefetching and set this parameter on
every attempt to probe the UFS device.
Tested-by: Stanley Chu <stanley.chu@mediatek.com>
Reviewed-by: Stanley Chu <stanley.chu@mediatek.com>
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Can Guo <cang@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Currently a 1 ms delay is applied before polling CONTROLLER_ENABLE
bit. This delay may not be required or can be changed in different
controllers. Make the delay as a changeable value in struct ufs_hba to
allow it customized by vendors.
Link: https://lore.kernel.org/r/20200318104016.28049-6-stanley.chu@mediatek.com
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Reviewed-by: Can Guo <cang@codeaurora.org>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Introduce a common delay function to provide flexible way for users to take
choices of udelay and usleep_range into consideration according to the
required delay time.
Link: https://lore.kernel.org/r/20200318104016.28049-4-stanley.chu@mediatek.com
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Reviewed-by: Can Guo <cang@codeaurora.org>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Use an enum to specify the host capabilities instead of #defines inside the
structure definition.
Link: https://lore.kernel.org/r/20200318104016.28049-3-stanley.chu@mediatek.com
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Reviewed-by: Can Guo <cang@codeaurora.org>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Use an enum to specify the various quirks instead of #defines inside the
structure definition.
[mkp: fix typo]
Link: https://lore.kernel.org/r/20200221140812.476338-3-hch@lst.de
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Remove various quirks that don't have users, as well as the dead code keyed
off them.
Link: https://lore.kernel.org/r/20200221140812.476338-2-hch@lst.de
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Many vendors would like to disable host TX LCC during initialization
flow. Introduce a common function for all users to make drivers easier to
read and maintained. This patch does not change any functionality.
Link: https://lore.kernel.org/r/20200207070357.17169-3-stanley.chu@mediatek.com
Reviewed-by: Can Guo <cang@codeaurora.org>
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Auto-Hibern8 may be disabled by some vendors or sysfs in runtime even if
Auto-Hibern8 capability is supported by host. If Auto-Hibern8 capability is
supported by host but not actually enabled, Auto-Hibern8 error shall not
happen.
To fix this, provide a way to detect if Auto-Hibern8 is actually enabled
first, and bypass Auto-Hibern8 disabling case in
ufshcd_is_auto_hibern8_error().
Fixes: 8217444039 ("scsi: ufs: Add error-handling of Auto-Hibernate")
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/r/20200129105251.12466-4-stanley.chu@mediatek.com
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Reviewed-by: Can Guo <cang@codeaurora.org>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Without variable is_init_prefetch, the current logic can guarantee
ufshcd_init_icc_levels() will execute only once, delete it now.
Link: https://lore.kernel.org/r/20200120130820.1737-7-huobean@gmail.com
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
In consideration of UFS host driver uses parameters of struct ufs_dev_desc,
move its parameters to struct ufs_dev_info, delete struct ufs_dev_desc.
Link: https://lore.kernel.org/r/20200120130820.1737-3-huobean@gmail.com
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com>
Reviewed-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Export below functions for vendor usage:
int ufshcd_hba_enable(struct ufs_hba *hba);
int ufshcd_make_hba_operational(struct ufs_hba *hba);
int ufshcd_uic_hibern8_exit(struct ufs_hba *hba);
Link: https://lore.kernel.org/r/20200117035108.19699-3-stanley.chu@mediatek.com
Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Pass UFS device information to vendor-specific variant callback
"apply_dev_quirks" because some platform vendors need to know such
information to apply special handling or quirks in specific devices.
At the same time, modify existing vendor implementations according to the
new interface for those vendor drivers which will be built-in or built as a
module alone with UFS core driver.
[mkp: clarified commit desc]
Cc: Alim Akhtar <alim.akhtar@samsung.com>
Cc: Asutosh Das <asutoshd@codeaurora.org>
Cc: Avri Altman <avri.altman@wdc.com>
Cc: Bart Van Assche <bvanassche@acm.org>
Cc: Bean Huo <beanhuo@micron.com>
Cc: Can Guo <cang@codeaurora.org>
Cc: Matthias Brugger <matthias.bgg@gmail.com>
Link: https://lore.kernel.org/r/1578726707-6596-2-git-send-email-stanley.chu@mediatek.com
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Device reset history shall be also added for vendor's device reset variant
operation implementation.
Cc: Alim Akhtar <alim.akhtar@samsung.com>
Cc: Asutosh Das <asutoshd@codeaurora.org>
Cc: Avri Altman <avri.altman@wdc.com>
Cc: Bart Van Assche <bvanassche@acm.org>
Cc: Bean Huo <beanhuo@micron.com>
Cc: Can Guo <cang@codeaurora.org>
Cc: Matthias Brugger <matthias.bgg@gmail.com>
Link: https://lore.kernel.org/r/1578147968-30938-3-git-send-email-stanley.chu@mediatek.com
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Export ufshcd_auto_hibern8_update to allow vendors to use common interface
to customize auto-hibernate timer.
Cc: Alim Akhtar <alim.akhtar@samsung.com>
Cc: Avri Altman <avri.altman@wdc.com>
Cc: Bart Van Assche <bvanassche@acm.org>
Cc: Bean Huo <beanhuo@micron.com>
Cc: Can Guo <cang@codeaurora.org>
Cc: Florian Fainelli <f.fainelli@gmail.com>
Cc: Matthias Brugger <matthias.bgg@gmail.com>
Link: https://lore.kernel.org/r/1577683950-1702-5-git-send-email-stanley.chu@mediatek.com
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
In UFS host reset and restore path, before probe, we stop and start the
host controller once. After host controller is stopped, the pending
requests, if any, are cleared from the doorbell, but no completion IRQ
would be raised due to the hba is stopped. These pending requests shall be
completed along with the first NOP_OUT command (as it is the first command
which can raise a transfer completion IRQ) sent during probe. Since the
OCSs of these pending requests are not SUCCESS (because they are not yet
literally finished), their UPIUs shall be dumped. When there are multiple
pending requests, the UPIU dump can be overwhelming and may lead to
stability issues because it is in atomic context. Therefore, before probe,
complete these pending requests right after host controller is stopped and
silence the UPIU dump from them.
Link: https://lore.kernel.org/r/1574751214-8321-5-git-send-email-cang@qti.qualcomm.com
Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Tested-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Can Guo <cang@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Manage TMF tags with blk_{get,put}_request() instead of
ufshcd_get_tm_free_slot() / ufshcd_put_tm_slot(). Store a per-request
completion pointer in request.end_io_data instead of using a waitqueue to
report TMF completion.
Cc: Can Guo <cang@codeaurora.org>
Cc: Stanley Chu <stanley.chu@mediatek.com>
Cc: Avri Altman <avri.altman@wdc.com>
Cc: Tomas Winkler <tomas.winkler@intel.com>
Link: https://lore.kernel.org/r/20191209181309.196233-3-bvanassche@acm.org
Tested-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Instead of tracking which tags are in use in the ufs_hba.lrb_in_use
bitmask, rely on the block layer tag allocation mechanism. This patch
removes the following busy-waiting loop if ufshcd_issue_devman_upiu_cmd()
and the block layer accidentally allocate the same tag for a SCSI request:
* ufshcd_queuecommand() returns SCSI_MLQUEUE_HOST_BUSY.
* The SCSI core requeues the SCSI command.
Cc: Can Guo <cang@codeaurora.org>
Cc: Stanley Chu <stanley.chu@mediatek.com>
Cc: Avri Altman <avri.altman@wdc.com>
Cc: Tomas Winkler <tomas.winkler@intel.com>
Link: https://lore.kernel.org/r/20191209181309.196233-2-bvanassche@acm.org
Tested-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Fix up possible unclocked register access to auto hibern8 register in
resume path and through sysfs entry. Meanwhile, enable auto hibern8 only
after device is fully initialized in probe path.
Link: https://lore.kernel.org/r/1573798172-20534-4-git-send-email-cang@codeaurora.org
Reviewed-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Can Guo <cang@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Rework from previous work by:
Sujit Reddy Thumma <sthumma@codeaurora.org>
Override auto suspend tunables for UFS device LUNs during initialization so
as to efficiently manage background operations and the power consumption.
Link: https://lore.kernel.org/r/1568649411-5127-3-git-send-email-stanley.chu@mediatek.com
Reviewed-by: Avri Altman <avri.altman@wdc.com>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Stanley Chu <stanley.chu@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Some UFS memory devices needs their reset line toggled in order to get them
into a good state for initialization. Provide a new vops to allow the
platform driver to implement this operation.
Link: https://lore.kernel.org/r/20190828191756.24312-2-bjorn.andersson@linaro.org
Signed-off-by: Bjorn Andersson <bjorn.andersson@linaro.org>
Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Stanley Chu <stanley.chu@mediatek.com>
Acked-by: Avri Altman <Avri.Altman@wdc.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Define new a type: uc_string_id for easier string handling and less
casting. Reduce number or string copies in price of a dynamic allocation.
Signed-off-by: Tomas Winkler <tomas.winkler@intel.com>
Tested-by: Avri Altman <avri.altman@wdc.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Kiwoong Kim