OpenCloudOS-Kernel/drivers/soc/qcom/rpmh-internal.h

144 lines
4.9 KiB
C
Raw Normal View History

drivers: qcom: rpmh-rsc: add RPMH controller for QCOM SoCs Add controller driver for QCOM SoCs that have hardware based shared resource management. The hardware IP known as RSC (Resource State Coordinator) houses multiple Direct Resource Voter (DRV) for different execution levels. A DRV is a unique voter on the state of a shared resource. A Trigger Control Set (TCS) is a bunch of slots that can house multiple resource state requests, that when triggered will issue those requests through an internal bus to the Resource Power Manager Hardened (RPMH) blocks. These hardware blocks are capable of adjusting clocks, voltages, etc. The resource state request from a DRV are aggregated along with state requests from other processors in the SoC and the aggregate value is applied on the resource. Some important aspects of the RPMH communication - - Requests are <addr, value> with some header information - Multiple requests (upto 16) may be sent through a TCS, at a time - Requests in a TCS are sent in sequence - Requests may be fire-n-forget or completion (response expected) - Multiple TCS from the same DRV may be triggered simultaneously - Cannot send a request if another request for the same addr is in progress from the same DRV - When all the requests from a TCS are complete, an IRQ is raised - The IRQ handler needs to clear the TCS before it is available for reuse - TCS configuration is specific to a DRV - Platform drivers may use DRV from different RSCs to make requests Resource state requests made when CPUs are active are called 'active' state requests. Requests made when all the CPUs are powered down (idle state) are called 'sleep' state requests. They are matched by a corresponding 'wake' state requests which puts the resources back in to previously requested active state before resuming any CPU. TCSes are dedicated for each type of requests. Active mode TCSes (AMC) are used to send requests immediately to the resource, while control TCS are used to provide specific information to the controller. Sleep and Wake TCS send sleep and wake requests, after and before the system halt respectively. Signed-off-by: Lina Iyer <ilina@codeaurora.org> Signed-off-by: Raju P.L.S.S.S.N <rplsssn@codeaurora.org> Signed-off-by: Andy Gross <andy.gross@linaro.org>
2018-06-20 21:26:58 +08:00
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Copyright (c) 2016-2018, The Linux Foundation. All rights reserved.
*/
#ifndef __RPM_INTERNAL_H__
#define __RPM_INTERNAL_H__
#include <linux/bitmap.h>
soc: qcom: rpmh-rsc: Sleep waiting for tcs slots to be free The busy loop in rpmh_rsc_send_data() is written with the assumption that the udelay will be preempted by the tcs_tx_done() irq handler when the TCS slots are all full. This doesn't hold true when the calling thread is an irqthread and the tcs_tx_done() irq is also an irqthread. That's because kernel irqthreads are SCHED_FIFO and thus need to voluntarily give up priority by calling into the scheduler so that other threads can run. I see RCU stalls when I boot with irqthreads on the kernel commandline because the modem remoteproc driver is trying to send an rpmh async message from an irqthread that needs to give up the CPU for the rpmh irqthread to run and clear out tcs slots. rcu: INFO: rcu_preempt self-detected stall on CPU rcu: 0-....: (1 GPs behind) idle=402/1/0x4000000000000002 softirq=2108/2109 fqs=4920 (t=21016 jiffies g=2933 q=590) Task dump for CPU 0: irq/11-smp2p R running task 0 148 2 0x00000028 Call trace: dump_backtrace+0x0/0x154 show_stack+0x20/0x2c sched_show_task+0xfc/0x108 dump_cpu_task+0x44/0x50 rcu_dump_cpu_stacks+0xa4/0xf8 rcu_sched_clock_irq+0x7dc/0xaa8 update_process_times+0x30/0x54 tick_sched_handle+0x50/0x64 tick_sched_timer+0x4c/0x8c __hrtimer_run_queues+0x21c/0x36c hrtimer_interrupt+0xf0/0x22c arch_timer_handler_phys+0x40/0x50 handle_percpu_devid_irq+0x114/0x25c __handle_domain_irq+0x84/0xc4 gic_handle_irq+0xd0/0x178 el1_irq+0xbc/0x180 save_return_addr+0x18/0x28 return_address+0x54/0x88 preempt_count_sub+0x40/0x88 _raw_spin_unlock_irqrestore+0x4c/0x6c ___ratelimit+0xd0/0x128 rpmh_rsc_send_data+0x24c/0x378 __rpmh_write+0x1b0/0x208 rpmh_write_async+0x90/0xbc rpmhpd_send_corner+0x60/0x8c rpmhpd_aggregate_corner+0x8c/0x124 rpmhpd_set_performance_state+0x8c/0xbc _genpd_set_performance_state+0xdc/0x1b8 dev_pm_genpd_set_performance_state+0xb8/0xf8 q6v5_pds_disable+0x34/0x60 [qcom_q6v5_mss] qcom_msa_handover+0x38/0x44 [qcom_q6v5_mss] q6v5_handover_interrupt+0x24/0x3c [qcom_q6v5] handle_nested_irq+0xd0/0x138 qcom_smp2p_intr+0x188/0x200 irq_thread_fn+0x2c/0x70 irq_thread+0xfc/0x14c kthread+0x11c/0x12c ret_from_fork+0x10/0x18 This busy loop naturally lends itself to using a wait queue so that each thread that tries to send a message will sleep waiting on the waitqueue and only be woken up when a free slot is available. This should make things more predictable too because the scheduler will be able to sleep tasks that are waiting on a free tcs instead of the busy loop we currently have today. Reviewed-by: Maulik Shah <mkshah@codeaurora.org> Reviewed-by: Douglas Anderson <dianders@chromium.org> Tested-by: Stanimir Varbanov <stanimir.varbanov@linaro.org> Cc: Douglas Anderson <dianders@chromium.org> Cc: Maulik Shah <mkshah@codeaurora.org> Cc: Lina Iyer <ilina@codeaurora.org> Signed-off-by: Stephen Boyd <swboyd@chromium.org> Link: https://lore.kernel.org/r/20200724211711.810009-1-sboyd@kernel.org Signed-off-by: Bjorn Andersson <bjorn.andersson@linaro.org>
2020-07-25 05:17:11 +08:00
#include <linux/wait.h>
drivers: qcom: rpmh-rsc: add RPMH controller for QCOM SoCs Add controller driver for QCOM SoCs that have hardware based shared resource management. The hardware IP known as RSC (Resource State Coordinator) houses multiple Direct Resource Voter (DRV) for different execution levels. A DRV is a unique voter on the state of a shared resource. A Trigger Control Set (TCS) is a bunch of slots that can house multiple resource state requests, that when triggered will issue those requests through an internal bus to the Resource Power Manager Hardened (RPMH) blocks. These hardware blocks are capable of adjusting clocks, voltages, etc. The resource state request from a DRV are aggregated along with state requests from other processors in the SoC and the aggregate value is applied on the resource. Some important aspects of the RPMH communication - - Requests are <addr, value> with some header information - Multiple requests (upto 16) may be sent through a TCS, at a time - Requests in a TCS are sent in sequence - Requests may be fire-n-forget or completion (response expected) - Multiple TCS from the same DRV may be triggered simultaneously - Cannot send a request if another request for the same addr is in progress from the same DRV - When all the requests from a TCS are complete, an IRQ is raised - The IRQ handler needs to clear the TCS before it is available for reuse - TCS configuration is specific to a DRV - Platform drivers may use DRV from different RSCs to make requests Resource state requests made when CPUs are active are called 'active' state requests. Requests made when all the CPUs are powered down (idle state) are called 'sleep' state requests. They are matched by a corresponding 'wake' state requests which puts the resources back in to previously requested active state before resuming any CPU. TCSes are dedicated for each type of requests. Active mode TCSes (AMC) are used to send requests immediately to the resource, while control TCS are used to provide specific information to the controller. Sleep and Wake TCS send sleep and wake requests, after and before the system halt respectively. Signed-off-by: Lina Iyer <ilina@codeaurora.org> Signed-off-by: Raju P.L.S.S.S.N <rplsssn@codeaurora.org> Signed-off-by: Andy Gross <andy.gross@linaro.org>
2018-06-20 21:26:58 +08:00
#include <soc/qcom/tcs.h>
#define TCS_TYPE_NR 4
#define MAX_CMDS_PER_TCS 16
#define MAX_TCS_PER_TYPE 3
#define MAX_TCS_NR (MAX_TCS_PER_TYPE * TCS_TYPE_NR)
#define MAX_TCS_SLOTS (MAX_CMDS_PER_TCS * MAX_TCS_PER_TYPE)
drivers: qcom: rpmh-rsc: add RPMH controller for QCOM SoCs Add controller driver for QCOM SoCs that have hardware based shared resource management. The hardware IP known as RSC (Resource State Coordinator) houses multiple Direct Resource Voter (DRV) for different execution levels. A DRV is a unique voter on the state of a shared resource. A Trigger Control Set (TCS) is a bunch of slots that can house multiple resource state requests, that when triggered will issue those requests through an internal bus to the Resource Power Manager Hardened (RPMH) blocks. These hardware blocks are capable of adjusting clocks, voltages, etc. The resource state request from a DRV are aggregated along with state requests from other processors in the SoC and the aggregate value is applied on the resource. Some important aspects of the RPMH communication - - Requests are <addr, value> with some header information - Multiple requests (upto 16) may be sent through a TCS, at a time - Requests in a TCS are sent in sequence - Requests may be fire-n-forget or completion (response expected) - Multiple TCS from the same DRV may be triggered simultaneously - Cannot send a request if another request for the same addr is in progress from the same DRV - When all the requests from a TCS are complete, an IRQ is raised - The IRQ handler needs to clear the TCS before it is available for reuse - TCS configuration is specific to a DRV - Platform drivers may use DRV from different RSCs to make requests Resource state requests made when CPUs are active are called 'active' state requests. Requests made when all the CPUs are powered down (idle state) are called 'sleep' state requests. They are matched by a corresponding 'wake' state requests which puts the resources back in to previously requested active state before resuming any CPU. TCSes are dedicated for each type of requests. Active mode TCSes (AMC) are used to send requests immediately to the resource, while control TCS are used to provide specific information to the controller. Sleep and Wake TCS send sleep and wake requests, after and before the system halt respectively. Signed-off-by: Lina Iyer <ilina@codeaurora.org> Signed-off-by: Raju P.L.S.S.S.N <rplsssn@codeaurora.org> Signed-off-by: Andy Gross <andy.gross@linaro.org>
2018-06-20 21:26:58 +08:00
struct rsc_drv;
/**
* struct tcs_group: group of Trigger Command Sets (TCS) to send state requests
* to the controller
*
* @drv: The controller.
* @type: Type of the TCS in this group - active, sleep, wake.
* @mask: Mask of the TCSes relative to all the TCSes in the RSC.
* @offset: Start of the TCS group relative to the TCSes in the RSC.
* @num_tcs: Number of TCSes in this type.
* @ncpt: Number of commands in each TCS.
* @req: Requests that are sent from the TCS; only used for ACTIVE_ONLY
* transfers (could be on a wake/sleep TCS if we are borrowing for
* an ACTIVE_ONLY transfer).
* Start: grab drv->lock, set req, set tcs_in_use, drop drv->lock,
* trigger
* End: get irq, access req,
* grab drv->lock, clear tcs_in_use, drop drv->lock
* @slots: Indicates which of @cmd_addr are occupied; only used for
* SLEEP / WAKE TCSs. Things are tightly packed in the
* case that (ncpt < MAX_CMDS_PER_TCS). That is if ncpt = 2 and
* MAX_CMDS_PER_TCS = 16 then bit[2] = the first bit in 2nd TCS.
drivers: qcom: rpmh-rsc: add RPMH controller for QCOM SoCs Add controller driver for QCOM SoCs that have hardware based shared resource management. The hardware IP known as RSC (Resource State Coordinator) houses multiple Direct Resource Voter (DRV) for different execution levels. A DRV is a unique voter on the state of a shared resource. A Trigger Control Set (TCS) is a bunch of slots that can house multiple resource state requests, that when triggered will issue those requests through an internal bus to the Resource Power Manager Hardened (RPMH) blocks. These hardware blocks are capable of adjusting clocks, voltages, etc. The resource state request from a DRV are aggregated along with state requests from other processors in the SoC and the aggregate value is applied on the resource. Some important aspects of the RPMH communication - - Requests are <addr, value> with some header information - Multiple requests (upto 16) may be sent through a TCS, at a time - Requests in a TCS are sent in sequence - Requests may be fire-n-forget or completion (response expected) - Multiple TCS from the same DRV may be triggered simultaneously - Cannot send a request if another request for the same addr is in progress from the same DRV - When all the requests from a TCS are complete, an IRQ is raised - The IRQ handler needs to clear the TCS before it is available for reuse - TCS configuration is specific to a DRV - Platform drivers may use DRV from different RSCs to make requests Resource state requests made when CPUs are active are called 'active' state requests. Requests made when all the CPUs are powered down (idle state) are called 'sleep' state requests. They are matched by a corresponding 'wake' state requests which puts the resources back in to previously requested active state before resuming any CPU. TCSes are dedicated for each type of requests. Active mode TCSes (AMC) are used to send requests immediately to the resource, while control TCS are used to provide specific information to the controller. Sleep and Wake TCS send sleep and wake requests, after and before the system halt respectively. Signed-off-by: Lina Iyer <ilina@codeaurora.org> Signed-off-by: Raju P.L.S.S.S.N <rplsssn@codeaurora.org> Signed-off-by: Andy Gross <andy.gross@linaro.org>
2018-06-20 21:26:58 +08:00
*/
struct tcs_group {
struct rsc_drv *drv;
int type;
u32 mask;
u32 offset;
int num_tcs;
int ncpt;
const struct tcs_request *req[MAX_TCS_PER_TYPE];
DECLARE_BITMAP(slots, MAX_TCS_SLOTS);
drivers: qcom: rpmh-rsc: add RPMH controller for QCOM SoCs Add controller driver for QCOM SoCs that have hardware based shared resource management. The hardware IP known as RSC (Resource State Coordinator) houses multiple Direct Resource Voter (DRV) for different execution levels. A DRV is a unique voter on the state of a shared resource. A Trigger Control Set (TCS) is a bunch of slots that can house multiple resource state requests, that when triggered will issue those requests through an internal bus to the Resource Power Manager Hardened (RPMH) blocks. These hardware blocks are capable of adjusting clocks, voltages, etc. The resource state request from a DRV are aggregated along with state requests from other processors in the SoC and the aggregate value is applied on the resource. Some important aspects of the RPMH communication - - Requests are <addr, value> with some header information - Multiple requests (upto 16) may be sent through a TCS, at a time - Requests in a TCS are sent in sequence - Requests may be fire-n-forget or completion (response expected) - Multiple TCS from the same DRV may be triggered simultaneously - Cannot send a request if another request for the same addr is in progress from the same DRV - When all the requests from a TCS are complete, an IRQ is raised - The IRQ handler needs to clear the TCS before it is available for reuse - TCS configuration is specific to a DRV - Platform drivers may use DRV from different RSCs to make requests Resource state requests made when CPUs are active are called 'active' state requests. Requests made when all the CPUs are powered down (idle state) are called 'sleep' state requests. They are matched by a corresponding 'wake' state requests which puts the resources back in to previously requested active state before resuming any CPU. TCSes are dedicated for each type of requests. Active mode TCSes (AMC) are used to send requests immediately to the resource, while control TCS are used to provide specific information to the controller. Sleep and Wake TCS send sleep and wake requests, after and before the system halt respectively. Signed-off-by: Lina Iyer <ilina@codeaurora.org> Signed-off-by: Raju P.L.S.S.S.N <rplsssn@codeaurora.org> Signed-off-by: Andy Gross <andy.gross@linaro.org>
2018-06-20 21:26:58 +08:00
};
/**
* struct rpmh_request: the message to be sent to rpmh-rsc
*
* @msg: the request
* @cmd: the payload that will be part of the @msg
* @completion: triggered when request is done
* @dev: the device making the request
* @err: err return from the controller
* @needs_free: check to free dynamically allocated request object
*/
struct rpmh_request {
struct tcs_request msg;
struct tcs_cmd cmd[MAX_RPMH_PAYLOAD];
struct completion *completion;
const struct device *dev;
int err;
bool needs_free;
};
/**
* struct rpmh_ctrlr: our representation of the controller
*
* @cache: the list of cached requests
* @cache_lock: synchronize access to the cache data
* @dirty: was the cache updated since flush
* @batch_cache: Cache sleep and wake requests sent as batch
*/
struct rpmh_ctrlr {
struct list_head cache;
spinlock_t cache_lock;
bool dirty;
struct list_head batch_cache;
};
drivers: qcom: rpmh-rsc: add RPMH controller for QCOM SoCs Add controller driver for QCOM SoCs that have hardware based shared resource management. The hardware IP known as RSC (Resource State Coordinator) houses multiple Direct Resource Voter (DRV) for different execution levels. A DRV is a unique voter on the state of a shared resource. A Trigger Control Set (TCS) is a bunch of slots that can house multiple resource state requests, that when triggered will issue those requests through an internal bus to the Resource Power Manager Hardened (RPMH) blocks. These hardware blocks are capable of adjusting clocks, voltages, etc. The resource state request from a DRV are aggregated along with state requests from other processors in the SoC and the aggregate value is applied on the resource. Some important aspects of the RPMH communication - - Requests are <addr, value> with some header information - Multiple requests (upto 16) may be sent through a TCS, at a time - Requests in a TCS are sent in sequence - Requests may be fire-n-forget or completion (response expected) - Multiple TCS from the same DRV may be triggered simultaneously - Cannot send a request if another request for the same addr is in progress from the same DRV - When all the requests from a TCS are complete, an IRQ is raised - The IRQ handler needs to clear the TCS before it is available for reuse - TCS configuration is specific to a DRV - Platform drivers may use DRV from different RSCs to make requests Resource state requests made when CPUs are active are called 'active' state requests. Requests made when all the CPUs are powered down (idle state) are called 'sleep' state requests. They are matched by a corresponding 'wake' state requests which puts the resources back in to previously requested active state before resuming any CPU. TCSes are dedicated for each type of requests. Active mode TCSes (AMC) are used to send requests immediately to the resource, while control TCS are used to provide specific information to the controller. Sleep and Wake TCS send sleep and wake requests, after and before the system halt respectively. Signed-off-by: Lina Iyer <ilina@codeaurora.org> Signed-off-by: Raju P.L.S.S.S.N <rplsssn@codeaurora.org> Signed-off-by: Andy Gross <andy.gross@linaro.org>
2018-06-20 21:26:58 +08:00
/**
* struct rsc_drv: the Direct Resource Voter (DRV) of the
* Resource State Coordinator controller (RSC)
*
* @name: Controller identifier.
* @base: Start address of the DRV registers in this controller.
* @tcs_base: Start address of the TCS registers in this controller.
* @id: Instance id in the controller (Direct Resource Voter).
* @num_tcs: Number of TCSes in this DRV.
* @rsc_pm: CPU PM notifier for controller.
* Used when solver mode is not present.
* @cpus_in_pm: Number of CPUs not in idle power collapse.
* Used when solver mode and "power-domains" is not present.
* @genpd_nb: PM Domain notifier for cluster genpd notifications.
* @tcs: TCS groups.
* @tcs_in_use: S/W state of the TCS; only set for ACTIVE_ONLY
* transfers, but might show a sleep/wake TCS in use if
* it was borrowed for an active_only transfer. You
soc: qcom: rpmh-rsc: Simplify locking by eliminating the per-TCS lock The rpmh-rsc code had both a driver-level lock (sometimes referred to in comments as drv->lock) and a lock per-TCS. The idea was supposed to be that there would be times where you could get by with just locking a TCS lock and therefor other RPMH users wouldn't be blocked. The above didn't work out so well. Looking at tcs_write() the bigger drv->lock was held for most of the function anyway. Only the __tcs_buffer_write() and __tcs_set_trigger() calls were called without holding the drv->lock. It actually turns out that in tcs_write() we don't need to hold the drv->lock for those function calls anyway even if the per-TCS lock isn't there anymore. From the newly added comments in the code, this is because: - We marked "tcs_in_use" under lock. - Once "tcs_in_use" has been marked nobody else could be writing to these registers until the interrupt goes off. - The interrupt can't go off until we trigger w/ the last line of __tcs_set_trigger(). Thus, from a tcs_write() point of view, the per-TCS lock was useless. Looking at rpmh_rsc_write_ctrl_data(), only the per-TCS lock was held. It turns out, though, that this function already needs to be called with the equivalent of the drv->lock held anyway (we either need to hold drv->lock as we will in a future patch or we need to know no other CPUs could be running as happens today). Specifically rpmh_rsc_write_ctrl_data() might be writing to a TCS that has been borrowed for writing an active transation but it never checks this. Let's eliminate this extra overhead and avoid possible AB BA locking headaches. Suggested-by: Maulik Shah <mkshah@codeaurora.org> Signed-off-by: Douglas Anderson <dianders@chromium.org> Reviewed-by: Stephen Boyd <swboyd@chromium.org> Link: https://lore.kernel.org/r/20200504104917.v6.4.Ib8dccfdb10bf6b1fb1d600ca1c21d9c0db1ef746@changeid Signed-off-by: Bjorn Andersson <bjorn.andersson@linaro.org>
2020-05-05 01:50:18 +08:00
* must hold the lock in this struct (AKA drv->lock) in
* order to update this.
* @lock: Synchronize state of the controller. If RPMH's cache
* lock will also be held, the order is: drv->lock then
* cache_lock.
soc: qcom: rpmh-rsc: Sleep waiting for tcs slots to be free The busy loop in rpmh_rsc_send_data() is written with the assumption that the udelay will be preempted by the tcs_tx_done() irq handler when the TCS slots are all full. This doesn't hold true when the calling thread is an irqthread and the tcs_tx_done() irq is also an irqthread. That's because kernel irqthreads are SCHED_FIFO and thus need to voluntarily give up priority by calling into the scheduler so that other threads can run. I see RCU stalls when I boot with irqthreads on the kernel commandline because the modem remoteproc driver is trying to send an rpmh async message from an irqthread that needs to give up the CPU for the rpmh irqthread to run and clear out tcs slots. rcu: INFO: rcu_preempt self-detected stall on CPU rcu: 0-....: (1 GPs behind) idle=402/1/0x4000000000000002 softirq=2108/2109 fqs=4920 (t=21016 jiffies g=2933 q=590) Task dump for CPU 0: irq/11-smp2p R running task 0 148 2 0x00000028 Call trace: dump_backtrace+0x0/0x154 show_stack+0x20/0x2c sched_show_task+0xfc/0x108 dump_cpu_task+0x44/0x50 rcu_dump_cpu_stacks+0xa4/0xf8 rcu_sched_clock_irq+0x7dc/0xaa8 update_process_times+0x30/0x54 tick_sched_handle+0x50/0x64 tick_sched_timer+0x4c/0x8c __hrtimer_run_queues+0x21c/0x36c hrtimer_interrupt+0xf0/0x22c arch_timer_handler_phys+0x40/0x50 handle_percpu_devid_irq+0x114/0x25c __handle_domain_irq+0x84/0xc4 gic_handle_irq+0xd0/0x178 el1_irq+0xbc/0x180 save_return_addr+0x18/0x28 return_address+0x54/0x88 preempt_count_sub+0x40/0x88 _raw_spin_unlock_irqrestore+0x4c/0x6c ___ratelimit+0xd0/0x128 rpmh_rsc_send_data+0x24c/0x378 __rpmh_write+0x1b0/0x208 rpmh_write_async+0x90/0xbc rpmhpd_send_corner+0x60/0x8c rpmhpd_aggregate_corner+0x8c/0x124 rpmhpd_set_performance_state+0x8c/0xbc _genpd_set_performance_state+0xdc/0x1b8 dev_pm_genpd_set_performance_state+0xb8/0xf8 q6v5_pds_disable+0x34/0x60 [qcom_q6v5_mss] qcom_msa_handover+0x38/0x44 [qcom_q6v5_mss] q6v5_handover_interrupt+0x24/0x3c [qcom_q6v5] handle_nested_irq+0xd0/0x138 qcom_smp2p_intr+0x188/0x200 irq_thread_fn+0x2c/0x70 irq_thread+0xfc/0x14c kthread+0x11c/0x12c ret_from_fork+0x10/0x18 This busy loop naturally lends itself to using a wait queue so that each thread that tries to send a message will sleep waiting on the waitqueue and only be woken up when a free slot is available. This should make things more predictable too because the scheduler will be able to sleep tasks that are waiting on a free tcs instead of the busy loop we currently have today. Reviewed-by: Maulik Shah <mkshah@codeaurora.org> Reviewed-by: Douglas Anderson <dianders@chromium.org> Tested-by: Stanimir Varbanov <stanimir.varbanov@linaro.org> Cc: Douglas Anderson <dianders@chromium.org> Cc: Maulik Shah <mkshah@codeaurora.org> Cc: Lina Iyer <ilina@codeaurora.org> Signed-off-by: Stephen Boyd <swboyd@chromium.org> Link: https://lore.kernel.org/r/20200724211711.810009-1-sboyd@kernel.org Signed-off-by: Bjorn Andersson <bjorn.andersson@linaro.org>
2020-07-25 05:17:11 +08:00
* @tcs_wait: Wait queue used to wait for @tcs_in_use to free up a
* slot
* @client: Handle to the DRV's client.
* @dev: RSC device.
drivers: qcom: rpmh-rsc: add RPMH controller for QCOM SoCs Add controller driver for QCOM SoCs that have hardware based shared resource management. The hardware IP known as RSC (Resource State Coordinator) houses multiple Direct Resource Voter (DRV) for different execution levels. A DRV is a unique voter on the state of a shared resource. A Trigger Control Set (TCS) is a bunch of slots that can house multiple resource state requests, that when triggered will issue those requests through an internal bus to the Resource Power Manager Hardened (RPMH) blocks. These hardware blocks are capable of adjusting clocks, voltages, etc. The resource state request from a DRV are aggregated along with state requests from other processors in the SoC and the aggregate value is applied on the resource. Some important aspects of the RPMH communication - - Requests are <addr, value> with some header information - Multiple requests (upto 16) may be sent through a TCS, at a time - Requests in a TCS are sent in sequence - Requests may be fire-n-forget or completion (response expected) - Multiple TCS from the same DRV may be triggered simultaneously - Cannot send a request if another request for the same addr is in progress from the same DRV - When all the requests from a TCS are complete, an IRQ is raised - The IRQ handler needs to clear the TCS before it is available for reuse - TCS configuration is specific to a DRV - Platform drivers may use DRV from different RSCs to make requests Resource state requests made when CPUs are active are called 'active' state requests. Requests made when all the CPUs are powered down (idle state) are called 'sleep' state requests. They are matched by a corresponding 'wake' state requests which puts the resources back in to previously requested active state before resuming any CPU. TCSes are dedicated for each type of requests. Active mode TCSes (AMC) are used to send requests immediately to the resource, while control TCS are used to provide specific information to the controller. Sleep and Wake TCS send sleep and wake requests, after and before the system halt respectively. Signed-off-by: Lina Iyer <ilina@codeaurora.org> Signed-off-by: Raju P.L.S.S.S.N <rplsssn@codeaurora.org> Signed-off-by: Andy Gross <andy.gross@linaro.org>
2018-06-20 21:26:58 +08:00
*/
struct rsc_drv {
const char *name;
void __iomem *base;
drivers: qcom: rpmh-rsc: add RPMH controller for QCOM SoCs Add controller driver for QCOM SoCs that have hardware based shared resource management. The hardware IP known as RSC (Resource State Coordinator) houses multiple Direct Resource Voter (DRV) for different execution levels. A DRV is a unique voter on the state of a shared resource. A Trigger Control Set (TCS) is a bunch of slots that can house multiple resource state requests, that when triggered will issue those requests through an internal bus to the Resource Power Manager Hardened (RPMH) blocks. These hardware blocks are capable of adjusting clocks, voltages, etc. The resource state request from a DRV are aggregated along with state requests from other processors in the SoC and the aggregate value is applied on the resource. Some important aspects of the RPMH communication - - Requests are <addr, value> with some header information - Multiple requests (upto 16) may be sent through a TCS, at a time - Requests in a TCS are sent in sequence - Requests may be fire-n-forget or completion (response expected) - Multiple TCS from the same DRV may be triggered simultaneously - Cannot send a request if another request for the same addr is in progress from the same DRV - When all the requests from a TCS are complete, an IRQ is raised - The IRQ handler needs to clear the TCS before it is available for reuse - TCS configuration is specific to a DRV - Platform drivers may use DRV from different RSCs to make requests Resource state requests made when CPUs are active are called 'active' state requests. Requests made when all the CPUs are powered down (idle state) are called 'sleep' state requests. They are matched by a corresponding 'wake' state requests which puts the resources back in to previously requested active state before resuming any CPU. TCSes are dedicated for each type of requests. Active mode TCSes (AMC) are used to send requests immediately to the resource, while control TCS are used to provide specific information to the controller. Sleep and Wake TCS send sleep and wake requests, after and before the system halt respectively. Signed-off-by: Lina Iyer <ilina@codeaurora.org> Signed-off-by: Raju P.L.S.S.S.N <rplsssn@codeaurora.org> Signed-off-by: Andy Gross <andy.gross@linaro.org>
2018-06-20 21:26:58 +08:00
void __iomem *tcs_base;
int id;
int num_tcs;
struct notifier_block rsc_pm;
struct notifier_block genpd_nb;
atomic_t cpus_in_pm;
drivers: qcom: rpmh-rsc: add RPMH controller for QCOM SoCs Add controller driver for QCOM SoCs that have hardware based shared resource management. The hardware IP known as RSC (Resource State Coordinator) houses multiple Direct Resource Voter (DRV) for different execution levels. A DRV is a unique voter on the state of a shared resource. A Trigger Control Set (TCS) is a bunch of slots that can house multiple resource state requests, that when triggered will issue those requests through an internal bus to the Resource Power Manager Hardened (RPMH) blocks. These hardware blocks are capable of adjusting clocks, voltages, etc. The resource state request from a DRV are aggregated along with state requests from other processors in the SoC and the aggregate value is applied on the resource. Some important aspects of the RPMH communication - - Requests are <addr, value> with some header information - Multiple requests (upto 16) may be sent through a TCS, at a time - Requests in a TCS are sent in sequence - Requests may be fire-n-forget or completion (response expected) - Multiple TCS from the same DRV may be triggered simultaneously - Cannot send a request if another request for the same addr is in progress from the same DRV - When all the requests from a TCS are complete, an IRQ is raised - The IRQ handler needs to clear the TCS before it is available for reuse - TCS configuration is specific to a DRV - Platform drivers may use DRV from different RSCs to make requests Resource state requests made when CPUs are active are called 'active' state requests. Requests made when all the CPUs are powered down (idle state) are called 'sleep' state requests. They are matched by a corresponding 'wake' state requests which puts the resources back in to previously requested active state before resuming any CPU. TCSes are dedicated for each type of requests. Active mode TCSes (AMC) are used to send requests immediately to the resource, while control TCS are used to provide specific information to the controller. Sleep and Wake TCS send sleep and wake requests, after and before the system halt respectively. Signed-off-by: Lina Iyer <ilina@codeaurora.org> Signed-off-by: Raju P.L.S.S.S.N <rplsssn@codeaurora.org> Signed-off-by: Andy Gross <andy.gross@linaro.org>
2018-06-20 21:26:58 +08:00
struct tcs_group tcs[TCS_TYPE_NR];
DECLARE_BITMAP(tcs_in_use, MAX_TCS_NR);
spinlock_t lock;
soc: qcom: rpmh-rsc: Sleep waiting for tcs slots to be free The busy loop in rpmh_rsc_send_data() is written with the assumption that the udelay will be preempted by the tcs_tx_done() irq handler when the TCS slots are all full. This doesn't hold true when the calling thread is an irqthread and the tcs_tx_done() irq is also an irqthread. That's because kernel irqthreads are SCHED_FIFO and thus need to voluntarily give up priority by calling into the scheduler so that other threads can run. I see RCU stalls when I boot with irqthreads on the kernel commandline because the modem remoteproc driver is trying to send an rpmh async message from an irqthread that needs to give up the CPU for the rpmh irqthread to run and clear out tcs slots. rcu: INFO: rcu_preempt self-detected stall on CPU rcu: 0-....: (1 GPs behind) idle=402/1/0x4000000000000002 softirq=2108/2109 fqs=4920 (t=21016 jiffies g=2933 q=590) Task dump for CPU 0: irq/11-smp2p R running task 0 148 2 0x00000028 Call trace: dump_backtrace+0x0/0x154 show_stack+0x20/0x2c sched_show_task+0xfc/0x108 dump_cpu_task+0x44/0x50 rcu_dump_cpu_stacks+0xa4/0xf8 rcu_sched_clock_irq+0x7dc/0xaa8 update_process_times+0x30/0x54 tick_sched_handle+0x50/0x64 tick_sched_timer+0x4c/0x8c __hrtimer_run_queues+0x21c/0x36c hrtimer_interrupt+0xf0/0x22c arch_timer_handler_phys+0x40/0x50 handle_percpu_devid_irq+0x114/0x25c __handle_domain_irq+0x84/0xc4 gic_handle_irq+0xd0/0x178 el1_irq+0xbc/0x180 save_return_addr+0x18/0x28 return_address+0x54/0x88 preempt_count_sub+0x40/0x88 _raw_spin_unlock_irqrestore+0x4c/0x6c ___ratelimit+0xd0/0x128 rpmh_rsc_send_data+0x24c/0x378 __rpmh_write+0x1b0/0x208 rpmh_write_async+0x90/0xbc rpmhpd_send_corner+0x60/0x8c rpmhpd_aggregate_corner+0x8c/0x124 rpmhpd_set_performance_state+0x8c/0xbc _genpd_set_performance_state+0xdc/0x1b8 dev_pm_genpd_set_performance_state+0xb8/0xf8 q6v5_pds_disable+0x34/0x60 [qcom_q6v5_mss] qcom_msa_handover+0x38/0x44 [qcom_q6v5_mss] q6v5_handover_interrupt+0x24/0x3c [qcom_q6v5] handle_nested_irq+0xd0/0x138 qcom_smp2p_intr+0x188/0x200 irq_thread_fn+0x2c/0x70 irq_thread+0xfc/0x14c kthread+0x11c/0x12c ret_from_fork+0x10/0x18 This busy loop naturally lends itself to using a wait queue so that each thread that tries to send a message will sleep waiting on the waitqueue and only be woken up when a free slot is available. This should make things more predictable too because the scheduler will be able to sleep tasks that are waiting on a free tcs instead of the busy loop we currently have today. Reviewed-by: Maulik Shah <mkshah@codeaurora.org> Reviewed-by: Douglas Anderson <dianders@chromium.org> Tested-by: Stanimir Varbanov <stanimir.varbanov@linaro.org> Cc: Douglas Anderson <dianders@chromium.org> Cc: Maulik Shah <mkshah@codeaurora.org> Cc: Lina Iyer <ilina@codeaurora.org> Signed-off-by: Stephen Boyd <swboyd@chromium.org> Link: https://lore.kernel.org/r/20200724211711.810009-1-sboyd@kernel.org Signed-off-by: Bjorn Andersson <bjorn.andersson@linaro.org>
2020-07-25 05:17:11 +08:00
wait_queue_head_t tcs_wait;
struct rpmh_ctrlr client;
struct device *dev;
drivers: qcom: rpmh-rsc: add RPMH controller for QCOM SoCs Add controller driver for QCOM SoCs that have hardware based shared resource management. The hardware IP known as RSC (Resource State Coordinator) houses multiple Direct Resource Voter (DRV) for different execution levels. A DRV is a unique voter on the state of a shared resource. A Trigger Control Set (TCS) is a bunch of slots that can house multiple resource state requests, that when triggered will issue those requests through an internal bus to the Resource Power Manager Hardened (RPMH) blocks. These hardware blocks are capable of adjusting clocks, voltages, etc. The resource state request from a DRV are aggregated along with state requests from other processors in the SoC and the aggregate value is applied on the resource. Some important aspects of the RPMH communication - - Requests are <addr, value> with some header information - Multiple requests (upto 16) may be sent through a TCS, at a time - Requests in a TCS are sent in sequence - Requests may be fire-n-forget or completion (response expected) - Multiple TCS from the same DRV may be triggered simultaneously - Cannot send a request if another request for the same addr is in progress from the same DRV - When all the requests from a TCS are complete, an IRQ is raised - The IRQ handler needs to clear the TCS before it is available for reuse - TCS configuration is specific to a DRV - Platform drivers may use DRV from different RSCs to make requests Resource state requests made when CPUs are active are called 'active' state requests. Requests made when all the CPUs are powered down (idle state) are called 'sleep' state requests. They are matched by a corresponding 'wake' state requests which puts the resources back in to previously requested active state before resuming any CPU. TCSes are dedicated for each type of requests. Active mode TCSes (AMC) are used to send requests immediately to the resource, while control TCS are used to provide specific information to the controller. Sleep and Wake TCS send sleep and wake requests, after and before the system halt respectively. Signed-off-by: Lina Iyer <ilina@codeaurora.org> Signed-off-by: Raju P.L.S.S.S.N <rplsssn@codeaurora.org> Signed-off-by: Andy Gross <andy.gross@linaro.org>
2018-06-20 21:26:58 +08:00
};
int rpmh_rsc_send_data(struct rsc_drv *drv, const struct tcs_request *msg);
int rpmh_rsc_write_ctrl_data(struct rsc_drv *drv,
const struct tcs_request *msg);
soc: qcom: rpmh-rsc: Caller handles tcs_invalidate() exclusivity Auditing tcs_invalidate() made me worried. Specifically I saw that it used spin_lock(), not spin_lock_irqsave(). That always worries me unless I can trace for sure that I'm in the interrupt handler or that someone else already disabled interrupts. Looking more at it, there is actually no reason for these locks anyway. Specifically the only reason you'd ever call rpmh_rsc_invalidate() is if you cared that the sleep/wake TCSes were empty. That means that they need to continue to be empty even after rpmh_rsc_invalidate() returns. The only way that can happen is if the caller already has done something to keep all other RPMH users out. It should be noted that even though the caller is only worried about making sleep/wake TCSes empty, they also need to worry about stopping active-only transfers if they need to handle the case where active-only transfers might borrow the wake TCS. At the moment rpmh_rsc_invalidate() is only called in PM code from the last CPU. If that later changes the caller will still need to solve the above problems themselves, so these locks will never be useful. Continuing to audit tcs_invalidate(), I found a bug. The function didn't properly check for a borrowed TCS if we hadn't recently written anything into the TCS. Specifically, if we've never written to the WAKE_TCS (or we've flushed it recently) then tcs->slots is empty. We'll early-out and we'll never call tcs_is_free(). I thought about fixing this bug by either deleting the early check for bitmap_empty() or possibly only doing it if we knew we weren't on a TCS that could be borrowed. However, I think it's better to just delete the checks. As argued above it's up to the caller to make sure that all other users of RPMH are quiet before tcs_invalidate() is called. Since callers need to handle the zero-active-TCS case anyway that means they need to make sure that the active-only transfers are quiet before calling too. The one way tcs_invalidate() gets called today is through rpmh_rsc_cpu_pm_callback() which calls rpmh_rsc_ctrlr_is_busy() to handle this. When we have another path to get to tcs_invalidate() it will also need to come up with something similar and it won't need this extra check either. If we later find some code path that actually needs this check back in (and somehow manages to be race free) we can always add it back in. Signed-off-by: Douglas Anderson <dianders@chromium.org> Reviewed-by: Maulik Shah <mkshah@codeaurora.org> Reviewed-by: Stephen Boyd <swboyd@chromium.org> Tested-by: Maulik Shah <mkshah@codeaurora.org> Link: https://lore.kernel.org/r/20200413100321.v4.9.I07c1f70e0e8f2dc0004bd38970b4e258acdc773e@changeid Signed-off-by: Bjorn Andersson <bjorn.andersson@linaro.org>
2020-04-14 01:04:14 +08:00
void rpmh_rsc_invalidate(struct rsc_drv *drv);
void rpmh_rsc_write_next_wakeup(struct rsc_drv *drv);
drivers: qcom: rpmh-rsc: add RPMH controller for QCOM SoCs Add controller driver for QCOM SoCs that have hardware based shared resource management. The hardware IP known as RSC (Resource State Coordinator) houses multiple Direct Resource Voter (DRV) for different execution levels. A DRV is a unique voter on the state of a shared resource. A Trigger Control Set (TCS) is a bunch of slots that can house multiple resource state requests, that when triggered will issue those requests through an internal bus to the Resource Power Manager Hardened (RPMH) blocks. These hardware blocks are capable of adjusting clocks, voltages, etc. The resource state request from a DRV are aggregated along with state requests from other processors in the SoC and the aggregate value is applied on the resource. Some important aspects of the RPMH communication - - Requests are <addr, value> with some header information - Multiple requests (upto 16) may be sent through a TCS, at a time - Requests in a TCS are sent in sequence - Requests may be fire-n-forget or completion (response expected) - Multiple TCS from the same DRV may be triggered simultaneously - Cannot send a request if another request for the same addr is in progress from the same DRV - When all the requests from a TCS are complete, an IRQ is raised - The IRQ handler needs to clear the TCS before it is available for reuse - TCS configuration is specific to a DRV - Platform drivers may use DRV from different RSCs to make requests Resource state requests made when CPUs are active are called 'active' state requests. Requests made when all the CPUs are powered down (idle state) are called 'sleep' state requests. They are matched by a corresponding 'wake' state requests which puts the resources back in to previously requested active state before resuming any CPU. TCSes are dedicated for each type of requests. Active mode TCSes (AMC) are used to send requests immediately to the resource, while control TCS are used to provide specific information to the controller. Sleep and Wake TCS send sleep and wake requests, after and before the system halt respectively. Signed-off-by: Lina Iyer <ilina@codeaurora.org> Signed-off-by: Raju P.L.S.S.S.N <rplsssn@codeaurora.org> Signed-off-by: Andy Gross <andy.gross@linaro.org>
2018-06-20 21:26:58 +08:00
void rpmh_tx_done(const struct tcs_request *msg, int r);
int rpmh_flush(struct rpmh_ctrlr *ctrlr);
drivers: qcom: rpmh-rsc: add RPMH controller for QCOM SoCs Add controller driver for QCOM SoCs that have hardware based shared resource management. The hardware IP known as RSC (Resource State Coordinator) houses multiple Direct Resource Voter (DRV) for different execution levels. A DRV is a unique voter on the state of a shared resource. A Trigger Control Set (TCS) is a bunch of slots that can house multiple resource state requests, that when triggered will issue those requests through an internal bus to the Resource Power Manager Hardened (RPMH) blocks. These hardware blocks are capable of adjusting clocks, voltages, etc. The resource state request from a DRV are aggregated along with state requests from other processors in the SoC and the aggregate value is applied on the resource. Some important aspects of the RPMH communication - - Requests are <addr, value> with some header information - Multiple requests (upto 16) may be sent through a TCS, at a time - Requests in a TCS are sent in sequence - Requests may be fire-n-forget or completion (response expected) - Multiple TCS from the same DRV may be triggered simultaneously - Cannot send a request if another request for the same addr is in progress from the same DRV - When all the requests from a TCS are complete, an IRQ is raised - The IRQ handler needs to clear the TCS before it is available for reuse - TCS configuration is specific to a DRV - Platform drivers may use DRV from different RSCs to make requests Resource state requests made when CPUs are active are called 'active' state requests. Requests made when all the CPUs are powered down (idle state) are called 'sleep' state requests. They are matched by a corresponding 'wake' state requests which puts the resources back in to previously requested active state before resuming any CPU. TCSes are dedicated for each type of requests. Active mode TCSes (AMC) are used to send requests immediately to the resource, while control TCS are used to provide specific information to the controller. Sleep and Wake TCS send sleep and wake requests, after and before the system halt respectively. Signed-off-by: Lina Iyer <ilina@codeaurora.org> Signed-off-by: Raju P.L.S.S.S.N <rplsssn@codeaurora.org> Signed-off-by: Andy Gross <andy.gross@linaro.org>
2018-06-20 21:26:58 +08:00
#endif /* __RPM_INTERNAL_H__ */