From 39ae3edda325e9cf9e978c9788affe88231f3b34 Mon Sep 17 00:00:00 2001 From: Mike Christie Date: Sat, 27 Feb 2021 11:00:06 -0600 Subject: scsi: target: core: Make completion affinity configurable It may not always be best to complete the IO on same CPU as it was submitted on. This commit allows userspace to configure it. This has been useful for vhost-scsi where we have a single thread for submissions and completions. If we force the completion on the submission CPU we may be adding conflicts with what the user has setup in the lower levels with settings like the block layer rq_affinity or the driver's IRQ or softirq (the network's rps_cpus value) settings. We may also want to set it up where the vhost thread runs on CPU N and does its submissions/completions there, and then have LIO do its completion booking on CPU M, but can't configure the lower levels due to issues like using dm-multipath with lots of paths (the path selector can throw commands all over the system because it's only taking into account latency/throughput at its level). The new setting is in: /sys/kernel/config/target/$fabric/$target/param/cmd_completion_affinity Writing: -1 -> Gives the current default behavior of completing on the submission CPU. -2 -> Completes the cmd on the CPU the lower layers sent it to us from. > 0 -> Completes on the CPU userspace has specified. Link: https://lore.kernel.org/r/20210227170006.5077-26-michael.christie@oracle.com Reviewed-by: Himanshu Madhani Signed-off-by: Mike Christie Signed-off-by: Martin K. Petersen --- include/target/target_core_base.h | 9 +++++++++ 1 file changed, 9 insertions(+) (limited to 'include/target') diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index cf445c3a551a..d1f7d2a45354 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -944,11 +944,20 @@ static inline struct se_portal_group *param_to_tpg(struct config_item *item) tpg_param_group); } +enum { + /* Use se_cmd's cpuid for completion */ + SE_COMPL_AFFINITY_CPUID = -1, + /* Complete on current CPU */ + SE_COMPL_AFFINITY_CURR_CPU = -2, +}; + struct se_wwn { struct target_fabric_configfs *wwn_tf; void *priv; struct config_group wwn_group; struct config_group fabric_stat_group; + struct config_group param_group; + int cmd_compl_affinity; }; static inline void atomic_inc_mb(atomic_t *v) -- cgit v1.2.3-58-ga151