diff options
| author | Paolo Valente <paolo.valente@linaro.org> | 2021-03-04 18:46:22 +0100 |
|---|---|---|
| committer | Jens Axboe <axboe@kernel.dk> | 2021-03-25 10:50:07 -0600 |
| commit | 2ec5a5c48373d4bc2f0699f86507a65bf0b9df35 (patch) | |
| tree | 5554b5da61f590cfd52111ce1f85c6052cabd65b /block/bfq-wf2q.c | |
| parent | 9cf1adc6d34f8bb12333afe189a2999131877ea3 (diff) | |
block, bfq: always inject I/O of queues blocked by wakers
Suppose that I/O dispatch is plugged, to wait for new I/O for the
in-service bfq-queue, say bfqq. Suppose then that there is a further
bfq_queue woken by bfqq, and that this woken queue has pending I/O. A
woken queue does not steal bandwidth from bfqq, because it remains
soon without I/O if bfqq is not served. So there is virtually no risk
of loss of bandwidth for bfqq if this woken queue has I/O dispatched
while bfqq is waiting for new I/O. In contrast, this extra I/O
injection boosts throughput. This commit performs this extra
injection.
Tested-by: Jan Kara <jack@suse.cz>
Signed-off-by: Paolo Valente <paolo.valente@linaro.org>
Tested-by: Oleksandr Natalenko <oleksandr@natalenko.name>
Link: https://lore.kernel.org/r/20210304174627.161-2-paolo.valente@linaro.org
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Diffstat (limited to 'block/bfq-wf2q.c')
| -rw-r--r-- | block/bfq-wf2q.c | 8 |
1 files changed, 8 insertions, 0 deletions
diff --git a/block/bfq-wf2q.c b/block/bfq-wf2q.c index 070e34a7feb1..7a462df71f68 100644 --- a/block/bfq-wf2q.c +++ b/block/bfq-wf2q.c @@ -1706,4 +1706,12 @@ void bfq_add_bfqq_busy(struct bfq_data *bfqd, struct bfq_queue *bfqq) if (bfqq->wr_coeff > 1) bfqd->wr_busy_queues++; + + /* Move bfqq to the head of the woken list of its waker */ + if (!hlist_unhashed(&bfqq->woken_list_node) && + &bfqq->woken_list_node != bfqq->waker_bfqq->woken_list.first) { + hlist_del_init(&bfqq->woken_list_node); + hlist_add_head(&bfqq->woken_list_node, + &bfqq->waker_bfqq->woken_list); + } } |