diff options
| author | Joe Damato <jdamato@fastly.com> | 2024-02-13 06:16:44 +0000 |
|---|---|---|
| committer | David S. Miller <davem@davemloft.net> | 2024-02-14 11:01:00 +0000 |
| commit | de57a251082211b68e8c01e0e8210a23c022ac57 (patch) | |
| tree | ec5e07f692cde151592ad63ea6c2b1b85e666fd0 /fs | |
| parent | c6aa2a7778d8e3ba7c6f84c8095f0b89f0617830 (diff) | |
eventpoll: Add per-epoll prefer busy poll option
When using epoll-based busy poll, the prefer_busy_poll option is hardcoded
to false. Users may want to enable prefer_busy_poll to be used in
conjunction with gro_flush_timeout and defer_hard_irqs_count to keep device
IRQs masked.
Other busy poll methods allow enabling or disabling prefer busy poll via
SO_PREFER_BUSY_POLL, but epoll-based busy polling uses a hardcoded value.
Fix this edge case by adding support for a per-epoll context
prefer_busy_poll option. The default is false, as it was hardcoded before
this change.
Signed-off-by: Joe Damato <jdamato@fastly.com>
Acked-by: Stanislav Fomichev <sdf@google.com>
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Diffstat (limited to 'fs')
| -rw-r--r-- | fs/eventpoll.c | 7 |
1 files changed, 5 insertions, 2 deletions
diff --git a/fs/eventpoll.c b/fs/eventpoll.c index ed83ae33dd45..1b8d01af0c2c 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -231,6 +231,7 @@ struct eventpoll { u32 busy_poll_usecs; /* busy poll packet budget */ u16 busy_poll_budget; + bool prefer_busy_poll; #endif #ifdef CONFIG_DEBUG_LOCK_ALLOC @@ -438,13 +439,14 @@ static bool ep_busy_loop(struct eventpoll *ep, int nonblock) { unsigned int napi_id = READ_ONCE(ep->napi_id); u16 budget = READ_ONCE(ep->busy_poll_budget); + bool prefer_busy_poll = READ_ONCE(ep->prefer_busy_poll); if (!budget) budget = BUSY_POLL_BUDGET; if (napi_id >= MIN_NAPI_ID && ep_busy_loop_on(ep)) { - napi_busy_loop(napi_id, nonblock ? NULL : ep_busy_loop_end, ep, false, - budget); + napi_busy_loop(napi_id, nonblock ? NULL : ep_busy_loop_end, + ep, prefer_busy_poll, budget); if (ep_events_available(ep)) return true; /* @@ -2098,6 +2100,7 @@ static int do_epoll_create(int flags) #ifdef CONFIG_NET_RX_BUSY_POLL ep->busy_poll_usecs = 0; ep->busy_poll_budget = 0; + ep->prefer_busy_poll = false; #endif ep->file = file; fd_install(fd, file); |