OPTIM: connection: disable receiving on disabled events when the run queue is too high
In order to save a lot on syscalls, we currently don't disable receiving
on a file descriptor anymore if its handler was already woken up. But if
the run queue is huge and the poller collects a lot of events, this causes
excess wakeups which take CPU time which is not used to flush these tasklets.
This patch simply considers the run queue size to decide whether or not to
stop receiving. Tests show that by stopping receiving when the run queue
reaches ~16 times its configured size, we can still hold maximal performance
in extreme situations like maxpollevents=20k for runqueue_depth=2, and still
totally avoid calling epoll_event under moderate load using default settings
on keep-alive connections.
diff --git a/src/connection.c b/src/connection.c
index fea092b..a6e2b34 100644
--- a/src/connection.c
+++ b/src/connection.c
@@ -127,6 +127,15 @@
if (!conn->subs->events)
conn->subs = NULL;
}
+ else if (tasks_run_queue_cur >= 16*global.tune.runqueue_depth) {
+ /* In order to save syscalls especially with epoll, we
+ * prefer *not* to disable receiving and instead let
+ * the handler do its job. But if the run queue becomes
+ * high, the excess of events may cause extra wakeups
+ * and in this case we'd rather flow-control ourselves.
+ */
+ fd_stop_recv(fd);
+ }
}
leave: