MAJOR: init: automatically set maxconn and/or maxsslconn when possible
If a memory size limit is enforced using "-n" on the command line and
one or both of maxconn / maxsslconn are not set, instead of using the
build-time values, haproxy now computes the number of sessions that can
be allocated depending on a number of parameters among which :
- global.maxconn (if set)
- global.maxsslconn (if set)
- maxzlibmem
- tune.ssl.cachesize
- presence of SSL in at least one frontend (bind lines)
- presence of SSL in at least one backend (server lines)
- tune.bufsize
- tune.cookie_len
The purpose is to ensure that not haproxy will not run out of memory
when maxing out all parameters. If neither maxconn nor maxsslconn are
used, it will consider that 100% of the sessions involve SSL on sides
where it's supported. That means that it will typically optimize maxconn
for SSL offloading or SSL bridging on all connections. This generally
means that the simple act of enabling SSL in a frontend or in a backend
will significantly reduce the global maxconn but in exchange of that, it
will guarantee that it will not fail.
All metrics may be enforced using #defines to accomodate variations in
SSL libraries or various allocation sizes.
diff --git a/doc/configuration.txt b/doc/configuration.txt
index d6a2df5..17f8860 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -775,7 +775,11 @@
the "select" poller cannot reliably use more than 1024 file descriptors on
some platforms. If your platform only supports select and reports "select
FAILED" on startup, you need to reduce maxconn until it works (slightly
- below 500 in general).
+ below 500 in general). If this value is not set, it will default to the value
+ set in DEFAULT_MAXCONN at build time (reported in haproxy -vv) if no memory
+ limit is enforced, or will be computed based on the memory limit, the buffer
+ size, memory allocated to compression, SSL cache size, and use or not of SSL
+ and the associated maxsslconn (which can also be automatic).
maxconnrate <number>
Sets the maximum per-process number of connections per second to <number>.
@@ -832,6 +836,13 @@
(since it unfortunately does not reliably check for such conditions). Note
that the limit applies both to incoming and outgoing connections, so one
connection which is deciphered then ciphered accounts for 2 SSL connections.
+ If this value is not set, but a memory limit is enforced, this value will be
+ automatically computed based on the memory limit, maxconn, the buffer size,
+ memory allocated to compression, SSL cache size, and use of SSL in either
+ frontends, backends or both. If neither maxconn nor maxsslconn are specified
+ when there is a memory limit, haproxy will automatically adjust these values
+ so that 100% of the connections can be made over SSL with no risk, and will
+ consider the sides where it is enabled (frontend, backend, both).
maxsslrate <number>
Sets the maximum per-process number of SSL sessions per second to <number>.
diff --git a/include/common/defaults.h b/include/common/defaults.h
index cd5edeb..0e37dac 100644
--- a/include/common/defaults.h
+++ b/include/common/defaults.h
@@ -256,6 +256,19 @@
#ifndef SSL_HANDSHAKE_MAX_COST
#define SSL_HANDSHAKE_MAX_COST (76*1024) // measured
#endif
+
+/* approximate session size (for maxconn estimate) */
+#ifndef SESSION_MAX_COST
+#define SESSION_MAX_COST (sizeof(struct session) + \
+ 2 * sizeof(struct channel) + \
+ 2 * sizeof(struct connection) + \
+ REQURI_LEN + \
+ 2 * global.tune.cookie_len)
+#endif
+
+/* available memory estimate : count about 3% of overhead in various structures */
+#ifndef MEM_USABLE_RATIO
+#define MEM_USABLE_RATIO 0.97
#endif
/* Number of samples used to compute the times reported in stats. A power of
diff --git a/src/haproxy.c b/src/haproxy.c
index 8a8e5f5..31447bf 100644
--- a/src/haproxy.c
+++ b/src/haproxy.c
@@ -784,8 +784,136 @@
global.pidfile = strdup(cfg_pidfile);
}
- if (global.maxconn == 0)
- global.maxconn = DEFAULT_MAXCONN;
+ /* Now we want to compute the maxconn and possibly maxsslconn values.
+ * It's a bit tricky. If memmax is not set, maxconn defaults to
+ * DEFAULT_MAXCONN and maxsslconn defaults to DEFAULT_MAXSSLCONN.
+ *
+ * If memmax is set, then it depends on which values are set. If
+ * maxsslconn is set, we use memmax to determine how many cleartext
+ * connections may be added, and set maxconn to the sum of the two.
+ * If maxconn is set and not maxsslconn, maxsslconn is computed from
+ * the remaining amount of memory between memmax and the cleartext
+ * connections. If neither are set, then it is considered that all
+ * connections are SSL-capable, and maxconn is computed based on this,
+ * then maxsslconn accordingly. We need to know if SSL is used on the
+ * frontends, backends, or both, because when it's used on both sides,
+ * we need twice the value for maxsslconn, but we only count the
+ * handshake once since it is not performed on the two sides at the
+ * same time (frontend-side is terminated before backend-side begins).
+ * The SSL stack is supposed to have filled ssl_session_cost and
+ * ssl_handshake_cost during its initialization.
+ */
+ if (!global.rlimit_memmax) {
+ if (global.maxconn == 0) {
+ global.maxconn = DEFAULT_MAXCONN;
+ if (global.mode & (MODE_VERBOSE|MODE_DEBUG))
+ fprintf(stderr, "Note: setting global.maxconn to %d.\n", global.maxconn);
+ }
+ }
+#ifdef USE_OPENSSL
+ else if (!global.maxconn && !global.maxsslconn &&
+ (global.ssl_used_frontend || global.ssl_used_backend)) {
+ /* memmax is set, compute everything automatically. Here we want
+ * to ensure that all SSL connections will be served. We take
+ * care of the number of sides where SSL is used, and consider
+ * the worst case : SSL used on both sides and doing a handshake
+ * simultaneously. Note that we can't have more than maxconn
+ * handshakes at a time by definition, so for the worst case of
+ * two SSL conns per connection, we count a single handshake.
+ */
+ int sides = !!global.ssl_used_frontend + !!global.ssl_used_backend;
+ int64_t mem = global.rlimit_memmax * 1048576ULL;
+
+ mem -= global.tune.sslcachesize * 200; // about 200 bytes per SSL cache entry
+ mem -= global.maxzlibmem;
+ mem = mem * MEM_USABLE_RATIO;
+
+ global.maxconn = mem /
+ ((SESSION_MAX_COST + 2 * global.tune.bufsize) + // session + 2 buffers per session
+ sides * global.ssl_session_max_cost + // SSL buffers, one per side
+ global.ssl_handshake_max_cost); // 1 handshake per connection max
+
+ global.maxconn = round_2dig(global.maxconn);
+ global.maxsslconn = sides * global.maxconn;
+ if (global.mode & (MODE_VERBOSE|MODE_DEBUG))
+ fprintf(stderr, "Note: setting global.maxconn to %d and global.maxsslconn to %d.\n",
+ global.maxconn, global.maxsslconn);
+ }
+ else if (!global.maxsslconn &&
+ (global.ssl_used_frontend || global.ssl_used_backend)) {
+ /* memmax and maxconn are known, compute maxsslconn automatically.
+ * maxsslconn being forced, we don't know how many of it will be
+ * on each side if both sides are being used. The worst case is
+ * when all connections use only one SSL instance because
+ * handshakes may be on two sides at the same time.
+ */
+ int sides = !!global.ssl_used_frontend + !!global.ssl_used_backend;
+ int64_t mem = global.rlimit_memmax * 1048576ULL;
+ int64_t sslmem;
+
+ mem -= global.tune.sslcachesize * 200; // about 200 bytes per SSL cache entry
+ mem -= global.maxzlibmem;
+ mem = mem * MEM_USABLE_RATIO;
+
+ sslmem = mem - global.maxconn * (int64_t)(SESSION_MAX_COST + 2 * global.tune.bufsize);
+ global.maxsslconn = sslmem / (global.ssl_session_max_cost + global.ssl_handshake_max_cost);
+ global.maxsslconn = round_2dig(global.maxsslconn);
+
+ if (sslmem <= 0 || global.maxsslconn < sides) {
+ Alert("Cannot compute the automatic maxsslconn because global.maxconn is already too "
+ "high for the global.memmax value (%d MB). The absolute maximum possible value "
+ "without SSL is %d, but %d was found and SSL is in use.\n",
+ global.rlimit_memmax,
+ (int)(mem / (SESSION_MAX_COST + 2 * global.tune.bufsize)),
+ global.maxconn);
+ exit(1);
+ }
+
+ if (global.maxsslconn > sides * global.maxconn)
+ global.maxsslconn = sides * global.maxconn;
+
+ if (global.mode & (MODE_VERBOSE|MODE_DEBUG))
+ fprintf(stderr, "Note: setting global.maxsslconn to %d\n", global.maxsslconn);
+ }
+#endif
+ else if (!global.maxconn) {
+ /* memmax and maxsslconn are known/unused, compute maxconn automatically */
+ int sides = !!global.ssl_used_frontend + !!global.ssl_used_backend;
+ int64_t mem = global.rlimit_memmax * 1048576ULL;
+ int64_t clearmem;
+
+ if (global.ssl_used_frontend || global.ssl_used_backend)
+ mem -= global.tune.sslcachesize * 200; // about 200 bytes per SSL cache entry
+
+ mem -= global.maxzlibmem;
+ mem = mem * MEM_USABLE_RATIO;
+
+ clearmem = mem;
+ if (sides)
+ clearmem -= (global.ssl_session_max_cost + global.ssl_handshake_max_cost) * (int64_t)global.maxsslconn;
+
+ global.maxconn = clearmem / (SESSION_MAX_COST + 2 * global.tune.bufsize);
+ global.maxconn = round_2dig(global.maxconn);
+
+ if (clearmem <= 0 || !global.maxconn) {
+ Alert("Cannot compute the automatic maxconn because global.maxsslconn is already too "
+ "high for the global.memmax value (%d MB). The absolute maximum possible value "
+ "is %d, but %d was found.\n",
+ global.rlimit_memmax,
+ (int)(mem / (global.ssl_session_max_cost + global.ssl_handshake_max_cost)),
+ global.maxsslconn);
+ exit(1);
+ }
+
+ if (global.mode & (MODE_VERBOSE|MODE_DEBUG)) {
+ if (sides && global.maxsslconn > sides * global.maxconn) {
+ fprintf(stderr, "Note: global.maxsslconn is forced to %d which causes global.maxconn "
+ "to be limited to %d. Better reduce global.maxsslconn to get more "
+ "room for extra connections.\n", global.maxsslconn, global.maxconn);
+ }
+ fprintf(stderr, "Note: setting global.maxconn to %d\n", global.maxconn);
+ }
+ }
if (!global.maxpipes) {
/* maxpipes not specified. Count how many frontends and backends