MEDIUM: mux-h1: Add the support of headers adjustment for bogus HTTP/1 apps
There is no standard case for HTTP header names because, as stated in the
RFC7230, they are case-insensitive. So applications must handle them in a
case-insensitive manner. But some bogus applications erroneously rely on the
case used by most browsers. This problem becomes critical with HTTP/2
because all header names must be exchanged in lowercase. And HAProxy uses the
same convention. All header names are sent in lowercase to clients and servers,
regardless of the HTTP version.
This design choice is linked to the HTX implementation. So, for previous
versions (2.0 and 1.9), a workaround is to disable the HTX mode to fall
back to the legacy HTTP mode.
Since the legacy HTTP mode was removed, some users reported interoperability
issues because their application was not able anymore to handle HTTP/1 message
received from HAProxy. So, we've decided to add a way to change the case of some
headers before sending them. It is now possible to define a "mapping" between a
lowercase header name and a version supported by the bogus application. To do
so, you must use the global directives "h1-case-adjust" and
"h1-case-adjust-file". Then options "h1-case-adjust-bogus-client" and
"h1-case-adjust-bogus-server" may be used in proxy sections to enable the
conversion. See the configuration manual for more info.
Of course, our advice is to urgently upgrade these applications for
interoperability concerns and because they may be vulnerable to various types of
content smuggling attacks. But, if your are really forced to use an unmaintained
bogus application, you may use these directive, at your own risks.
If it is relevant, this feature may be backported to 2.0.
(cherry picked from commit 98fbe9531a923499f165599203ef46b78e8f5ad7)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
diff --git a/doc/configuration.txt b/doc/configuration.txt
index e1e9810..c8f308b 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -584,6 +584,8 @@
- gid
- group
- hard-stop-after
+ - h1-case-adjust
+ - h1-case-adjust-file
- log
- log-tag
- log-send-hostname
@@ -838,6 +840,55 @@
global
hard-stop-after 30s
+h1-case-adjust <from> <to>
+ Defines the case adjustment to apply, when enabled, to the header name
+ <from>, to change it to <to> before sending it to HTTP/1 clients or
+ servers. <from> must be in lower case, and <from> and <to> must not differ
+ except for their case. It may be repeated if several header names need to be
+ ajusted. Duplicate entries are not allowed. If a lot of header names have to
+ be adjusted, it might be more convenient to use "h1-case-adjust-file".
+ Please note that no transformation will be applied unless "option
+ h1-case-adjust-bogus-client" or "option h1-case-adjust-bogus-server" is
+ specified in a proxy.
+
+ There is no standard case for header names because, as stated in RFC7230,
+ they are case-insensitive. So applications must handle them in a case-
+ insensitive manner. But some bogus applications violate the standards and
+ erroneously rely on the cases most commonly used by browsers. This problem
+ becomes critical with HTTP/2 because all header names must be exchanged in
+ lower case, and HAProxy follows the same convention. All header names are
+ sent in lower case to clients and servers, regardless of the HTTP version.
+
+ Applications which fail to properly process requests or responses may require
+ to temporarily use such workarounds to adjust header names sent to them for
+ the time it takes the application to be fixed. Please note that an
+ application which requires such workarounds might be vulnerable to content
+ smuggling attacks and must absolutely be fixed.
+
+ Example:
+ global
+ h1-case-adjust content-length Content-Length
+
+ See "h1-case-adjust-file", "option h1-case-adjust-bogus-client" and
+ "option h1-case-adjust-bogus-server".
+
+h1-case-adjust-file <hdrs-file>
+ Defines a file containing a list of key/value pairs used to adjust the case
+ of some header names before sending them to HTTP/1 clients or servers. The
+ file <hdrs-file> must contain 2 header names per line. The first one must be
+ in lower case and both must not differ except for their case. Lines which
+ start with '#' are ignored, just like empty lines. Leading and trailing tabs
+ and spaces are stripped. Duplicate entries are not allowed. Please note that
+ no transformation will be applied unless "option h1-case-adjust-bogus-client"
+ or "option h1-case-adjust-bogus-server" is specified in a proxy.
+
+ If this directive is repeated, only the last one will be processed. It is an
+ alternative to the directive "h1-case-adjust" if a lot of header names need
+ to be adjusted. Please read the risks associated with using this.
+
+ See "h1-case-adjust", "option h1-case-adjust-bogus-client" and
+ "option h1-case-adjust-bogus-server".
+
group <group name>
Similar to "gid" but uses the GID of group name <group name> from /etc/group.
See also "gid" and "user".
@@ -2404,6 +2455,8 @@
option dontlognull (*) X X X -
-- keyword -------------------------- defaults - frontend - listen -- backend -
option forwardfor X X X X
+option h1-case-adjust-bogus-client (*) X X X -
+option h1-case-adjust-bogus-server (*) X - X X
option http-buffer-request (*) X X X X
option http-ignore-probes (*) X X X -
option http-keep-alive (*) X X X X
@@ -6204,6 +6257,76 @@
"option http-keep-alive"
+option h1-case-adjust-bogus-client
+no option h1-case-adjust-bogus-client
+ Enable or disable the case adjustment of HTTP/1 headers sent to bogus clients
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+ Arguments : none
+
+ There is no standard case for header names because, as stated in RFC7230,
+ they are case-insensitive. So applications must handle them in a case-
+ insensitive manner. But some bogus applications violate the standards and
+ erroneously rely on the cases most commonly used by browsers. This problem
+ becomes critical with HTTP/2 because all header names must be exchanged in
+ lower case, and HAProxy follows the same convention. All header names are
+ sent in lower case to clients and servers, regardless of the HTTP version.
+
+ When HAProxy receives an HTTP/1 response, its header names are converted to
+ lower case and manipulated and sent this way to the clients. If a client is
+ known to violate the HTTP standards and to fail to process a response coming
+ from HAProxy, it is possible to transform the lower case header names to a
+ different format when the response is formatted and sent to the client, by
+ enabling this option and specifying the list of headers to be reformatted
+ using the global directives "h1-case-adjust" or "h1-case-adjust-file". This
+ must only be a temporary workaround for the time it takes the client to be
+ fixed, because clients which require such workarounds might be vulnerable to
+ content smuggling attacks and must absolutely be fixed.
+
+ Please note that this option will not affect standards-compliant clients.
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+ See also: "option h1-case-adjust-bogus-server", "h1-case-adjust",
+ "h1-case-adjust-file".
+
+
+option h1-case-adjust-bogus-server
+no option h1-case-adjust-bogus-server
+ Enable or disable the case adjustment of HTTP/1 headers sent to bogus servers
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+ Arguments : none
+
+ There is no standard case for header names because, as stated in RFC7230,
+ they are case-insensitive. So applications must handle them in a case-
+ insensitive manner. But some bogus applications violate the standards and
+ erroneously rely on the cases most commonly used by browsers. This problem
+ becomes critical with HTTP/2 because all header names must be exchanged in
+ lower case, and HAProxy follows the same convention. All header names are
+ sent in lower case to clients and servers, regardless of the HTTP version.
+
+ When HAProxy receives an HTTP/1 request, its header names are converted to
+ lower case and manipulated and sent this way to the servers. If a server is
+ known to violate the HTTP standards and to fail to process a request coming
+ from HAProxy, it is possible to transform the lower case header names to a
+ different format when the request is formatted and sent to the server, by
+ enabling this option and specifying the list of headers to be reformatted
+ using the global directives "h1-case-adjust" or "h1-case-adjust-file". This
+ must only be a temporary workaround for the time it takes the server to be
+ fixed, because servers which require such workarounds might be vulnerable to
+ content smuggling attacks and must absolutely be fixed.
+
+ Please note that this option will not affect standards-compliant servers.
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+ See also: "option h1-case-adjust-bogus-client", "h1-case-adjust",
+ "h1-case-adjust-file".
+
+
option http-buffer-request
no option http-buffer-request
Enable or disable waiting for whole HTTP request body before proceeding
diff --git a/include/types/proxy.h b/include/types/proxy.h
index 2518f88..6aa0de1 100644
--- a/include/types/proxy.h
+++ b/include/types/proxy.h
@@ -143,7 +143,10 @@
#define PR_O2_INDEPSTR 0x00001000 /* independent streams, don't update rex on write */
#define PR_O2_SOCKSTAT 0x00002000 /* collect & provide separate statistics for sockets */
-/* unused: 0x00004000 0x00008000 0x00010000 */
+#define PR_O2_H1_ADJ_BUGCLI 0x00008000 /* adjust the case of h1 headers of the response for bogus clients */
+#define PR_O2_H1_ADJ_BUGSRV 0x00004000 /* adjust the case of h1 headers of the request for bogus servers */
+
+/* unused: 0x00010000 */
#define PR_O2_NODELAY 0x00020000 /* fully interactive mode, never delay outgoing data */
#define PR_O2_USE_PXHDR 0x00040000 /* use Proxy-Connection for proxy requests */
diff --git a/src/mux_h1.c b/src/mux_h1.c
index c25a0b9..b3b66f4 100644
--- a/src/mux_h1.c
+++ b/src/mux_h1.c
@@ -16,6 +16,8 @@
#include <common/htx.h>
#include <common/initcall.h>
+#include <ebistree.h>
+
#include <types/pipe.h>
#include <types/proxy.h>
#include <types/session.h>
@@ -107,6 +109,22 @@
uint16_t status; /* HTTP response status */
};
+/* Map of headers used to convert outgoing headers */
+struct h1_hdrs_map {
+ char *name;
+ struct eb_root map;
+};
+
+/* An entry in a headers map */
+struct h1_hdr_entry {
+ struct ist name;
+ struct ebpt_node node;
+};
+
+/* Declare the headers map */
+static struct h1_hdrs_map hdrs_map = { .name = NULL, .map = EB_ROOT };
+
+
/* the h1c and h1s pools */
DECLARE_STATIC_POOL(pool_head_h1c, "h1c", sizeof(struct h1c));
DECLARE_STATIC_POOL(pool_head_h1s, "h1s", sizeof(struct h1s));
@@ -803,6 +821,34 @@
h1_update_res_conn_value(h1s, h1m, conn_val);
}
+/* Try to adjust the case of the message header name using the global map
+ * <hdrs_map>.
+ */
+static void h1_adjust_case_outgoing_hdr(struct h1s *h1s, struct h1m *h1m, struct ist *name)
+{
+ struct ebpt_node *node;
+ struct h1_hdr_entry *entry;
+
+ /* No entry in the map, do nothing */
+ if (eb_is_empty(&hdrs_map.map))
+ return;
+
+ /* No conversion fo the request headers */
+ if (!(h1m->flags & H1_MF_RESP) && !(h1s->h1c->px->options2 & PR_O2_H1_ADJ_BUGSRV))
+ return;
+
+ /* No conversion fo the response headers */
+ if ((h1m->flags & H1_MF_RESP) && !(h1s->h1c->px->options2 & PR_O2_H1_ADJ_BUGCLI))
+ return;
+
+ node = ebis_lookup_len(&hdrs_map.map, name->ptr, name->len);
+ if (!node)
+ return;
+ entry = container_of(node, struct h1_hdr_entry, node);
+ name->ptr = entry->name.ptr;
+ name->len = entry->name.len;
+}
+
/* Append the description of what is present in error snapshot <es> into <out>.
* The description must be small enough to always fit in a buffer. The output
* buffer may be the trash so the trash must not be used inside this function.
@@ -1673,6 +1719,9 @@
goto skip_hdr;
}
+ /* Try to adjust the case of the header name */
+ if (h1c->px->options2 & (PR_O2_H1_ADJ_BUGCLI|PR_O2_H1_ADJ_BUGSRV))
+ h1_adjust_case_outgoing_hdr(h1s, h1m, &n);
if (!htx_hdr_to_h1(n, v, &tmp))
goto full;
skip_hdr:
@@ -1692,6 +1741,9 @@
v = ist("");
h1_process_output_conn_mode(h1s, h1m, &v);
if (v.len) {
+ /* Try to adjust the case of the header name */
+ if (h1c->px->options2 & (PR_O2_H1_ADJ_BUGCLI|PR_O2_H1_ADJ_BUGSRV))
+ h1_adjust_case_outgoing_hdr(h1s, h1m, &n);
if (!htx_hdr_to_h1(n, v, &tmp))
goto full;
}
@@ -1781,6 +1833,10 @@
else { // HTX_BLK_TLR
n = htx_get_blk_name(chn_htx, blk);
v = htx_get_blk_value(chn_htx, blk);
+
+ /* Try to adjust the case of the header name */
+ if (h1c->px->options2 & (PR_O2_H1_ADJ_BUGCLI|PR_O2_H1_ADJ_BUGSRV))
+ h1_adjust_case_outgoing_hdr(h1s, h1m, &n);
if (!htx_hdr_to_h1(n, v, &tmp))
goto full;
}
@@ -2555,8 +2611,197 @@
chunk_appendf(msg, " .cs.flg=0x%08x .cs.data=%p",
h1s->cs->flags, h1s->cs->data);
}
+}
+
+
+/* Add an entry in the headers map. Returns -1 on error and 0 on success. */
+static int add_hdr_case_adjust(const char *from, const char *to, char **err)
+{
+ struct h1_hdr_entry *entry;
+
+ /* Be sure there is a non-empty <to> */
+ if (!strlen(to)) {
+ memprintf(err, "expect <to>");
+ return -1;
+ }
+
+ /* Be sure only the case differs between <from> and <to> */
+ if (strcasecmp(from, to)) {
+ memprintf(err, "<from> and <to> must not differ execpt the case");
+ return -1;
+ }
+
+ /* Be sure <from> does not already existsin the tree */
+ if (ebis_lookup(&hdrs_map.map, from)) {
+ memprintf(err, "duplicate entry '%s'", from);
+ return -1;
+ }
+
+ /* Create the entry and insert it in the tree */
+ entry = malloc(sizeof(*entry));
+ if (!entry) {
+ memprintf(err, "out of memory");
+ return -1;
+ }
+
+ entry->node.key = strdup(from);
+ entry->name.ptr = strdup(to);
+ entry->name.len = strlen(to);
+ if (!entry->node.key || !entry->name.ptr) {
+ free(entry->node.key);
+ free(entry->name.ptr);
+ free(entry);
+ memprintf(err, "out of memory");
+ return -1;
+ }
+ ebis_insert(&hdrs_map.map, &entry->node);
+ return 0;
+}
+
+static void h1_hdeaders_case_adjust_deinit()
+{
+ struct ebpt_node *node, *next;
+ struct h1_hdr_entry *entry;
+
+ node = ebpt_first(&hdrs_map.map);
+ while (node) {
+ next = ebpt_next(node);
+ ebpt_delete(node);
+ entry = container_of(node, struct h1_hdr_entry, node);
+ free(entry->node.key);
+ free(entry->name.ptr);
+ free(entry);
+ node = next;
+ }
+ free(hdrs_map.name);
}
+static int cfg_h1_headers_case_adjust_postparser()
+{
+ FILE *file = NULL;
+ char *c, *key_beg, *key_end, *value_beg, *value_end;
+ char *err;
+ int rc, line = 0, err_code = 0;
+
+ if (!hdrs_map.name)
+ goto end;
+
+ file = fopen(hdrs_map.name, "r");
+ if (!file) {
+ ha_alert("config : h1-outgoing-headers-case-adjust-file '%s': failed to open file.\n",
+ hdrs_map.name);
+ err_code |= ERR_ALERT | ERR_FATAL;
+ goto end;
+ }
+
+ /* now parse all lines. The file may contain only two header name per
+ * line, separated by spaces. All heading and trailing spaces will be
+ * ignored. Lines starting with a # are ignored.
+ */
+ while (fgets(trash.area, trash.size, file) != NULL) {
+ line++;
+ c = trash.area;
+
+ /* strip leading spaces and tabs */
+ while (*c == ' ' || *c == '\t')
+ c++;
+
+ /* ignore emptu lines, or lines beginning with a dash */
+ if (*c == '#' || *c == '\0' || *c == '\r' || *c == '\n')
+ continue;
+
+ /* look for the end of the key */
+ key_beg = c;
+ while (*c != '\0' && *c != ' ' && *c != '\t' && *c != '\n' && *c != '\r')
+ c++;
+ key_end = c;
+
+ /* strip middle spaces and tabs */
+ while (*c == ' ' || *c == '\t')
+ c++;
+
+ /* look for the end of the value, it is the end of the line */
+ value_beg = c;
+ while (*c && *c != '\n' && *c != '\r')
+ c++;
+ value_end = c;
+
+ /* trim possibly trailing spaces and tabs */
+ while (value_end > value_beg && (value_end[-1] == ' ' || value_end[-1] == '\t'))
+ value_end--;
+
+ /* set final \0 and check entries */
+ *key_end = '\0';
+ *value_end = '\0';
+
+ err = NULL;
+ rc = add_hdr_case_adjust(key_beg, value_beg, &err);
+ if (rc < 0) {
+ free(err);
+ ha_alert("config : h1-outgoing-headers-case-adjust-file '%s' : %s at line %d.\n",
+ hdrs_map.name, err, line);
+ err_code |= ERR_ALERT | ERR_FATAL;
+ goto end;
+ }
+ if (rc > 0) {
+ free(err);
+ ha_warning("config : h1-outgoing-headers-case-adjust-file '%s' : %s at line %d.\n",
+ hdrs_map.name, err, line);
+ err_code |= ERR_WARN;
+ }
+ }
+
+ end:
+ if (file)
+ fclose(file);
+ hap_register_post_deinit(h1_hdeaders_case_adjust_deinit);
+ return err_code;
+}
+
+
+/* config parser for global "h1-outgoing-header-case-adjust" */
+static int cfg_parse_h1_header_case_adjust(char **args, int section_type, struct proxy *curpx,
+ struct proxy *defpx, const char *file, int line,
+ char **err)
+{
+ if (too_many_args(2, args, err, NULL))
+ return -1;
+ if (!*(args[1]) || !*(args[2])) {
+ memprintf(err, "'%s' expects <from> and <to> as argument.", args[0]);
+ return -1;
+ }
+ return add_hdr_case_adjust(args[1], args[2], err);
+}
+
+/* config parser for global "h1-outgoing-headers-case-adjust-file" */
+static int cfg_parse_h1_headers_case_adjust_file(char **args, int section_type, struct proxy *curpx,
+ struct proxy *defpx, const char *file, int line,
+ char **err)
+{
+ if (too_many_args(1, args, err, NULL))
+ return -1;
+ if (!*(args[1])) {
+ memprintf(err, "'%s' expects <file> as argument.", args[0]);
+ return -1;
+ }
+ free(hdrs_map.name);
+ hdrs_map.name = strdup(args[1]);
+ return 0;
+}
+
+
+/* config keyword parsers */
+static struct cfg_kw_list cfg_kws = {{ }, {
+ { CFG_GLOBAL, "h1-case-adjust", cfg_parse_h1_header_case_adjust },
+ { CFG_GLOBAL, "h1-case-adjust-file", cfg_parse_h1_headers_case_adjust_file },
+ { 0, NULL, NULL },
+ }
+};
+
+INITCALL1(STG_REGISTER, cfg_register_keywords, &cfg_kws);
+REGISTER_CONFIG_POSTPARSER("h1-headers-map", cfg_h1_headers_case_adjust_postparser);
+
+
/****************************************/
/* MUX initialization and instanciation */
/****************************************/
diff --git a/src/proxy.c b/src/proxy.c
index 4aff044..301d4d6 100644
--- a/src/proxy.c
+++ b/src/proxy.c
@@ -112,6 +112,8 @@
{ "http-pretend-keepalive", PR_O2_FAKE_KA, PR_CAP_BE, 0, PR_MODE_HTTP },
{ "http-no-delay", PR_O2_NODELAY, PR_CAP_FE|PR_CAP_BE, 0, PR_MODE_HTTP },
{ "http-use-htx", PR_O2_USE_HTX, PR_CAP_FE|PR_CAP_BE, 0, 0 },
+ {"h1-case-adjust-bogus-client", PR_O2_H1_ADJ_BUGCLI, PR_CAP_FE, 0, PR_MODE_HTTP },
+ {"h1-case-adjust-bogus-server", PR_O2_H1_ADJ_BUGSRV, PR_CAP_BE, 0, PR_MODE_HTTP },
{ NULL, 0, 0, 0 }
};