willy tarreau | 0174f31 | 2005-12-18 01:02:42 +0100 | [diff] [blame] | 1 | ------------------- |
Willy Tarreau | 94b4591 | 2006-05-31 06:40:15 +0200 | [diff] [blame] | 2 | HAProxy |
willy tarreau | 0174f31 | 2005-12-18 01:02:42 +0100 | [diff] [blame] | 3 | Reference Manual |
| 4 | ------------------- |
Willy Tarreau | 7b4c5ae | 2008-04-19 21:06:14 +0200 | [diff] [blame] | 5 | version 1.3.15 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 6 | willy tarreau |
Willy Tarreau | 7b4c5ae | 2008-04-19 21:06:14 +0200 | [diff] [blame] | 7 | 2008/04/19 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 8 | |
Willy Tarreau | a080eca | 2009-10-14 20:30:15 +0200 | [diff] [blame] | 9 | |
| 10 | !!!! NOTE: THIS DOCUMENT IS OUTDATED !!!! |
| 11 | |
| 12 | Please use "configuration.txt" from the same directory, or download |
| 13 | an up-to-date version from the following location : |
| 14 | |
Willy Tarreau | be6008f | 2009-10-14 22:22:03 +0200 | [diff] [blame] | 15 | http://haproxy.1wt.eu/download/1.4/doc/configuration.txt |
Willy Tarreau | a080eca | 2009-10-14 20:30:15 +0200 | [diff] [blame] | 16 | |
| 17 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 18 | ============ |
| 19 | | Abstract | |
| 20 | ============ |
| 21 | |
Willy Tarreau | 94b4591 | 2006-05-31 06:40:15 +0200 | [diff] [blame] | 22 | HAProxy is a TCP/HTTP reverse proxy which is particularly suited for high |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 23 | availability environments. Indeed, it can : |
| 24 | - route HTTP requests depending on statically assigned cookies ; |
| 25 | - spread the load among several servers while assuring server persistence |
| 26 | through the use of HTTP cookies ; |
| 27 | - switch to backup servers in the event a main one fails ; |
| 28 | - accept connections to special ports dedicated to service monitoring ; |
| 29 | - stop accepting connections without breaking existing ones ; |
| 30 | - add/modify/delete HTTP headers both ways ; |
| 31 | - block requests matching a particular pattern ; |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 32 | - hold clients to the right application server depending on application |
| 33 | cookies |
willy tarreau | 481132e | 2006-05-21 21:43:10 +0200 | [diff] [blame] | 34 | - report detailed status as HTML pages to authenticated users from an URI |
| 35 | intercepted from the application. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 36 | |
| 37 | It needs very little resource. Its event-driven architecture allows it to easily |
| 38 | handle thousands of simultaneous connections on hundreds of instances without |
| 39 | risking the system's stability. |
| 40 | |
| 41 | ==================== |
| 42 | | Start parameters | |
| 43 | ==================== |
| 44 | |
| 45 | There are only a few command line options : |
| 46 | |
| 47 | -f <configuration file> |
| 48 | -n <high limit for the total number of simultaneous connections> |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 49 | = 'maxconn' in 'global' section |
| 50 | -N <high limit for the per-listener number of simultaneous connections> |
| 51 | = 'maxconn' in 'listen' or 'default' sections |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 52 | -d starts in foregreound with debugging mode enabled |
| 53 | -D starts in daemon mode |
Marc-Antoine Perennou | 992709b | 2013-02-12 10:53:52 +0100 | [diff] [blame] | 54 | -Ds starts in systemd daemon mode |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 55 | -q disable messages on output |
| 56 | -V displays messages on output even when -q or 'quiet' are specified. |
| 57 | -c only checks config file and exits with code 0 if no error was found, or |
| 58 | exits with code 1 if a syntax error was found. |
willy tarreau | fe2c5c1 | 2005-12-17 14:14:34 +0100 | [diff] [blame] | 59 | -p <pidfile> asks the process to write down each of its children's |
| 60 | pids to this file in daemon mode. |
willy tarreau | 34f4530 | 2006-04-15 21:37:14 +0200 | [diff] [blame] | 61 | -sf specifies a list of pids to send a FINISH signal to after startup. |
| 62 | -st specifies a list of pids to send a TERMINATE signal to after startup. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 63 | -s shows statistics (only if compiled in) |
| 64 | -l shows even more statistics (implies '-s') |
Willy Tarreau | de99e99 | 2007-04-16 00:53:59 +0200 | [diff] [blame] | 65 | -dk disables use of kqueue() |
willy tarreau | 64a3cc3 | 2005-12-18 01:13:11 +0100 | [diff] [blame] | 66 | -de disables use of epoll() |
| 67 | -dp disables use of poll() |
willy tarreau | 34f4530 | 2006-04-15 21:37:14 +0200 | [diff] [blame] | 68 | -db disables background mode (stays in foreground, useful for debugging) |
| 69 | -m <megs> enforces a memory usage limit to a maximum of <megs> megabytes. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 70 | |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 71 | The maximal number of connections per proxy instance is used as the default |
| 72 | parameter for each instance for which the 'maxconn' paramter is not set in the |
| 73 | 'listen' section. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 74 | |
| 75 | The maximal number of total connections limits the number of connections used by |
| 76 | the whole process if the 'maxconn' parameter is not set in the 'global' section. |
| 77 | |
| 78 | The debugging mode has the same effect as the 'debug' option in the 'global' |
| 79 | section. When the proxy runs in this mode, it dumps every connections, |
| 80 | disconnections, timestamps, and HTTP headers to stdout. This should NEVER |
| 81 | be used in an init script since it will prevent the system from starting up. |
| 82 | |
willy tarreau | 34f4530 | 2006-04-15 21:37:14 +0200 | [diff] [blame] | 83 | For debugging, the '-db' option is very useful as it temporarily disables |
| 84 | daemon mode and multi-process mode. The service can then be stopped by simply |
| 85 | pressing Ctrl-C, without having to edit the config nor run full debug. |
| 86 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 87 | Statistics are only available if compiled in with the 'STATTIME' option. It's |
willy tarreau | 481132e | 2006-05-21 21:43:10 +0200 | [diff] [blame] | 88 | only used during code optimization phases, and will soon disappear. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 89 | |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 90 | The '-st' and '-sf' options are used for hot reconfiguration (see below). |
willy tarreau | 34f4530 | 2006-04-15 21:37:14 +0200 | [diff] [blame] | 91 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 92 | ====================== |
| 93 | | Configuration file | |
| 94 | ====================== |
| 95 | |
| 96 | Structure |
| 97 | ========= |
| 98 | |
| 99 | The configuration file parser ignores empty lines, spaces, tabs. Anything |
| 100 | between a sharp ('#') not following a backslash ('\'), and the end of a line |
| 101 | constitutes a comment and is ignored too. |
| 102 | |
| 103 | The configuration file is segmented in sections. A section begins whenever |
| 104 | one of these 3 keywords are encountered : |
| 105 | |
| 106 | - 'global' |
| 107 | - 'listen' |
| 108 | - 'defaults' |
| 109 | |
| 110 | Every parameter refer to the section beginning at the last one of these 3 |
| 111 | keywords. |
| 112 | |
| 113 | |
| 114 | 1) Global parameters |
| 115 | ==================== |
| 116 | |
| 117 | Global parameters affect the whole process behaviour. They are all set in the |
| 118 | 'global' section. There may be several 'global' sections if needed, but their |
| 119 | parameters will only be merged. Allowed parameters in 'global' section include |
| 120 | the following ones : |
| 121 | |
| 122 | - log <address> <facility> [max_level] |
| 123 | - maxconn <number> |
| 124 | - uid <user id> |
| 125 | - gid <group id> |
Willy Tarreau | 95c20ac | 2007-03-25 15:39:23 +0200 | [diff] [blame] | 126 | - user <user name> |
| 127 | - group <group name> |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 128 | - chroot <directory> |
| 129 | - nbproc <number> |
| 130 | - daemon |
| 131 | - debug |
Willy Tarreau | de99e99 | 2007-04-16 00:53:59 +0200 | [diff] [blame] | 132 | - nokqueue |
willy tarreau | 64a3cc3 | 2005-12-18 01:13:11 +0100 | [diff] [blame] | 133 | - noepoll |
| 134 | - nopoll |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 135 | - quiet |
willy tarreau | fe2c5c1 | 2005-12-17 14:14:34 +0100 | [diff] [blame] | 136 | - pidfile <file> |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 137 | - ulimit-n <number> |
willy tarreau | 598da41 | 2005-12-18 01:07:29 +0100 | [diff] [blame] | 138 | - stats |
Willy Tarreau | 1db3771 | 2007-06-03 17:16:49 +0200 | [diff] [blame] | 139 | - tune.maxpollevents <number> |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 140 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 141 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 142 | 1.1) Event logging |
| 143 | ------------------ |
| 144 | Most events are logged : start, stop, servers going up and down, connections and |
| 145 | errors. Each event generates a syslog message which can be sent to up to 2 |
| 146 | servers. The syntax is : |
| 147 | |
| 148 | log <ip_address> <facility> [max_level] |
| 149 | |
| 150 | Connections are logged at level "info". Services initialization and servers |
| 151 | going up are logged at level "notice", termination signals are logged at |
| 152 | "warning", and definitive service termination, as well as loss of servers are |
| 153 | logged at level "alert". The optional parameter <max_level> specifies above |
| 154 | what level messages should be sent. Level can take one of these 8 values : |
| 155 | |
| 156 | emerg, alert, crit, err, warning, notice, info, debug |
| 157 | |
| 158 | For backwards compatibility with versions 1.1.16 and earlier, the default level |
| 159 | value is "debug" if not specified. |
| 160 | |
| 161 | Permitted facilities are : |
| 162 | kern, user, mail, daemon, auth, syslog, lpr, news, |
| 163 | uucp, cron, auth2, ftp, ntp, audit, alert, cron2, |
| 164 | local0, local1, local2, local3, local4, local5, local6, local7 |
| 165 | |
| 166 | According to RFC3164, messages are truncated to 1024 bytes before being emitted. |
| 167 | |
| 168 | Example : |
| 169 | --------- |
| 170 | global |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 171 | log 192.168.2.200 local3 |
| 172 | log 127.0.0.1 local4 notice |
| 173 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 174 | |
| 175 | 1.2) limiting the number of connections |
| 176 | --------------------------------------- |
| 177 | It is possible and recommended to limit the global number of per-process |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 178 | connections using the 'maxconn' global keyword. Since one connection includes |
| 179 | both a client and a server, it means that the max number of TCP sessions will |
| 180 | be about the double of this number. It's important to understand this when |
| 181 | trying to find best values for 'ulimit -n' before starting the proxy. To |
| 182 | anticipate the number of sockets needed, all these parameters must be counted : |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 183 | |
| 184 | - 1 socket per incoming connection |
| 185 | - 1 socket per outgoing connection |
| 186 | - 1 socket per address/port/proxy tuple. |
| 187 | - 1 socket per server being health-checked |
| 188 | - 1 socket for all logs |
| 189 | |
| 190 | In simple configurations where each proxy only listens one one address/port, |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 191 | set the limit of file descriptors (ulimit -n) to |
| 192 | (2 * maxconn + nbproxies + nbservers + 1). Starting with versions 1.1.32/1.2.6, |
| 193 | it is now possible to set the limit in the configuration using the 'ulimit-n' |
| 194 | global keyword, provided the proxy is started as root. This puts an end to the |
| 195 | recurrent problem of ensuring that the system limits are adapted to the proxy |
| 196 | values. Note that these limits are per-process. |
| 197 | |
| 198 | Example : |
| 199 | --------- |
| 200 | global |
| 201 | maxconn 32000 |
| 202 | ulimit-n 65536 |
| 203 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 204 | |
| 205 | 1.3) Drop of priviledges |
| 206 | ------------------------ |
| 207 | In order to reduce the risk and consequences of attacks, in the event where a |
| 208 | yet non-identified vulnerability would be successfully exploited, it's possible |
| 209 | to lower the process priviledges and even isolate it in a riskless directory. |
| 210 | |
| 211 | In the 'global' section, the 'uid' parameter sets a numerical user identifier |
| 212 | which the process will switch to after binding its listening sockets. The value |
| 213 | '0', which normally represents the super-user, here indicates that the UID must |
| 214 | not change during startup. It's the default behaviour. The 'gid' parameter does |
Willy Tarreau | 95c20ac | 2007-03-25 15:39:23 +0200 | [diff] [blame] | 215 | the same for the group identifier. If setting an uid is not possible because of |
| 216 | deployment constraints, it is possible to set a user name with the 'user' |
| 217 | keyword followed by a valid user name. The same is true for the gid. It is |
| 218 | possible to specify a group name after the 'group' keyword. |
| 219 | |
| 220 | It is particularly advised against use of generic accounts such as 'nobody' |
| 221 | because it has the same consequences as using 'root' if other services use |
| 222 | them. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 223 | |
| 224 | The 'chroot' parameter makes the process isolate itself in an empty directory |
| 225 | just before switching its UID. This type of isolation (chroot) can sometimes |
| 226 | be worked around on certain OS (Linux, Solaris), provided that the attacker |
| 227 | has gained 'root' priviledges and has the ability to use or create a directory. |
| 228 | For this reason, it's capital to use a dedicated directory and not to share one |
| 229 | between several services of different nature. To make isolation more resistant, |
| 230 | it's recommended to use an empty directory without any right, and to change the |
| 231 | UID of the process so that it cannot do anything there. |
| 232 | |
| 233 | Note: in the event where such a vulnerability would be exploited, it's most |
| 234 | likely that first attempts would kill the process due to 'Segmentation Fault', |
| 235 | 'Bus Error' or 'Illegal Instruction' signals. Eventhough it's true that |
| 236 | isolating the server reduces the risks of intrusion, it's sometimes useful to |
| 237 | find why a process dies, via the analysis of a 'core' file, although very rare |
| 238 | (the last bug of this sort was fixed in 1.1.9). For security reasons, most |
| 239 | systems disable the generation of core file when a process changes its UID. So |
| 240 | the two workarounds are either to start the process from a restricted user |
| 241 | account, which will not be able to chroot itself, or start it as root and not |
| 242 | change the UID. In both cases the core will be either in the start or the chroot |
| 243 | directories. Do not forget to allow core dumps prior to start the process : |
| 244 | |
| 245 | # ulimit -c unlimited |
| 246 | |
| 247 | Example : |
| 248 | --------- |
| 249 | |
Willy Tarreau | 95c20ac | 2007-03-25 15:39:23 +0200 | [diff] [blame] | 250 | # with uid/gid |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 251 | global |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 252 | uid 30000 |
| 253 | gid 30000 |
| 254 | chroot /var/chroot/haproxy |
| 255 | |
Willy Tarreau | 95c20ac | 2007-03-25 15:39:23 +0200 | [diff] [blame] | 256 | # with user/group |
| 257 | global |
| 258 | user haproxy |
| 259 | group public |
| 260 | chroot /var/chroot/haproxy |
| 261 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 262 | |
| 263 | 1.4) Startup modes |
| 264 | ------------------ |
willy tarreau | 34f4530 | 2006-04-15 21:37:14 +0200 | [diff] [blame] | 265 | The service can start in several different modes : |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 266 | - foreground / background |
| 267 | - quiet / normal / debug |
| 268 | |
| 269 | The default mode is normal, foreground, which means that the program doesn't |
| 270 | return once started. NEVER EVER use this mode in a system startup script, or |
| 271 | the system won't boot. It needs to be started in background, so that it |
| 272 | returns immediately after forking. That's accomplished by the 'daemon' option |
| 273 | in the 'global' section, which is the equivalent of the '-D' command line |
| 274 | argument. |
| 275 | |
willy tarreau | 34f4530 | 2006-04-15 21:37:14 +0200 | [diff] [blame] | 276 | The '-db' command line argument overrides the 'daemon' and 'nbproc' global |
| 277 | options to make the process run in normal, foreground mode. |
| 278 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 279 | Moreover, certain alert messages are still sent to the standard output even |
| 280 | in 'daemon' mode. To make them disappear, simply add the 'quiet' option in the |
| 281 | 'global' section. This option has no command-line equivalent. |
| 282 | |
| 283 | Last, the 'debug' mode, enabled with the 'debug' option in the 'global' section, |
| 284 | and which is equivalent of the '-d' option, allows deep TCP/HTTP analysis, with |
| 285 | timestamped display of each connection, disconnection, and HTTP headers for both |
| 286 | ways. This mode is incompatible with 'daemon' and 'quiet' modes for obvious |
| 287 | reasons. |
| 288 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 289 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 290 | 1.5) Increasing the overall processing power |
| 291 | -------------------------------------------- |
| 292 | On multi-processor systems, it may seem to be a shame to use only one processor, |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 293 | eventhough the load needed to saturate a recent processor is far above common |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 294 | usage. Anyway, for very specific needs, the proxy can start several processes |
| 295 | between which the operating system will spread the incoming connections. The |
| 296 | number of processes is controlled by the 'nbproc' parameter in the 'global' |
willy tarreau | 4302f49 | 2005-12-18 01:00:37 +0100 | [diff] [blame] | 297 | section. It defaults to 1, and obviously works only in 'daemon' mode. One |
| 298 | typical usage of this parameter has been to workaround the default per-process |
| 299 | file-descriptor limit that Solaris imposes to user processes. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 300 | |
| 301 | Example : |
| 302 | --------- |
| 303 | |
| 304 | global |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 305 | daemon |
| 306 | quiet |
| 307 | nbproc 2 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 308 | |
| 309 | |
willy tarreau | fe2c5c1 | 2005-12-17 14:14:34 +0100 | [diff] [blame] | 310 | 1.6) Helping process management |
| 311 | ------------------------------- |
| 312 | Haproxy now supports the notion of pidfile. If the '-p' command line argument, |
| 313 | or the 'pidfile' global option is followed with a file name, this file will be |
| 314 | removed, then filled with all children's pids, one per line (only in daemon |
| 315 | mode). This file is NOT within the chroot, which allows to work with a readonly |
| 316 | chroot. It will be owned by the user starting the process, and will have |
| 317 | permissions 0644. |
| 318 | |
| 319 | Example : |
| 320 | --------- |
| 321 | |
| 322 | global |
| 323 | daemon |
| 324 | quiet |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 325 | nbproc 2 |
willy tarreau | fe2c5c1 | 2005-12-17 14:14:34 +0100 | [diff] [blame] | 326 | pidfile /var/run/haproxy-private.pid |
| 327 | |
| 328 | # to stop only those processes among others : |
| 329 | # kill $(</var/run/haproxy-private.pid) |
| 330 | |
willy tarreau | 34f4530 | 2006-04-15 21:37:14 +0200 | [diff] [blame] | 331 | # to reload a new configuration with minimal service impact and without |
| 332 | # breaking existing sessions : |
Willy Tarreau | 10806d5 | 2007-09-09 23:49:18 +0200 | [diff] [blame] | 333 | # haproxy -f haproxy.cfg -p /var/run/haproxy-private.pid -sf $(</var/run/haproxy-private.pid) |
willy tarreau | fe2c5c1 | 2005-12-17 14:14:34 +0100 | [diff] [blame] | 334 | |
willy tarreau | 64a3cc3 | 2005-12-18 01:13:11 +0100 | [diff] [blame] | 335 | 1.7) Polling mechanisms |
| 336 | ----------------------- |
| 337 | Starting from version 1.2.5, haproxy supports the poll() and epoll() polling |
| 338 | mechanisms. On systems where select() is limited by FD_SETSIZE (like Solaris), |
| 339 | poll() can be an interesting alternative. Performance tests show that Solaris' |
| 340 | poll() performance does not decay as fast as the numbers of sockets increase, |
| 341 | making it a safe solution for high loads. However, Solaris already uses poll() |
| 342 | to emulate select(), so as long as the number of sockets has no reason to go |
| 343 | higher than FD_SETSIZE, poll() should not provide any better performance. On |
| 344 | Linux systems with the epoll() patch (or any 2.6 version), haproxy will use |
| 345 | epoll() which is extremely fast and non dependant on the number of sockets. |
| 346 | Tests have shown constant performance from 1 to 20000 simultaneous sessions. |
Willy Tarreau | de99e99 | 2007-04-16 00:53:59 +0200 | [diff] [blame] | 347 | Version 1.3.9 introduced kqueue() for FreeBSD/OpenBSD, and speculative epoll() |
| 348 | which consists in trying to perform I/O before queuing the events via syscalls. |
willy tarreau | 64a3cc3 | 2005-12-18 01:13:11 +0100 | [diff] [blame] | 349 | |
Willy Tarreau | 1db3771 | 2007-06-03 17:16:49 +0200 | [diff] [blame] | 350 | In order to optimize latency, it is now possible to limit the number of events |
| 351 | returned by a single call to poll. The limit is fixed to 200 by default. If a |
| 352 | smaller latency is seeked, it may be useful to reduce this value by using the |
| 353 | 'tune.maxpollevents' parameter in the 'global' section. Increasing it will |
| 354 | slightly save CPU cycles in presence of large number of connections. |
| 355 | |
Willy Tarreau | de99e99 | 2007-04-16 00:53:59 +0200 | [diff] [blame] | 356 | Haproxy will use kqueue() or speculative epoll() when available, then epoll(), |
| 357 | and will fall back to poll(), then to select(). However, if for any reason you |
| 358 | need to disable epoll() or poll() (eg. because of a bug or just to compare |
Willy Tarreau | e9f49e7 | 2012-11-11 17:42:00 +0100 | [diff] [blame] | 359 | performance), new global options have been created for this matter : 'nopoll', |
| 360 | 'nokqueue', and 'noepoll'. |
willy tarreau | 64a3cc3 | 2005-12-18 01:13:11 +0100 | [diff] [blame] | 361 | |
| 362 | Example : |
| 363 | --------- |
| 364 | |
| 365 | global |
| 366 | # use only select() |
| 367 | noepoll |
| 368 | nopoll |
Willy Tarreau | 1db3771 | 2007-06-03 17:16:49 +0200 | [diff] [blame] | 369 | tune.maxpollevents 100 |
willy tarreau | 64a3cc3 | 2005-12-18 01:13:11 +0100 | [diff] [blame] | 370 | |
| 371 | Note : |
| 372 | ------ |
| 373 | For the sake of configuration file portability, these options are accepted but |
| 374 | ignored if the poll() or epoll() mechanisms have not been enabled at compile |
| 375 | time. |
| 376 | |
Willy Tarreau | de99e99 | 2007-04-16 00:53:59 +0200 | [diff] [blame] | 377 | To make debugging easier, the '-de' runtime argument disables epoll support, |
| 378 | the '-dp' argument disables poll support, '-dk' disables kqueue and '-ds' |
| 379 | disables speculative epoll(). They are respectively equivalent to 'noepoll', |
Willy Tarreau | e9f49e7 | 2012-11-11 17:42:00 +0100 | [diff] [blame] | 380 | 'nopoll', and 'nokqueue'. |
willy tarreau | 64a3cc3 | 2005-12-18 01:13:11 +0100 | [diff] [blame] | 381 | |
| 382 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 383 | 2) Declaration of a listening service |
| 384 | ===================================== |
| 385 | |
| 386 | Service sections start with the 'listen' keyword : |
| 387 | |
| 388 | listen <instance_name> [ <IP_address>:<port_range>[,...] ] |
| 389 | |
| 390 | - <instance_name> is the name of the instance. This name will be reported in |
| 391 | logs, so it is good to have it reflect the proxied service. No unicity test |
| 392 | is done on this name, and it's not mandatory for it to be unique, but highly |
| 393 | recommended. |
| 394 | |
| 395 | - <IP_address> is the IP address the proxy binds to. Empty address, '*' and |
| 396 | '0.0.0.0' all mean that the proxy listens to all valid addresses on the |
| 397 | system. |
| 398 | |
| 399 | - <port_range> is either a unique port, or a port range for which the proxy will |
| 400 | accept connections for the IP address specified above. This range can be : |
| 401 | - a numerical port (ex: '80') |
| 402 | - a dash-delimited ports range explicitly stating the lower and upper bounds |
| 403 | (ex: '2000-2100') which are included in the range. |
| 404 | |
| 405 | Particular care must be taken against port ranges, because every <addr:port> |
| 406 | couple consumes one socket (=a file descriptor), so it's easy to eat lots of |
| 407 | descriptors with a simple range. The <addr:port> couple must be used only once |
| 408 | among all instances running on a same system. Please note that attaching to |
| 409 | ports lower than 1024 need particular priviledges to start the program, which |
Jamie Gloudon | 801a0a3 | 2012-08-25 00:18:33 -0400 | [diff] [blame] | 410 | are independent of the 'uid' parameter. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 411 | |
| 412 | - the <IP_address>:<port_range> couple may be repeated indefinitely to require |
| 413 | the proxy to listen to other addresses and/or ports. To achieve this, simply |
| 414 | separate them with a coma. |
| 415 | |
| 416 | Examples : |
| 417 | --------- |
| 418 | listen http_proxy :80 |
| 419 | listen x11_proxy 127.0.0.1:6000-6009 |
| 420 | listen smtp_proxy 127.0.0.1:25,127.0.0.1:587 |
| 421 | listen ldap_proxy :389,:663 |
| 422 | |
| 423 | In the event that all addresses do not fit line width, it's preferable to |
| 424 | detach secondary addresses on other lines with the 'bind' keyword. If this |
| 425 | keyword is used, it's not even necessary to specify the first address on the |
| 426 | 'listen' line, which sometimes makes multiple configuration handling easier : |
| 427 | |
| 428 | bind [ <IP_address>:<port_range>[,...] ] |
| 429 | |
| 430 | Examples : |
| 431 | ---------- |
| 432 | listen http_proxy |
| 433 | bind :80,:443 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 434 | bind 10.0.0.1:10080,10.0.0.1:10443 |
| 435 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 436 | |
| 437 | 2.1) Inhibiting a service |
| 438 | ------------------------- |
| 439 | A service may be disabled for maintenance reasons, without needing to comment |
| 440 | out the whole section, simply by specifying the 'disabled' keyword in the |
| 441 | section to be disabled : |
| 442 | |
| 443 | listen smtp_proxy 0.0.0.0:25 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 444 | disabled |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 445 | |
| 446 | Note: the 'enabled' keyword allows to enable a service which has been disabled |
| 447 | previously by a default configuration. |
| 448 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 449 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 450 | 2.2) Modes of operation |
| 451 | ----------------------- |
| 452 | A service can work in 3 different distinct modes : |
| 453 | - TCP |
| 454 | - HTTP |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 455 | - health |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 456 | |
| 457 | TCP mode |
| 458 | -------- |
| 459 | In this mode, the service relays TCP connections as soon as they're established, |
| 460 | towards one or several servers. No processing is done on the stream. It's only |
| 461 | an association of source(addr:port) -> destination(addr:port). To use this mode, |
| 462 | you must specify 'mode tcp' in the 'listen' section. This is the default mode. |
| 463 | |
| 464 | Example : |
| 465 | --------- |
| 466 | listen smtp_proxy 0.0.0.0:25 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 467 | mode tcp |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 468 | |
| 469 | HTTP mode |
| 470 | --------- |
| 471 | In this mode, the service relays TCP connections towards one or several servers, |
| 472 | when it has enough informations to decide, which normally means that all HTTP |
| 473 | headers have been read. Some of them may be scanned for a cookie or a pattern |
| 474 | matching a regex. To use this mode, specify 'mode http' in the 'listen' section. |
| 475 | |
| 476 | Example : |
| 477 | --------- |
| 478 | listen http_proxy 0.0.0.0:80 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 479 | mode http |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 480 | |
| 481 | Health-checking mode |
| 482 | -------------------- |
| 483 | This mode provides a way for external components to check the proxy's health. |
| 484 | It is meant to be used with intelligent load-balancers which can use send/expect |
| 485 | scripts to check for all of their servers' availability. This one simply accepts |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 486 | the connection, returns the word 'OK' and closes it. If the 'option httpchk' is |
| 487 | set, then the reply will be 'HTTP/1.0 200 OK' with no data, so that it can be |
| 488 | tested from a tool which supports HTTP health-checks. To enable it, simply |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 489 | specify 'health' as the working mode : |
| 490 | |
| 491 | Example : |
| 492 | --------- |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 493 | # simple response : 'OK' |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 494 | listen health_check 0.0.0.0:60000 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 495 | mode health |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 496 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 497 | # HTTP response : 'HTTP/1.0 200 OK' |
| 498 | listen http_health_check 0.0.0.0:60001 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 499 | mode health |
| 500 | option httpchk |
| 501 | |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 502 | 2.2.1 Monitoring |
| 503 | ---------------- |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 504 | Versions 1.1.32 and 1.2.6 provide a new solution to check the proxy's |
| 505 | availability without perturbating the service. The 'monitor-net' keyword was |
| 506 | created to specify a network of equipments which CANNOT use the service for |
| 507 | anything but health-checks. This is particularly suited to TCP proxies, because |
| 508 | it prevents the proxy from relaying the monitor's connection to the remote |
| 509 | server. |
| 510 | |
| 511 | When used with TCP, the connection is accepted then closed and nothing is |
| 512 | logged. This is enough for a front-end load-balancer to detect the service as |
| 513 | available. |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 514 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 515 | When used with HTTP, the connection is accepted, nothing is logged, the |
| 516 | following response is sent, then the session is closed : "HTTP/1.0 200 OK". |
| 517 | This is normally enough for any front-end HTTP load-balancer to detect the |
| 518 | service as available too, both with TCP and HTTP checks. |
| 519 | |
| 520 | Proxies using the "monitor-net" keyword can remove the "option dontlognull", as |
| 521 | it will make them log empty connections from hosts outside the monitoring |
| 522 | network. |
| 523 | |
| 524 | Example : |
| 525 | --------- |
| 526 | |
| 527 | listen tse-proxy |
| 528 | bind :3389,:1494,:5900 # TSE, ICA and VNC at once. |
| 529 | mode tcp |
| 530 | balance roundrobin |
| 531 | server tse-farm 192.168.1.10 |
| 532 | monitor-net 192.168.1.252/31 # L4 load-balancers on .252 and .253 |
| 533 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 534 | |
Willy Tarreau | 1c47f85 | 2006-07-09 08:22:27 +0200 | [diff] [blame] | 535 | When the system executing the checks is located behind a proxy, the monitor-net |
| 536 | keyword cannot be used because haproxy will always see the proxy's address. To |
| 537 | overcome this limitation, version 1.2.15 brought the 'monitor-uri' keyword. It |
| 538 | defines an URI which will not be forwarded nor logged, but for which haproxy |
| 539 | will immediately send an "HTTP/1.0 200 OK" response. This makes it possible to |
| 540 | check the validity of the reverse-proxy->haproxy chain with one request. It can |
| 541 | be used in HTTPS checks in front of an stunnel -> haproxy combination for |
| 542 | instance. Obviously, this keyword is only valid in HTTP mode, otherwise there |
| 543 | is no notion of URI. Note that the method and HTTP versions are simply ignored. |
| 544 | |
| 545 | Example : |
| 546 | --------- |
| 547 | |
| 548 | listen stunnel_backend :8080 |
| 549 | mode http |
| 550 | balance roundrobin |
| 551 | server web1 192.168.1.10:80 check |
| 552 | server web2 192.168.1.11:80 check |
| 553 | monitor-uri /haproxy_test |
| 554 | |
| 555 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 556 | 2.3) Limiting the number of simultaneous connections |
| 557 | ---------------------------------------------------- |
| 558 | The 'maxconn' parameter allows a proxy to refuse connections above a certain |
| 559 | amount of simultaneous ones. When the limit is reached, it simply stops |
| 560 | listening, but the system may still be accepting them because of the back log |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 561 | queue. These connections will be processed later when other ones have freed |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 562 | some slots. This provides a serialization effect which helps very fragile |
willy tarreau | 34f4530 | 2006-04-15 21:37:14 +0200 | [diff] [blame] | 563 | servers resist to high loads. See further for system limitations. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 564 | |
| 565 | Example : |
| 566 | --------- |
| 567 | listen tiny_server 0.0.0.0:80 |
| 568 | maxconn 10 |
| 569 | |
| 570 | |
| 571 | 2.4) Soft stop |
| 572 | -------------- |
| 573 | It is possible to stop services without breaking existing connections by the |
willy tarreau | 22739ef | 2006-01-20 20:43:32 +0100 | [diff] [blame] | 574 | sending of the SIGUSR1 signal to the process. All services are then put into |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 575 | soft-stop state, which means that they will refuse to accept new connections, |
| 576 | except for those which have a non-zero value in the 'grace' parameter, in which |
| 577 | case they will still accept connections for the specified amount of time, in |
willy tarreau | 22739ef | 2006-01-20 20:43:32 +0100 | [diff] [blame] | 578 | milliseconds. This makes it possible to tell a load-balancer that the service |
| 579 | is failing, while still doing the job during the time it needs to detect it. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 580 | |
| 581 | Note: active connections are never killed. In the worst case, the user will have |
| 582 | to wait for all of them to close or to time-out, or simply kill the process |
willy tarreau | 22739ef | 2006-01-20 20:43:32 +0100 | [diff] [blame] | 583 | normally (SIGTERM). The default 'grace' value is '0'. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 584 | |
| 585 | Example : |
| 586 | --------- |
| 587 | # enter soft stop after 'killall -USR1 haproxy' |
| 588 | # the service will still run 10 seconds after the signal |
| 589 | listen http_proxy 0.0.0.0:80 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 590 | mode http |
| 591 | grace 10000 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 592 | |
| 593 | # this port is dedicated to a load-balancer, and must fail immediately |
| 594 | listen health_check 0.0.0.0:60000 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 595 | mode health |
| 596 | grace 0 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 597 | |
| 598 | |
willy tarreau | 39df2dc | 2006-01-29 21:56:05 +0100 | [diff] [blame] | 599 | As of version 1.2.8, a new soft-reconfiguration mechanism has been introduced. |
willy tarreau | 22739ef | 2006-01-20 20:43:32 +0100 | [diff] [blame] | 600 | It is now possible to "pause" all the proxies by sending a SIGTTOU signal to |
| 601 | the processes. This will disable the listening socket without breaking existing |
| 602 | connections. After that, sending a SIGTTIN signal to those processes enables |
| 603 | the listening sockets again. This is very useful to try to load a new |
| 604 | configuration or even a new version of haproxy without breaking existing |
| 605 | connections. If the load succeeds, then simply send a SIGUSR1 which will make |
| 606 | the previous proxies exit immediately once their sessions are closed ; and if |
| 607 | the load fails, then simply send a SIGTTIN to restore the service immediately. |
| 608 | Please note that the 'grace' parameter is ignored for SIGTTOU, as well as for |
| 609 | SIGUSR1 when the process was in the pause mode. Please also note that it would |
| 610 | be useful to save the pidfile before starting a new instance. |
| 611 | |
willy tarreau | 34f4530 | 2006-04-15 21:37:14 +0200 | [diff] [blame] | 612 | This mechanism fully exploited since 1.2.11 with the '-st' and '-sf' options |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 613 | (see below). |
| 614 | |
| 615 | 2.4.1) Hot reconfiguration |
| 616 | -------------------------- |
| 617 | The '-st' and '-sf' command line options are used to inform previously running |
| 618 | processes that a configuration is being reloaded. They will receive the SIGTTOU |
| 619 | signal to ask them to temporarily stop listening to the ports so that the new |
| 620 | process can grab them. If anything wrong happens, the new process will send |
| 621 | them a SIGTTIN to tell them to re-listen to the ports and continue their normal |
| 622 | work. Otherwise, it will either ask them to finish (-sf) their work then softly |
| 623 | exit, or immediately terminate (-st), breaking existing sessions. A typical use |
| 624 | of this allows a configuration reload without service interruption : |
| 625 | |
| 626 | # haproxy -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid) |
| 627 | |
willy tarreau | 22739ef | 2006-01-20 20:43:32 +0100 | [diff] [blame] | 628 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 629 | 2.5) Connections expiration time |
| 630 | -------------------------------- |
| 631 | It is possible (and recommended) to configure several time-outs on TCP |
Jamie Gloudon | 801a0a3 | 2012-08-25 00:18:33 -0400 | [diff] [blame] | 632 | connections. Three independent timers are adjustable with values specified |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 633 | in milliseconds. A session will be terminated if either one of these timers |
| 634 | expire. |
| 635 | |
| 636 | - the time we accept to wait for data from the client, or for the client to |
| 637 | accept data : 'clitimeout' : |
| 638 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 639 | # client time-out set to 2mn30. |
| 640 | clitimeout 150000 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 641 | |
| 642 | - the time we accept to wait for data from the server, or for the server to |
| 643 | accept data : 'srvtimeout' : |
| 644 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 645 | # server time-out set to 30s. |
| 646 | srvtimeout 30000 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 647 | |
| 648 | - the time we accept to wait for a connection to establish on a server : |
| 649 | 'contimeout' : |
| 650 | |
| 651 | # we give up if the connection does not complete within 4 seconds |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 652 | contimeout 4000 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 653 | |
| 654 | Notes : |
| 655 | ------- |
| 656 | - 'contimeout' and 'srvtimeout' have no sense on 'health' mode servers ; |
| 657 | - under high loads, or with a saturated or defective network, it's possible |
| 658 | that some packets get lost. Since the first TCP retransmit only happens |
| 659 | after 3 seconds, a time-out equal to, or lower than 3 seconds cannot |
| 660 | compensate for a packet loss. A 4 seconds time-out seems a reasonable |
| 661 | minimum which will considerably reduce connection failures. |
Willy Tarreau | befdff1 | 2007-12-02 22:27:38 +0100 | [diff] [blame] | 662 | - starting with version 1.3.14, it is possible to specify timeouts in |
| 663 | arbitrary time units among { us, ms, s, m, h, d }. For this, the integer |
| 664 | value just has to be suffixed with the unit. |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 665 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 666 | 2.6) Attempts to reconnect |
| 667 | -------------------------- |
| 668 | After a connection failure to a server, it is possible to retry, potentially |
| 669 | on another server. This is useful if health-checks are too rare and you don't |
| 670 | want the clients to see the failures. The number of attempts to reconnect is |
| 671 | set by the 'retries' paramter. |
| 672 | |
| 673 | Example : |
| 674 | --------- |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 675 | # we can retry 3 times max after a failure |
| 676 | retries 3 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 677 | |
willy tarreau | 34f4530 | 2006-04-15 21:37:14 +0200 | [diff] [blame] | 678 | Please note that the reconnection attempt may lead to getting the connection |
| 679 | sent to a new server if the original one died between connection attempts. |
| 680 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 681 | |
| 682 | 2.7) Address of the dispatch server (deprecated) |
| 683 | ------------------------------------------------ |
| 684 | The server which will be sent all new connections is defined by the 'dispatch' |
| 685 | parameter, in the form <address>:<port>. It generally is dedicated to unknown |
| 686 | connections and will assign them a cookie, in case of HTTP persistence mode, |
| 687 | or simply is a single server in case of generic TCP proxy. This old mode is only |
| 688 | provided for backwards compatibility, but doesn't allow to check remote servers |
| 689 | state, and has a rather limited usage. All new setups should switch to 'balance' |
| 690 | mode. The principle of the dispatcher is to be able to perform the load |
| 691 | balancing itself, but work only on new clients so that the server doesn't need |
| 692 | to be a big machine. |
| 693 | |
| 694 | Example : |
| 695 | --------- |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 696 | # all new connections go there |
| 697 | dispatch 192.168.1.2:80 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 698 | |
| 699 | Note : |
| 700 | ------ |
| 701 | This parameter has no sense for 'health' servers, and is incompatible with |
| 702 | 'balance' mode. |
| 703 | |
| 704 | |
| 705 | 2.8) Outgoing source address |
| 706 | ---------------------------- |
| 707 | It is often necessary to bind to a particular address when connecting to some |
| 708 | remote hosts. This is done via the 'source' parameter which is a per-proxy |
| 709 | parameter. A newer version may allow to fix different sources to reach different |
| 710 | servers. The syntax is 'source <address>[:<port>]', where <address> is a valid |
| 711 | local address (or '0.0.0.0' or '*' or empty to let the system choose), and |
| 712 | <port> is an optional parameter allowing the user to force the source port for |
| 713 | very specific needs. If the port is not specified or is '0', the system will |
| 714 | choose a free port. Note that as of version 1.1.18, the servers health checks |
| 715 | are also performed from the same source. |
| 716 | |
| 717 | Examples : |
| 718 | ---------- |
| 719 | listen http_proxy *:80 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 720 | # all connections take 192.168.1.200 as source address |
| 721 | source 192.168.1.200:0 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 722 | |
| 723 | listen rlogin_proxy *:513 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 724 | # use address 192.168.1.200 and the reserved port 900 (needs to be root) |
| 725 | source 192.168.1.200:900 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 726 | |
| 727 | |
| 728 | 2.9) Setting the cookie name |
| 729 | ---------------------------- |
| 730 | In HTTP mode, it is possible to look for a particular cookie which will contain |
| 731 | a server identifier which should handle the connection. The cookie name is set |
| 732 | via the 'cookie' parameter. |
| 733 | |
| 734 | Example : |
| 735 | --------- |
| 736 | listen http_proxy :80 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 737 | mode http |
| 738 | cookie SERVERID |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 739 | |
| 740 | It is possible to change the cookie behaviour to get a smarter persistence, |
| 741 | depending on applications. It is notably possible to delete or modify a cookie |
| 742 | emitted by a server, insert a cookie identifying the server in an HTTP response |
| 743 | and even add a header to tell upstream caches not to cache this response. |
| 744 | |
| 745 | Examples : |
| 746 | ---------- |
| 747 | |
| 748 | To remove the cookie for direct accesses (ie when the server matches the one |
| 749 | which was specified in the client cookie) : |
| 750 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 751 | cookie SERVERID indirect |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 752 | |
| 753 | To replace the cookie value with the one assigned to the server if any (no |
| 754 | cookie will be created if the server does not provide one, nor if the |
| 755 | configuration does not provide one). This lets the application put the cookie |
| 756 | exactly on certain pages (eg: successful authentication) : |
| 757 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 758 | cookie SERVERID rewrite |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 759 | |
| 760 | To create a new cookie and assign the server identifier to it (in this case, all |
| 761 | servers should be associated with a valid cookie, since no cookie will simply |
| 762 | delete the cookie from the client's browser) : |
| 763 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 764 | cookie SERVERID insert |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 765 | |
willy tarreau | 0174f31 | 2005-12-18 01:02:42 +0100 | [diff] [blame] | 766 | To reuse an existing application cookie and prefix it with the server's |
| 767 | identifier, and remove it in the request, use the 'prefix' option. This allows |
| 768 | to insert a haproxy in front of an application without risking to break clients |
| 769 | which does not support more than one cookie : |
| 770 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 771 | cookie JSESSIONID prefix |
willy tarreau | 0174f31 | 2005-12-18 01:02:42 +0100 | [diff] [blame] | 772 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 773 | To insert a cookie and ensure that no upstream cache will store it, add the |
| 774 | 'nocache' option : |
| 775 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 776 | cookie SERVERID insert nocache |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 777 | |
| 778 | To insert a cookie only after a POST request, add 'postonly' after 'insert'. |
| 779 | This has the advantage that there's no risk of caching, and that all pages |
| 780 | seen before the POST one can still be cached : |
| 781 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 782 | cookie SERVERID insert postonly |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 783 | |
| 784 | Notes : |
| 785 | ----------- |
| 786 | - it is possible to combine 'insert' with 'indirect' or 'rewrite' to adapt to |
| 787 | applications which already generate the cookie with an invalid content. |
| 788 | |
| 789 | - in the case where 'insert' and 'indirect' are both specified, the cookie is |
willy tarreau | 0174f31 | 2005-12-18 01:02:42 +0100 | [diff] [blame] | 790 | never transmitted to the server, since it wouldn't understand it. This is the |
| 791 | most application-transparent mode. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 792 | |
| 793 | - it is particularly recommended to use 'nocache' in 'insert' mode if any |
| 794 | upstream HTTP/1.0 cache is susceptible to cache the result, because this may |
| 795 | lead to many clients going to the same server, or even worse, some clients |
| 796 | having their server changed while retrieving a page from the cache. |
| 797 | |
willy tarreau | 0174f31 | 2005-12-18 01:02:42 +0100 | [diff] [blame] | 798 | - the 'prefix' mode normally does not need 'indirect', 'nocache', nor |
| 799 | 'postonly', because just as in the 'rewrite' mode, it relies on the |
| 800 | application to know when a cookie can be emitted. However, since it has to |
| 801 | fix the cookie name in every subsequent requests, you must ensure that the |
| 802 | proxy will be used without any "HTTP keep-alive". Use option "httpclose" if |
| 803 | unsure. |
| 804 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 805 | - when the application is well known and controlled, the best method is to |
| 806 | only add the persistence cookie on a POST form because it's up to the |
willy tarreau | 0174f31 | 2005-12-18 01:02:42 +0100 | [diff] [blame] | 807 | application to select which page it wants the upstream servers to cache. In |
| 808 | this case, you would use 'insert postonly indirect'. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 809 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 810 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 811 | 2.10) Associating a cookie value with a server |
| 812 | ---------------------------------------------- |
| 813 | In HTTP mode, it's possible to associate a cookie value to each server. This |
| 814 | was initially used in combination with 'dispatch' mode to handle direct accesses |
| 815 | but it is now the standard way of doing the load balancing. The syntax is : |
| 816 | |
| 817 | server <identifier> <address>:<port> cookie <value> |
| 818 | |
| 819 | - <identifier> is any name which can be used to identify the server in the logs. |
| 820 | - <address>:<port> specifies where the server is bound. |
| 821 | - <value> is the value to put in or to read from the cookie. |
| 822 | |
| 823 | Example : the 'SERVERID' cookie can be either 'server01' or 'server02' |
| 824 | --------- |
| 825 | listen http_proxy :80 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 826 | mode http |
| 827 | cookie SERVERID |
| 828 | dispatch 192.168.1.100:80 |
| 829 | server web1 192.168.1.1:80 cookie server01 |
| 830 | server web2 192.168.1.2:80 cookie server02 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 831 | |
| 832 | Warning : the syntax has changed since version 1.0 ! |
| 833 | --------- |
| 834 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 835 | |
willy tarreau | 598da41 | 2005-12-18 01:07:29 +0100 | [diff] [blame] | 836 | 2.11) Application Cookies |
| 837 | ------------------------- |
| 838 | Since 1.2.4 it is possible to catch the cookie that comes from an |
| 839 | application server in order to apply "application session stickyness". |
| 840 | The server's response is searched for 'appsession' cookie, the first |
| 841 | 'len' bytes are used for matching and it is stored for a period of |
| 842 | 'timeout'. |
| 843 | The syntax is: |
| 844 | |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 845 | appsession <session_cookie> len <match_length> timeout <holdtime> |
willy tarreau | 598da41 | 2005-12-18 01:07:29 +0100 | [diff] [blame] | 846 | |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 847 | - <session_cookie> is the cookie, the server uses for it's session-handling |
| 848 | - <match_length> how many bytes/characters should be used for matching equal |
willy tarreau | 598da41 | 2005-12-18 01:07:29 +0100 | [diff] [blame] | 849 | sessions |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 850 | - <holdtime> after this inactivaty time, in ms, the cookie will be deleted |
willy tarreau | 598da41 | 2005-12-18 01:07:29 +0100 | [diff] [blame] | 851 | from the sessionstore |
Willy Tarreau | befdff1 | 2007-12-02 22:27:38 +0100 | [diff] [blame] | 852 | - starting with version 1.3.14, it is possible to specify timeouts in |
| 853 | arbitrary time units among { us, ms, s, m, h, d }. For this, the integer |
| 854 | value just has to be prefixed with the unit. |
willy tarreau | 598da41 | 2005-12-18 01:07:29 +0100 | [diff] [blame] | 855 | |
| 856 | The appsession is only per 'listen' section possible. |
| 857 | |
| 858 | Example : |
| 859 | --------- |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 860 | listen http_lb1 192.168.3.4:80 |
| 861 | mode http |
| 862 | capture request header Cookie len 200 |
| 863 | # Havind a ServerID cookie on the client allows him to reach |
| 864 | # the right server even after expiration of the appsession. |
| 865 | cookie ServerID insert nocache indirect |
| 866 | # Will memorize 52 bytes of the cookie 'JSESSIONID' and keep them |
| 867 | # for 3 hours. It will match it in the cookie and the URL field. |
Willy Tarreau | befdff1 | 2007-12-02 22:27:38 +0100 | [diff] [blame] | 868 | appsession JSESSIONID len 52 timeout 3h |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 869 | server first1 10.3.9.2:10805 check inter 3000 cookie first |
| 870 | server secon1 10.3.9.3:10805 check inter 3000 cookie secon |
| 871 | server first1 10.3.9.4:10805 check inter 3000 cookie first |
| 872 | server secon2 10.3.9.5:10805 check inter 3000 cookie secon |
| 873 | option httpchk GET /test.jsp |
willy tarreau | 598da41 | 2005-12-18 01:07:29 +0100 | [diff] [blame] | 874 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 875 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 876 | 3) Autonomous load balancer |
| 877 | =========================== |
| 878 | |
| 879 | The proxy can perform the load-balancing itself, both in TCP and in HTTP modes. |
| 880 | This is the most interesting mode which obsoletes the old 'dispatch' mode |
| 881 | described above. It has advantages such as server health monitoring, multiple |
| 882 | port binding and port mapping. To use this mode, the 'balance' keyword is used, |
willy tarreau | 34f4530 | 2006-04-15 21:37:14 +0200 | [diff] [blame] | 883 | followed by the selected algorithm. Up to version 1.2.11, only 'roundrobin' was |
| 884 | available, which is also the default value if unspecified. Starting with |
Willy Tarreau | 2fcb500 | 2007-05-08 13:35:26 +0200 | [diff] [blame] | 885 | version 1.2.12, a new 'source' keyword appeared. A new 'uri' keyword was added |
| 886 | in version 1.3.10. In this mode, there will be no dispatch address, but the |
| 887 | proxy needs at least one server. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 888 | |
| 889 | Example : same as the last one, with internal load balancer |
| 890 | --------- |
| 891 | |
| 892 | listen http_proxy :80 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 893 | mode http |
| 894 | cookie SERVERID |
| 895 | balance roundrobin |
| 896 | server web1 192.168.1.1:80 cookie server01 |
| 897 | server web2 192.168.1.2:80 cookie server02 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 898 | |
| 899 | |
| 900 | Since version 1.1.22, it is possible to automatically determine on which port |
| 901 | the server will get the connection, depending on the port the client connected |
| 902 | to. Indeed, there now are 4 possible combinations for the server's <port> field: |
| 903 | |
| 904 | - unspecified or '0' : |
| 905 | the connection will be sent to the same port as the one on which the proxy |
| 906 | received the client connection itself. |
| 907 | |
| 908 | - numerical value (the only one supported in versions earlier than 1.1.22) : |
| 909 | the connection will always be sent to the specified port. |
| 910 | |
| 911 | - '+' followed by a numerical value : |
| 912 | the connection will be sent to the same port as the one on which the proxy |
| 913 | received the connection, plus this value. |
| 914 | |
| 915 | - '-' followed by a numerical value : |
| 916 | the connection will be sent to the same port as the one on which the proxy |
| 917 | received the connection, minus this value. |
| 918 | |
| 919 | Examples : |
| 920 | ---------- |
| 921 | |
| 922 | # same as previous example |
| 923 | |
| 924 | listen http_proxy :80 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 925 | mode http |
| 926 | cookie SERVERID |
| 927 | balance roundrobin |
| 928 | server web1 192.168.1.1 cookie server01 |
| 929 | server web2 192.168.1.2 cookie server02 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 930 | |
| 931 | # simultaneous relaying of ports 80, 81 and 8080-8089 |
| 932 | |
| 933 | listen http_proxy :80,:81,:8080-8089 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 934 | mode http |
| 935 | cookie SERVERID |
| 936 | balance roundrobin |
| 937 | server web1 192.168.1.1 cookie server01 |
| 938 | server web2 192.168.1.2 cookie server02 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 939 | |
| 940 | # relaying of TCP ports 25, 389 and 663 to ports 1025, 1389 and 1663 |
| 941 | |
| 942 | listen http_proxy :25,:389,:663 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 943 | mode tcp |
| 944 | balance roundrobin |
| 945 | server srv1 192.168.1.1:+1000 |
| 946 | server srv2 192.168.1.2:+1000 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 947 | |
willy tarreau | 34f4530 | 2006-04-15 21:37:14 +0200 | [diff] [blame] | 948 | As previously stated, version 1.2.12 brought the 'source' keyword. When this |
| 949 | keyword is used, the client's IP address is hashed and evenly distributed among |
| 950 | the available servers so that a same source IP will always go to the same |
| 951 | server as long as there are no change in the number of available servers. This |
| 952 | can be used for instance to bind HTTP and HTTPS to the same server. It can also |
| 953 | be used to improve stickyness when one part of the client population does not |
| 954 | accept cookies. In this case, only those ones will be perturbated should a |
| 955 | server fail. |
| 956 | |
| 957 | NOTE: It is important to consider the fact that many clients surf the net |
| 958 | through proxy farms which assign different IP addresses for each |
| 959 | request. Others use dialup connections with a different IP at each |
| 960 | connection. Thus, the 'source' parameter should be used with extreme |
| 961 | care. |
| 962 | |
| 963 | Examples : |
| 964 | ---------- |
| 965 | |
| 966 | # make a same IP go to the same server whatever the service |
| 967 | |
| 968 | listen http_proxy |
| 969 | bind :80,:443 |
| 970 | mode http |
| 971 | balance source |
| 972 | server web1 192.168.1.1 |
| 973 | server web2 192.168.1.2 |
| 974 | |
| 975 | # try to improve client-server binding by using both source IP and cookie : |
| 976 | |
| 977 | listen http_proxy :80 |
| 978 | mode http |
| 979 | cookie SERVERID |
| 980 | balance source |
| 981 | server web1 192.168.1.1 cookie server01 |
| 982 | server web2 192.168.1.2 cookie server02 |
| 983 | |
Willy Tarreau | 2fcb500 | 2007-05-08 13:35:26 +0200 | [diff] [blame] | 984 | As indicated above, the 'uri' keyword was introduced in version 1.3.10. It is |
| 985 | useful when load-balancing between reverse proxy-caches, because it will hash |
| 986 | the URI and use the hash result to select a server, thus optimizing the hit |
| 987 | rate on the caches, because the same URI will always reach the same cache. This |
| 988 | keyword is only allowed in HTTP mode. |
| 989 | |
| 990 | Example : |
| 991 | --------- |
| 992 | |
| 993 | # Always send a given URI to the same server |
| 994 | |
| 995 | listen http_proxy |
| 996 | bind :3128 |
| 997 | mode http |
| 998 | balance uri |
| 999 | server squid1 192.168.1.1 |
| 1000 | server squid2 192.168.1.2 |
| 1001 | |
Willy Tarreau | 0173280 | 2007-11-01 22:48:15 +0100 | [diff] [blame] | 1002 | Version 1.3.14 introduced the "balance url_param" method. It consists in |
| 1003 | relying on a parameter passed in the URL to perform a hash. This is mostly |
| 1004 | useful for applications which do not have strict persistence requirements, |
| 1005 | but for which it still provides a performance boost due to local caching. |
| 1006 | Some of these applications may not be able to use a cookie for whatever reason, |
| 1007 | but may be able to look for a parameter passed in the URL. If the parameter is |
| 1008 | missing from the URL, then the 'round robin' method applies. |
| 1009 | |
matt.farnsworth@nokia.com | 1c2ab96 | 2008-04-14 20:47:37 +0200 | [diff] [blame] | 1010 | A modifier may be added to specify that parameters in POST requests may be |
| 1011 | found in the messsage body if the URL lacks a '?' separator character. |
| 1012 | A wait limit may also be applied, if no limit is requested then |
| 1013 | the default value is 48 octets, the minimum is 3. HAProxy may wait, until 48 |
| 1014 | octets are received. If Content-Length is missing, or zero it need not |
| 1015 | wait for more data then the client promissed to send. When Content-Length is |
| 1016 | present, and more than <max_wait>; then waiting is limited and it is assumed this |
| 1017 | will be enough data to search for the presence of a parameter. If |
| 1018 | Transfer-Encoding: chunked is used (unlikely), then the length of the first chunk |
| 1019 | is the maximum number of bytes to wait for. |
| 1020 | |
| 1021 | balance url_param <param> [check_post [<max_wait>]] |
| 1022 | |
| 1023 | Caveats for using the check_post extension: |
| 1024 | |
| 1025 | - all POST requests are eligable for consideration, because there is |
| 1026 | no way to determine if the parameters will be found in the body or |
| 1027 | entity which may contain binary data. Therefore another method may be |
| 1028 | required to restrict consideration of POST requests that have no URL |
| 1029 | parameters in the body. (see acl reqideny http_end) |
| 1030 | |
| 1031 | Limitations on inspecting the entity body of a POST: |
| 1032 | |
| 1033 | - Content-Encoding is not supported, the parameter search will probably fail; |
| 1034 | and load balancing will fall back to Round Robin. |
| 1035 | |
| 1036 | - Expect: 100-continue is not supported, load balancing will fall back to |
| 1037 | Round Robin. |
| 1038 | |
| 1039 | - Transfer-Encoding(RFC2616 3.6.1) is only supported in the first chunk. If |
| 1040 | the entire parameter value is not present in the first chunk, the selection |
| 1041 | of server is undefined (actually, defined by how little actually appeared in |
| 1042 | the first chunk). |
| 1043 | |
| 1044 | - This feature does not support generation of a 100, 411 or 501 response. |
| 1045 | |
| 1046 | - In some cases, requesting check_post MAY attempt to scan the entire contents |
| 1047 | of a message body. Scaning normally terminates when linear white space or |
| 1048 | control characters are found, indicating the end of what might be a URL parameter |
| 1049 | list. This is probably not a concern with SGML type message bodies. |
| 1050 | |
| 1051 | |
Willy Tarreau | 0173280 | 2007-11-01 22:48:15 +0100 | [diff] [blame] | 1052 | Example : |
| 1053 | --------- |
| 1054 | |
| 1055 | # Hash the "basket_id" argument from the URL to determine the server |
| 1056 | |
| 1057 | listen http_proxy |
| 1058 | bind :3128 |
| 1059 | mode http |
| 1060 | balance url_param basket_id |
| 1061 | server ebiz1 192.168.1.1 |
| 1062 | server ebiz2 192.168.1.2 |
| 1063 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1064 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 1065 | 3.1) Server monitoring |
| 1066 | ---------------------- |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1067 | It is possible to check the servers status by trying to establish TCP |
| 1068 | connections or even sending HTTP requests to them. A server which fails to |
| 1069 | reply to health checks as expected will not be used by the load balancing |
| 1070 | algorithms. To enable monitoring, add the 'check' keyword on a server line. |
| 1071 | It is possible to specify the interval between tests (in milliseconds) with |
| 1072 | the 'inter' parameter, the number of failures supported before declaring that |
| 1073 | the server has fallen down with the 'fall' parameter, and the number of valid |
| 1074 | checks needed for the server to fully get up with the 'rise' parameter. Since |
| 1075 | version 1.1.22, it is also possible to send checks to a different port |
| 1076 | (mandatory when none is specified) with the 'port' parameter. The default |
| 1077 | values are the following ones : |
| 1078 | |
| 1079 | - inter : 2000 |
| 1080 | - rise : 2 |
| 1081 | - fall : 3 |
| 1082 | - port : default server port |
Willy Tarreau | 2ea3abb | 2007-03-25 16:45:16 +0200 | [diff] [blame] | 1083 | - addr : specific address for the test (default = address server) |
| 1084 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1085 | The default mode consists in establishing TCP connections only. But in certain |
| 1086 | types of application failures, it is often that the server continues to accept |
| 1087 | connections because the system does it itself while the application is running |
| 1088 | an endless loop, or is completely stuck. So in version 1.1.16 were introduced |
| 1089 | HTTP health checks which only performed simple lightweight requests and analysed |
| 1090 | the response. Now, as of version 1.1.23, it is possible to change the HTTP |
| 1091 | method, the URI, and the HTTP version string (which even allows to send headers |
| 1092 | with a dirty trick). To enable HTTP health-checks, use 'option httpchk'. |
| 1093 | |
| 1094 | By default, requests use the 'OPTIONS' method because it's very light and easy |
| 1095 | to filter from logs, and does it on '/'. Only HTTP responses 2xx and 3xx are |
| 1096 | considered valid ones, and only if they come before the time to send a new |
| 1097 | request is reached ('inter' parameter). If some servers block this type of |
| 1098 | request, 3 other forms help to forge a request : |
| 1099 | |
| 1100 | - option httpchk -> OPTIONS / HTTP/1.0 |
| 1101 | - option httpchk URI -> OPTIONS <URI> HTTP/1.0 |
| 1102 | - option httpchk METH URI -> <METH> <URI> HTTP/1.0 |
| 1103 | - option httpchk METH URI VER -> <METH> <URI> <VER> |
| 1104 | |
Willy Tarreau | f3c6920 | 2006-07-09 16:42:34 +0200 | [diff] [blame] | 1105 | Some people are using HAProxy to relay various TCP-based protocols such as |
| 1106 | HTTPS, SMTP or LDAP, with the most common one being HTTPS. One problem commonly |
| 1107 | encountered in data centers is the need to forward the traffic to far remote |
| 1108 | servers while providing server fail-over. Often, TCP-only checks are not enough |
| 1109 | because intermediate firewalls, load balancers or proxies might acknowledge the |
| 1110 | connection before it reaches the real server. The only solution to this problem |
| 1111 | is to send application-level health checks. Since the demand for HTTPS checks |
| 1112 | is high, it has been implemented in 1.2.15 based on SSLv3 Client Hello packets. |
| 1113 | To enable it, use 'option ssl-hello-chk'. It will send SSL CLIENT HELLO packets |
| 1114 | to the servers, announcing support for most common cipher suites. If the server |
| 1115 | responds what looks like a SERVER HELLO or an ALERT (refuses the ciphers) then |
| 1116 | the response is considered as valid. Note that Apache does not generate a log |
| 1117 | when it receives only an HELLO message, which makes this type of message |
| 1118 | perfectly suit this need. |
| 1119 | |
Willy Tarreau | 2367790 | 2007-05-08 23:50:35 +0200 | [diff] [blame] | 1120 | Version 1.3.10 introduced the SMTP health check. By default, it sends |
| 1121 | "HELO localhost" to the servers, and waits for the 250 message. Note that it |
| 1122 | can also send a specific request : |
| 1123 | |
| 1124 | - option smtpchk -> sends "HELO localhost" |
| 1125 | - option smtpchk EHLO mail.mydomain.com -> sends this ESMTP greeting |
| 1126 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1127 | See examples below. |
| 1128 | |
| 1129 | Since version 1.1.17, it is possible to specify backup servers. These servers |
| 1130 | are only sollicited when no other server is available. This may only be useful |
| 1131 | to serve a maintenance page, or define one active and one backup server (seldom |
| 1132 | used in TCP mode). To make a server a backup one, simply add the 'backup' option |
| 1133 | on its line. These servers also support cookies, so if a cookie is specified for |
| 1134 | a backup server, clients assigned to this server will stick to it even when the |
| 1135 | other ones come back. Conversely, if no cookie is assigned to such a server, |
| 1136 | the clients will get their cookies removed (empty cookie = removal), and will |
| 1137 | be balanced against other servers once they come back. Please note that there |
Willy TARREAU | 3481c46 | 2006-03-01 22:37:57 +0100 | [diff] [blame] | 1138 | is no load-balancing among backup servers by default. If there are several |
| 1139 | backup servers, the second one will only be used when the first one dies, and |
| 1140 | so on. To force load-balancing between backup servers, specify the 'allbackups' |
| 1141 | option. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1142 | |
Willy Tarreau | 2ea3abb | 2007-03-25 16:45:16 +0200 | [diff] [blame] | 1143 | Since version 1.1.22, it is possible to send health checks to a different port |
| 1144 | than the service. It is mainly needed in setups where the server does not have |
| 1145 | any predefined port, for instance when the port is deduced from the listening |
| 1146 | port. For this, use the 'port' parameter followed by the port number which must |
| 1147 | respond to health checks. It is also possible to send health checks to a |
| 1148 | different address than the service. It makes it easier to use a dedicated check |
| 1149 | daemon on the servers, for instance, check return contents and stop several |
| 1150 | farms at once in the event of an error anywhere. |
| 1151 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1152 | Since version 1.1.17, it is also possible to visually check the status of all |
| 1153 | servers at once. For this, you just have to send a SIGHUP signal to the proxy. |
| 1154 | The servers status will be dumped into the logs at the 'notice' level, as well |
| 1155 | as on <stderr> if not closed. For this reason, it's always a good idea to have |
| 1156 | one local log server at the 'notice' level. |
| 1157 | |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 1158 | Since version 1.1.28 and 1.2.1, if an instance loses all its servers, an |
willy tarreau | 0174f31 | 2005-12-18 01:02:42 +0100 | [diff] [blame] | 1159 | emergency message will be sent in the logs to inform the administator that an |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 1160 | immediate action must be taken. |
| 1161 | |
willy tarreau | 0174f31 | 2005-12-18 01:02:42 +0100 | [diff] [blame] | 1162 | Since version 1.1.30 and 1.2.3, several servers can share the same cookie |
| 1163 | value. This is particularly useful in backup mode, to select alternate paths |
| 1164 | for a given server for example, to provide soft-stop, or to direct the clients |
| 1165 | to a temporary page during an application restart. The principle is that when |
| 1166 | a server is dead, the proxy will first look for another server which shares the |
| 1167 | same cookie value for every client which presents the cookie. If there is no |
| 1168 | standard server for this cookie, it will then look for a backup server which |
| 1169 | shares the same name. Please consult the architecture guide for more information. |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 1170 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1171 | Examples : |
| 1172 | ---------- |
| 1173 | # same setup as in paragraph 3) with TCP monitoring |
| 1174 | listen http_proxy 0.0.0.0:80 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1175 | mode http |
| 1176 | cookie SERVERID |
| 1177 | balance roundrobin |
| 1178 | server web1 192.168.1.1:80 cookie server01 check |
| 1179 | server web2 192.168.1.2:80 cookie server02 check inter 500 rise 1 fall 2 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1180 | |
| 1181 | # same with HTTP monitoring via 'OPTIONS / HTTP/1.0' |
| 1182 | listen http_proxy 0.0.0.0:80 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1183 | mode http |
| 1184 | cookie SERVERID |
| 1185 | balance roundrobin |
| 1186 | option httpchk |
| 1187 | server web1 192.168.1.1:80 cookie server01 check |
| 1188 | server web2 192.168.1.2:80 cookie server02 check inter 500 rise 1 fall 2 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1189 | |
| 1190 | # same with HTTP monitoring via 'OPTIONS /index.html HTTP/1.0' |
| 1191 | listen http_proxy 0.0.0.0:80 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1192 | mode http |
| 1193 | cookie SERVERID |
| 1194 | balance roundrobin |
| 1195 | option httpchk /index.html |
| 1196 | server web1 192.168.1.1:80 cookie server01 check |
| 1197 | server web2 192.168.1.2:80 cookie server02 check inter 500 rise 1 fall 2 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1198 | |
| 1199 | # same with HTTP monitoring via 'HEAD /index.jsp? HTTP/1.1\r\nHost: www' |
| 1200 | listen http_proxy 0.0.0.0:80 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1201 | mode http |
| 1202 | cookie SERVERID |
| 1203 | balance roundrobin |
| 1204 | option httpchk HEAD /index.jsp? HTTP/1.1\r\nHost:\ www |
| 1205 | server web1 192.168.1.1:80 cookie server01 check |
| 1206 | server web2 192.168.1.2:80 cookie server02 check inter 500 rise 1 fall 2 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1207 | |
willy tarreau | 0174f31 | 2005-12-18 01:02:42 +0100 | [diff] [blame] | 1208 | # Load-balancing with 'prefixed cookie' persistence, and soft-stop using an |
| 1209 | # alternate port 81 on the server for health-checks. |
| 1210 | listen http_proxy 0.0.0.0:80 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1211 | mode http |
| 1212 | cookie JSESSIONID prefix |
| 1213 | balance roundrobin |
| 1214 | option httpchk HEAD /index.jsp? HTTP/1.1\r\nHost:\ www |
| 1215 | server web1-norm 192.168.1.1:80 cookie s1 check port 81 |
| 1216 | server web2-norm 192.168.1.2:80 cookie s2 check port 81 |
| 1217 | server web1-stop 192.168.1.1:80 cookie s1 check port 80 backup |
| 1218 | server web2-stop 192.168.1.2:80 cookie s2 check port 80 backup |
willy tarreau | 0174f31 | 2005-12-18 01:02:42 +0100 | [diff] [blame] | 1219 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1220 | # automatic insertion of a cookie in the server's response, and automatic |
| 1221 | # deletion of the cookie in the client request, while asking upstream caches |
| 1222 | # not to cache replies. |
| 1223 | listen web_appl 0.0.0.0:80 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1224 | mode http |
| 1225 | cookie SERVERID insert nocache indirect |
| 1226 | balance roundrobin |
| 1227 | server web1 192.168.1.1:80 cookie server01 check |
| 1228 | server web2 192.168.1.2:80 cookie server02 check |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1229 | |
| 1230 | # same with off-site application backup and local error pages server |
| 1231 | listen web_appl 0.0.0.0:80 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1232 | mode http |
| 1233 | cookie SERVERID insert nocache indirect |
| 1234 | balance roundrobin |
| 1235 | server web1 192.168.1.1:80 cookie server01 check |
| 1236 | server web2 192.168.1.2:80 cookie server02 check |
| 1237 | server web-backup 192.168.2.1:80 cookie server03 check backup |
| 1238 | server web-excuse 192.168.3.1:80 check backup |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1239 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1240 | # SMTP+TLS relaying with health-checks and backup servers |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1241 | |
| 1242 | listen http_proxy :25,:587 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1243 | mode tcp |
| 1244 | balance roundrobin |
| 1245 | server srv1 192.168.1.1 check port 25 inter 30000 rise 1 fall 2 |
| 1246 | server srv2 192.168.1.2 backup |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1247 | |
Willy Tarreau | f3c6920 | 2006-07-09 16:42:34 +0200 | [diff] [blame] | 1248 | # HTTPS relaying with health-checks and backup servers |
| 1249 | |
| 1250 | listen http_proxy :443 |
| 1251 | mode tcp |
| 1252 | option ssl-hello-chk |
| 1253 | balance roundrobin |
| 1254 | server srv1 192.168.1.1 check inter 30000 rise 1 fall 2 |
| 1255 | server srv2 192.168.1.2 backup |
| 1256 | |
Willy TARREAU | 3481c46 | 2006-03-01 22:37:57 +0100 | [diff] [blame] | 1257 | # Load-balancing using a backup pool (requires haproxy 1.2.9) |
| 1258 | listen http_proxy 0.0.0.0:80 |
| 1259 | mode http |
| 1260 | balance roundrobin |
| 1261 | option httpchk |
| 1262 | server inst1 192.168.1.1:80 cookie s1 check |
| 1263 | server inst2 192.168.1.2:80 cookie s2 check |
| 1264 | server inst3 192.168.1.3:80 cookie s3 check |
| 1265 | server back1 192.168.1.10:80 check backup |
| 1266 | server back2 192.168.1.11:80 check backup |
| 1267 | option allbackups # all backups will be used |
| 1268 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1269 | |
| 1270 | 3.2) Redistribute connections in case of failure |
| 1271 | ------------------------------------------------ |
| 1272 | In HTTP mode, if a server designated by a cookie does not respond, the clients |
| 1273 | may definitely stick to it because they cannot flush the cookie, so they will |
| 1274 | not be able to access the service anymore. Specifying 'redispatch' will allow |
| 1275 | the proxy to break their persistence and redistribute them to working servers. |
| 1276 | |
| 1277 | Example : |
| 1278 | --------- |
| 1279 | listen http_proxy 0.0.0.0:80 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1280 | mode http |
| 1281 | cookie SERVERID |
| 1282 | dispatch 192.168.1.100:80 |
| 1283 | server web1 192.168.1.1:80 cookie server01 |
| 1284 | server web2 192.168.1.2:80 cookie server02 |
| 1285 | redispatch # send back to dispatch in case of connection failure |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1286 | |
| 1287 | Up to, and including version 1.1.16, this parameter only applied to connection |
| 1288 | failures. Since version 1.1.17, it also applies to servers which have been |
| 1289 | detected as failed by the health check mechanism. Indeed, a server may be broken |
| 1290 | but still accepting connections, which would not solve every case. But it is |
| 1291 | possible to conserve the old behaviour, that is, make a client insist on trying |
| 1292 | to connect to a server even if it is said to be down, by setting the 'persist' |
| 1293 | option : |
| 1294 | |
| 1295 | listen http_proxy 0.0.0.0:80 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1296 | mode http |
| 1297 | option persist |
| 1298 | cookie SERVERID |
| 1299 | dispatch 192.168.1.100:80 |
| 1300 | server web1 192.168.1.1:80 cookie server01 |
| 1301 | server web2 192.168.1.2:80 cookie server02 |
| 1302 | redispatch # send back to dispatch in case of connection failure |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1303 | |
| 1304 | |
willy tarreau | 34f4530 | 2006-04-15 21:37:14 +0200 | [diff] [blame] | 1305 | 3.3) Assigning different weights to servers |
| 1306 | ------------------------------------------- |
| 1307 | Sometimes you will need to bring new servers to increase your server farm's |
| 1308 | capacity, but the new server will be either smaller (emergency use of anything |
| 1309 | that fits) or bigger (when investing in new hardware). For this reason, it |
| 1310 | might be wise to be able to send more clients to biggest servers. Till version |
| 1311 | 1.2.11, it was necessary to replicate the same server multiple times in the |
| 1312 | configuration. Starting with 1.2.12, the 'weight' option is available. HAProxy |
| 1313 | then computes the most homogenous possible map of servers based on their |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 1314 | weights so that the load gets distributed as smoothly as possible among them. |
| 1315 | The weight, between 1 and 256, should reflect one server's capacity relative to |
| 1316 | others. Weight 1 represents the lowest frequency and 256 the highest. This way, |
| 1317 | if a server fails, the remaining capacities are still respected. |
willy tarreau | 34f4530 | 2006-04-15 21:37:14 +0200 | [diff] [blame] | 1318 | |
| 1319 | Example : |
| 1320 | --------- |
| 1321 | # fair distribution among two opterons and one old pentium3 |
| 1322 | |
| 1323 | listen web_appl 0.0.0.0:80 |
| 1324 | mode http |
| 1325 | cookie SERVERID insert nocache indirect |
| 1326 | balance roundrobin |
| 1327 | server pentium3-800 192.168.1.1:80 cookie server01 weight 8 check |
| 1328 | server opteron-2.0G 192.168.1.2:80 cookie server02 weight 20 check |
| 1329 | server opteron-2.4G 192.168.1.3:80 cookie server03 weight 24 check |
| 1330 | server web-backup1 192.168.2.1:80 cookie server04 check backup |
| 1331 | server web-excuse 192.168.3.1:80 check backup |
| 1332 | |
| 1333 | Notes : |
| 1334 | ------- |
| 1335 | - if unspecified, the default weight is 1 |
| 1336 | |
| 1337 | - the weight does not impact health checks, so it is cleaner to use weights |
| 1338 | than replicating the same server several times |
| 1339 | |
| 1340 | - weights also work on backup servers if the 'allbackups' option is used |
| 1341 | |
| 1342 | - the weights also apply to the source address load balancing |
| 1343 | ('balance source'). |
| 1344 | |
| 1345 | - whatever the weights, the first server will always be assigned first. This |
| 1346 | is helpful for troubleshooting. |
| 1347 | |
| 1348 | - for the purists, the map calculation algorithm gives precedence to first |
| 1349 | server, so the map is the most uniform when servers are declared in |
| 1350 | ascending order relative to their weights. |
| 1351 | |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 1352 | The load distribution will follow exactly this sequence : |
| 1353 | |
| 1354 | Request| 1 1 1 1 |
| 1355 | number | 1 2 3 4 5 6 7 8 9 0 1 2 3 |
| 1356 | --------+--------------------------- |
| 1357 | p3-800 | X . . . . . . X . . . . . |
| 1358 | opt-20 | . X . X . X . . . X . X . |
| 1359 | opt-24 | . . X . X . X . X . X . X |
| 1360 | |
| 1361 | |
| 1362 | 3.4) Limiting the number of concurrent sessions on each server |
| 1363 | -------------------------------------------------------------- |
| 1364 | Some pre-forked servers such as Apache suffer from too many concurrent |
| 1365 | sessions, because it's very expensive to run hundreds or thousands of |
| 1366 | processes on one system. One solution is to increase the number of servers |
| 1367 | and load-balance between them, but it is a problem when the only goal is |
| 1368 | to resist to short surges. |
| 1369 | |
| 1370 | To solve this problem, a new feature was implemented in HAProxy 1.2.13. |
| 1371 | It's a per-server 'maxconn', associated with a per-server and a per-proxy |
| 1372 | queue. This transforms haproxy into a request buffer between the thousands of |
| 1373 | clients and the few servers. On many circumstances, lowering the maxconn value |
| 1374 | will increase the server's performance and decrease the overall response times |
| 1375 | because the servers will be less congested. |
| 1376 | |
| 1377 | When a request tries to reach any server, the first non-saturated server is |
| 1378 | used, respective to the load balancing algorithm. If all servers are saturated, |
| 1379 | then the request gets queued into the instance's global queue. It will be |
| 1380 | dequeued once a server will have freed a session and all previously queued |
| 1381 | requests have been processed. |
| 1382 | |
| 1383 | If a request references a particular server (eg: source hashing, or persistence |
| 1384 | cookie), and if this server is full, then the request will be queued into the |
| 1385 | server's dedicated queue. This queue has higher priority than the global queue, |
| 1386 | so it's easier for already registered users to enter the site than for new |
| 1387 | users. |
| 1388 | |
| 1389 | For this, the logs have been enhanced to show the number of sessions per |
| 1390 | server, the request's position in the queue and the time spent in the queue. |
| 1391 | This helps doing capacity planning. See the 'logs' section below for more info. |
| 1392 | |
| 1393 | Example : |
| 1394 | --------- |
| 1395 | # be nice with P3 which only has 256 MB of RAM. |
| 1396 | listen web_appl 0.0.0.0:80 |
| 1397 | maxconn 10000 |
| 1398 | mode http |
| 1399 | cookie SERVERID insert nocache indirect |
| 1400 | balance roundrobin |
| 1401 | server pentium3-800 192.168.1.1:80 cookie s1 weight 8 maxconn 100 check |
| 1402 | server opteron-2.0G 192.168.1.2:80 cookie s2 weight 20 maxconn 300 check |
| 1403 | server opteron-2.4G 192.168.1.3:80 cookie s3 weight 24 maxconn 300 check |
| 1404 | server web-backup1 192.168.2.1:80 cookie s4 check maxconn 200 backup |
| 1405 | server web-excuse 192.168.3.1:80 check backup |
| 1406 | |
willy tarreau | f76e6ca | 2006-05-21 21:09:55 +0200 | [diff] [blame] | 1407 | |
| 1408 | This was so much efficient at reducing the server's response time that some |
| 1409 | users wanted to use low values to improve their server's performance. However, |
| 1410 | they were not able anymore to handle very large loads because it was not |
| 1411 | possible anymore to saturate the servers. For this reason, version 1.2.14 has |
| 1412 | brought dynamic limitation with the addition of the parameter 'minconn'. When |
| 1413 | this parameter is set along with maxconn, it will enable dynamic limitation |
| 1414 | based on the instance's load. The maximum number of concurrent sessions on a |
| 1415 | server will be proportionnal to the number of sessions on the instance relative |
| 1416 | to its maxconn. A minimum of <minconn> will be allowed whatever the load. This |
| 1417 | will ensure that servers will perform at their best level under normal loads, |
| 1418 | while still handling surges when needed. The dynamic limit is computed like |
| 1419 | this : |
| 1420 | |
| 1421 | srv.dyn_limit = max(srv.minconn, srv.maxconn * inst.sess / inst.maxconn) |
| 1422 | |
| 1423 | Example : |
| 1424 | --------- |
| 1425 | # be nice with P3 which only has 256 MB of RAM. |
| 1426 | listen web_appl 0.0.0.0:80 |
| 1427 | maxconn 10000 |
| 1428 | mode http |
| 1429 | cookie SERVERID insert nocache indirect |
| 1430 | balance roundrobin |
| 1431 | server pentium3-800 192.168.1.1:80 cookie s1 weight 8 minconn 10 maxconn 100 check |
| 1432 | server opteron-2.0G 192.168.1.2:80 cookie s2 weight 20 minconn 30 maxconn 300 check |
| 1433 | server opteron-2.4G 192.168.1.3:80 cookie s3 weight 24 minconn 30 maxconn 300 check |
| 1434 | server web-backup1 192.168.2.1:80 cookie s4 check maxconn 200 backup |
| 1435 | server web-excuse 192.168.3.1:80 check backup |
| 1436 | |
| 1437 | In the example above, the server 'pentium3-800' will receive at most 100 |
| 1438 | simultaneous sessions when the proxy instance will reach 10000 sessions, and |
| 1439 | will receive only 10 simultaneous sessions when the proxy will be under 1000 |
| 1440 | sessions. |
| 1441 | |
Elijah Epifanov | acafc5f | 2007-10-25 20:15:38 +0200 | [diff] [blame] | 1442 | It is possible to limit server queue length in order to rebalance excess |
| 1443 | sessions between less busy application servers IF session affinity isn't |
| 1444 | hard functional requirement (for example it just gives huge performance boost |
| 1445 | by keeping server-local caches hot and compact). 'maxqueue' option sets a |
| 1446 | queue limit on a server, as in example below: |
| 1447 | |
| 1448 | ... (just the same as in example above) |
| 1449 | server pentium3-800 192.168.1.1:80 cookie s1 weight 8 minconn 10 maxconn 100 check maxqueue 50 |
| 1450 | server opteron-2.0G 192.168.1.2:80 cookie s2 weight 20 minconn 30 maxconn 300 check maxqueue 200 |
| 1451 | server opteron-2.4G 192.168.1.3:80 cookie s3 weight 24 minconn 30 maxconn 300 check |
| 1452 | |
| 1453 | Absence of 'maxqueue' option means unlimited queue. When queue gets filled |
| 1454 | up to 'maxqueue' client session is moved from server-local queue to a global |
| 1455 | one. |
| 1456 | |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 1457 | Notes : |
| 1458 | ------- |
| 1459 | - The requests will not stay indefinitely in the queue, they follow the |
| 1460 | 'contimeout' parameter, and if a request cannot be dequeued within this |
| 1461 | timeout because the server is saturated or because the queue is filled, |
| 1462 | the session will expire with a 503 error. |
| 1463 | |
willy tarreau | f76e6ca | 2006-05-21 21:09:55 +0200 | [diff] [blame] | 1464 | - if only <minconn> is specified, it has the same effect as <maxconn> |
| 1465 | |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 1466 | - setting too low values for maxconn might improve performance but might also |
| 1467 | allow slow users to block access to the server for other users. |
| 1468 | |
willy tarreau | 34f4530 | 2006-04-15 21:37:14 +0200 | [diff] [blame] | 1469 | |
willy tarreau | e0bdd62 | 2006-05-21 20:51:54 +0200 | [diff] [blame] | 1470 | 3.5) Dropping aborted requests |
| 1471 | ------------------------------ |
| 1472 | In presence of very high loads, the servers will take some time to respond. The |
| 1473 | per-proxy's connection queue will inflate, and the response time will increase |
| 1474 | respective to the size of the queue times the average per-session response |
| 1475 | time. When clients will wait for more than a few seconds, they will often hit |
| 1476 | the 'STOP' button on their browser, leaving a useless request in the queue, and |
| 1477 | slowing down other users. |
| 1478 | |
| 1479 | As there is no way to distinguish between a full STOP and a simple |
| 1480 | shutdown(SHUT_WR) on the client side, HTTP agents should be conservative and |
| 1481 | consider that the client might only have closed its output channel while |
| 1482 | waiting for the response. However, this introduces risks of congestion when |
| 1483 | lots of users do the same, and is completely useless nowadays because probably |
| 1484 | no client at all will close the session while waiting for the response. Some |
| 1485 | HTTP agents support this (Squid, Apache, HAProxy), and others do not (TUX, most |
| 1486 | hardware-based load balancers). So the probability for a closed input channel |
| 1487 | to represent a user hitting the 'STOP' button is close to 100%, and it is very |
| 1488 | tempting to be able to abort the session early without polluting the servers. |
| 1489 | |
| 1490 | For this reason, a new option "abortonclose" was introduced in version 1.2.14. |
| 1491 | By default (without the option) the behaviour is HTTP-compliant. But when the |
| 1492 | option is specified, a session with an incoming channel closed will be aborted |
| 1493 | if it's still possible, which means that it's either waiting for a connect() to |
| 1494 | establish or it is queued waiting for a connection slot. This considerably |
| 1495 | reduces the queue size and the load on saturated servers when users are tempted |
| 1496 | to click on STOP, which in turn reduces the response time for other users. |
| 1497 | |
| 1498 | Example : |
| 1499 | --------- |
| 1500 | listen web_appl 0.0.0.0:80 |
| 1501 | maxconn 10000 |
| 1502 | mode http |
| 1503 | cookie SERVERID insert nocache indirect |
| 1504 | balance roundrobin |
| 1505 | server web1 192.168.1.1:80 cookie s1 weight 10 maxconn 100 check |
| 1506 | server web2 192.168.1.2:80 cookie s2 weight 10 maxconn 100 check |
| 1507 | server web3 192.168.1.3:80 cookie s3 weight 10 maxconn 100 check |
| 1508 | server bck1 192.168.2.1:80 cookie s4 check maxconn 200 backup |
| 1509 | option abortonclose |
| 1510 | |
| 1511 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1512 | 4) Additionnal features |
| 1513 | ======================= |
| 1514 | |
willy tarreau | 481132e | 2006-05-21 21:43:10 +0200 | [diff] [blame] | 1515 | Other features are available. They are transparent mode, event logging, header |
| 1516 | rewriting/filtering, and the status as an HTML page. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1517 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1518 | |
willy tarreau | 0174f31 | 2005-12-18 01:02:42 +0100 | [diff] [blame] | 1519 | 4.1) Network features |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1520 | --------------------- |
willy tarreau | 0174f31 | 2005-12-18 01:02:42 +0100 | [diff] [blame] | 1521 | 4.1.1) Transparent mode |
| 1522 | ----------------------- |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1523 | In HTTP mode, the 'transparent' keyword allows to intercept sessions which are |
| 1524 | routed through the system hosting the proxy. This mode was implemented as a |
| 1525 | replacement for the 'dispatch' mode, since connections without cookie will be |
| 1526 | sent to the original address while known cookies will be sent to the servers. |
| 1527 | This mode implies that the system can redirect sessions to a local port. |
| 1528 | |
| 1529 | Example : |
| 1530 | --------- |
| 1531 | listen http_proxy 0.0.0.0:65000 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1532 | mode http |
| 1533 | transparent |
| 1534 | cookie SERVERID |
| 1535 | server server01 192.168.1.1:80 |
| 1536 | server server02 192.168.1.2:80 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1537 | |
| 1538 | # iptables -t nat -A PREROUTING -i eth0 -p tcp -d 192.168.1.100 \ |
| 1539 | --dport 80 -j REDIRECT --to-ports 65000 |
| 1540 | |
| 1541 | Note : |
| 1542 | ------ |
| 1543 | If the port is left unspecified on the server, the port the client connected to |
| 1544 | will be used. This allows to relay a full port range without using transparent |
| 1545 | mode nor thousands of file descriptors, provided that the system can redirect |
| 1546 | sessions to local ports. |
| 1547 | |
| 1548 | Example : |
| 1549 | --------- |
| 1550 | # redirect all ports to local port 65000, then forward to the server on the |
| 1551 | # original port. |
| 1552 | listen http_proxy 0.0.0.0:65000 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1553 | mode tcp |
| 1554 | server server01 192.168.1.1 check port 60000 |
| 1555 | server server02 192.168.1.2 check port 60000 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1556 | |
| 1557 | # iptables -t nat -A PREROUTING -i eth0 -p tcp -d 192.168.1.100 \ |
| 1558 | -j REDIRECT --to-ports 65000 |
| 1559 | |
willy tarreau | 0174f31 | 2005-12-18 01:02:42 +0100 | [diff] [blame] | 1560 | 4.1.2) Per-server source address binding |
| 1561 | ---------------------------------------- |
| 1562 | As of versions 1.1.30 and 1.2.3, it is possible to specify a particular source |
| 1563 | to reach each server. This is useful when reaching backup servers from a |
| 1564 | different LAN, or to use an alternate path to reach the same server. It is also |
| 1565 | usable to provide source load-balancing for outgoing connections. Obviously, |
| 1566 | the same source address is used to send health-checks. |
| 1567 | |
| 1568 | Example : |
| 1569 | --------- |
| 1570 | # use a particular source to reach both servers |
| 1571 | listen http_proxy 0.0.0.0:65000 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1572 | mode http |
| 1573 | balance roundrobin |
| 1574 | server server01 192.168.1.1:80 source 192.168.2.13 |
| 1575 | server server02 192.168.1.2:80 source 192.168.2.13 |
willy tarreau | 0174f31 | 2005-12-18 01:02:42 +0100 | [diff] [blame] | 1576 | |
| 1577 | Example : |
| 1578 | --------- |
| 1579 | # use a particular source to reach each servers |
| 1580 | listen http_proxy 0.0.0.0:65000 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1581 | mode http |
| 1582 | balance roundrobin |
| 1583 | server server01 192.168.1.1:80 source 192.168.1.1 |
| 1584 | server server02 192.168.2.1:80 source 192.168.2.1 |
willy tarreau | 0174f31 | 2005-12-18 01:02:42 +0100 | [diff] [blame] | 1585 | |
| 1586 | Example : |
| 1587 | --------- |
| 1588 | # provide source load-balancing to reach the same proxy through 2 WAN links |
| 1589 | listen http_proxy 0.0.0.0:65000 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1590 | mode http |
| 1591 | balance roundrobin |
| 1592 | server remote-proxy-way1 192.168.1.1:3128 source 192.168.2.1 |
| 1593 | server remote-proxy-way2 192.168.1.1:3128 source 192.168.3.1 |
willy tarreau | 0174f31 | 2005-12-18 01:02:42 +0100 | [diff] [blame] | 1594 | |
| 1595 | Example : |
| 1596 | --------- |
| 1597 | # force a TCP connection to bind to a specific port |
| 1598 | listen http_proxy 0.0.0.0:2000 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1599 | mode tcp |
| 1600 | balance roundrobin |
| 1601 | server srv1 192.168.1.1:80 source 192.168.2.1:20 |
| 1602 | server srv2 192.168.1.2:80 source 192.168.2.1:20 |
willy tarreau | 0174f31 | 2005-12-18 01:02:42 +0100 | [diff] [blame] | 1603 | |
willy tarreau | b952e1d | 2005-12-18 01:31:20 +0100 | [diff] [blame] | 1604 | 4.1.3) TCP keep-alive |
| 1605 | --------------------- |
| 1606 | With version 1.2.7, it becomes possible to enable TCP keep-alives on both the |
| 1607 | client and server sides. This makes it possible to prevent long sessions from |
| 1608 | expiring on external layer 4 components such as firewalls and load-balancers. |
| 1609 | It also allows the system to terminate dead sessions when no timeout has been |
| 1610 | set (not recommanded). The proxy cannot set the keep-alive probes intervals nor |
| 1611 | maximal count, consult your operating system manual for this. There are 3 |
| 1612 | options to enable TCP keep-alive : |
| 1613 | |
| 1614 | option tcpka # enables keep-alive both on client and server side |
| 1615 | option clitcpka # enables keep-alive only on client side |
| 1616 | option srvtcpka # enables keep-alive only on server side |
| 1617 | |
Alexandre Cassen | 87ea548 | 2007-10-11 20:48:58 +0200 | [diff] [blame] | 1618 | 4.1.4) TCP lingering |
| 1619 | -------------------- |
| 1620 | It is possible to disable the system's lingering of data unacked by the client |
| 1621 | at the end of a session. This is sometimes required when haproxy is used as a |
| 1622 | front-end with lots of unreliable clients, and you observe thousands of sockets |
| 1623 | in the FIN_WAIT state on the machine. This may be used in a frontend to affect |
| 1624 | the client-side connection, as well as in a backend for the server-side |
| 1625 | connection : |
| 1626 | |
| 1627 | option nolinger # disables data lingering |
| 1628 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1629 | |
| 1630 | 4.2) Event logging |
| 1631 | ------------------ |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1632 | |
| 1633 | HAProxy's strength certainly lies in its precise logs. It probably provides the |
| 1634 | finest level of information available for such a product, which is very |
| 1635 | important for troubleshooting complex environments. Standard log information |
| 1636 | include client ports, TCP/HTTP state timers, precise session state at |
| 1637 | termination and precise termination cause, information about decisions to |
| 1638 | direct trafic to a server, and of course the ability to capture arbitrary |
| 1639 | headers. |
| 1640 | |
| 1641 | In order to improve administrators reactivity, it offers a great transparency |
| 1642 | about encountered problems, both internal and external, and it is possible to |
| 1643 | send logs to different sources at the same time with different level filters : |
| 1644 | |
| 1645 | - global process-level logs (system errors, start/stop, etc..) |
| 1646 | - per-listener system and internal errors (lack of resource, bugs, ...) |
| 1647 | - per-listener external troubles (servers up/down, max connections) |
| 1648 | - per-listener activity (client connections), either at the establishment or |
| 1649 | at the termination. |
| 1650 | |
| 1651 | The ability to distribute different levels of logs to different log servers |
| 1652 | allow several production teams to interact and to fix their problems as soon |
| 1653 | as possible. For example, the system team might monitor system-wide errors, |
| 1654 | while the application team might be monitoring the up/down for their servers in |
| 1655 | real time, and the security team might analyze the activity logs with one hour |
| 1656 | delay. |
| 1657 | |
willy tarreau | c1cae63 | 2005-12-17 14:12:23 +0100 | [diff] [blame] | 1658 | 4.2.1) Log levels |
| 1659 | ----------------- |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 1660 | TCP and HTTP connections can be logged with informations such as date, time, |
| 1661 | source IP address, destination address, connection duration, response times, |
| 1662 | HTTP request, the HTTP return code, number of bytes transmitted, the conditions |
| 1663 | in which the session ended, and even exchanged cookies values, to track a |
| 1664 | particular user's problems for example. All messages are sent to up to two |
| 1665 | syslog servers. Consult section 1.1 for more info about log facilities. The |
| 1666 | syntax follows : |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1667 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 1668 | log <address_1> <facility_1> [max_level_1] |
| 1669 | log <address_2> <facility_2> [max_level_2] |
| 1670 | or |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1671 | log global |
| 1672 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 1673 | Note : |
| 1674 | ------ |
| 1675 | The particular syntax 'log global' means that the same log configuration as the |
| 1676 | 'global' section will be used. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1677 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 1678 | Example : |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1679 | --------- |
| 1680 | listen http_proxy 0.0.0.0:80 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1681 | mode http |
| 1682 | log 192.168.2.200 local3 |
| 1683 | log 192.168.2.201 local4 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1684 | |
willy tarreau | c1cae63 | 2005-12-17 14:12:23 +0100 | [diff] [blame] | 1685 | 4.2.2) Log format |
| 1686 | ----------------- |
| 1687 | By default, connections are logged at the TCP level, as soon as the session |
| 1688 | establishes between the client and the proxy. By enabling the 'tcplog' option, |
| 1689 | the proxy will wait until the session ends to generate an enhanced log |
| 1690 | containing more information such as session duration and its state during the |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 1691 | disconnection. The number of remaining session after disconnection is also |
| 1692 | indicated (for the server, the listener, and the process). |
willy tarreau | c1cae63 | 2005-12-17 14:12:23 +0100 | [diff] [blame] | 1693 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1694 | Example of TCP logging : |
| 1695 | ------------------------ |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 1696 | listen relais-tcp 0.0.0.0:8000 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1697 | mode tcp |
| 1698 | option tcplog |
| 1699 | log 192.168.2.200 local3 |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 1700 | |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 1701 | >>> haproxy[18989]: 127.0.0.1:34550 [15/Oct/2003:15:24:28] relais-tcp Srv1 0/0/5007 0 -- 1/1/1 0/0 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1702 | |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 1703 | Field Format Example |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1704 | |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 1705 | 1 process_name '[' pid ']:' haproxy[18989]: |
| 1706 | 2 client_ip ':' client_port 127.0.0.1:34550 |
| 1707 | 3 '[' date ']' [15/Oct/2003:15:24:28] |
| 1708 | 4 listener_name relais-tcp |
| 1709 | 5 server_name Srv1 |
| 1710 | 6 queue_time '/' connect_time '/' total_time 0/0/5007 |
| 1711 | 7 bytes_read 0 |
| 1712 | 8 termination_state -- |
| 1713 | 9 srv_conn '/' listener_conn '/' process_conn 1/1/1 |
| 1714 | 10 position in srv_queue / listener_queue 0/0 |
| 1715 | |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 1716 | |
willy tarreau | c1cae63 | 2005-12-17 14:12:23 +0100 | [diff] [blame] | 1717 | Another option, 'httplog', provides more detailed information about HTTP |
| 1718 | contents, such as the request and some cookies. In the event where an external |
| 1719 | component would establish frequent connections to check the service, logs may be |
| 1720 | full of useless lines. So it is possible not to log any session which didn't |
| 1721 | transfer any data, by the setting of the 'dontlognull' option. This only has |
| 1722 | effect on sessions which are established then closed. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1723 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1724 | Example of HTTP logging : |
| 1725 | ------------------------- |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1726 | listen http_proxy 0.0.0.0:80 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1727 | mode http |
| 1728 | option httplog |
| 1729 | option dontlognull |
| 1730 | log 192.168.2.200 local3 |
| 1731 | |
Krzysztof Piotr Oledzki | 25b501a | 2008-01-06 16:36:16 +0100 | [diff] [blame] | 1732 | >>> haproxy[674]: 127.0.0.1:33319 [15/Oct/2003:08:31:57] relais-http Srv1 9/0/7/147/723 200 243 - - ---- 34/34/15/8/3 0/0 "HEAD / HTTP/1.0" |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1733 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1734 | More complete example |
Krzysztof Piotr Oledzki | 25b501a | 2008-01-06 16:36:16 +0100 | [diff] [blame] | 1735 | haproxy[18989]: 10.0.0.1:34552 [15/Oct/2003:15:26:31] relais-http Srv1 3183/-1/-1/-1/11215 503 0 - - SC-- 205/202/150/137/+4 0/0 {w.ods.org|Mozilla} {} "HEAD / HTTP/1.0" |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1736 | |
Krzysztof Piotr Oledzki | 25b501a | 2008-01-06 16:36:16 +0100 | [diff] [blame] | 1737 | Field Format Example |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1738 | |
Krzysztof Piotr Oledzki | 25b501a | 2008-01-06 16:36:16 +0100 | [diff] [blame] | 1739 | 1 process_name '[' pid ']:' haproxy[18989]: |
| 1740 | 2 client_ip ':' client_port 10.0.0.1:34552 |
| 1741 | 3 '[' date ']' [15/Oct/2003:15:26:31] |
| 1742 | 4 listener_name relais-http |
| 1743 | 5 server_name Srv1 |
| 1744 | 6 Tq '/' Tw '/' Tc '/' Tr '/' Tt 3183/-1/-1/-1/11215 |
| 1745 | 7 HTTP_return_code 503 |
| 1746 | 8 bytes_read 0 |
| 1747 | 9 captured_request_cookie - |
| 1748 | 10 captured_response_cookie - |
| 1749 | 11 termination_state SC-- |
| 1750 | 12 actconn '/' feconn '/' beconn '/' srv_conn '/' retries 205/202/150/137/+4 |
| 1751 | 13 position in srv_queue / listener_queue 0/0 |
| 1752 | 14 '{' captured_request_headers '}' {w.ods.org|Mozilla} |
| 1753 | 15 '{' captured_response_headers '}' {} |
| 1754 | 16 '"' HTTP_request '"' "HEAD / HTTP/1.0" |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1755 | |
| 1756 | Note for log parsers: the URI is ALWAYS the end of the line starting with the |
| 1757 | first double quote '"'. |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 1758 | |
Krzysztof Piotr Oledzki | 25b501a | 2008-01-06 16:36:16 +0100 | [diff] [blame] | 1759 | The retries count may have additional '+' sign means that the connection had been |
| 1760 | redispatched from one server to another shortly before retries limit (retries 4 |
| 1761 | in above example) was depleted. |
| 1762 | |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 1763 | The problem when logging at end of connection is that you have no clue about |
| 1764 | what is happening during very long sessions. To workaround this problem, a |
| 1765 | new option 'logasap' has been introduced in 1.1.28/1.2.1. When specified, the |
| 1766 | proxy will log as soon as possible, just before data transfer begins. This means |
| 1767 | that in case of TCP, it will still log the connection status to the server, and |
| 1768 | in case of HTTP, it will log just after processing the server headers. In this |
| 1769 | case, the number of bytes reported is the number of header bytes sent to the |
| 1770 | client. |
| 1771 | |
| 1772 | In order to avoid confusion with normal logs, the total time field and the |
Krzysztof Piotr Oledzki | 25b501a | 2008-01-06 16:36:16 +0100 | [diff] [blame] | 1773 | number of bytes are prefixed with a '+' sign which means that real numbers are |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 1774 | certainly bigger. |
| 1775 | |
| 1776 | Example : |
| 1777 | --------- |
| 1778 | |
| 1779 | listen http_proxy 0.0.0.0:80 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1780 | mode http |
| 1781 | option httplog |
| 1782 | option dontlognull |
| 1783 | option logasap |
| 1784 | log 192.168.2.200 local3 |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 1785 | |
Krzysztof Piotr Oledzki | 25b501a | 2008-01-06 16:36:16 +0100 | [diff] [blame] | 1786 | >>> haproxy[674]: 127.0.0.1:33320 [15/Oct/2003:08:32:17] relais-http Srv1 9/10/7/14/+30 200 +243 - - ---- 3/1/1/1/0 1/0 "GET /image.iso HTTP/1.0" |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 1787 | |
willy tarreau | c1cae63 | 2005-12-17 14:12:23 +0100 | [diff] [blame] | 1788 | 4.2.3) Timing events |
| 1789 | -------------------- |
| 1790 | Timers provide a great help in trouble shooting network problems. All values |
| 1791 | are reported in milliseconds (ms). In HTTP mode, four control points are |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 1792 | reported under the form 'Tq/Tw/Tc/Tr/Tt' : |
willy tarreau | c1cae63 | 2005-12-17 14:12:23 +0100 | [diff] [blame] | 1793 | |
| 1794 | - Tq: total time to get the client request. |
| 1795 | It's the time elapsed between the moment the client connection was accepted |
| 1796 | and the moment the proxy received the last HTTP header. The value '-1' |
| 1797 | indicates that the end of headers (empty line) has never been seen. |
| 1798 | |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 1799 | - Tw: total time spent in the queues waiting for a connection slot. It |
| 1800 | accounts for listener's queue as well as the server's queue, and depends |
| 1801 | on the queue size, and the time needed for the server to complete previous |
| 1802 | sessions. The value '-1' means that the request was killed before reaching |
| 1803 | the queue. |
| 1804 | |
willy tarreau | c1cae63 | 2005-12-17 14:12:23 +0100 | [diff] [blame] | 1805 | - Tc: total time to establish the TCP connection to the server. |
| 1806 | It's the time elapsed between the moment the proxy sent the connection |
| 1807 | request, and the moment it was acknowledged, or between the TCP SYN packet |
| 1808 | and the matching SYN/ACK in return. The value '-1' means that the |
| 1809 | connection never established. |
| 1810 | |
| 1811 | - Tr: server response time. It's the time elapsed between the moment the |
| 1812 | TCP connection was established to the server and the moment it send its |
| 1813 | complete response header. It purely shows its request processing time, |
| 1814 | without the network overhead due to the data transmission. The value '-1' |
| 1815 | means that the last the response header (empty line) was never seen. |
| 1816 | |
| 1817 | - Tt: total session duration time, between the moment the proxy accepted it |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 1818 | and the moment both ends were closed. The exception is when the 'logasap' |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 1819 | option is specified. In this case, it only equals (Tq+Tw+Tc+Tr), and is |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 1820 | prefixed with a '+' sign. From this field, we can deduce Td, the data |
| 1821 | transmission time, by substracting other timers when valid : |
willy tarreau | c1cae63 | 2005-12-17 14:12:23 +0100 | [diff] [blame] | 1822 | |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 1823 | Td = Tt - (Tq + Tw + Tc + Tr) |
willy tarreau | c1cae63 | 2005-12-17 14:12:23 +0100 | [diff] [blame] | 1824 | |
| 1825 | Timers with '-1' values have to be excluded from this equation. |
| 1826 | |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 1827 | In TCP mode ('option tcplog'), only Tw, Tc and Tt are reported. |
willy tarreau | c1cae63 | 2005-12-17 14:12:23 +0100 | [diff] [blame] | 1828 | |
| 1829 | These timers provide precious indications on trouble causes. Since the TCP |
| 1830 | protocol defines retransmit delays of 3, 6, 12... seconds, we know for sure |
| 1831 | that timers close to multiples of 3s are nearly always related to packets lost |
| 1832 | due to network problems (wires or negociation). Moreover, if <Tt> is close to |
| 1833 | a timeout value specified in the configuration, it often means that a session |
| 1834 | has been aborted on time-out. |
| 1835 | |
| 1836 | Most common cases : |
| 1837 | |
| 1838 | - If Tq is close to 3000, a packet has probably been lost between the client |
| 1839 | and the proxy. |
| 1840 | - If Tc is close to 3000, a packet has probably been lost between the server |
| 1841 | and the proxy during the server connection phase. This one should always be |
| 1842 | very low (less than a few tens). |
| 1843 | - If Tr is nearly always lower than 3000 except some rare values which seem to |
| 1844 | be the average majored by 3000, there are probably some packets lost between |
| 1845 | the proxy and the server. |
| 1846 | - If Tt is often slightly higher than a time-out, it's often because the |
| 1847 | client and the server use HTTP keep-alive and the session is maintained |
| 1848 | after the response ends. Se further for how to disable HTTP keep-alive. |
| 1849 | |
| 1850 | Other cases ('xx' means any value to be ignored) : |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 1851 | -1/xx/xx/xx/Tt: the client was not able to send its complete request in time, |
| 1852 | or that it aborted it too early. |
| 1853 | Tq/-1/xx/xx/Tt: it was not possible to process the request, maybe because |
| 1854 | servers were out of order. |
| 1855 | Tq/Tw/-1/xx/Tt: the connection could not establish on the server. Either it |
| 1856 | refused it or it timed out after Tt-(Tq+Tw) ms. |
| 1857 | Tq/Tw/Tc/-1/Tt: the server has accepted the connection but did not return a |
| 1858 | complete response in time, or it closed its connexion |
| 1859 | unexpectedly, after Tt-(Tq+Tw+Tc) ms. |
willy tarreau | c1cae63 | 2005-12-17 14:12:23 +0100 | [diff] [blame] | 1860 | |
| 1861 | 4.2.4) Session state at disconnection |
| 1862 | ------------------------------------- |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1863 | TCP and HTTP logs provide a session completion indicator in the |
| 1864 | <termination_state> field, just before the number of active |
| 1865 | connections. It is 2-characters long in TCP, and 4-characters long in |
| 1866 | HTTP, each of which has a special meaning : |
| 1867 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 1868 | - On the first character, a code reporting the first event which caused the |
| 1869 | session to terminate : |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1870 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1871 | C : the TCP session was unexpectedly aborted by the client. |
| 1872 | |
| 1873 | S : the TCP session was unexpectedly aborted by the server, or the |
| 1874 | server explicitly refused it. |
| 1875 | |
| 1876 | P : the session was prematurely aborted by the proxy, because of a |
| 1877 | connection limit enforcement, because a DENY filter was matched, |
| 1878 | or because of a security check which detected and blocked a |
| 1879 | dangerous error in server response which might have caused |
| 1880 | information leak (eg: cacheable cookie). |
| 1881 | |
| 1882 | R : a resource on the proxy has been exhausted (memory, sockets, source |
| 1883 | ports, ...). Usually, this appears during the connection phase, and |
| 1884 | system logs should contain a copy of the precise error. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1885 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1886 | I : an internal error was identified by the proxy during a self-check. |
| 1887 | This should NEVER happen, and you are encouraged to report any log |
| 1888 | containing this, because this is a bug. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1889 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1890 | c : the client-side time-out expired first. |
| 1891 | |
| 1892 | s : the server-side time-out expired first. |
| 1893 | |
| 1894 | - : normal session completion. |
| 1895 | |
| 1896 | - on the second character, the TCP/HTTP session state when it was closed : |
| 1897 | |
| 1898 | R : waiting for complete REQUEST from the client (HTTP only). Nothing |
| 1899 | was sent to any server. |
| 1900 | |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 1901 | Q : waiting in the QUEUE for a connection slot. This can only happen on |
| 1902 | servers which have a 'maxconn' parameter set. No connection attempt |
| 1903 | was made to any server. |
| 1904 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1905 | C : waiting for CONNECTION to establish on the server. The server might |
| 1906 | at most have noticed a connection attempt. |
| 1907 | |
| 1908 | H : waiting for, receiving and processing server HEADERS (HTTP only). |
| 1909 | |
| 1910 | D : the session was in the DATA phase. |
| 1911 | |
| 1912 | L : the proxy was still transmitting LAST data to the client while the |
| 1913 | server had already finished. |
| 1914 | |
Willy Tarreau | 2272dc1 | 2006-09-03 10:19:38 +0200 | [diff] [blame] | 1915 | T : the request was tarpitted. It has been held open on with the client |
Willy Tarreau | 08fa2e3 | 2006-09-03 10:47:37 +0200 | [diff] [blame] | 1916 | during the whole contimeout duration or untill the client closed. |
Willy Tarreau | 2272dc1 | 2006-09-03 10:19:38 +0200 | [diff] [blame] | 1917 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1918 | - : normal session completion after end of data transfer. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1919 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 1920 | - the third character tells whether the persistence cookie was provided by |
willy tarreau | c1cae63 | 2005-12-17 14:12:23 +0100 | [diff] [blame] | 1921 | the client (only in HTTP mode) : |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1922 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1923 | N : the client provided NO cookie. This is usually the case on new |
| 1924 | connections. |
| 1925 | |
| 1926 | I : the client provided an INVALID cookie matching no known |
| 1927 | server. This might be caused by a recent configuration change, |
| 1928 | mixed cookies between HTTP/HTTPS sites, or an attack. |
| 1929 | |
| 1930 | D : the client provided a cookie designating a server which was DOWN, |
| 1931 | so either the 'persist' option was used and the client was sent to |
| 1932 | this server, or it was not set and the client was redispatched to |
| 1933 | another server. |
| 1934 | |
| 1935 | V : the client provided a valid cookie, and was sent to the associated |
| 1936 | server. |
| 1937 | |
| 1938 | - : does not apply (no cookie set in configuration). |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1939 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 1940 | - the last character reports what operations were performed on the persistence |
willy tarreau | c1cae63 | 2005-12-17 14:12:23 +0100 | [diff] [blame] | 1941 | cookie returned by the server (only in HTTP mode) : |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1942 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1943 | N : NO cookie was provided by the server, and none was inserted either. |
| 1944 | |
| 1945 | I : no cookie was provided by the server, and the proxy INSERTED one. |
| 1946 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 1947 | P : a cookie was PROVIDED by the server and transmitted as-is. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1948 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1949 | R : the cookie provided by the server was REWRITTEN by the proxy. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1950 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1951 | D : the cookie provided by the server was DELETED by the proxy. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1952 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1953 | - : does not apply (no cookie set in configuration). |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1954 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1955 | The combination of the two first flags give a lot of information about what was |
| 1956 | happening when the session terminated. It can be helpful to detect server |
| 1957 | saturation, network troubles, local system resource starvation, attacks, etc... |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1958 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1959 | The most common termination flags combinations are indicated here. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1960 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1961 | Flags Reason |
| 1962 | CR The client aborted before sending a full request. Most probably the |
| 1963 | request was done by hand using a telnet client, and aborted early. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1964 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1965 | cR The client timed out before sending a full request. This is sometimes |
| 1966 | caused by too large TCP MSS values on the client side for PPPoE |
| 1967 | networks which cannot transport full-sized packets, or by clients |
| 1968 | sending requests by hand and not typing fast enough. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1969 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1970 | SC The server explicitly refused the connection (the proxy received a |
| 1971 | TCP RST or an ICMP in return). Under some circumstances, it can |
| 1972 | also be the network stack telling the proxy that the server is |
| 1973 | unreachable (eg: no route, or no ARP response on local network). |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 1974 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1975 | sC The connection to the server did not complete during contimeout. |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 1976 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1977 | PC The proxy refused to establish a connection to the server because the |
| 1978 | maxconn limit has been reached. The listener's maxconn parameter may |
| 1979 | be increased in the proxy configuration, as well as the global |
| 1980 | maxconn parameter. |
willy tarreau | c1cae63 | 2005-12-17 14:12:23 +0100 | [diff] [blame] | 1981 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1982 | RC A local resource has been exhausted (memory, sockets, source ports) |
| 1983 | preventing the connection to the server from establishing. The error |
| 1984 | logs will tell precisely what was missing. Anyway, this can only be |
| 1985 | solved by system tuning. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 1986 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 1987 | cH The client timed out during a POST request. This is sometimes caused |
| 1988 | by too large TCP MSS values for PPPoE networks which cannot transport |
| 1989 | full-sized packets. |
willy tarreau | c1cae63 | 2005-12-17 14:12:23 +0100 | [diff] [blame] | 1990 | |
willy tarreau | 078c79a | 2006-05-13 12:23:58 +0200 | [diff] [blame] | 1991 | CH The client aborted while waiting for the server to start responding. |
| 1992 | It might be the server taking too long to respond or the client |
| 1993 | clicking the 'Stop' button too fast. |
| 1994 | |
| 1995 | CQ The client aborted while its session was queued, waiting for a server |
| 1996 | with enough empty slots to accept it. It might be that either all the |
| 1997 | servers were saturated or the assigned server taking too long to |
| 1998 | respond. |
| 1999 | |
Willy Tarreau | 08fa2e3 | 2006-09-03 10:47:37 +0200 | [diff] [blame] | 2000 | CT The client aborted while its session was tarpitted. |
| 2001 | |
willy tarreau | 078c79a | 2006-05-13 12:23:58 +0200 | [diff] [blame] | 2002 | sQ The session spent too much time in queue and has been expired. |
| 2003 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2004 | SH The server aborted before sending its full headers, or it crashed. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2005 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2006 | sH The server failed to reply during the srvtimeout delay, which |
| 2007 | indicates too long transactions, probably caused by back-end |
| 2008 | saturation. The only solutions are to fix the problem on the |
| 2009 | application or to increase the 'srvtimeout' parameter to support |
| 2010 | longer delays (at the risk of the client giving up anyway). |
| 2011 | |
| 2012 | PR The proxy blocked the client's request, either because of an invalid |
| 2013 | HTTP syntax, in which case it returned an HTTP 400 error to the |
| 2014 | client, or because a deny filter matched, in which case it returned |
| 2015 | an HTTP 403 error. |
| 2016 | |
| 2017 | PH The proxy blocked the server's response, because it was invalid, |
| 2018 | incomplete, dangerous (cache control), or matched a security filter. |
| 2019 | In any case, an HTTP 502 error is sent to the client. |
| 2020 | |
Willy Tarreau | 2272dc1 | 2006-09-03 10:19:38 +0200 | [diff] [blame] | 2021 | PT The proxy blocked the client's request and has tarpitted its |
| 2022 | connection before returning it a 500 server error. Nothing was sent |
| 2023 | to the server. |
| 2024 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2025 | cD The client did not read any data for as long as the clitimeout delay. |
| 2026 | This is often caused by network failures on the client side. |
| 2027 | |
| 2028 | CD The client unexpectedly aborted during data transfer. This is either |
| 2029 | caused by a browser crash, or by a keep-alive session between the |
| 2030 | server and the client terminated first by the client. |
| 2031 | |
| 2032 | sD The server did nothing during the srvtimeout delay. This is often |
| 2033 | caused by too short timeouts on L4 equipements before the server |
| 2034 | (firewalls, load-balancers, ...). |
| 2035 | |
| 2036 | 4.2.5) Non-printable characters |
willy tarreau | 4302f49 | 2005-12-18 01:00:37 +0100 | [diff] [blame] | 2037 | ------------------------------- |
| 2038 | As of version 1.1.29, non-printable characters are not sent as-is into log |
| 2039 | files, but are converted to their two-digits hexadecimal representation, |
| 2040 | prefixed by the character '#'. The only characters that can now be logged |
| 2041 | without being escaped are between 32 and 126 (inclusive). Obviously, the |
| 2042 | escape character '#' is also encoded to avoid any ambiguity. It is the same for |
| 2043 | the character '"', as well as '{', '|' and '}' when logging headers. |
| 2044 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2045 | 4.2.6) Capturing HTTP headers and cookies |
| 2046 | ----------------------------------------- |
| 2047 | Version 1.1.23 brought cookie capture, and 1.1.29 the header capture. All this |
| 2048 | is performed using the 'capture' keyword. |
| 2049 | |
| 2050 | Cookie capture makes it easy to track a complete user session. The syntax is : |
| 2051 | |
| 2052 | capture cookie <cookie_prefix> len <capture_length> |
| 2053 | |
| 2054 | This will enable cookie capture from both requests and responses. This way, |
| 2055 | it's easy to detect when a user switches to a new session for example, because |
| 2056 | the server will reassign it a new cookie. |
| 2057 | |
| 2058 | The FIRST cookie whose name starts with <cookie_prefix> will be captured, and |
| 2059 | logged as 'NAME=value', without exceeding <capture_length> characters (64 max). |
| 2060 | When the cookie name is fixed and known, it's preferable to suffix '=' to it to |
| 2061 | ensure that no other cookie will be logged. |
| 2062 | |
| 2063 | Examples : |
| 2064 | ---------- |
| 2065 | # capture the first cookie whose name starts with "ASPSESSION" |
| 2066 | capture cookie ASPSESSION len 32 |
| 2067 | |
| 2068 | # capture the first cookie whose name is exactly "vgnvisitor" |
| 2069 | capture cookie vgnvisitor= len 32 |
| 2070 | |
| 2071 | In the logs, the field preceeding the completion indicator contains the cookie |
| 2072 | value as sent by the server, preceeded by the cookie value as sent by the |
| 2073 | client. Each of these field is replaced with '-' when no cookie was seen or |
| 2074 | when the option is disabled. |
| 2075 | |
| 2076 | Header captures have a different goal. They are useful to track unique request |
| 2077 | identifiers set by a previous proxy, virtual host names, user-agents, POST |
| 2078 | content-length, referrers, etc. In the response, one can search for information |
| 2079 | about the response length, how the server asked the cache to behave, or an |
| 2080 | object location during a redirection. As for cookie captures, it is both |
| 2081 | possible to include request headers and response headers at the same time. The |
| 2082 | syntax is : |
willy tarreau | 4302f49 | 2005-12-18 01:00:37 +0100 | [diff] [blame] | 2083 | |
| 2084 | capture request header <name> len <max length> |
| 2085 | capture response header <name> len <max length> |
| 2086 | |
| 2087 | Note: Header names are not case-sensitive. |
| 2088 | |
| 2089 | Examples: |
| 2090 | --------- |
| 2091 | # keep the name of the virtual server |
| 2092 | capture request header Host len 20 |
| 2093 | # keep the amount of data uploaded during a POST |
| 2094 | capture request header Content-Length len 10 |
| 2095 | |
| 2096 | # note the expected cache behaviour on the response |
| 2097 | capture response header Cache-Control len 8 |
| 2098 | # note the URL location during a redirection |
| 2099 | capture response header Location len 20 |
| 2100 | |
| 2101 | Non-existant headers are logged as empty strings, and if one header appears more |
| 2102 | than once, only its last occurence will be kept. Request headers are grouped |
| 2103 | within braces '{' and '}' in the same order as they were declared, and delimited |
| 2104 | with a vertical bar '|' without any space. Response headers follow the same |
| 2105 | representation, but are displayed after a space following the request headers |
| 2106 | block. These blocks are displayed just before the HTTP request in the logs. |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2107 | |
willy tarreau | 4302f49 | 2005-12-18 01:00:37 +0100 | [diff] [blame] | 2108 | Example : |
| 2109 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2110 | Config: |
| 2111 | |
| 2112 | capture request header Host len 20 |
| 2113 | capture request header Content-Length len 10 |
| 2114 | capture request header Referer len 20 |
| 2115 | capture response header Server len 20 |
| 2116 | capture response header Content-Length len 10 |
| 2117 | capture response header Cache-Control len 8 |
| 2118 | capture response header Via len 20 |
| 2119 | capture response header Location len 20 |
| 2120 | |
| 2121 | Log : |
| 2122 | |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 2123 | Aug 9 20:26:09 localhost haproxy[2022]: 127.0.0.1:34014 [09/Aug/2004:20:26:09] relais-http netcache 0/0/0/162/+162 200 +350 - - ---- 0/0/0 0/0 {fr.adserver.yahoo.co||http://fr.f416.mail.} {|864|private||} "GET http://fr.adserver.yahoo.com/" |
| 2124 | Aug 9 20:30:46 localhost haproxy[2022]: 127.0.0.1:34020 [09/Aug/2004:20:30:46] relais-http netcache 0/0/0/182/+182 200 +279 - - ---- 0/0/0 0/0 {w.ods.org||} {Formilux/0.1.8|3495|||} "GET http://w.ods.org/sytadin.html HTTP/1.1" |
| 2125 | Aug 9 20:30:46 localhost haproxy[2022]: 127.0.0.1:34028 [09/Aug/2004:20:30:46] relais-http netcache 0/0/2/126/+128 200 +223 - - ---- 0/0/0 0/0 {www.infotrafic.com||http://w.ods.org/syt} {Apache/2.0.40 (Red H|9068|||} "GET http://www.infotrafic.com/images/live/cartesidf/grandes/idf_ne.png HTTP/1.1" |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2126 | |
| 2127 | |
| 2128 | 4.2.7) Examples of logs |
| 2129 | ----------------------- |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 2130 | - haproxy[674]: 127.0.0.1:33319 [15/Oct/2003:08:31:57] relais-http Srv1 6559/0/7/147/6723 200 243 - - ---- 1/3/5 0/0 "HEAD / HTTP/1.0" |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2131 | => long request (6.5s) entered by hand through 'telnet'. The server replied |
| 2132 | in 147 ms, and the session ended normally ('----') |
| 2133 | |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 2134 | - haproxy[674]: 127.0.0.1:33319 [15/Oct/2003:08:31:57] relais-http Srv1 6559/1230/7/147/6870 200 243 - - ---- 99/239/324 0/9 "HEAD / HTTP/1.0" |
| 2135 | => Idem, but the request was queued in the global queue behind 9 other |
| 2136 | requests, and waited there for 1230 ms. |
| 2137 | |
| 2138 | - haproxy[674]: 127.0.0.1:33320 [15/Oct/2003:08:32:17] relais-http Srv1 9/0/7/14/+30 200 +243 - - ---- 1/3/3 0/0 "GET /image.iso HTTP/1.0" |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2139 | => request for a long data transfer. The 'logasap' option was specified, so |
| 2140 | the log was produced just before transfering data. The server replied in |
| 2141 | 14 ms, 243 bytes of headers were sent to the client, and total time from |
| 2142 | accept to first data byte is 30 ms. |
| 2143 | |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 2144 | - haproxy[674]: 127.0.0.1:33320 [15/Oct/2003:08:32:17] relais-http Srv1 9/0/7/14/30 502 243 - - PH-- 0/2/3 0/0 "GET /cgi-bin/bug.cgi? HTTP/1.0" |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2145 | => the proxy blocked a server response either because of an 'rspdeny' or |
| 2146 | 'rspideny' filter, or because it blocked sensible information which risked |
| 2147 | being cached. In this case, the response is replaced with a '502 bad |
| 2148 | gateway'. |
| 2149 | |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 2150 | - haproxy[18113]: 127.0.0.1:34548 [15/Oct/2003:15:18:55] relais-http <NOSRV> -1/-1/-1/-1/8490 -1 0 - - CR-- 0/2/2 0/0 "" |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2151 | => the client never completed its request and aborted itself ('C---') after |
| 2152 | 8.5s, while the proxy was waiting for the request headers ('-R--'). |
| 2153 | Nothing was sent to the server. |
| 2154 | |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 2155 | - haproxy[18113]: 127.0.0.1:34549 [15/Oct/2003:15:19:06] relais-http <NOSRV> -1/-1/-1/-1/50001 408 0 - - cR-- 2/2 0/0 "" |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2156 | => The client never completed its request, which was aborted by the time-out |
| 2157 | ('c---') after 50s, while the proxy was waiting for the request headers ('-R--'). |
| 2158 | Nothing was sent to the server, but the proxy could send a 408 return code |
| 2159 | to the client. |
willy tarreau | 4302f49 | 2005-12-18 01:00:37 +0100 | [diff] [blame] | 2160 | |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 2161 | - haproxy[18989]: 127.0.0.1:34550 [15/Oct/2003:15:24:28] relais-tcp Srv1 0/0/5007 0 cD 0/0/0 0/0 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2162 | => This is a 'tcplog' entry. Client-side time-out ('c----') occured after 5s. |
willy tarreau | 4302f49 | 2005-12-18 01:00:37 +0100 | [diff] [blame] | 2163 | |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 2164 | - haproxy[18989]: 10.0.0.1:34552 [15/Oct/2003:15:26:31] relais-http Srv1 3183/-1/-1/-1/11215 503 0 - - SC-- 115/202/205 0/0 "HEAD / HTTP/1.0" |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2165 | => The request took 3s to complete (probably a network problem), and the |
| 2166 | connection to the server failed ('SC--') after 4 attemps of 2 seconds |
| 2167 | (config says 'retries 3'), then a 503 error code was sent to the client. |
willy tarreau | 532bb55 | 2006-05-13 18:40:37 +0200 | [diff] [blame] | 2168 | There were 115 connections on this server, 202 connections on this proxy, |
| 2169 | and 205 on the global process. It is possible that the server refused the |
| 2170 | connection because of too many already established. |
willy tarreau | 4302f49 | 2005-12-18 01:00:37 +0100 | [diff] [blame] | 2171 | |
willy tarreau | 4302f49 | 2005-12-18 01:00:37 +0100 | [diff] [blame] | 2172 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2173 | 4.3) HTTP header manipulation |
| 2174 | ----------------------------- |
| 2175 | In HTTP mode, it is possible to rewrite, add or delete some of the request and |
| 2176 | response headers based on regular expressions. It is also possible to block a |
| 2177 | request or a response if a particular header matches a regular expression, |
| 2178 | which is enough to stops most elementary protocol attacks, and to protect |
| 2179 | against information leak from the internal network. But there is a limitation |
| 2180 | to this : since haproxy's HTTP engine knows nothing about keep-alive, only |
| 2181 | headers passed during the first request of a TCP session will be seen. All |
| 2182 | subsequent headers will be considered data only and not analyzed. Furthermore, |
| 2183 | haproxy doesn't touch data contents, it stops at the end of headers. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2184 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2185 | The syntax is : |
| 2186 | reqadd <string> to add a header to the request |
| 2187 | reqrep <search> <replace> to modify the request |
| 2188 | reqirep <search> <replace> same, but ignoring the case |
| 2189 | reqdel <search> to delete a header in the request |
| 2190 | reqidel <search> same, but ignoring the case |
| 2191 | reqallow <search> definitely allow a request if a header matches <search> |
| 2192 | reqiallow <search> same, but ignoring the case |
| 2193 | reqdeny <search> denies a request if a header matches <search> |
| 2194 | reqideny <search> same, but ignoring the case |
| 2195 | reqpass <search> ignore a header matching <search> |
| 2196 | reqipass <search> same, but ignoring the case |
Willy Tarreau | 2272dc1 | 2006-09-03 10:19:38 +0200 | [diff] [blame] | 2197 | reqtarpit <search> tarpit a request matching <search> |
| 2198 | reqitarpit <search> same, but ignoring the case |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2199 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2200 | rspadd <string> to add a header to the response |
| 2201 | rsprep <search> <replace> to modify the response |
| 2202 | rspirep <search> <replace> same, but ignoring the case |
| 2203 | rspdel <search> to delete the response |
| 2204 | rspidel <search> same, but ignoring the case |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 2205 | rspdeny <search> replaces a response with a HTTP 502 if a header matches <search> |
| 2206 | rspideny <search> same, but ignoring the case |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2207 | |
| 2208 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2209 | <search> is a POSIX regular expression (regex) which supports grouping through |
| 2210 | parenthesis (without the backslash). Spaces and other delimiters must be |
| 2211 | prefixed with a backslash ('\') to avoid confusion with a field delimiter. |
| 2212 | Other characters may be prefixed with a backslash to change their meaning : |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2213 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2214 | \t for a tab |
| 2215 | \r for a carriage return (CR) |
| 2216 | \n for a new line (LF) |
| 2217 | \ to mark a space and differentiate it from a delimiter |
| 2218 | \# to mark a sharp and differentiate it from a comment |
| 2219 | \\ to use a backslash in a regex |
| 2220 | \\\\ to use a backslash in the text (*2 for regex, *2 for haproxy) |
| 2221 | \xXX to write the ASCII hex code XX as in the C language |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2222 | |
| 2223 | |
Willy Tarreau | 2272dc1 | 2006-09-03 10:19:38 +0200 | [diff] [blame] | 2224 | <replace> contains the string to be used to replace the largest portion of text |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2225 | matching the regex. It can make use of the special characters above, and can |
| 2226 | reference a substring delimited by parenthesis in the regex, by the group |
Willy Tarreau | 2272dc1 | 2006-09-03 10:19:38 +0200 | [diff] [blame] | 2227 | numerical order from 0 to 9 (0 being the entire line). In this case, you would |
| 2228 | write a backslash ('\') immediately followed by one digit indicating the group |
| 2229 | position. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2230 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2231 | <string> represents the string which will systematically be added after the last |
| 2232 | header line. It can also use special characters above. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2233 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2234 | Notes : |
| 2235 | ------- |
| 2236 | - the first line is considered as a header, which makes it possible to rewrite |
| 2237 | or filter HTTP requests URIs or response codes. |
| 2238 | - 'reqrep' is the equivalent of 'cliexp' in version 1.0, and 'rsprep' is the |
| 2239 | equivalent of 'srvexp' in 1.0. Those names are still supported but |
| 2240 | deprecated. |
| 2241 | - for performances reasons, the number of characters added to a request or to |
| 2242 | a response is limited to 4096 since version 1.1.5 (it was 256 before). This |
| 2243 | value is easy to modify in the code if needed (#define). If it is too short |
| 2244 | on occasional uses, it is possible to gain some space by removing some |
| 2245 | useless headers before adding new ones. |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 2246 | - a denied request will generate an "HTTP 403 forbidden" response, while a |
| 2247 | denied response will generate an "HTTP 502 Bad gateway" response. |
Willy Tarreau | 2272dc1 | 2006-09-03 10:19:38 +0200 | [diff] [blame] | 2248 | - a tarpitted request will be held open on the client side for a duration |
Willy Tarreau | 08fa2e3 | 2006-09-03 10:47:37 +0200 | [diff] [blame] | 2249 | defined in the contimeout parameter, or untill the client aborts. Nothing |
| 2250 | will be sent to any server. When the timeout is reached, the proxy will |
| 2251 | reply with a 500 server error response so that the attacker does not |
| 2252 | suspect it has been tarpitted. The logs may report the 500, but the |
| 2253 | termination flags will indicate 'PT' in this case. |
Willy Tarreau | 2272dc1 | 2006-09-03 10:19:38 +0200 | [diff] [blame] | 2254 | |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2255 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2256 | Examples : |
| 2257 | ---------- |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2258 | ###### a few examples ###### |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2259 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2260 | # rewrite 'online.fr' instead of 'free.fr' for GET and POST requests |
| 2261 | reqrep ^(GET\ .*)(.free.fr)(.*) \1.online.fr\3 |
| 2262 | reqrep ^(POST\ .*)(.free.fr)(.*) \1.online.fr\3 |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2263 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2264 | # force proxy connections to close |
| 2265 | reqirep ^Proxy-Connection:.* Proxy-Connection:\ close |
| 2266 | # rewrite locations |
| 2267 | rspirep ^(Location:\ )([^:]*://[^/]*)(.*) \1\3 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2268 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2269 | ###### A full configuration being used on production ###### |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2270 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2271 | # Every header should end with a colon followed by one space. |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2272 | reqideny ^[^:\ ]*[\ ]*$ |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2273 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2274 | # block Apache chunk exploit |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2275 | reqideny ^Transfer-Encoding:[\ ]*chunked |
| 2276 | reqideny ^Host:\ apache- |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2277 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2278 | # block annoying worms that fill the logs... |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2279 | reqideny ^[^:\ ]*\ .*(\.|%2e)(\.|%2e)(%2f|%5c|/|\\\\) |
| 2280 | reqideny ^[^:\ ]*\ ([^\ ]*\ [^\ ]*\ |.*%00) |
| 2281 | reqideny ^[^:\ ]*\ .*<script |
| 2282 | reqideny ^[^:\ ]*\ .*/(root\.exe\?|cmd\.exe\?|default\.ida\?) |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2283 | |
Willy Tarreau | 2272dc1 | 2006-09-03 10:19:38 +0200 | [diff] [blame] | 2284 | # tarpit attacks on the login page. |
| 2285 | reqtarpit ^[^:\ ]*\ .*\.php?login=[^0-9] |
| 2286 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2287 | # allow other syntactically valid requests, and block any other method |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2288 | reqipass ^(GET|POST|HEAD|OPTIONS)\ /.*\ HTTP/1\.[01]$ |
| 2289 | reqipass ^OPTIONS\ \\*\ HTTP/1\.[01]$ |
| 2290 | reqideny ^[^:\ ]*\ |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2291 | |
| 2292 | # force connection:close, thus disabling HTTP keep-alive |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2293 | option httpclose |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2294 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2295 | # change the server name |
| 2296 | rspidel ^Server:\ |
| 2297 | rspadd Server:\ Formilux/0.1.8 |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2298 | |
| 2299 | |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 2300 | Also, the 'forwardfor' option creates an HTTP 'X-Forwarded-For' header which |
willy tarreau | c1cae63 | 2005-12-17 14:12:23 +0100 | [diff] [blame] | 2301 | contains the client's IP address. This is useful to let the final web server |
Willy Tarreau | 7ac51f6 | 2007-03-25 16:00:04 +0200 | [diff] [blame] | 2302 | know what the client address was (eg for statistics on domains). Starting with |
| 2303 | version 1.3.8, it is possible to specify the "except" keyword followed by a |
| 2304 | source IP address or network for which no header will be added. This is very |
| 2305 | useful when another reverse-proxy which already adds the header runs on the |
| 2306 | same machine or in a known DMZ, the most common case being the local use of |
| 2307 | stunnel on the same system. |
willy tarreau | c1cae63 | 2005-12-17 14:12:23 +0100 | [diff] [blame] | 2308 | |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 2309 | Last, the 'httpclose' option removes any 'Connection' header both ways, and |
| 2310 | adds a 'Connection: close' header in each direction. This makes it easier to |
Willy TARREAU | 767ba71 | 2006-03-01 22:40:50 +0100 | [diff] [blame] | 2311 | disable HTTP keep-alive than the previous 4-rules block. |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 2312 | |
willy tarreau | c1cae63 | 2005-12-17 14:12:23 +0100 | [diff] [blame] | 2313 | Example : |
| 2314 | --------- |
| 2315 | listen http_proxy 0.0.0.0:80 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2316 | mode http |
| 2317 | log global |
| 2318 | option httplog |
| 2319 | option dontlognull |
Willy Tarreau | 7ac51f6 | 2007-03-25 16:00:04 +0200 | [diff] [blame] | 2320 | option forwardfor except 127.0.0.1/8 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2321 | option httpclose |
| 2322 | |
Willy TARREAU | 767ba71 | 2006-03-01 22:40:50 +0100 | [diff] [blame] | 2323 | Note that some HTTP servers do not necessarily close the connections when they |
| 2324 | receive the 'Connection: close', and if the client does not close either, then |
| 2325 | the connection will be maintained up to the time-out. This translates into high |
| 2326 | number of simultaneous sessions and high global session times in the logs. To |
| 2327 | workaround this, a new option 'forceclose' appeared in version 1.2.9 to enforce |
| 2328 | the closing of the outgoing server channel as soon as the server begins to |
| 2329 | reply and only if the request buffer is empty. Note that this should NOT be |
| 2330 | used if CONNECT requests are expected between the client and the server. The |
| 2331 | 'forceclose' option implies the 'httpclose' option. |
| 2332 | |
| 2333 | Example : |
| 2334 | --------- |
| 2335 | listen http_proxy 0.0.0.0:80 |
| 2336 | mode http |
| 2337 | log global |
| 2338 | option httplog |
| 2339 | option dontlognull |
| 2340 | option forwardfor |
| 2341 | option forceclose |
| 2342 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2343 | |
| 2344 | 4.4) Load balancing with persistence |
| 2345 | ------------------------------------ |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2346 | Combining cookie insertion with internal load balancing allows to transparently |
| 2347 | bring persistence to applications. The principle is quite simple : |
| 2348 | - assign a cookie value to each server |
| 2349 | - enable the load balancing between servers |
| 2350 | - insert a cookie into responses resulting from the balancing algorithm |
| 2351 | (indirect accesses), end ensure that no upstream proxy will cache it. |
| 2352 | - remove the cookie in the request headers so that the application never sees |
| 2353 | it. |
| 2354 | |
| 2355 | Example : |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2356 | --------- |
| 2357 | listen application 0.0.0.0:80 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2358 | mode http |
| 2359 | cookie SERVERID insert nocache indirect |
| 2360 | balance roundrobin |
| 2361 | server srv1 192.168.1.1:80 cookie server01 check |
| 2362 | server srv2 192.168.1.2:80 cookie server02 check |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2363 | |
willy tarreau | 0174f31 | 2005-12-18 01:02:42 +0100 | [diff] [blame] | 2364 | The other solution brought by versions 1.1.30 and 1.2.3 is to reuse a cookie |
| 2365 | from the server, and prefix the server's name to it. In this case, don't forget |
| 2366 | to force "httpclose" mode so that you can be assured that every subsequent |
| 2367 | request will have its cookie fixed. |
| 2368 | |
| 2369 | listen application 0.0.0.0:80 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2370 | mode http |
| 2371 | cookie JSESSIONID prefix |
| 2372 | balance roundrobin |
| 2373 | server srv1 192.168.1.1:80 cookie srv1 check |
| 2374 | server srv2 192.168.1.2:80 cookie srv2 check |
| 2375 | option httpclose |
willy tarreau | 0174f31 | 2005-12-18 01:02:42 +0100 | [diff] [blame] | 2376 | |
| 2377 | |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 2378 | 4.5) Protection against information leak from the servers |
| 2379 | --------------------------------------------------------- |
| 2380 | In versions 1.1.28/1.2.1, a new option 'checkcache' was created. It carefully |
| 2381 | checks 'Cache-control', 'Pragma' and 'Set-cookie' headers in server response |
| 2382 | to check if there's a risk of caching a cookie on a client-side proxy. When this |
| 2383 | option is enabled, the only responses which can be delivered to the client are : |
| 2384 | - all those without 'Set-Cookie' header ; |
| 2385 | - all those with a return code other than 200, 203, 206, 300, 301, 410, |
| 2386 | provided that the server has not set a 'Cache-control: public' header ; |
| 2387 | - all those that come from a POST request, provided that the server has not |
| 2388 | set a 'Cache-Control: public' header ; |
| 2389 | - those with a 'Pragma: no-cache' header |
| 2390 | - those with a 'Cache-control: private' header |
| 2391 | - those with a 'Cache-control: no-store' header |
| 2392 | - those with a 'Cache-control: max-age=0' header |
| 2393 | - those with a 'Cache-control: s-maxage=0' header |
| 2394 | - those with a 'Cache-control: no-cache' header |
| 2395 | - those with a 'Cache-control: no-cache="set-cookie"' header |
| 2396 | - those with a 'Cache-control: no-cache="set-cookie,' header |
| 2397 | (allowing other fields after set-cookie) |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2398 | |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 2399 | If a response doesn't respect these requirements, then it will be blocked just |
| 2400 | as if it was from an 'rspdeny' filter, with an "HTTP 502 bad gateway". The |
| 2401 | session state shows "PH--" meaning that the proxy blocked the response during |
| 2402 | headers processing. Additionnaly, an alert will be sent in the logs so that |
| 2403 | admins are told that there's something to be done. |
| 2404 | |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2405 | |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 2406 | 4.6) Customizing errors |
| 2407 | ----------------------- |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2408 | Some situations can make haproxy return an HTTP error code to the client : |
| 2409 | - invalid or too long request => HTTP 400 |
| 2410 | - request not completely sent in time => HTTP 408 |
| 2411 | - forbidden request (matches a deny filter) => HTTP 403 |
| 2412 | - internal error in haproxy => HTTP 500 |
| 2413 | - the server returned an invalid or incomplete response => HTTP 502 |
| 2414 | - no server was available to handle the request => HTTP 503 |
| 2415 | - the server failed to reply in time => HTTP 504 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2416 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2417 | A succint error message taken from the RFC accompanies these return codes. |
| 2418 | But depending on the clients knowledge, it may be better to return custom, user |
Willy Tarreau | 3f49b30 | 2007-06-11 00:29:26 +0200 | [diff] [blame] | 2419 | friendly, error pages. This is made possible in two ways, one involving a |
| 2420 | redirection to a known server, and another one consisting in returning a local |
| 2421 | file. |
| 2422 | |
| 2423 | 4.6.1) Relocation |
| 2424 | ----------------- |
| 2425 | An error relocation is achieved using the 'errorloc' command : |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2426 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2427 | errorloc <HTTP_code> <location> |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2428 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2429 | Instead of generating an HTTP error <HTTP_code> among those above, the proxy |
| 2430 | will return a temporary redirection code (HTTP 302) towards the address |
| 2431 | specified in <location>. This address may be either relative to the site or |
| 2432 | absolute. Since this request will be handled by the client's browser, it's |
| 2433 | mandatory that the returned address be reachable from the outside. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2434 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2435 | Example : |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2436 | --------- |
| 2437 | listen application 0.0.0.0:80 |
| 2438 | errorloc 400 /badrequest.html |
| 2439 | errorloc 403 /forbidden.html |
| 2440 | errorloc 408 /toolong.html |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2441 | errorloc 500 http://haproxy.domain.net/bugreport.html |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2442 | errorloc 502 http://192.168.114.58/error50x.html |
| 2443 | errorloc 503 http://192.168.114.58/error50x.html |
| 2444 | errorloc 504 http://192.168.114.58/error50x.html |
| 2445 | |
willy tarreau | c1f4753 | 2005-12-18 01:08:26 +0100 | [diff] [blame] | 2446 | Note: RFC2616 says that a client must reuse the same method to fetch the |
| 2447 | Location returned by a 302, which causes problems with the POST method. |
| 2448 | The return code 303 was designed explicitly to force the client to fetch the |
| 2449 | Location URL with the GET method, but there are some browsers pre-dating |
| 2450 | HTTP/1.1 which don't support it. Anyway, most browsers still behave with 302 as |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2451 | if it was a 303. In order to allow the user to chose, versions 1.1.31 and 1.2.5 |
| 2452 | bring two new keywords to replace 'errorloc' : 'errorloc302' and 'errorloc303'. |
willy tarreau | c1f4753 | 2005-12-18 01:08:26 +0100 | [diff] [blame] | 2453 | |
| 2454 | They are preffered over errorloc (which still does 302). Consider using |
| 2455 | errorloc303 everytime you know that your clients support HTTP 303 responses.. |
| 2456 | |
Willy Tarreau | 3f49b30 | 2007-06-11 00:29:26 +0200 | [diff] [blame] | 2457 | 4.6.2) Local files |
| 2458 | ------------------ |
| 2459 | Sometimes, it is desirable to change the returned error without resorting to |
| 2460 | redirections. The second method consists in loading local files during startup |
| 2461 | and send them as pure HTTP content upon error. This is what the 'errorfile' |
| 2462 | keyword does. |
| 2463 | |
| 2464 | Warning, there are traps to consider : |
| 2465 | - The files are loaded while parsing configuration, before doing a chroot(). |
| 2466 | Thus, they are relative to the real filesystem. For this reason, it is |
| 2467 | recommended to pass an absolute path to those files. |
| 2468 | |
| 2469 | - The contents of those files is not HTML, but real HTTP protocol with |
| 2470 | possible HTML body. So the first line and headers are mandatory. Ideally, |
| 2471 | every line in the HTTP part should end with CR-LF for maximum compatibility. |
| 2472 | |
| 2473 | - The response is limited to the buffer size (BUSIZE), generally 8 or 16 kB. |
| 2474 | |
| 2475 | - The response should not include references to the local server, in order to |
| 2476 | avoid infinite loops on the browser in case of local failure. |
| 2477 | |
| 2478 | Example : |
| 2479 | --------- |
| 2480 | errorfile 400 /etc/haproxy/errorfiles/400badreq.http |
| 2481 | errorfile 403 /etc/haproxy/errorfiles/403forbid.http |
| 2482 | errorfile 503 /etc/haproxy/errorfiles/503sorry.http |
| 2483 | |
willy tarreau | c1f4753 | 2005-12-18 01:08:26 +0100 | [diff] [blame] | 2484 | |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 2485 | 4.7) Modifying default values |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2486 | ----------------------------- |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2487 | Version 1.1.22 introduced the notion of default values, which eliminates the |
| 2488 | pain of often repeating common parameters between many instances, such as |
| 2489 | logs, timeouts, modes, etc... |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2490 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2491 | Default values are set in a 'defaults' section. Each of these section clears |
| 2492 | all previously set default parameters, so there may be as many default |
| 2493 | parameters as needed. Only the last one before a 'listen' section will be |
| 2494 | used for this section. The 'defaults' section uses the same syntax as the |
| 2495 | 'listen' section, for the supported parameters. The 'defaults' keyword ignores |
| 2496 | everything on its command line, so that fake instance names can be specified |
| 2497 | there for better clarity. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2498 | |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 2499 | In version 1.1.28/1.2.1, only those parameters can be preset in the 'default' |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2500 | section : |
| 2501 | - log (the first and second one) |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2502 | - mode { tcp, http, health } |
| 2503 | - balance { roundrobin } |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2504 | - disabled (to disable every further instances) |
| 2505 | - enabled (to enable every further instances, this is the default) |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2506 | - contimeout, clitimeout, srvtimeout, grace, retries, maxconn |
willy tarreau | 982249e | 2005-12-18 00:57:06 +0100 | [diff] [blame] | 2507 | - option { redispatch, transparent, keepalive, forwardfor, logasap, httpclose, |
| 2508 | checkcache, httplog, tcplog, dontlognull, persist, httpchk } |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2509 | - redispatch, redisp, transparent, source { addr:port } |
| 2510 | - cookie, capture |
| 2511 | - errorloc |
| 2512 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2513 | As of 1.1.24, it is not possible to put certain parameters in a 'defaults' |
| 2514 | section, mainly regular expressions and server configurations : |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2515 | - dispatch, server, |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2516 | - req*, rsp* |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2517 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2518 | Last, there's no way yet to change a boolean option from its assigned default |
| 2519 | value. So if an 'option' statement is set in a 'defaults' section, the only |
| 2520 | way to flush it is to redefine a new 'defaults' section without this 'option'. |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2521 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2522 | Examples : |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2523 | ---------- |
| 2524 | defaults applications TCP |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2525 | log global |
| 2526 | mode tcp |
| 2527 | balance roundrobin |
| 2528 | clitimeout 180000 |
| 2529 | srvtimeout 180000 |
| 2530 | contimeout 4000 |
| 2531 | retries 3 |
| 2532 | redispatch |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2533 | |
| 2534 | listen app_tcp1 10.0.0.1:6000-6063 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2535 | server srv1 192.168.1.1 check port 6000 inter 10000 |
| 2536 | server srv2 192.168.1.2 backup |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2537 | |
| 2538 | listen app_tcp2 10.0.0.2:6000-6063 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2539 | server srv1 192.168.2.1 check port 6000 inter 10000 |
| 2540 | server srv2 192.168.2.2 backup |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2541 | |
| 2542 | defaults applications HTTP |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2543 | log global |
| 2544 | mode http |
| 2545 | option httplog |
| 2546 | option forwardfor |
| 2547 | option dontlognull |
| 2548 | balance roundrobin |
| 2549 | clitimeout 20000 |
| 2550 | srvtimeout 20000 |
| 2551 | contimeout 4000 |
| 2552 | retries 3 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2553 | |
| 2554 | listen app_http1 10.0.0.1:80-81 |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2555 | cookie SERVERID postonly insert indirect |
| 2556 | capture cookie userid= len 10 |
| 2557 | server srv1 192.168.1.1:+8000 cookie srv1 check port 8080 inter 1000 |
| 2558 | server srv1 192.168.1.2:+8000 cookie srv2 check port 8080 inter 1000 |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2559 | |
| 2560 | defaults |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2561 | # this empty section voids all default parameters |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2562 | |
willy tarreau | 481132e | 2006-05-21 21:43:10 +0200 | [diff] [blame] | 2563 | |
| 2564 | 4.8) Status report in HTML page |
| 2565 | ------------------------------- |
| 2566 | Starting with 1.2.14, it is possible for HAProxy to intercept requests for a |
| 2567 | particular URI and return a full report of the proxy's activity and servers |
| 2568 | statistics. This is available through the 'stats' keyword, associated to any |
| 2569 | such options : |
| 2570 | |
| 2571 | - stats enable |
| 2572 | - stats uri <uri prefix> |
| 2573 | - stats realm <authentication realm> |
| 2574 | - stats auth <user:password> |
| 2575 | - stats scope <proxy_id> | '.' |
| 2576 | |
| 2577 | By default, the status report is disabled. Specifying any combination above |
| 2578 | enables it for the proxy instance referencing it. The easiest solution is to |
| 2579 | use "stats enable" which will enable the report with default parameters : |
| 2580 | |
| 2581 | - default URI : "/haproxy?stats" (CONFIG_STATS_DEFAULT_URI) |
| 2582 | - default auth : unspecified (no authentication) |
| 2583 | - default realm : "HAProxy Statistics" (CONFIG_STATS_DEFAULT_REALM) |
| 2584 | - default scope : unspecified (access to all instances) |
| 2585 | |
| 2586 | The "stats uri <uri_prefix>" option allows one to intercept another URI prefix. |
| 2587 | Note that any URI that BEGINS with this string will match. For instance, one |
| 2588 | proxy instance might be dedicated to status page only and would reply to any |
| 2589 | URI. |
| 2590 | |
| 2591 | Example : |
| 2592 | --------- |
| 2593 | # catches any URI and returns the status page. |
| 2594 | listen stats :8080 |
| 2595 | mode http |
| 2596 | stats uri / |
| 2597 | |
| 2598 | The "stats auth <user:password>" option enables Basic authentication and adds a |
| 2599 | valid user:password combination to the list of authorized accounts. The user |
| 2600 | and password are passed in the configuration file as clear text, and since this |
| 2601 | is HTTP Basic authentication, you should be aware that it transits as clear |
| 2602 | text on the network, so you must not use any sensible account. The list is |
| 2603 | unlimited in order to provide easy accesses to developpers or customers. |
| 2604 | |
| 2605 | The "stats realm <realm>" option defines the "realm" name which is displayed |
| 2606 | in the popup box when the browser asks for a password. It's important to ensure |
| 2607 | that this one is not used by the application, otherwise the browser will try to |
| 2608 | use a cached one from the application. Note that any space in the realm name |
| 2609 | should be escaped with a backslash ('\'). |
| 2610 | |
| 2611 | The "stats scope <proxy_id>" option limits the scope of the status report. By |
| 2612 | default, all proxy instances are listed. But under some circumstances, it would |
| 2613 | be better to limit the listing to some proxies or only to the current one. This |
| 2614 | is what this option does. The special proxy name "." (a single dot) references |
| 2615 | the current proxy. The proxy name can be repeated multiple times, even for |
| 2616 | proxies defined later in the configuration or some which do not exist. The name |
| 2617 | is the one which appears after the 'listen' keyword. |
| 2618 | |
| 2619 | Example : |
| 2620 | --------- |
| 2621 | # simple application with authenticated embedded status report |
| 2622 | listen app1 192.168.1.100:80 |
| 2623 | mode http |
willy tarreau | d4ba08d | 2006-05-21 21:54:14 +0200 | [diff] [blame] | 2624 | option httpclose |
willy tarreau | 481132e | 2006-05-21 21:43:10 +0200 | [diff] [blame] | 2625 | balance roundrobin |
| 2626 | cookie SERVERID postonly insert indirect |
| 2627 | server srv1 192.168.1.1:8080 cookie srv1 check inter 1000 |
| 2628 | server srv1 192.168.1.2:8080 cookie srv2 check inter 1000 |
| 2629 | stats uri /my_stats |
willy tarreau | d4ba08d | 2006-05-21 21:54:14 +0200 | [diff] [blame] | 2630 | stats realm Statistics\ for\ MyApp1-2 |
| 2631 | stats auth guest:guest |
| 2632 | stats auth admin:AdMiN123 |
| 2633 | stats scope . |
| 2634 | stats scope app2 |
willy tarreau | 481132e | 2006-05-21 21:43:10 +0200 | [diff] [blame] | 2635 | |
| 2636 | # simple application with anonymous embedded status report |
| 2637 | listen app2 192.168.2.100:80 |
| 2638 | mode http |
willy tarreau | d4ba08d | 2006-05-21 21:54:14 +0200 | [diff] [blame] | 2639 | option httpclose |
willy tarreau | 481132e | 2006-05-21 21:43:10 +0200 | [diff] [blame] | 2640 | balance roundrobin |
| 2641 | cookie SERVERID postonly insert indirect |
| 2642 | server srv1 192.168.2.1:8080 cookie srv1 check inter 1000 |
| 2643 | server srv1 192.168.2.2:8080 cookie srv2 check inter 1000 |
| 2644 | stats uri /my_stats |
willy tarreau | d4ba08d | 2006-05-21 21:54:14 +0200 | [diff] [blame] | 2645 | stats realm Statistics\ for\ MyApp2 |
| 2646 | stats scope . |
willy tarreau | 481132e | 2006-05-21 21:43:10 +0200 | [diff] [blame] | 2647 | |
| 2648 | listen admin_page :8080 |
| 2649 | mode http |
| 2650 | stats uri /my_stats |
willy tarreau | d4ba08d | 2006-05-21 21:54:14 +0200 | [diff] [blame] | 2651 | stats realm Global\ statistics |
| 2652 | stats auth admin:AdMiN123 |
willy tarreau | 481132e | 2006-05-21 21:43:10 +0200 | [diff] [blame] | 2653 | |
| 2654 | Notes : |
| 2655 | ------- |
| 2656 | - The 'stats' options can also be specified in the 'defaults' section, in |
| 2657 | which case it will provide the exact same configuration to all further |
| 2658 | instances (hence the usefulness of the scope "."). However, if an instance |
| 2659 | redefines any 'stats' parameter, defaults will not be used for this |
| 2660 | instance. |
| 2661 | |
| 2662 | - HTTP Basic authentication is very basic and unsecure from snooping. No |
| 2663 | sensible password should be used, and be aware that there is no way to |
| 2664 | remove it from the browser so it will be sent to the whole application |
| 2665 | upon further accesses. |
| 2666 | |
willy tarreau | d4ba08d | 2006-05-21 21:54:14 +0200 | [diff] [blame] | 2667 | - It is very important that the 'option httpclose' is specified, otherwise |
| 2668 | the proxy will not be able to detect the URI within keep-alive sessions |
| 2669 | maintained between the browser and the servers, so the stats URI will be |
| 2670 | forwarded unmodified to the server as if the option was not set. |
| 2671 | |
willy tarreau | 481132e | 2006-05-21 21:43:10 +0200 | [diff] [blame] | 2672 | |
Willy Tarreau | 726c2bf | 2007-05-09 01:31:45 +0200 | [diff] [blame] | 2673 | 5) Access lists |
| 2674 | =============== |
| 2675 | |
| 2676 | With version 1.3.10, a new concept of access lists (acl) was born. As it was |
| 2677 | not necesary to reinvent the wheel, and because even long thoughts lead to |
| 2678 | unsatisfying proposals, it was finally decided that something close to what |
| 2679 | Squid provides would be a good compromise between features and ease of use. |
| 2680 | |
| 2681 | The principle is very simple : acls are declared with a name, a test and a list |
| 2682 | of valid values to check against during the test. Conditions are applied on |
| 2683 | various actions, and those conditions apply a logical AND between acls. The |
| 2684 | condition is then only met if all acls are true. |
| 2685 | |
| 2686 | It is possible to use the reserved keyword "OR" in conditions, and it is |
| 2687 | possible for an acl to be specified multiple times, even with various tests, in |
| 2688 | which case the first one which returns true validates the ACL. |
| 2689 | |
Willy Tarreau | ae8b796 | 2007-06-09 23:10:04 +0200 | [diff] [blame] | 2690 | As of 1.3.12, only the following tests have been implemented : |
Willy Tarreau | 726c2bf | 2007-05-09 01:31:45 +0200 | [diff] [blame] | 2691 | |
| 2692 | Layer 3/4 : |
| 2693 | src <ipv4_address>[/mask] ... : match IPv4 source address |
| 2694 | dst <ipv4_address>[/mask] ... : match IPv4 destination address |
Willy Tarreau | ae8b796 | 2007-06-09 23:10:04 +0200 | [diff] [blame] | 2695 | src_port <range> ... : match source port range |
| 2696 | dst_port <range> ... : match destination port range |
| 2697 | dst_conn <range> ... : match #connections on frontend |
Willy Tarreau | 726c2bf | 2007-05-09 01:31:45 +0200 | [diff] [blame] | 2698 | |
| 2699 | Layer 7 : |
| 2700 | method <HTTP method> ... : match HTTP method |
| 2701 | req_ver <1.0|1.1> ... : match HTTP request version |
| 2702 | resp_ver <1.0|1.1> ... : match HTTP response version |
Willy Tarreau | ae8b796 | 2007-06-09 23:10:04 +0200 | [diff] [blame] | 2703 | status <range> ... : match HTTP response status code in range |
Willy Tarreau | 726c2bf | 2007-05-09 01:31:45 +0200 | [diff] [blame] | 2704 | url <string> ... : exact string match on URI |
| 2705 | url_reg <regex> ... : regex string match on URI |
| 2706 | url_beg <string> ... : true if URI begins with <string> |
| 2707 | url_end <string> ... : true if URI ends with <string> |
| 2708 | url_sub <string> ... : true if URI contains <string> |
| 2709 | url_dir <string> ... : true if URI contains <string> between slashes |
| 2710 | url_dom <string> ... : true if URI contains <string> between slashes or dots |
| 2711 | |
Willy Tarreau | ae8b796 | 2007-06-09 23:10:04 +0200 | [diff] [blame] | 2712 | A 'range' is one or two integers which may be prefixed by an operator. |
| 2713 | The syntax is : |
| 2714 | |
| 2715 | [<op>] <low>[:<high>] |
| 2716 | |
| 2717 | Where <op> can be : |
| 2718 | 'eq' : the tested value must be equal to <low> or within <low>..<high> |
| 2719 | 'le' : the tested value must be lower than or equal to <low> |
| 2720 | 'lt' : the tested value must be lower than <low> |
| 2721 | 'ge' : the tested value must be greater than or equal to <low> |
| 2722 | 'gt' : the tested value must be greater than <low> |
| 2723 | |
| 2724 | When no operator is defined, 'eq' is assumed. Note that when the operator is |
| 2725 | specified, it applies to all subsequent ranges of values until the end of the |
| 2726 | line is reached or another operator is specified. Example : |
| 2727 | |
| 2728 | acl status_error status 400:599 |
| 2729 | acl saturated_frt dst_conn ge 1000 |
| 2730 | acl invalid_ports src_port lt 512 ge 65535 |
| 2731 | |
Willy Tarreau | 726c2bf | 2007-05-09 01:31:45 +0200 | [diff] [blame] | 2732 | Other ones are coming (headers, cookies, time, auth), it's just a matter of |
| 2733 | time. It is also planned to be able to read the patterns from a file, as well |
| 2734 | as to ignore the case for some of them. |
| 2735 | |
| 2736 | The only command supporting a condition right now is the "block" command, which |
| 2737 | blocks a request and returns a 403 if its condition is true (with the "if" |
| 2738 | keyword), or if it is false (with the "unless" keyword). |
| 2739 | |
| 2740 | Example : |
| 2741 | --------- |
| 2742 | |
| 2743 | acl options_uris url * |
| 2744 | acl meth_option method OPTIONS |
| 2745 | acl http_1.1 req_ver 1.1 |
| 2746 | acl allowed_meth method GET HEAD POST OPTIONS CONNECT |
| 2747 | acl connect_meth method CONNECT |
| 2748 | acl proxy_url url_beg http:// |
| 2749 | |
| 2750 | # block if reserved URI "*" used with a method other than "OPTIONS" |
| 2751 | block if options_uris !meth_option |
| 2752 | |
| 2753 | # block if the OPTIONS method is used with HTTP 1.0 |
| 2754 | block if meth_option !http_1.1 |
| 2755 | |
| 2756 | # allow non-proxy url with anything but the CONNECT method |
| 2757 | block if !connect_meth !proxy_url |
| 2758 | |
| 2759 | # block all unknown methods |
| 2760 | block unless allowed_meth |
| 2761 | |
| 2762 | Note: this documentation is very light but should permit one to start and above |
| 2763 | all it should permit to work on the project without being slowed down too much |
| 2764 | with the doc. |
| 2765 | |
| 2766 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2767 | ========================= |
| 2768 | | System-specific setup | |
| 2769 | ========================= |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2770 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2771 | Linux 2.4 |
| 2772 | ========= |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2773 | |
| 2774 | -- cut here -- |
| 2775 | #!/bin/sh |
| 2776 | # set this to about 256/4M (16384 for 256M machine) |
| 2777 | MAXFILES=16384 |
| 2778 | echo $MAXFILES > /proc/sys/fs/file-max |
| 2779 | ulimit -n $MAXFILES |
| 2780 | |
| 2781 | if [ -e /proc/sys/net/ipv4/ip_conntrack_max ]; then |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2782 | echo 65536 > /proc/sys/net/ipv4/ip_conntrack_max |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2783 | fi |
| 2784 | |
| 2785 | if [ -e /proc/sys/net/ipv4/netfilter/ip_ct_tcp_timeout_fin_wait ]; then |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2786 | # 30 seconds for fin, 15 for time wait |
| 2787 | echo 3000 > /proc/sys/net/ipv4/netfilter/ip_ct_tcp_timeout_fin_wait |
| 2788 | echo 1500 > /proc/sys/net/ipv4/netfilter/ip_ct_tcp_timeout_time_wait |
| 2789 | echo 0 > /proc/sys/net/ipv4/netfilter/ip_ct_tcp_log_invalid_scale |
| 2790 | echo 0 > /proc/sys/net/ipv4/netfilter/ip_ct_tcp_log_out_of_window |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2791 | fi |
| 2792 | |
| 2793 | echo 1024 60999 > /proc/sys/net/ipv4/ip_local_port_range |
| 2794 | echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout |
| 2795 | echo 4096 > /proc/sys/net/ipv4/tcp_max_syn_backlog |
| 2796 | echo 262144 > /proc/sys/net/ipv4/tcp_max_tw_buckets |
| 2797 | echo 262144 > /proc/sys/net/ipv4/tcp_max_orphans |
| 2798 | echo 300 > /proc/sys/net/ipv4/tcp_keepalive_time |
| 2799 | echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle |
| 2800 | echo 0 > /proc/sys/net/ipv4/tcp_timestamps |
| 2801 | echo 0 > /proc/sys/net/ipv4/tcp_ecn |
willy tarreau | c5f73ed | 2005-12-18 01:26:38 +0100 | [diff] [blame] | 2802 | echo 1 > /proc/sys/net/ipv4/tcp_sack |
willy tarreau | eedaa9f | 2005-12-17 14:08:03 +0100 | [diff] [blame] | 2803 | echo 0 > /proc/sys/net/ipv4/tcp_dsack |
| 2804 | |
| 2805 | # auto-tuned on 2.4 |
| 2806 | #echo 262143 > /proc/sys/net/core/rmem_max |
| 2807 | #echo 262143 > /proc/sys/net/core/rmem_default |
| 2808 | |
| 2809 | echo 16384 65536 524288 > /proc/sys/net/ipv4/tcp_rmem |
| 2810 | echo 16384 349520 699040 > /proc/sys/net/ipv4/tcp_wmem |
| 2811 | |
| 2812 | -- cut here -- |
| 2813 | |
willy tarreau | 197e8ec | 2005-12-17 14:10:59 +0100 | [diff] [blame] | 2814 | |
| 2815 | FreeBSD |
| 2816 | ======= |
| 2817 | |
| 2818 | A FreeBSD port of HA-Proxy is now available and maintained, thanks to |
| 2819 | Clement Laforet <sheepkiller@cultdeadsheep.org>. |
| 2820 | |
| 2821 | For more information : |
| 2822 | http://www.freebsd.org/cgi/url.cgi?ports/net/haproxy/pkg-descr |
| 2823 | http://www.freebsd.org/cgi/cvsweb.cgi/ports/net/haproxy/ |
| 2824 | http://www.freshports.org/net/haproxy |
| 2825 | |
| 2826 | |
| 2827 | -- end -- |