blob: 3d2a2be7dcaaffd4f354ec464c963c3747952a63 [file] [log] [blame]
willy tarreau0174f312005-12-18 01:02:42 +01001 -------------------
Willy Tarreau94b45912006-05-31 06:40:15 +02002 HAProxy
willy tarreau0174f312005-12-18 01:02:42 +01003 Reference Manual
4 -------------------
Willy Tarreau7b4c5ae2008-04-19 21:06:14 +02005 version 1.3.15
willy tarreauc5f73ed2005-12-18 01:26:38 +01006 willy tarreau
Willy Tarreau7b4c5ae2008-04-19 21:06:14 +02007 2008/04/19
willy tarreaueedaa9f2005-12-17 14:08:03 +01008
Willy Tarreaua080eca2009-10-14 20:30:15 +02009
10 !!!! NOTE: THIS DOCUMENT IS OUTDATED !!!!
11
12 Please use "configuration.txt" from the same directory, or download
13 an up-to-date version from the following location :
14
Willy Tarreaube6008f2009-10-14 22:22:03 +020015 http://haproxy.1wt.eu/download/1.4/doc/configuration.txt
Willy Tarreaua080eca2009-10-14 20:30:15 +020016
17
willy tarreaueedaa9f2005-12-17 14:08:03 +010018============
19| Abstract |
20============
21
Willy Tarreau94b45912006-05-31 06:40:15 +020022HAProxy is a TCP/HTTP reverse proxy which is particularly suited for high
willy tarreaueedaa9f2005-12-17 14:08:03 +010023availability environments. Indeed, it can :
24 - route HTTP requests depending on statically assigned cookies ;
25 - spread the load among several servers while assuring server persistence
26 through the use of HTTP cookies ;
27 - switch to backup servers in the event a main one fails ;
28 - accept connections to special ports dedicated to service monitoring ;
29 - stop accepting connections without breaking existing ones ;
30 - add/modify/delete HTTP headers both ways ;
31 - block requests matching a particular pattern ;
willy tarreau532bb552006-05-13 18:40:37 +020032 - hold clients to the right application server depending on application
33 cookies
willy tarreau481132e2006-05-21 21:43:10 +020034 - report detailed status as HTML pages to authenticated users from an URI
35 intercepted from the application.
willy tarreaueedaa9f2005-12-17 14:08:03 +010036
37It needs very little resource. Its event-driven architecture allows it to easily
38handle thousands of simultaneous connections on hundreds of instances without
39risking the system's stability.
40
41====================
42| Start parameters |
43====================
44
45There are only a few command line options :
46
47 -f <configuration file>
48 -n <high limit for the total number of simultaneous connections>
willy tarreau532bb552006-05-13 18:40:37 +020049 = 'maxconn' in 'global' section
50 -N <high limit for the per-listener number of simultaneous connections>
51 = 'maxconn' in 'listen' or 'default' sections
willy tarreaueedaa9f2005-12-17 14:08:03 +010052 -d starts in foregreound with debugging mode enabled
53 -D starts in daemon mode
willy tarreau982249e2005-12-18 00:57:06 +010054 -q disable messages on output
55 -V displays messages on output even when -q or 'quiet' are specified.
56 -c only checks config file and exits with code 0 if no error was found, or
57 exits with code 1 if a syntax error was found.
willy tarreaufe2c5c12005-12-17 14:14:34 +010058 -p <pidfile> asks the process to write down each of its children's
59 pids to this file in daemon mode.
willy tarreau34f45302006-04-15 21:37:14 +020060 -sf specifies a list of pids to send a FINISH signal to after startup.
61 -st specifies a list of pids to send a TERMINATE signal to after startup.
willy tarreaueedaa9f2005-12-17 14:08:03 +010062 -s shows statistics (only if compiled in)
63 -l shows even more statistics (implies '-s')
Willy Tarreaude99e992007-04-16 00:53:59 +020064 -dk disables use of kqueue()
willy tarreau64a3cc32005-12-18 01:13:11 +010065 -de disables use of epoll()
66 -dp disables use of poll()
willy tarreau34f45302006-04-15 21:37:14 +020067 -db disables background mode (stays in foreground, useful for debugging)
68 -m <megs> enforces a memory usage limit to a maximum of <megs> megabytes.
willy tarreaueedaa9f2005-12-17 14:08:03 +010069
willy tarreau532bb552006-05-13 18:40:37 +020070The maximal number of connections per proxy instance is used as the default
71parameter for each instance for which the 'maxconn' paramter is not set in the
72'listen' section.
willy tarreaueedaa9f2005-12-17 14:08:03 +010073
74The maximal number of total connections limits the number of connections used by
75the whole process if the 'maxconn' parameter is not set in the 'global' section.
76
77The debugging mode has the same effect as the 'debug' option in the 'global'
78section. When the proxy runs in this mode, it dumps every connections,
79disconnections, timestamps, and HTTP headers to stdout. This should NEVER
80be used in an init script since it will prevent the system from starting up.
81
willy tarreau34f45302006-04-15 21:37:14 +020082For debugging, the '-db' option is very useful as it temporarily disables
83daemon mode and multi-process mode. The service can then be stopped by simply
84pressing Ctrl-C, without having to edit the config nor run full debug.
85
willy tarreaueedaa9f2005-12-17 14:08:03 +010086Statistics are only available if compiled in with the 'STATTIME' option. It's
willy tarreau481132e2006-05-21 21:43:10 +020087only used during code optimization phases, and will soon disappear.
willy tarreaueedaa9f2005-12-17 14:08:03 +010088
willy tarreau532bb552006-05-13 18:40:37 +020089The '-st' and '-sf' options are used for hot reconfiguration (see below).
willy tarreau34f45302006-04-15 21:37:14 +020090
willy tarreaueedaa9f2005-12-17 14:08:03 +010091======================
92| Configuration file |
93======================
94
95Structure
96=========
97
98The configuration file parser ignores empty lines, spaces, tabs. Anything
99between a sharp ('#') not following a backslash ('\'), and the end of a line
100constitutes a comment and is ignored too.
101
102The configuration file is segmented in sections. A section begins whenever
103one of these 3 keywords are encountered :
104
105 - 'global'
106 - 'listen'
107 - 'defaults'
108
109Every parameter refer to the section beginning at the last one of these 3
110keywords.
111
112
1131) Global parameters
114====================
115
116Global parameters affect the whole process behaviour. They are all set in the
117'global' section. There may be several 'global' sections if needed, but their
118parameters will only be merged. Allowed parameters in 'global' section include
119the following ones :
120
121 - log <address> <facility> [max_level]
122 - maxconn <number>
123 - uid <user id>
124 - gid <group id>
Willy Tarreau95c20ac2007-03-25 15:39:23 +0200125 - user <user name>
126 - group <group name>
willy tarreaueedaa9f2005-12-17 14:08:03 +0100127 - chroot <directory>
128 - nbproc <number>
129 - daemon
130 - debug
Willy Tarreaude99e992007-04-16 00:53:59 +0200131 - nokqueue
willy tarreau64a3cc32005-12-18 01:13:11 +0100132 - noepoll
133 - nopoll
willy tarreaueedaa9f2005-12-17 14:08:03 +0100134 - quiet
willy tarreaufe2c5c12005-12-17 14:14:34 +0100135 - pidfile <file>
willy tarreauc5f73ed2005-12-18 01:26:38 +0100136 - ulimit-n <number>
willy tarreau598da412005-12-18 01:07:29 +0100137 - stats
Willy Tarreau1db37712007-06-03 17:16:49 +0200138 - tune.maxpollevents <number>
willy tarreaueedaa9f2005-12-17 14:08:03 +0100139
willy tarreauc5f73ed2005-12-18 01:26:38 +0100140
willy tarreaueedaa9f2005-12-17 14:08:03 +01001411.1) Event logging
142------------------
143Most events are logged : start, stop, servers going up and down, connections and
144errors. Each event generates a syslog message which can be sent to up to 2
145servers. The syntax is :
146
147 log <ip_address> <facility> [max_level]
148
149Connections are logged at level "info". Services initialization and servers
150going up are logged at level "notice", termination signals are logged at
151"warning", and definitive service termination, as well as loss of servers are
152logged at level "alert". The optional parameter <max_level> specifies above
153what level messages should be sent. Level can take one of these 8 values :
154
155 emerg, alert, crit, err, warning, notice, info, debug
156
157For backwards compatibility with versions 1.1.16 and earlier, the default level
158value is "debug" if not specified.
159
160Permitted facilities are :
161 kern, user, mail, daemon, auth, syslog, lpr, news,
162 uucp, cron, auth2, ftp, ntp, audit, alert, cron2,
163 local0, local1, local2, local3, local4, local5, local6, local7
164
165According to RFC3164, messages are truncated to 1024 bytes before being emitted.
166
167Example :
168---------
169 global
willy tarreauc5f73ed2005-12-18 01:26:38 +0100170 log 192.168.2.200 local3
171 log 127.0.0.1 local4 notice
172
willy tarreaueedaa9f2005-12-17 14:08:03 +0100173
1741.2) limiting the number of connections
175---------------------------------------
176It is possible and recommended to limit the global number of per-process
willy tarreauc5f73ed2005-12-18 01:26:38 +0100177connections using the 'maxconn' global keyword. Since one connection includes
178both a client and a server, it means that the max number of TCP sessions will
179be about the double of this number. It's important to understand this when
180trying to find best values for 'ulimit -n' before starting the proxy. To
181anticipate the number of sockets needed, all these parameters must be counted :
willy tarreaueedaa9f2005-12-17 14:08:03 +0100182
183 - 1 socket per incoming connection
184 - 1 socket per outgoing connection
185 - 1 socket per address/port/proxy tuple.
186 - 1 socket per server being health-checked
187 - 1 socket for all logs
188
189In simple configurations where each proxy only listens one one address/port,
willy tarreauc5f73ed2005-12-18 01:26:38 +0100190set the limit of file descriptors (ulimit -n) to
191(2 * maxconn + nbproxies + nbservers + 1). Starting with versions 1.1.32/1.2.6,
192it is now possible to set the limit in the configuration using the 'ulimit-n'
193global keyword, provided the proxy is started as root. This puts an end to the
194recurrent problem of ensuring that the system limits are adapted to the proxy
195values. Note that these limits are per-process.
196
197Example :
198---------
199 global
200 maxconn 32000
201 ulimit-n 65536
202
willy tarreaueedaa9f2005-12-17 14:08:03 +0100203
2041.3) Drop of priviledges
205------------------------
206In order to reduce the risk and consequences of attacks, in the event where a
207yet non-identified vulnerability would be successfully exploited, it's possible
208to lower the process priviledges and even isolate it in a riskless directory.
209
210In the 'global' section, the 'uid' parameter sets a numerical user identifier
211which the process will switch to after binding its listening sockets. The value
212'0', which normally represents the super-user, here indicates that the UID must
213not change during startup. It's the default behaviour. The 'gid' parameter does
Willy Tarreau95c20ac2007-03-25 15:39:23 +0200214the same for the group identifier. If setting an uid is not possible because of
215deployment constraints, it is possible to set a user name with the 'user'
216keyword followed by a valid user name. The same is true for the gid. It is
217possible to specify a group name after the 'group' keyword.
218
219It is particularly advised against use of generic accounts such as 'nobody'
220because it has the same consequences as using 'root' if other services use
221them.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100222
223The 'chroot' parameter makes the process isolate itself in an empty directory
224just before switching its UID. This type of isolation (chroot) can sometimes
225be worked around on certain OS (Linux, Solaris), provided that the attacker
226has gained 'root' priviledges and has the ability to use or create a directory.
227For this reason, it's capital to use a dedicated directory and not to share one
228between several services of different nature. To make isolation more resistant,
229it's recommended to use an empty directory without any right, and to change the
230UID of the process so that it cannot do anything there.
231
232Note: in the event where such a vulnerability would be exploited, it's most
233likely that first attempts would kill the process due to 'Segmentation Fault',
234'Bus Error' or 'Illegal Instruction' signals. Eventhough it's true that
235isolating the server reduces the risks of intrusion, it's sometimes useful to
236find why a process dies, via the analysis of a 'core' file, although very rare
237(the last bug of this sort was fixed in 1.1.9). For security reasons, most
238systems disable the generation of core file when a process changes its UID. So
239the two workarounds are either to start the process from a restricted user
240account, which will not be able to chroot itself, or start it as root and not
241change the UID. In both cases the core will be either in the start or the chroot
242directories. Do not forget to allow core dumps prior to start the process :
243
244# ulimit -c unlimited
245
246Example :
247---------
248
Willy Tarreau95c20ac2007-03-25 15:39:23 +0200249 # with uid/gid
willy tarreaueedaa9f2005-12-17 14:08:03 +0100250 global
willy tarreauc5f73ed2005-12-18 01:26:38 +0100251 uid 30000
252 gid 30000
253 chroot /var/chroot/haproxy
254
Willy Tarreau95c20ac2007-03-25 15:39:23 +0200255 # with user/group
256 global
257 user haproxy
258 group public
259 chroot /var/chroot/haproxy
260
willy tarreaueedaa9f2005-12-17 14:08:03 +0100261
2621.4) Startup modes
263------------------
willy tarreau34f45302006-04-15 21:37:14 +0200264The service can start in several different modes :
willy tarreaueedaa9f2005-12-17 14:08:03 +0100265 - foreground / background
266 - quiet / normal / debug
267
268The default mode is normal, foreground, which means that the program doesn't
269return once started. NEVER EVER use this mode in a system startup script, or
270the system won't boot. It needs to be started in background, so that it
271returns immediately after forking. That's accomplished by the 'daemon' option
272in the 'global' section, which is the equivalent of the '-D' command line
273argument.
274
willy tarreau34f45302006-04-15 21:37:14 +0200275The '-db' command line argument overrides the 'daemon' and 'nbproc' global
276options to make the process run in normal, foreground mode.
277
willy tarreaueedaa9f2005-12-17 14:08:03 +0100278Moreover, certain alert messages are still sent to the standard output even
279in 'daemon' mode. To make them disappear, simply add the 'quiet' option in the
280'global' section. This option has no command-line equivalent.
281
282Last, the 'debug' mode, enabled with the 'debug' option in the 'global' section,
283and which is equivalent of the '-d' option, allows deep TCP/HTTP analysis, with
284timestamped display of each connection, disconnection, and HTTP headers for both
285ways. This mode is incompatible with 'daemon' and 'quiet' modes for obvious
286reasons.
287
willy tarreauc5f73ed2005-12-18 01:26:38 +0100288
willy tarreaueedaa9f2005-12-17 14:08:03 +01002891.5) Increasing the overall processing power
290--------------------------------------------
291On multi-processor systems, it may seem to be a shame to use only one processor,
willy tarreau982249e2005-12-18 00:57:06 +0100292eventhough the load needed to saturate a recent processor is far above common
willy tarreaueedaa9f2005-12-17 14:08:03 +0100293usage. Anyway, for very specific needs, the proxy can start several processes
294between which the operating system will spread the incoming connections. The
295number of processes is controlled by the 'nbproc' parameter in the 'global'
willy tarreau4302f492005-12-18 01:00:37 +0100296section. It defaults to 1, and obviously works only in 'daemon' mode. One
297typical usage of this parameter has been to workaround the default per-process
298file-descriptor limit that Solaris imposes to user processes.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100299
300Example :
301---------
302
303 global
willy tarreauc5f73ed2005-12-18 01:26:38 +0100304 daemon
305 quiet
306 nbproc 2
willy tarreaueedaa9f2005-12-17 14:08:03 +0100307
308
willy tarreaufe2c5c12005-12-17 14:14:34 +01003091.6) Helping process management
310-------------------------------
311Haproxy now supports the notion of pidfile. If the '-p' command line argument,
312or the 'pidfile' global option is followed with a file name, this file will be
313removed, then filled with all children's pids, one per line (only in daemon
314mode). This file is NOT within the chroot, which allows to work with a readonly
315 chroot. It will be owned by the user starting the process, and will have
316permissions 0644.
317
318Example :
319---------
320
321 global
322 daemon
323 quiet
willy tarreauc5f73ed2005-12-18 01:26:38 +0100324 nbproc 2
willy tarreaufe2c5c12005-12-17 14:14:34 +0100325 pidfile /var/run/haproxy-private.pid
326
327 # to stop only those processes among others :
328 # kill $(</var/run/haproxy-private.pid)
329
willy tarreau34f45302006-04-15 21:37:14 +0200330 # to reload a new configuration with minimal service impact and without
331 # breaking existing sessions :
Willy Tarreau10806d52007-09-09 23:49:18 +0200332 # haproxy -f haproxy.cfg -p /var/run/haproxy-private.pid -sf $(</var/run/haproxy-private.pid)
willy tarreaufe2c5c12005-12-17 14:14:34 +0100333
willy tarreau64a3cc32005-12-18 01:13:11 +01003341.7) Polling mechanisms
335-----------------------
336Starting from version 1.2.5, haproxy supports the poll() and epoll() polling
337mechanisms. On systems where select() is limited by FD_SETSIZE (like Solaris),
338poll() can be an interesting alternative. Performance tests show that Solaris'
339poll() performance does not decay as fast as the numbers of sockets increase,
340making it a safe solution for high loads. However, Solaris already uses poll()
341to emulate select(), so as long as the number of sockets has no reason to go
342higher than FD_SETSIZE, poll() should not provide any better performance. On
343Linux systems with the epoll() patch (or any 2.6 version), haproxy will use
344epoll() which is extremely fast and non dependant on the number of sockets.
345Tests have shown constant performance from 1 to 20000 simultaneous sessions.
Willy Tarreaude99e992007-04-16 00:53:59 +0200346Version 1.3.9 introduced kqueue() for FreeBSD/OpenBSD, and speculative epoll()
347which consists in trying to perform I/O before queuing the events via syscalls.
willy tarreau64a3cc32005-12-18 01:13:11 +0100348
Willy Tarreau1db37712007-06-03 17:16:49 +0200349In order to optimize latency, it is now possible to limit the number of events
350returned by a single call to poll. The limit is fixed to 200 by default. If a
351smaller latency is seeked, it may be useful to reduce this value by using the
352'tune.maxpollevents' parameter in the 'global' section. Increasing it will
353slightly save CPU cycles in presence of large number of connections.
354
Willy Tarreaude99e992007-04-16 00:53:59 +0200355Haproxy will use kqueue() or speculative epoll() when available, then epoll(),
356and will fall back to poll(), then to select(). However, if for any reason you
357need to disable epoll() or poll() (eg. because of a bug or just to compare
Willy Tarreaue9f49e72012-11-11 17:42:00 +0100358performance), new global options have been created for this matter : 'nopoll',
359'nokqueue', and 'noepoll'.
willy tarreau64a3cc32005-12-18 01:13:11 +0100360
361Example :
362---------
363
364 global
365 # use only select()
366 noepoll
367 nopoll
Willy Tarreau1db37712007-06-03 17:16:49 +0200368 tune.maxpollevents 100
willy tarreau64a3cc32005-12-18 01:13:11 +0100369
370Note :
371------
372For the sake of configuration file portability, these options are accepted but
373ignored if the poll() or epoll() mechanisms have not been enabled at compile
374time.
375
Willy Tarreaude99e992007-04-16 00:53:59 +0200376To make debugging easier, the '-de' runtime argument disables epoll support,
377the '-dp' argument disables poll support, '-dk' disables kqueue and '-ds'
378disables speculative epoll(). They are respectively equivalent to 'noepoll',
Willy Tarreaue9f49e72012-11-11 17:42:00 +0100379'nopoll', and 'nokqueue'.
willy tarreau64a3cc32005-12-18 01:13:11 +0100380
381
willy tarreaueedaa9f2005-12-17 14:08:03 +01003822) Declaration of a listening service
383=====================================
384
385Service sections start with the 'listen' keyword :
386
387 listen <instance_name> [ <IP_address>:<port_range>[,...] ]
388
389- <instance_name> is the name of the instance. This name will be reported in
390 logs, so it is good to have it reflect the proxied service. No unicity test
391 is done on this name, and it's not mandatory for it to be unique, but highly
392 recommended.
393
394- <IP_address> is the IP address the proxy binds to. Empty address, '*' and
395 '0.0.0.0' all mean that the proxy listens to all valid addresses on the
396 system.
397
398- <port_range> is either a unique port, or a port range for which the proxy will
399 accept connections for the IP address specified above. This range can be :
400 - a numerical port (ex: '80')
401 - a dash-delimited ports range explicitly stating the lower and upper bounds
402 (ex: '2000-2100') which are included in the range.
403
404 Particular care must be taken against port ranges, because every <addr:port>
405 couple consumes one socket (=a file descriptor), so it's easy to eat lots of
406 descriptors with a simple range. The <addr:port> couple must be used only once
407 among all instances running on a same system. Please note that attaching to
408 ports lower than 1024 need particular priviledges to start the program, which
Jamie Gloudon801a0a32012-08-25 00:18:33 -0400409 are independent of the 'uid' parameter.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100410
411- the <IP_address>:<port_range> couple may be repeated indefinitely to require
412 the proxy to listen to other addresses and/or ports. To achieve this, simply
413 separate them with a coma.
414
415Examples :
416---------
417 listen http_proxy :80
418 listen x11_proxy 127.0.0.1:6000-6009
419 listen smtp_proxy 127.0.0.1:25,127.0.0.1:587
420 listen ldap_proxy :389,:663
421
422In the event that all addresses do not fit line width, it's preferable to
423detach secondary addresses on other lines with the 'bind' keyword. If this
424keyword is used, it's not even necessary to specify the first address on the
425'listen' line, which sometimes makes multiple configuration handling easier :
426
427 bind [ <IP_address>:<port_range>[,...] ]
428
429Examples :
430----------
431 listen http_proxy
432 bind :80,:443
willy tarreauc5f73ed2005-12-18 01:26:38 +0100433 bind 10.0.0.1:10080,10.0.0.1:10443
434
willy tarreaueedaa9f2005-12-17 14:08:03 +0100435
4362.1) Inhibiting a service
437-------------------------
438A service may be disabled for maintenance reasons, without needing to comment
439out the whole section, simply by specifying the 'disabled' keyword in the
440section to be disabled :
441
442 listen smtp_proxy 0.0.0.0:25
willy tarreauc5f73ed2005-12-18 01:26:38 +0100443 disabled
willy tarreaueedaa9f2005-12-17 14:08:03 +0100444
445Note: the 'enabled' keyword allows to enable a service which has been disabled
446 previously by a default configuration.
447
willy tarreauc5f73ed2005-12-18 01:26:38 +0100448
willy tarreaueedaa9f2005-12-17 14:08:03 +01004492.2) Modes of operation
450-----------------------
451A service can work in 3 different distinct modes :
452 - TCP
453 - HTTP
willy tarreau532bb552006-05-13 18:40:37 +0200454 - health
willy tarreaueedaa9f2005-12-17 14:08:03 +0100455
456TCP mode
457--------
458In this mode, the service relays TCP connections as soon as they're established,
459towards one or several servers. No processing is done on the stream. It's only
460an association of source(addr:port) -> destination(addr:port). To use this mode,
461you must specify 'mode tcp' in the 'listen' section. This is the default mode.
462
463Example :
464---------
465 listen smtp_proxy 0.0.0.0:25
willy tarreauc5f73ed2005-12-18 01:26:38 +0100466 mode tcp
willy tarreaueedaa9f2005-12-17 14:08:03 +0100467
468HTTP mode
469---------
470In this mode, the service relays TCP connections towards one or several servers,
471when it has enough informations to decide, which normally means that all HTTP
472headers have been read. Some of them may be scanned for a cookie or a pattern
473matching a regex. To use this mode, specify 'mode http' in the 'listen' section.
474
475Example :
476---------
477 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +0100478 mode http
willy tarreaueedaa9f2005-12-17 14:08:03 +0100479
480Health-checking mode
481--------------------
482This mode provides a way for external components to check the proxy's health.
483It is meant to be used with intelligent load-balancers which can use send/expect
484scripts to check for all of their servers' availability. This one simply accepts
willy tarreau197e8ec2005-12-17 14:10:59 +0100485the connection, returns the word 'OK' and closes it. If the 'option httpchk' is
486set, then the reply will be 'HTTP/1.0 200 OK' with no data, so that it can be
487tested from a tool which supports HTTP health-checks. To enable it, simply
willy tarreaueedaa9f2005-12-17 14:08:03 +0100488specify 'health' as the working mode :
489
490Example :
491---------
willy tarreau197e8ec2005-12-17 14:10:59 +0100492 # simple response : 'OK'
willy tarreaueedaa9f2005-12-17 14:08:03 +0100493 listen health_check 0.0.0.0:60000
willy tarreauc5f73ed2005-12-18 01:26:38 +0100494 mode health
willy tarreaueedaa9f2005-12-17 14:08:03 +0100495
willy tarreau197e8ec2005-12-17 14:10:59 +0100496 # HTTP response : 'HTTP/1.0 200 OK'
497 listen http_health_check 0.0.0.0:60001
willy tarreauc5f73ed2005-12-18 01:26:38 +0100498 mode health
499 option httpchk
500
willy tarreau532bb552006-05-13 18:40:37 +02005012.2.1 Monitoring
502----------------
willy tarreauc5f73ed2005-12-18 01:26:38 +0100503Versions 1.1.32 and 1.2.6 provide a new solution to check the proxy's
504availability without perturbating the service. The 'monitor-net' keyword was
505created to specify a network of equipments which CANNOT use the service for
506anything but health-checks. This is particularly suited to TCP proxies, because
507it prevents the proxy from relaying the monitor's connection to the remote
508server.
509
510When used with TCP, the connection is accepted then closed and nothing is
511logged. This is enough for a front-end load-balancer to detect the service as
512available.
willy tarreau197e8ec2005-12-17 14:10:59 +0100513
willy tarreauc5f73ed2005-12-18 01:26:38 +0100514When used with HTTP, the connection is accepted, nothing is logged, the
515following response is sent, then the session is closed : "HTTP/1.0 200 OK".
516This is normally enough for any front-end HTTP load-balancer to detect the
517service as available too, both with TCP and HTTP checks.
518
519Proxies using the "monitor-net" keyword can remove the "option dontlognull", as
520it will make them log empty connections from hosts outside the monitoring
521network.
522
523Example :
524---------
525
526 listen tse-proxy
527 bind :3389,:1494,:5900 # TSE, ICA and VNC at once.
528 mode tcp
529 balance roundrobin
530 server tse-farm 192.168.1.10
531 monitor-net 192.168.1.252/31 # L4 load-balancers on .252 and .253
532
willy tarreaueedaa9f2005-12-17 14:08:03 +0100533
Willy Tarreau1c47f852006-07-09 08:22:27 +0200534When the system executing the checks is located behind a proxy, the monitor-net
535keyword cannot be used because haproxy will always see the proxy's address. To
536overcome this limitation, version 1.2.15 brought the 'monitor-uri' keyword. It
537defines an URI which will not be forwarded nor logged, but for which haproxy
538will immediately send an "HTTP/1.0 200 OK" response. This makes it possible to
539check the validity of the reverse-proxy->haproxy chain with one request. It can
540be used in HTTPS checks in front of an stunnel -> haproxy combination for
541instance. Obviously, this keyword is only valid in HTTP mode, otherwise there
542is no notion of URI. Note that the method and HTTP versions are simply ignored.
543
544Example :
545---------
546
547 listen stunnel_backend :8080
548 mode http
549 balance roundrobin
550 server web1 192.168.1.10:80 check
551 server web2 192.168.1.11:80 check
552 monitor-uri /haproxy_test
553
554
willy tarreaueedaa9f2005-12-17 14:08:03 +01005552.3) Limiting the number of simultaneous connections
556----------------------------------------------------
557The 'maxconn' parameter allows a proxy to refuse connections above a certain
558amount of simultaneous ones. When the limit is reached, it simply stops
559listening, but the system may still be accepting them because of the back log
willy tarreau982249e2005-12-18 00:57:06 +0100560queue. These connections will be processed later when other ones have freed
willy tarreaueedaa9f2005-12-17 14:08:03 +0100561some slots. This provides a serialization effect which helps very fragile
willy tarreau34f45302006-04-15 21:37:14 +0200562servers resist to high loads. See further for system limitations.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100563
564Example :
565---------
566 listen tiny_server 0.0.0.0:80
567 maxconn 10
568
569
5702.4) Soft stop
571--------------
572It is possible to stop services without breaking existing connections by the
willy tarreau22739ef2006-01-20 20:43:32 +0100573sending of the SIGUSR1 signal to the process. All services are then put into
willy tarreaueedaa9f2005-12-17 14:08:03 +0100574soft-stop state, which means that they will refuse to accept new connections,
575except for those which have a non-zero value in the 'grace' parameter, in which
576case they will still accept connections for the specified amount of time, in
willy tarreau22739ef2006-01-20 20:43:32 +0100577milliseconds. This makes it possible to tell a load-balancer that the service
578is failing, while still doing the job during the time it needs to detect it.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100579
580Note: active connections are never killed. In the worst case, the user will have
581to wait for all of them to close or to time-out, or simply kill the process
willy tarreau22739ef2006-01-20 20:43:32 +0100582normally (SIGTERM). The default 'grace' value is '0'.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100583
584Example :
585---------
586 # enter soft stop after 'killall -USR1 haproxy'
587 # the service will still run 10 seconds after the signal
588 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +0100589 mode http
590 grace 10000
willy tarreaueedaa9f2005-12-17 14:08:03 +0100591
592 # this port is dedicated to a load-balancer, and must fail immediately
593 listen health_check 0.0.0.0:60000
willy tarreauc5f73ed2005-12-18 01:26:38 +0100594 mode health
595 grace 0
willy tarreaueedaa9f2005-12-17 14:08:03 +0100596
597
willy tarreau39df2dc2006-01-29 21:56:05 +0100598As of version 1.2.8, a new soft-reconfiguration mechanism has been introduced.
willy tarreau22739ef2006-01-20 20:43:32 +0100599It is now possible to "pause" all the proxies by sending a SIGTTOU signal to
600the processes. This will disable the listening socket without breaking existing
601connections. After that, sending a SIGTTIN signal to those processes enables
602the listening sockets again. This is very useful to try to load a new
603configuration or even a new version of haproxy without breaking existing
604connections. If the load succeeds, then simply send a SIGUSR1 which will make
605the previous proxies exit immediately once their sessions are closed ; and if
606the load fails, then simply send a SIGTTIN to restore the service immediately.
607Please note that the 'grace' parameter is ignored for SIGTTOU, as well as for
608SIGUSR1 when the process was in the pause mode. Please also note that it would
609be useful to save the pidfile before starting a new instance.
610
willy tarreau34f45302006-04-15 21:37:14 +0200611This mechanism fully exploited since 1.2.11 with the '-st' and '-sf' options
willy tarreau532bb552006-05-13 18:40:37 +0200612(see below).
613
6142.4.1) Hot reconfiguration
615--------------------------
616The '-st' and '-sf' command line options are used to inform previously running
617processes that a configuration is being reloaded. They will receive the SIGTTOU
618signal to ask them to temporarily stop listening to the ports so that the new
619process can grab them. If anything wrong happens, the new process will send
620them a SIGTTIN to tell them to re-listen to the ports and continue their normal
621work. Otherwise, it will either ask them to finish (-sf) their work then softly
622exit, or immediately terminate (-st), breaking existing sessions. A typical use
623of this allows a configuration reload without service interruption :
624
625 # haproxy -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid)
626
willy tarreau22739ef2006-01-20 20:43:32 +0100627
willy tarreaueedaa9f2005-12-17 14:08:03 +01006282.5) Connections expiration time
629--------------------------------
630It is possible (and recommended) to configure several time-outs on TCP
Jamie Gloudon801a0a32012-08-25 00:18:33 -0400631connections. Three independent timers are adjustable with values specified
willy tarreaueedaa9f2005-12-17 14:08:03 +0100632in milliseconds. A session will be terminated if either one of these timers
633expire.
634
635 - the time we accept to wait for data from the client, or for the client to
636 accept data : 'clitimeout' :
637
willy tarreauc5f73ed2005-12-18 01:26:38 +0100638 # client time-out set to 2mn30.
639 clitimeout 150000
willy tarreaueedaa9f2005-12-17 14:08:03 +0100640
641 - the time we accept to wait for data from the server, or for the server to
642 accept data : 'srvtimeout' :
643
willy tarreauc5f73ed2005-12-18 01:26:38 +0100644 # server time-out set to 30s.
645 srvtimeout 30000
willy tarreaueedaa9f2005-12-17 14:08:03 +0100646
647 - the time we accept to wait for a connection to establish on a server :
648 'contimeout' :
649
650 # we give up if the connection does not complete within 4 seconds
willy tarreauc5f73ed2005-12-18 01:26:38 +0100651 contimeout 4000
willy tarreaueedaa9f2005-12-17 14:08:03 +0100652
653Notes :
654-------
655 - 'contimeout' and 'srvtimeout' have no sense on 'health' mode servers ;
656 - under high loads, or with a saturated or defective network, it's possible
657 that some packets get lost. Since the first TCP retransmit only happens
658 after 3 seconds, a time-out equal to, or lower than 3 seconds cannot
659 compensate for a packet loss. A 4 seconds time-out seems a reasonable
660 minimum which will considerably reduce connection failures.
Willy Tarreaubefdff12007-12-02 22:27:38 +0100661 - starting with version 1.3.14, it is possible to specify timeouts in
662 arbitrary time units among { us, ms, s, m, h, d }. For this, the integer
663 value just has to be suffixed with the unit.
willy tarreauc5f73ed2005-12-18 01:26:38 +0100664
willy tarreaueedaa9f2005-12-17 14:08:03 +01006652.6) Attempts to reconnect
666--------------------------
667After a connection failure to a server, it is possible to retry, potentially
668on another server. This is useful if health-checks are too rare and you don't
669want the clients to see the failures. The number of attempts to reconnect is
670set by the 'retries' paramter.
671
672Example :
673---------
willy tarreauc5f73ed2005-12-18 01:26:38 +0100674 # we can retry 3 times max after a failure
675 retries 3
willy tarreaueedaa9f2005-12-17 14:08:03 +0100676
willy tarreau34f45302006-04-15 21:37:14 +0200677Please note that the reconnection attempt may lead to getting the connection
678sent to a new server if the original one died between connection attempts.
679
willy tarreaueedaa9f2005-12-17 14:08:03 +0100680
6812.7) Address of the dispatch server (deprecated)
682------------------------------------------------
683The server which will be sent all new connections is defined by the 'dispatch'
684parameter, in the form <address>:<port>. It generally is dedicated to unknown
685connections and will assign them a cookie, in case of HTTP persistence mode,
686or simply is a single server in case of generic TCP proxy. This old mode is only
687provided for backwards compatibility, but doesn't allow to check remote servers
688state, and has a rather limited usage. All new setups should switch to 'balance'
689mode. The principle of the dispatcher is to be able to perform the load
690balancing itself, but work only on new clients so that the server doesn't need
691to be a big machine.
692
693Example :
694---------
willy tarreauc5f73ed2005-12-18 01:26:38 +0100695 # all new connections go there
696 dispatch 192.168.1.2:80
willy tarreaueedaa9f2005-12-17 14:08:03 +0100697
698Note :
699------
700This parameter has no sense for 'health' servers, and is incompatible with
701'balance' mode.
702
703
7042.8) Outgoing source address
705----------------------------
706It is often necessary to bind to a particular address when connecting to some
707remote hosts. This is done via the 'source' parameter which is a per-proxy
708parameter. A newer version may allow to fix different sources to reach different
709servers. The syntax is 'source <address>[:<port>]', where <address> is a valid
710local address (or '0.0.0.0' or '*' or empty to let the system choose), and
711<port> is an optional parameter allowing the user to force the source port for
712very specific needs. If the port is not specified or is '0', the system will
713choose a free port. Note that as of version 1.1.18, the servers health checks
714are also performed from the same source.
715
716Examples :
717----------
718 listen http_proxy *:80
willy tarreauc5f73ed2005-12-18 01:26:38 +0100719 # all connections take 192.168.1.200 as source address
720 source 192.168.1.200:0
willy tarreaueedaa9f2005-12-17 14:08:03 +0100721
722 listen rlogin_proxy *:513
willy tarreauc5f73ed2005-12-18 01:26:38 +0100723 # use address 192.168.1.200 and the reserved port 900 (needs to be root)
724 source 192.168.1.200:900
willy tarreaueedaa9f2005-12-17 14:08:03 +0100725
726
7272.9) Setting the cookie name
728----------------------------
729In HTTP mode, it is possible to look for a particular cookie which will contain
730a server identifier which should handle the connection. The cookie name is set
731via the 'cookie' parameter.
732
733Example :
734---------
735 listen http_proxy :80
willy tarreauc5f73ed2005-12-18 01:26:38 +0100736 mode http
737 cookie SERVERID
willy tarreaueedaa9f2005-12-17 14:08:03 +0100738
739It is possible to change the cookie behaviour to get a smarter persistence,
740depending on applications. It is notably possible to delete or modify a cookie
741emitted by a server, insert a cookie identifying the server in an HTTP response
742and even add a header to tell upstream caches not to cache this response.
743
744Examples :
745----------
746
747To remove the cookie for direct accesses (ie when the server matches the one
748which was specified in the client cookie) :
749
willy tarreauc5f73ed2005-12-18 01:26:38 +0100750 cookie SERVERID indirect
willy tarreaueedaa9f2005-12-17 14:08:03 +0100751
752To replace the cookie value with the one assigned to the server if any (no
753cookie will be created if the server does not provide one, nor if the
754configuration does not provide one). This lets the application put the cookie
755exactly on certain pages (eg: successful authentication) :
756
willy tarreauc5f73ed2005-12-18 01:26:38 +0100757 cookie SERVERID rewrite
willy tarreaueedaa9f2005-12-17 14:08:03 +0100758
759To create a new cookie and assign the server identifier to it (in this case, all
760servers should be associated with a valid cookie, since no cookie will simply
761delete the cookie from the client's browser) :
762
willy tarreauc5f73ed2005-12-18 01:26:38 +0100763 cookie SERVERID insert
willy tarreaueedaa9f2005-12-17 14:08:03 +0100764
willy tarreau0174f312005-12-18 01:02:42 +0100765To reuse an existing application cookie and prefix it with the server's
766identifier, and remove it in the request, use the 'prefix' option. This allows
767to insert a haproxy in front of an application without risking to break clients
768which does not support more than one cookie :
769
willy tarreauc5f73ed2005-12-18 01:26:38 +0100770 cookie JSESSIONID prefix
willy tarreau0174f312005-12-18 01:02:42 +0100771
willy tarreaueedaa9f2005-12-17 14:08:03 +0100772To insert a cookie and ensure that no upstream cache will store it, add the
773'nocache' option :
774
willy tarreauc5f73ed2005-12-18 01:26:38 +0100775 cookie SERVERID insert nocache
willy tarreaueedaa9f2005-12-17 14:08:03 +0100776
777To insert a cookie only after a POST request, add 'postonly' after 'insert'.
778This has the advantage that there's no risk of caching, and that all pages
779seen before the POST one can still be cached :
780
willy tarreauc5f73ed2005-12-18 01:26:38 +0100781 cookie SERVERID insert postonly
willy tarreaueedaa9f2005-12-17 14:08:03 +0100782
783Notes :
784-----------
785- it is possible to combine 'insert' with 'indirect' or 'rewrite' to adapt to
786 applications which already generate the cookie with an invalid content.
787
788- in the case where 'insert' and 'indirect' are both specified, the cookie is
willy tarreau0174f312005-12-18 01:02:42 +0100789 never transmitted to the server, since it wouldn't understand it. This is the
790 most application-transparent mode.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100791
792- it is particularly recommended to use 'nocache' in 'insert' mode if any
793 upstream HTTP/1.0 cache is susceptible to cache the result, because this may
794 lead to many clients going to the same server, or even worse, some clients
795 having their server changed while retrieving a page from the cache.
796
willy tarreau0174f312005-12-18 01:02:42 +0100797- the 'prefix' mode normally does not need 'indirect', 'nocache', nor
798 'postonly', because just as in the 'rewrite' mode, it relies on the
799 application to know when a cookie can be emitted. However, since it has to
800 fix the cookie name in every subsequent requests, you must ensure that the
801 proxy will be used without any "HTTP keep-alive". Use option "httpclose" if
802 unsure.
803
willy tarreaueedaa9f2005-12-17 14:08:03 +0100804- when the application is well known and controlled, the best method is to
805 only add the persistence cookie on a POST form because it's up to the
willy tarreau0174f312005-12-18 01:02:42 +0100806 application to select which page it wants the upstream servers to cache. In
807 this case, you would use 'insert postonly indirect'.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100808
willy tarreauc5f73ed2005-12-18 01:26:38 +0100809
willy tarreaueedaa9f2005-12-17 14:08:03 +01008102.10) Associating a cookie value with a server
811----------------------------------------------
812In HTTP mode, it's possible to associate a cookie value to each server. This
813was initially used in combination with 'dispatch' mode to handle direct accesses
814but it is now the standard way of doing the load balancing. The syntax is :
815
816 server <identifier> <address>:<port> cookie <value>
817
818- <identifier> is any name which can be used to identify the server in the logs.
819- <address>:<port> specifies where the server is bound.
820- <value> is the value to put in or to read from the cookie.
821
822Example : the 'SERVERID' cookie can be either 'server01' or 'server02'
823---------
824 listen http_proxy :80
willy tarreauc5f73ed2005-12-18 01:26:38 +0100825 mode http
826 cookie SERVERID
827 dispatch 192.168.1.100:80
828 server web1 192.168.1.1:80 cookie server01
829 server web2 192.168.1.2:80 cookie server02
willy tarreaueedaa9f2005-12-17 14:08:03 +0100830
831Warning : the syntax has changed since version 1.0 !
832---------
833
willy tarreauc5f73ed2005-12-18 01:26:38 +0100834
willy tarreau598da412005-12-18 01:07:29 +01008352.11) Application Cookies
836-------------------------
837Since 1.2.4 it is possible to catch the cookie that comes from an
838application server in order to apply "application session stickyness".
839The server's response is searched for 'appsession' cookie, the first
840'len' bytes are used for matching and it is stored for a period of
841'timeout'.
842The syntax is:
843
willy tarreau532bb552006-05-13 18:40:37 +0200844 appsession <session_cookie> len <match_length> timeout <holdtime>
willy tarreau598da412005-12-18 01:07:29 +0100845
willy tarreau532bb552006-05-13 18:40:37 +0200846- <session_cookie> is the cookie, the server uses for it's session-handling
847- <match_length> how many bytes/characters should be used for matching equal
willy tarreau598da412005-12-18 01:07:29 +0100848 sessions
willy tarreau532bb552006-05-13 18:40:37 +0200849- <holdtime> after this inactivaty time, in ms, the cookie will be deleted
willy tarreau598da412005-12-18 01:07:29 +0100850 from the sessionstore
Willy Tarreaubefdff12007-12-02 22:27:38 +0100851- starting with version 1.3.14, it is possible to specify timeouts in
852 arbitrary time units among { us, ms, s, m, h, d }. For this, the integer
853 value just has to be prefixed with the unit.
willy tarreau598da412005-12-18 01:07:29 +0100854
855The appsession is only per 'listen' section possible.
856
857Example :
858---------
willy tarreau532bb552006-05-13 18:40:37 +0200859 listen http_lb1 192.168.3.4:80
860 mode http
861 capture request header Cookie len 200
862 # Havind a ServerID cookie on the client allows him to reach
863 # the right server even after expiration of the appsession.
864 cookie ServerID insert nocache indirect
865 # Will memorize 52 bytes of the cookie 'JSESSIONID' and keep them
866 # for 3 hours. It will match it in the cookie and the URL field.
Willy Tarreaubefdff12007-12-02 22:27:38 +0100867 appsession JSESSIONID len 52 timeout 3h
willy tarreau532bb552006-05-13 18:40:37 +0200868 server first1 10.3.9.2:10805 check inter 3000 cookie first
869 server secon1 10.3.9.3:10805 check inter 3000 cookie secon
870 server first1 10.3.9.4:10805 check inter 3000 cookie first
871 server secon2 10.3.9.5:10805 check inter 3000 cookie secon
872 option httpchk GET /test.jsp
willy tarreau598da412005-12-18 01:07:29 +0100873
willy tarreauc5f73ed2005-12-18 01:26:38 +0100874
willy tarreaueedaa9f2005-12-17 14:08:03 +01008753) Autonomous load balancer
876===========================
877
878The proxy can perform the load-balancing itself, both in TCP and in HTTP modes.
879This is the most interesting mode which obsoletes the old 'dispatch' mode
880described above. It has advantages such as server health monitoring, multiple
881port binding and port mapping. To use this mode, the 'balance' keyword is used,
willy tarreau34f45302006-04-15 21:37:14 +0200882followed by the selected algorithm. Up to version 1.2.11, only 'roundrobin' was
883available, which is also the default value if unspecified. Starting with
Willy Tarreau2fcb5002007-05-08 13:35:26 +0200884version 1.2.12, a new 'source' keyword appeared. A new 'uri' keyword was added
885in version 1.3.10. In this mode, there will be no dispatch address, but the
886proxy needs at least one server.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100887
888Example : same as the last one, with internal load balancer
889---------
890
891 listen http_proxy :80
willy tarreauc5f73ed2005-12-18 01:26:38 +0100892 mode http
893 cookie SERVERID
894 balance roundrobin
895 server web1 192.168.1.1:80 cookie server01
896 server web2 192.168.1.2:80 cookie server02
willy tarreaueedaa9f2005-12-17 14:08:03 +0100897
898
899Since version 1.1.22, it is possible to automatically determine on which port
900the server will get the connection, depending on the port the client connected
901to. Indeed, there now are 4 possible combinations for the server's <port> field:
902
903 - unspecified or '0' :
904 the connection will be sent to the same port as the one on which the proxy
905 received the client connection itself.
906
907 - numerical value (the only one supported in versions earlier than 1.1.22) :
908 the connection will always be sent to the specified port.
909
910 - '+' followed by a numerical value :
911 the connection will be sent to the same port as the one on which the proxy
912 received the connection, plus this value.
913
914 - '-' followed by a numerical value :
915 the connection will be sent to the same port as the one on which the proxy
916 received the connection, minus this value.
917
918Examples :
919----------
920
921# same as previous example
922
923 listen http_proxy :80
willy tarreauc5f73ed2005-12-18 01:26:38 +0100924 mode http
925 cookie SERVERID
926 balance roundrobin
927 server web1 192.168.1.1 cookie server01
928 server web2 192.168.1.2 cookie server02
willy tarreaueedaa9f2005-12-17 14:08:03 +0100929
930# simultaneous relaying of ports 80, 81 and 8080-8089
931
932 listen http_proxy :80,:81,:8080-8089
willy tarreauc5f73ed2005-12-18 01:26:38 +0100933 mode http
934 cookie SERVERID
935 balance roundrobin
936 server web1 192.168.1.1 cookie server01
937 server web2 192.168.1.2 cookie server02
willy tarreaueedaa9f2005-12-17 14:08:03 +0100938
939# relaying of TCP ports 25, 389 and 663 to ports 1025, 1389 and 1663
940
941 listen http_proxy :25,:389,:663
willy tarreauc5f73ed2005-12-18 01:26:38 +0100942 mode tcp
943 balance roundrobin
944 server srv1 192.168.1.1:+1000
945 server srv2 192.168.1.2:+1000
willy tarreaueedaa9f2005-12-17 14:08:03 +0100946
willy tarreau34f45302006-04-15 21:37:14 +0200947As previously stated, version 1.2.12 brought the 'source' keyword. When this
948keyword is used, the client's IP address is hashed and evenly distributed among
949the available servers so that a same source IP will always go to the same
950server as long as there are no change in the number of available servers. This
951can be used for instance to bind HTTP and HTTPS to the same server. It can also
952be used to improve stickyness when one part of the client population does not
953accept cookies. In this case, only those ones will be perturbated should a
954server fail.
955
956NOTE: It is important to consider the fact that many clients surf the net
957 through proxy farms which assign different IP addresses for each
958 request. Others use dialup connections with a different IP at each
959 connection. Thus, the 'source' parameter should be used with extreme
960 care.
961
962Examples :
963----------
964
965# make a same IP go to the same server whatever the service
966
967 listen http_proxy
968 bind :80,:443
969 mode http
970 balance source
971 server web1 192.168.1.1
972 server web2 192.168.1.2
973
974# try to improve client-server binding by using both source IP and cookie :
975
976 listen http_proxy :80
977 mode http
978 cookie SERVERID
979 balance source
980 server web1 192.168.1.1 cookie server01
981 server web2 192.168.1.2 cookie server02
982
Willy Tarreau2fcb5002007-05-08 13:35:26 +0200983As indicated above, the 'uri' keyword was introduced in version 1.3.10. It is
984useful when load-balancing between reverse proxy-caches, because it will hash
985the URI and use the hash result to select a server, thus optimizing the hit
986rate on the caches, because the same URI will always reach the same cache. This
987keyword is only allowed in HTTP mode.
988
989Example :
990---------
991
992# Always send a given URI to the same server
993
994 listen http_proxy
995 bind :3128
996 mode http
997 balance uri
998 server squid1 192.168.1.1
999 server squid2 192.168.1.2
1000
Willy Tarreau01732802007-11-01 22:48:15 +01001001Version 1.3.14 introduced the "balance url_param" method. It consists in
1002relying on a parameter passed in the URL to perform a hash. This is mostly
1003useful for applications which do not have strict persistence requirements,
1004but for which it still provides a performance boost due to local caching.
1005Some of these applications may not be able to use a cookie for whatever reason,
1006but may be able to look for a parameter passed in the URL. If the parameter is
1007missing from the URL, then the 'round robin' method applies.
1008
matt.farnsworth@nokia.com1c2ab962008-04-14 20:47:37 +02001009A modifier may be added to specify that parameters in POST requests may be
1010found in the messsage body if the URL lacks a '?' separator character.
1011A wait limit may also be applied, if no limit is requested then
1012the default value is 48 octets, the minimum is 3. HAProxy may wait, until 48
1013octets are received. If Content-Length is missing, or zero it need not
1014wait for more data then the client promissed to send. When Content-Length is
1015present, and more than <max_wait>; then waiting is limited and it is assumed this
1016will be enough data to search for the presence of a parameter. If
1017Transfer-Encoding: chunked is used (unlikely), then the length of the first chunk
1018is the maximum number of bytes to wait for.
1019
1020balance url_param <param> [check_post [<max_wait>]]
1021
1022Caveats for using the check_post extension:
1023
1024 - all POST requests are eligable for consideration, because there is
1025 no way to determine if the parameters will be found in the body or
1026 entity which may contain binary data. Therefore another method may be
1027 required to restrict consideration of POST requests that have no URL
1028 parameters in the body. (see acl reqideny http_end)
1029
1030Limitations on inspecting the entity body of a POST:
1031
1032 - Content-Encoding is not supported, the parameter search will probably fail;
1033 and load balancing will fall back to Round Robin.
1034
1035 - Expect: 100-continue is not supported, load balancing will fall back to
1036 Round Robin.
1037
1038 - Transfer-Encoding(RFC2616 3.6.1) is only supported in the first chunk. If
1039 the entire parameter value is not present in the first chunk, the selection
1040 of server is undefined (actually, defined by how little actually appeared in
1041 the first chunk).
1042
1043 - This feature does not support generation of a 100, 411 or 501 response.
1044
1045 - In some cases, requesting check_post MAY attempt to scan the entire contents
1046 of a message body. Scaning normally terminates when linear white space or
1047 control characters are found, indicating the end of what might be a URL parameter
1048 list. This is probably not a concern with SGML type message bodies.
1049
1050
Willy Tarreau01732802007-11-01 22:48:15 +01001051Example :
1052---------
1053
1054# Hash the "basket_id" argument from the URL to determine the server
1055
1056 listen http_proxy
1057 bind :3128
1058 mode http
1059 balance url_param basket_id
1060 server ebiz1 192.168.1.1
1061 server ebiz2 192.168.1.2
1062
willy tarreaueedaa9f2005-12-17 14:08:03 +01001063
willy tarreau197e8ec2005-12-17 14:10:59 +010010643.1) Server monitoring
1065----------------------
willy tarreaueedaa9f2005-12-17 14:08:03 +01001066It is possible to check the servers status by trying to establish TCP
1067connections or even sending HTTP requests to them. A server which fails to
1068reply to health checks as expected will not be used by the load balancing
1069algorithms. To enable monitoring, add the 'check' keyword on a server line.
1070It is possible to specify the interval between tests (in milliseconds) with
1071the 'inter' parameter, the number of failures supported before declaring that
1072the server has fallen down with the 'fall' parameter, and the number of valid
1073checks needed for the server to fully get up with the 'rise' parameter. Since
1074version 1.1.22, it is also possible to send checks to a different port
1075(mandatory when none is specified) with the 'port' parameter. The default
1076values are the following ones :
1077
1078 - inter : 2000
1079 - rise : 2
1080 - fall : 3
1081 - port : default server port
Willy Tarreau2ea3abb2007-03-25 16:45:16 +02001082 - addr : specific address for the test (default = address server)
1083
willy tarreaueedaa9f2005-12-17 14:08:03 +01001084The default mode consists in establishing TCP connections only. But in certain
1085types of application failures, it is often that the server continues to accept
1086connections because the system does it itself while the application is running
1087an endless loop, or is completely stuck. So in version 1.1.16 were introduced
1088HTTP health checks which only performed simple lightweight requests and analysed
1089the response. Now, as of version 1.1.23, it is possible to change the HTTP
1090method, the URI, and the HTTP version string (which even allows to send headers
1091with a dirty trick). To enable HTTP health-checks, use 'option httpchk'.
1092
1093By default, requests use the 'OPTIONS' method because it's very light and easy
1094to filter from logs, and does it on '/'. Only HTTP responses 2xx and 3xx are
1095considered valid ones, and only if they come before the time to send a new
1096request is reached ('inter' parameter). If some servers block this type of
1097request, 3 other forms help to forge a request :
1098
1099 - option httpchk -> OPTIONS / HTTP/1.0
1100 - option httpchk URI -> OPTIONS <URI> HTTP/1.0
1101 - option httpchk METH URI -> <METH> <URI> HTTP/1.0
1102 - option httpchk METH URI VER -> <METH> <URI> <VER>
1103
Willy Tarreauf3c69202006-07-09 16:42:34 +02001104Some people are using HAProxy to relay various TCP-based protocols such as
1105HTTPS, SMTP or LDAP, with the most common one being HTTPS. One problem commonly
1106encountered in data centers is the need to forward the traffic to far remote
1107servers while providing server fail-over. Often, TCP-only checks are not enough
1108because intermediate firewalls, load balancers or proxies might acknowledge the
1109connection before it reaches the real server. The only solution to this problem
1110is to send application-level health checks. Since the demand for HTTPS checks
1111is high, it has been implemented in 1.2.15 based on SSLv3 Client Hello packets.
1112To enable it, use 'option ssl-hello-chk'. It will send SSL CLIENT HELLO packets
1113to the servers, announcing support for most common cipher suites. If the server
1114responds what looks like a SERVER HELLO or an ALERT (refuses the ciphers) then
1115the response is considered as valid. Note that Apache does not generate a log
1116when it receives only an HELLO message, which makes this type of message
1117perfectly suit this need.
1118
Willy Tarreau23677902007-05-08 23:50:35 +02001119Version 1.3.10 introduced the SMTP health check. By default, it sends
1120"HELO localhost" to the servers, and waits for the 250 message. Note that it
1121can also send a specific request :
1122
1123 - option smtpchk -> sends "HELO localhost"
1124 - option smtpchk EHLO mail.mydomain.com -> sends this ESMTP greeting
1125
willy tarreaueedaa9f2005-12-17 14:08:03 +01001126See examples below.
1127
1128Since version 1.1.17, it is possible to specify backup servers. These servers
1129are only sollicited when no other server is available. This may only be useful
1130to serve a maintenance page, or define one active and one backup server (seldom
1131used in TCP mode). To make a server a backup one, simply add the 'backup' option
1132on its line. These servers also support cookies, so if a cookie is specified for
1133a backup server, clients assigned to this server will stick to it even when the
1134other ones come back. Conversely, if no cookie is assigned to such a server,
1135the clients will get their cookies removed (empty cookie = removal), and will
1136be balanced against other servers once they come back. Please note that there
Willy TARREAU3481c462006-03-01 22:37:57 +01001137is no load-balancing among backup servers by default. If there are several
1138backup servers, the second one will only be used when the first one dies, and
1139so on. To force load-balancing between backup servers, specify the 'allbackups'
1140option.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001141
Willy Tarreau2ea3abb2007-03-25 16:45:16 +02001142Since version 1.1.22, it is possible to send health checks to a different port
1143than the service. It is mainly needed in setups where the server does not have
1144any predefined port, for instance when the port is deduced from the listening
1145port. For this, use the 'port' parameter followed by the port number which must
1146respond to health checks. It is also possible to send health checks to a
1147different address than the service. It makes it easier to use a dedicated check
1148daemon on the servers, for instance, check return contents and stop several
1149farms at once in the event of an error anywhere.
1150
willy tarreaueedaa9f2005-12-17 14:08:03 +01001151Since version 1.1.17, it is also possible to visually check the status of all
1152servers at once. For this, you just have to send a SIGHUP signal to the proxy.
1153The servers status will be dumped into the logs at the 'notice' level, as well
1154as on <stderr> if not closed. For this reason, it's always a good idea to have
1155one local log server at the 'notice' level.
1156
willy tarreau982249e2005-12-18 00:57:06 +01001157Since version 1.1.28 and 1.2.1, if an instance loses all its servers, an
willy tarreau0174f312005-12-18 01:02:42 +01001158emergency message will be sent in the logs to inform the administator that an
willy tarreau982249e2005-12-18 00:57:06 +01001159immediate action must be taken.
1160
willy tarreau0174f312005-12-18 01:02:42 +01001161Since version 1.1.30 and 1.2.3, several servers can share the same cookie
1162value. This is particularly useful in backup mode, to select alternate paths
1163for a given server for example, to provide soft-stop, or to direct the clients
1164to a temporary page during an application restart. The principle is that when
1165a server is dead, the proxy will first look for another server which shares the
1166same cookie value for every client which presents the cookie. If there is no
1167standard server for this cookie, it will then look for a backup server which
1168shares the same name. Please consult the architecture guide for more information.
willy tarreau982249e2005-12-18 00:57:06 +01001169
willy tarreaueedaa9f2005-12-17 14:08:03 +01001170Examples :
1171----------
1172# same setup as in paragraph 3) with TCP monitoring
1173 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001174 mode http
1175 cookie SERVERID
1176 balance roundrobin
1177 server web1 192.168.1.1:80 cookie server01 check
1178 server web2 192.168.1.2:80 cookie server02 check inter 500 rise 1 fall 2
willy tarreaueedaa9f2005-12-17 14:08:03 +01001179
1180# same with HTTP monitoring via 'OPTIONS / HTTP/1.0'
1181 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001182 mode http
1183 cookie SERVERID
1184 balance roundrobin
1185 option httpchk
1186 server web1 192.168.1.1:80 cookie server01 check
1187 server web2 192.168.1.2:80 cookie server02 check inter 500 rise 1 fall 2
willy tarreaueedaa9f2005-12-17 14:08:03 +01001188
1189# same with HTTP monitoring via 'OPTIONS /index.html HTTP/1.0'
1190 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001191 mode http
1192 cookie SERVERID
1193 balance roundrobin
1194 option httpchk /index.html
1195 server web1 192.168.1.1:80 cookie server01 check
1196 server web2 192.168.1.2:80 cookie server02 check inter 500 rise 1 fall 2
willy tarreaueedaa9f2005-12-17 14:08:03 +01001197
1198# same with HTTP monitoring via 'HEAD /index.jsp? HTTP/1.1\r\nHost: www'
1199 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001200 mode http
1201 cookie SERVERID
1202 balance roundrobin
1203 option httpchk HEAD /index.jsp? HTTP/1.1\r\nHost:\ www
1204 server web1 192.168.1.1:80 cookie server01 check
1205 server web2 192.168.1.2:80 cookie server02 check inter 500 rise 1 fall 2
willy tarreaueedaa9f2005-12-17 14:08:03 +01001206
willy tarreau0174f312005-12-18 01:02:42 +01001207# Load-balancing with 'prefixed cookie' persistence, and soft-stop using an
1208# alternate port 81 on the server for health-checks.
1209 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001210 mode http
1211 cookie JSESSIONID prefix
1212 balance roundrobin
1213 option httpchk HEAD /index.jsp? HTTP/1.1\r\nHost:\ www
1214 server web1-norm 192.168.1.1:80 cookie s1 check port 81
1215 server web2-norm 192.168.1.2:80 cookie s2 check port 81
1216 server web1-stop 192.168.1.1:80 cookie s1 check port 80 backup
1217 server web2-stop 192.168.1.2:80 cookie s2 check port 80 backup
willy tarreau0174f312005-12-18 01:02:42 +01001218
willy tarreaueedaa9f2005-12-17 14:08:03 +01001219# automatic insertion of a cookie in the server's response, and automatic
1220# deletion of the cookie in the client request, while asking upstream caches
1221# not to cache replies.
1222 listen web_appl 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001223 mode http
1224 cookie SERVERID insert nocache indirect
1225 balance roundrobin
1226 server web1 192.168.1.1:80 cookie server01 check
1227 server web2 192.168.1.2:80 cookie server02 check
willy tarreaueedaa9f2005-12-17 14:08:03 +01001228
1229# same with off-site application backup and local error pages server
1230 listen web_appl 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001231 mode http
1232 cookie SERVERID insert nocache indirect
1233 balance roundrobin
1234 server web1 192.168.1.1:80 cookie server01 check
1235 server web2 192.168.1.2:80 cookie server02 check
1236 server web-backup 192.168.2.1:80 cookie server03 check backup
1237 server web-excuse 192.168.3.1:80 check backup
willy tarreaueedaa9f2005-12-17 14:08:03 +01001238
willy tarreauc5f73ed2005-12-18 01:26:38 +01001239# SMTP+TLS relaying with health-checks and backup servers
willy tarreaueedaa9f2005-12-17 14:08:03 +01001240
1241 listen http_proxy :25,:587
willy tarreauc5f73ed2005-12-18 01:26:38 +01001242 mode tcp
1243 balance roundrobin
1244 server srv1 192.168.1.1 check port 25 inter 30000 rise 1 fall 2
1245 server srv2 192.168.1.2 backup
willy tarreaueedaa9f2005-12-17 14:08:03 +01001246
Willy Tarreauf3c69202006-07-09 16:42:34 +02001247# HTTPS relaying with health-checks and backup servers
1248
1249 listen http_proxy :443
1250 mode tcp
1251 option ssl-hello-chk
1252 balance roundrobin
1253 server srv1 192.168.1.1 check inter 30000 rise 1 fall 2
1254 server srv2 192.168.1.2 backup
1255
Willy TARREAU3481c462006-03-01 22:37:57 +01001256# Load-balancing using a backup pool (requires haproxy 1.2.9)
1257 listen http_proxy 0.0.0.0:80
1258 mode http
1259 balance roundrobin
1260 option httpchk
1261 server inst1 192.168.1.1:80 cookie s1 check
1262 server inst2 192.168.1.2:80 cookie s2 check
1263 server inst3 192.168.1.3:80 cookie s3 check
1264 server back1 192.168.1.10:80 check backup
1265 server back2 192.168.1.11:80 check backup
1266 option allbackups # all backups will be used
1267
willy tarreaueedaa9f2005-12-17 14:08:03 +01001268
12693.2) Redistribute connections in case of failure
1270------------------------------------------------
1271In HTTP mode, if a server designated by a cookie does not respond, the clients
1272may definitely stick to it because they cannot flush the cookie, so they will
1273not be able to access the service anymore. Specifying 'redispatch' will allow
1274the proxy to break their persistence and redistribute them to working servers.
1275
1276Example :
1277---------
1278 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001279 mode http
1280 cookie SERVERID
1281 dispatch 192.168.1.100:80
1282 server web1 192.168.1.1:80 cookie server01
1283 server web2 192.168.1.2:80 cookie server02
1284 redispatch # send back to dispatch in case of connection failure
willy tarreaueedaa9f2005-12-17 14:08:03 +01001285
1286Up to, and including version 1.1.16, this parameter only applied to connection
1287failures. Since version 1.1.17, it also applies to servers which have been
1288detected as failed by the health check mechanism. Indeed, a server may be broken
1289but still accepting connections, which would not solve every case. But it is
1290possible to conserve the old behaviour, that is, make a client insist on trying
1291to connect to a server even if it is said to be down, by setting the 'persist'
1292option :
1293
1294 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001295 mode http
1296 option persist
1297 cookie SERVERID
1298 dispatch 192.168.1.100:80
1299 server web1 192.168.1.1:80 cookie server01
1300 server web2 192.168.1.2:80 cookie server02
1301 redispatch # send back to dispatch in case of connection failure
willy tarreaueedaa9f2005-12-17 14:08:03 +01001302
1303
willy tarreau34f45302006-04-15 21:37:14 +020013043.3) Assigning different weights to servers
1305-------------------------------------------
1306Sometimes you will need to bring new servers to increase your server farm's
1307capacity, but the new server will be either smaller (emergency use of anything
1308that fits) or bigger (when investing in new hardware). For this reason, it
1309might be wise to be able to send more clients to biggest servers. Till version
13101.2.11, it was necessary to replicate the same server multiple times in the
1311configuration. Starting with 1.2.12, the 'weight' option is available. HAProxy
1312then computes the most homogenous possible map of servers based on their
willy tarreau532bb552006-05-13 18:40:37 +02001313weights so that the load gets distributed as smoothly as possible among them.
1314The weight, between 1 and 256, should reflect one server's capacity relative to
1315others. Weight 1 represents the lowest frequency and 256 the highest. This way,
1316if a server fails, the remaining capacities are still respected.
willy tarreau34f45302006-04-15 21:37:14 +02001317
1318Example :
1319---------
1320# fair distribution among two opterons and one old pentium3
1321
1322 listen web_appl 0.0.0.0:80
1323 mode http
1324 cookie SERVERID insert nocache indirect
1325 balance roundrobin
1326 server pentium3-800 192.168.1.1:80 cookie server01 weight 8 check
1327 server opteron-2.0G 192.168.1.2:80 cookie server02 weight 20 check
1328 server opteron-2.4G 192.168.1.3:80 cookie server03 weight 24 check
1329 server web-backup1 192.168.2.1:80 cookie server04 check backup
1330 server web-excuse 192.168.3.1:80 check backup
1331
1332Notes :
1333-------
1334 - if unspecified, the default weight is 1
1335
1336 - the weight does not impact health checks, so it is cleaner to use weights
1337 than replicating the same server several times
1338
1339 - weights also work on backup servers if the 'allbackups' option is used
1340
1341 - the weights also apply to the source address load balancing
1342 ('balance source').
1343
1344 - whatever the weights, the first server will always be assigned first. This
1345 is helpful for troubleshooting.
1346
1347 - for the purists, the map calculation algorithm gives precedence to first
1348 server, so the map is the most uniform when servers are declared in
1349 ascending order relative to their weights.
1350
willy tarreau532bb552006-05-13 18:40:37 +02001351The load distribution will follow exactly this sequence :
1352
1353 Request| 1 1 1 1
1354 number | 1 2 3 4 5 6 7 8 9 0 1 2 3
1355 --------+---------------------------
1356 p3-800 | X . . . . . . X . . . . .
1357 opt-20 | . X . X . X . . . X . X .
1358 opt-24 | . . X . X . X . X . X . X
1359
1360
13613.4) Limiting the number of concurrent sessions on each server
1362--------------------------------------------------------------
1363Some pre-forked servers such as Apache suffer from too many concurrent
1364sessions, because it's very expensive to run hundreds or thousands of
1365processes on one system. One solution is to increase the number of servers
1366and load-balance between them, but it is a problem when the only goal is
1367to resist to short surges.
1368
1369To solve this problem, a new feature was implemented in HAProxy 1.2.13.
1370It's a per-server 'maxconn', associated with a per-server and a per-proxy
1371queue. This transforms haproxy into a request buffer between the thousands of
1372clients and the few servers. On many circumstances, lowering the maxconn value
1373will increase the server's performance and decrease the overall response times
1374because the servers will be less congested.
1375
1376When a request tries to reach any server, the first non-saturated server is
1377used, respective to the load balancing algorithm. If all servers are saturated,
1378then the request gets queued into the instance's global queue. It will be
1379dequeued once a server will have freed a session and all previously queued
1380requests have been processed.
1381
1382If a request references a particular server (eg: source hashing, or persistence
1383cookie), and if this server is full, then the request will be queued into the
1384server's dedicated queue. This queue has higher priority than the global queue,
1385so it's easier for already registered users to enter the site than for new
1386users.
1387
1388For this, the logs have been enhanced to show the number of sessions per
1389server, the request's position in the queue and the time spent in the queue.
1390This helps doing capacity planning. See the 'logs' section below for more info.
1391
1392Example :
1393---------
1394 # be nice with P3 which only has 256 MB of RAM.
1395 listen web_appl 0.0.0.0:80
1396 maxconn 10000
1397 mode http
1398 cookie SERVERID insert nocache indirect
1399 balance roundrobin
1400 server pentium3-800 192.168.1.1:80 cookie s1 weight 8 maxconn 100 check
1401 server opteron-2.0G 192.168.1.2:80 cookie s2 weight 20 maxconn 300 check
1402 server opteron-2.4G 192.168.1.3:80 cookie s3 weight 24 maxconn 300 check
1403 server web-backup1 192.168.2.1:80 cookie s4 check maxconn 200 backup
1404 server web-excuse 192.168.3.1:80 check backup
1405
willy tarreauf76e6ca2006-05-21 21:09:55 +02001406
1407This was so much efficient at reducing the server's response time that some
1408users wanted to use low values to improve their server's performance. However,
1409they were not able anymore to handle very large loads because it was not
1410possible anymore to saturate the servers. For this reason, version 1.2.14 has
1411brought dynamic limitation with the addition of the parameter 'minconn'. When
1412this parameter is set along with maxconn, it will enable dynamic limitation
1413based on the instance's load. The maximum number of concurrent sessions on a
1414server will be proportionnal to the number of sessions on the instance relative
1415to its maxconn. A minimum of <minconn> will be allowed whatever the load. This
1416will ensure that servers will perform at their best level under normal loads,
1417while still handling surges when needed. The dynamic limit is computed like
1418this :
1419
1420 srv.dyn_limit = max(srv.minconn, srv.maxconn * inst.sess / inst.maxconn)
1421
1422Example :
1423---------
1424 # be nice with P3 which only has 256 MB of RAM.
1425 listen web_appl 0.0.0.0:80
1426 maxconn 10000
1427 mode http
1428 cookie SERVERID insert nocache indirect
1429 balance roundrobin
1430 server pentium3-800 192.168.1.1:80 cookie s1 weight 8 minconn 10 maxconn 100 check
1431 server opteron-2.0G 192.168.1.2:80 cookie s2 weight 20 minconn 30 maxconn 300 check
1432 server opteron-2.4G 192.168.1.3:80 cookie s3 weight 24 minconn 30 maxconn 300 check
1433 server web-backup1 192.168.2.1:80 cookie s4 check maxconn 200 backup
1434 server web-excuse 192.168.3.1:80 check backup
1435
1436In the example above, the server 'pentium3-800' will receive at most 100
1437simultaneous sessions when the proxy instance will reach 10000 sessions, and
1438will receive only 10 simultaneous sessions when the proxy will be under 1000
1439sessions.
1440
Elijah Epifanovacafc5f2007-10-25 20:15:38 +02001441It is possible to limit server queue length in order to rebalance excess
1442sessions between less busy application servers IF session affinity isn't
1443hard functional requirement (for example it just gives huge performance boost
1444by keeping server-local caches hot and compact). 'maxqueue' option sets a
1445queue limit on a server, as in example below:
1446
1447... (just the same as in example above)
1448 server pentium3-800 192.168.1.1:80 cookie s1 weight 8 minconn 10 maxconn 100 check maxqueue 50
1449 server opteron-2.0G 192.168.1.2:80 cookie s2 weight 20 minconn 30 maxconn 300 check maxqueue 200
1450 server opteron-2.4G 192.168.1.3:80 cookie s3 weight 24 minconn 30 maxconn 300 check
1451
1452Absence of 'maxqueue' option means unlimited queue. When queue gets filled
1453up to 'maxqueue' client session is moved from server-local queue to a global
1454one.
1455
willy tarreau532bb552006-05-13 18:40:37 +02001456Notes :
1457-------
1458 - The requests will not stay indefinitely in the queue, they follow the
1459 'contimeout' parameter, and if a request cannot be dequeued within this
1460 timeout because the server is saturated or because the queue is filled,
1461 the session will expire with a 503 error.
1462
willy tarreauf76e6ca2006-05-21 21:09:55 +02001463 - if only <minconn> is specified, it has the same effect as <maxconn>
1464
willy tarreau532bb552006-05-13 18:40:37 +02001465 - setting too low values for maxconn might improve performance but might also
1466 allow slow users to block access to the server for other users.
1467
willy tarreau34f45302006-04-15 21:37:14 +02001468
willy tarreaue0bdd622006-05-21 20:51:54 +020014693.5) Dropping aborted requests
1470------------------------------
1471In presence of very high loads, the servers will take some time to respond. The
1472per-proxy's connection queue will inflate, and the response time will increase
1473respective to the size of the queue times the average per-session response
1474time. When clients will wait for more than a few seconds, they will often hit
1475the 'STOP' button on their browser, leaving a useless request in the queue, and
1476slowing down other users.
1477
1478As there is no way to distinguish between a full STOP and a simple
1479shutdown(SHUT_WR) on the client side, HTTP agents should be conservative and
1480consider that the client might only have closed its output channel while
1481waiting for the response. However, this introduces risks of congestion when
1482lots of users do the same, and is completely useless nowadays because probably
1483no client at all will close the session while waiting for the response. Some
1484HTTP agents support this (Squid, Apache, HAProxy), and others do not (TUX, most
1485hardware-based load balancers). So the probability for a closed input channel
1486to represent a user hitting the 'STOP' button is close to 100%, and it is very
1487tempting to be able to abort the session early without polluting the servers.
1488
1489For this reason, a new option "abortonclose" was introduced in version 1.2.14.
1490By default (without the option) the behaviour is HTTP-compliant. But when the
1491option is specified, a session with an incoming channel closed will be aborted
1492if it's still possible, which means that it's either waiting for a connect() to
1493establish or it is queued waiting for a connection slot. This considerably
1494reduces the queue size and the load on saturated servers when users are tempted
1495to click on STOP, which in turn reduces the response time for other users.
1496
1497Example :
1498---------
1499 listen web_appl 0.0.0.0:80
1500 maxconn 10000
1501 mode http
1502 cookie SERVERID insert nocache indirect
1503 balance roundrobin
1504 server web1 192.168.1.1:80 cookie s1 weight 10 maxconn 100 check
1505 server web2 192.168.1.2:80 cookie s2 weight 10 maxconn 100 check
1506 server web3 192.168.1.3:80 cookie s3 weight 10 maxconn 100 check
1507 server bck1 192.168.2.1:80 cookie s4 check maxconn 200 backup
1508 option abortonclose
1509
1510
willy tarreaueedaa9f2005-12-17 14:08:03 +010015114) Additionnal features
1512=======================
1513
willy tarreau481132e2006-05-21 21:43:10 +02001514Other features are available. They are transparent mode, event logging, header
1515rewriting/filtering, and the status as an HTML page.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001516
willy tarreauc5f73ed2005-12-18 01:26:38 +01001517
willy tarreau0174f312005-12-18 01:02:42 +010015184.1) Network features
willy tarreaueedaa9f2005-12-17 14:08:03 +01001519---------------------
willy tarreau0174f312005-12-18 01:02:42 +010015204.1.1) Transparent mode
1521-----------------------
willy tarreaueedaa9f2005-12-17 14:08:03 +01001522In HTTP mode, the 'transparent' keyword allows to intercept sessions which are
1523routed through the system hosting the proxy. This mode was implemented as a
1524replacement for the 'dispatch' mode, since connections without cookie will be
1525sent to the original address while known cookies will be sent to the servers.
1526This mode implies that the system can redirect sessions to a local port.
1527
1528Example :
1529---------
1530 listen http_proxy 0.0.0.0:65000
willy tarreauc5f73ed2005-12-18 01:26:38 +01001531 mode http
1532 transparent
1533 cookie SERVERID
1534 server server01 192.168.1.1:80
1535 server server02 192.168.1.2:80
willy tarreaueedaa9f2005-12-17 14:08:03 +01001536
1537 # iptables -t nat -A PREROUTING -i eth0 -p tcp -d 192.168.1.100 \
1538 --dport 80 -j REDIRECT --to-ports 65000
1539
1540Note :
1541------
1542If the port is left unspecified on the server, the port the client connected to
1543will be used. This allows to relay a full port range without using transparent
1544mode nor thousands of file descriptors, provided that the system can redirect
1545sessions to local ports.
1546
1547Example :
1548---------
1549 # redirect all ports to local port 65000, then forward to the server on the
1550 # original port.
1551 listen http_proxy 0.0.0.0:65000
willy tarreauc5f73ed2005-12-18 01:26:38 +01001552 mode tcp
1553 server server01 192.168.1.1 check port 60000
1554 server server02 192.168.1.2 check port 60000
willy tarreaueedaa9f2005-12-17 14:08:03 +01001555
1556 # iptables -t nat -A PREROUTING -i eth0 -p tcp -d 192.168.1.100 \
1557 -j REDIRECT --to-ports 65000
1558
willy tarreau0174f312005-12-18 01:02:42 +010015594.1.2) Per-server source address binding
1560----------------------------------------
1561As of versions 1.1.30 and 1.2.3, it is possible to specify a particular source
1562to reach each server. This is useful when reaching backup servers from a
1563different LAN, or to use an alternate path to reach the same server. It is also
1564usable to provide source load-balancing for outgoing connections. Obviously,
1565the same source address is used to send health-checks.
1566
1567Example :
1568---------
1569 # use a particular source to reach both servers
1570 listen http_proxy 0.0.0.0:65000
willy tarreauc5f73ed2005-12-18 01:26:38 +01001571 mode http
1572 balance roundrobin
1573 server server01 192.168.1.1:80 source 192.168.2.13
1574 server server02 192.168.1.2:80 source 192.168.2.13
willy tarreau0174f312005-12-18 01:02:42 +01001575
1576Example :
1577---------
1578 # use a particular source to reach each servers
1579 listen http_proxy 0.0.0.0:65000
willy tarreauc5f73ed2005-12-18 01:26:38 +01001580 mode http
1581 balance roundrobin
1582 server server01 192.168.1.1:80 source 192.168.1.1
1583 server server02 192.168.2.1:80 source 192.168.2.1
willy tarreau0174f312005-12-18 01:02:42 +01001584
1585Example :
1586---------
1587 # provide source load-balancing to reach the same proxy through 2 WAN links
1588 listen http_proxy 0.0.0.0:65000
willy tarreauc5f73ed2005-12-18 01:26:38 +01001589 mode http
1590 balance roundrobin
1591 server remote-proxy-way1 192.168.1.1:3128 source 192.168.2.1
1592 server remote-proxy-way2 192.168.1.1:3128 source 192.168.3.1
willy tarreau0174f312005-12-18 01:02:42 +01001593
1594Example :
1595---------
1596 # force a TCP connection to bind to a specific port
1597 listen http_proxy 0.0.0.0:2000
willy tarreauc5f73ed2005-12-18 01:26:38 +01001598 mode tcp
1599 balance roundrobin
1600 server srv1 192.168.1.1:80 source 192.168.2.1:20
1601 server srv2 192.168.1.2:80 source 192.168.2.1:20
willy tarreau0174f312005-12-18 01:02:42 +01001602
willy tarreaub952e1d2005-12-18 01:31:20 +010016034.1.3) TCP keep-alive
1604---------------------
1605With version 1.2.7, it becomes possible to enable TCP keep-alives on both the
1606client and server sides. This makes it possible to prevent long sessions from
1607expiring on external layer 4 components such as firewalls and load-balancers.
1608It also allows the system to terminate dead sessions when no timeout has been
1609set (not recommanded). The proxy cannot set the keep-alive probes intervals nor
1610maximal count, consult your operating system manual for this. There are 3
1611options to enable TCP keep-alive :
1612
1613 option tcpka # enables keep-alive both on client and server side
1614 option clitcpka # enables keep-alive only on client side
1615 option srvtcpka # enables keep-alive only on server side
1616
Alexandre Cassen87ea5482007-10-11 20:48:58 +020016174.1.4) TCP lingering
1618--------------------
1619It is possible to disable the system's lingering of data unacked by the client
1620at the end of a session. This is sometimes required when haproxy is used as a
1621front-end with lots of unreliable clients, and you observe thousands of sockets
1622in the FIN_WAIT state on the machine. This may be used in a frontend to affect
1623the client-side connection, as well as in a backend for the server-side
1624connection :
1625
1626 option nolinger # disables data lingering
1627
willy tarreaueedaa9f2005-12-17 14:08:03 +01001628
16294.2) Event logging
1630------------------
willy tarreauc5f73ed2005-12-18 01:26:38 +01001631
1632HAProxy's strength certainly lies in its precise logs. It probably provides the
1633finest level of information available for such a product, which is very
1634important for troubleshooting complex environments. Standard log information
1635include client ports, TCP/HTTP state timers, precise session state at
1636termination and precise termination cause, information about decisions to
1637direct trafic to a server, and of course the ability to capture arbitrary
1638headers.
1639
1640In order to improve administrators reactivity, it offers a great transparency
1641about encountered problems, both internal and external, and it is possible to
1642send logs to different sources at the same time with different level filters :
1643
1644 - global process-level logs (system errors, start/stop, etc..)
1645 - per-listener system and internal errors (lack of resource, bugs, ...)
1646 - per-listener external troubles (servers up/down, max connections)
1647 - per-listener activity (client connections), either at the establishment or
1648 at the termination.
1649
1650The ability to distribute different levels of logs to different log servers
1651allow several production teams to interact and to fix their problems as soon
1652as possible. For example, the system team might monitor system-wide errors,
1653while the application team might be monitoring the up/down for their servers in
1654real time, and the security team might analyze the activity logs with one hour
1655delay.
1656
willy tarreauc1cae632005-12-17 14:12:23 +010016574.2.1) Log levels
1658-----------------
willy tarreau197e8ec2005-12-17 14:10:59 +01001659TCP and HTTP connections can be logged with informations such as date, time,
1660source IP address, destination address, connection duration, response times,
1661HTTP request, the HTTP return code, number of bytes transmitted, the conditions
1662in which the session ended, and even exchanged cookies values, to track a
1663particular user's problems for example. All messages are sent to up to two
1664syslog servers. Consult section 1.1 for more info about log facilities. The
1665syntax follows :
willy tarreaueedaa9f2005-12-17 14:08:03 +01001666
willy tarreau197e8ec2005-12-17 14:10:59 +01001667 log <address_1> <facility_1> [max_level_1]
1668 log <address_2> <facility_2> [max_level_2]
1669or
willy tarreaueedaa9f2005-12-17 14:08:03 +01001670 log global
1671
willy tarreau197e8ec2005-12-17 14:10:59 +01001672Note :
1673------
1674The particular syntax 'log global' means that the same log configuration as the
1675'global' section will be used.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001676
willy tarreau197e8ec2005-12-17 14:10:59 +01001677Example :
willy tarreaueedaa9f2005-12-17 14:08:03 +01001678---------
1679 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001680 mode http
1681 log 192.168.2.200 local3
1682 log 192.168.2.201 local4
willy tarreaueedaa9f2005-12-17 14:08:03 +01001683
willy tarreauc1cae632005-12-17 14:12:23 +010016844.2.2) Log format
1685-----------------
1686By default, connections are logged at the TCP level, as soon as the session
1687establishes between the client and the proxy. By enabling the 'tcplog' option,
1688the proxy will wait until the session ends to generate an enhanced log
1689containing more information such as session duration and its state during the
willy tarreau532bb552006-05-13 18:40:37 +02001690disconnection. The number of remaining session after disconnection is also
1691indicated (for the server, the listener, and the process).
willy tarreauc1cae632005-12-17 14:12:23 +01001692
willy tarreauc5f73ed2005-12-18 01:26:38 +01001693Example of TCP logging :
1694------------------------
willy tarreau982249e2005-12-18 00:57:06 +01001695 listen relais-tcp 0.0.0.0:8000
willy tarreauc5f73ed2005-12-18 01:26:38 +01001696 mode tcp
1697 option tcplog
1698 log 192.168.2.200 local3
willy tarreau982249e2005-12-18 00:57:06 +01001699
willy tarreau532bb552006-05-13 18:40:37 +02001700>>> haproxy[18989]: 127.0.0.1:34550 [15/Oct/2003:15:24:28] relais-tcp Srv1 0/0/5007 0 -- 1/1/1 0/0
willy tarreauc5f73ed2005-12-18 01:26:38 +01001701
willy tarreau532bb552006-05-13 18:40:37 +02001702 Field Format Example
willy tarreauc5f73ed2005-12-18 01:26:38 +01001703
willy tarreau532bb552006-05-13 18:40:37 +02001704 1 process_name '[' pid ']:' haproxy[18989]:
1705 2 client_ip ':' client_port 127.0.0.1:34550
1706 3 '[' date ']' [15/Oct/2003:15:24:28]
1707 4 listener_name relais-tcp
1708 5 server_name Srv1
1709 6 queue_time '/' connect_time '/' total_time 0/0/5007
1710 7 bytes_read 0
1711 8 termination_state --
1712 9 srv_conn '/' listener_conn '/' process_conn 1/1/1
1713 10 position in srv_queue / listener_queue 0/0
1714
willy tarreau982249e2005-12-18 00:57:06 +01001715
willy tarreauc1cae632005-12-17 14:12:23 +01001716Another option, 'httplog', provides more detailed information about HTTP
1717contents, such as the request and some cookies. In the event where an external
1718component would establish frequent connections to check the service, logs may be
1719full of useless lines. So it is possible not to log any session which didn't
1720transfer any data, by the setting of the 'dontlognull' option. This only has
1721effect on sessions which are established then closed.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001722
willy tarreauc5f73ed2005-12-18 01:26:38 +01001723Example of HTTP logging :
1724-------------------------
willy tarreaueedaa9f2005-12-17 14:08:03 +01001725 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001726 mode http
1727 option httplog
1728 option dontlognull
1729 log 192.168.2.200 local3
1730
Krzysztof Piotr Oledzki25b501a2008-01-06 16:36:16 +01001731>>> haproxy[674]: 127.0.0.1:33319 [15/Oct/2003:08:31:57] relais-http Srv1 9/0/7/147/723 200 243 - - ---- 34/34/15/8/3 0/0 "HEAD / HTTP/1.0"
willy tarreaueedaa9f2005-12-17 14:08:03 +01001732
willy tarreauc5f73ed2005-12-18 01:26:38 +01001733More complete example
Krzysztof Piotr Oledzki25b501a2008-01-06 16:36:16 +01001734 haproxy[18989]: 10.0.0.1:34552 [15/Oct/2003:15:26:31] relais-http Srv1 3183/-1/-1/-1/11215 503 0 - - SC-- 205/202/150/137/+4 0/0 {w.ods.org|Mozilla} {} "HEAD / HTTP/1.0"
willy tarreauc5f73ed2005-12-18 01:26:38 +01001735
Krzysztof Piotr Oledzki25b501a2008-01-06 16:36:16 +01001736 Field Format Example
willy tarreauc5f73ed2005-12-18 01:26:38 +01001737
Krzysztof Piotr Oledzki25b501a2008-01-06 16:36:16 +01001738 1 process_name '[' pid ']:' haproxy[18989]:
1739 2 client_ip ':' client_port 10.0.0.1:34552
1740 3 '[' date ']' [15/Oct/2003:15:26:31]
1741 4 listener_name relais-http
1742 5 server_name Srv1
1743 6 Tq '/' Tw '/' Tc '/' Tr '/' Tt 3183/-1/-1/-1/11215
1744 7 HTTP_return_code 503
1745 8 bytes_read 0
1746 9 captured_request_cookie -
1747 10 captured_response_cookie -
1748 11 termination_state SC--
1749 12 actconn '/' feconn '/' beconn '/' srv_conn '/' retries 205/202/150/137/+4
1750 13 position in srv_queue / listener_queue 0/0
1751 14 '{' captured_request_headers '}' {w.ods.org|Mozilla}
1752 15 '{' captured_response_headers '}' {}
1753 16 '"' HTTP_request '"' "HEAD / HTTP/1.0"
willy tarreauc5f73ed2005-12-18 01:26:38 +01001754
1755Note for log parsers: the URI is ALWAYS the end of the line starting with the
1756 first double quote '"'.
willy tarreau982249e2005-12-18 00:57:06 +01001757
Krzysztof Piotr Oledzki25b501a2008-01-06 16:36:16 +01001758The retries count may have additional '+' sign means that the connection had been
1759redispatched from one server to another shortly before retries limit (retries 4
1760in above example) was depleted.
1761
willy tarreau982249e2005-12-18 00:57:06 +01001762The problem when logging at end of connection is that you have no clue about
1763what is happening during very long sessions. To workaround this problem, a
1764new option 'logasap' has been introduced in 1.1.28/1.2.1. When specified, the
1765proxy will log as soon as possible, just before data transfer begins. This means
1766that in case of TCP, it will still log the connection status to the server, and
1767in case of HTTP, it will log just after processing the server headers. In this
1768case, the number of bytes reported is the number of header bytes sent to the
1769client.
1770
1771In order to avoid confusion with normal logs, the total time field and the
Krzysztof Piotr Oledzki25b501a2008-01-06 16:36:16 +01001772number of bytes are prefixed with a '+' sign which means that real numbers are
willy tarreau982249e2005-12-18 00:57:06 +01001773certainly bigger.
1774
1775Example :
1776---------
1777
1778 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001779 mode http
1780 option httplog
1781 option dontlognull
1782 option logasap
1783 log 192.168.2.200 local3
willy tarreau982249e2005-12-18 00:57:06 +01001784
Krzysztof Piotr Oledzki25b501a2008-01-06 16:36:16 +01001785>>> haproxy[674]: 127.0.0.1:33320 [15/Oct/2003:08:32:17] relais-http Srv1 9/10/7/14/+30 200 +243 - - ---- 3/1/1/1/0 1/0 "GET /image.iso HTTP/1.0"
willy tarreau982249e2005-12-18 00:57:06 +01001786
willy tarreauc1cae632005-12-17 14:12:23 +010017874.2.3) Timing events
1788--------------------
1789Timers provide a great help in trouble shooting network problems. All values
1790are reported in milliseconds (ms). In HTTP mode, four control points are
willy tarreau532bb552006-05-13 18:40:37 +02001791reported under the form 'Tq/Tw/Tc/Tr/Tt' :
willy tarreauc1cae632005-12-17 14:12:23 +01001792
1793 - Tq: total time to get the client request.
1794 It's the time elapsed between the moment the client connection was accepted
1795 and the moment the proxy received the last HTTP header. The value '-1'
1796 indicates that the end of headers (empty line) has never been seen.
1797
willy tarreau532bb552006-05-13 18:40:37 +02001798 - Tw: total time spent in the queues waiting for a connection slot. It
1799 accounts for listener's queue as well as the server's queue, and depends
1800 on the queue size, and the time needed for the server to complete previous
1801 sessions. The value '-1' means that the request was killed before reaching
1802 the queue.
1803
willy tarreauc1cae632005-12-17 14:12:23 +01001804 - Tc: total time to establish the TCP connection to the server.
1805 It's the time elapsed between the moment the proxy sent the connection
1806 request, and the moment it was acknowledged, or between the TCP SYN packet
1807 and the matching SYN/ACK in return. The value '-1' means that the
1808 connection never established.
1809
1810 - Tr: server response time. It's the time elapsed between the moment the
1811 TCP connection was established to the server and the moment it send its
1812 complete response header. It purely shows its request processing time,
1813 without the network overhead due to the data transmission. The value '-1'
1814 means that the last the response header (empty line) was never seen.
1815
1816 - Tt: total session duration time, between the moment the proxy accepted it
willy tarreau982249e2005-12-18 00:57:06 +01001817 and the moment both ends were closed. The exception is when the 'logasap'
willy tarreau532bb552006-05-13 18:40:37 +02001818 option is specified. In this case, it only equals (Tq+Tw+Tc+Tr), and is
willy tarreau982249e2005-12-18 00:57:06 +01001819 prefixed with a '+' sign. From this field, we can deduce Td, the data
1820 transmission time, by substracting other timers when valid :
willy tarreauc1cae632005-12-17 14:12:23 +01001821
willy tarreau532bb552006-05-13 18:40:37 +02001822 Td = Tt - (Tq + Tw + Tc + Tr)
willy tarreauc1cae632005-12-17 14:12:23 +01001823
1824 Timers with '-1' values have to be excluded from this equation.
1825
willy tarreau532bb552006-05-13 18:40:37 +02001826In TCP mode ('option tcplog'), only Tw, Tc and Tt are reported.
willy tarreauc1cae632005-12-17 14:12:23 +01001827
1828These timers provide precious indications on trouble causes. Since the TCP
1829protocol defines retransmit delays of 3, 6, 12... seconds, we know for sure
1830that timers close to multiples of 3s are nearly always related to packets lost
1831due to network problems (wires or negociation). Moreover, if <Tt> is close to
1832a timeout value specified in the configuration, it often means that a session
1833has been aborted on time-out.
1834
1835Most common cases :
1836
1837 - If Tq is close to 3000, a packet has probably been lost between the client
1838 and the proxy.
1839 - If Tc is close to 3000, a packet has probably been lost between the server
1840 and the proxy during the server connection phase. This one should always be
1841 very low (less than a few tens).
1842 - If Tr is nearly always lower than 3000 except some rare values which seem to
1843 be the average majored by 3000, there are probably some packets lost between
1844 the proxy and the server.
1845 - If Tt is often slightly higher than a time-out, it's often because the
1846 client and the server use HTTP keep-alive and the session is maintained
1847 after the response ends. Se further for how to disable HTTP keep-alive.
1848
1849Other cases ('xx' means any value to be ignored) :
willy tarreau532bb552006-05-13 18:40:37 +02001850 -1/xx/xx/xx/Tt: the client was not able to send its complete request in time,
1851 or that it aborted it too early.
1852 Tq/-1/xx/xx/Tt: it was not possible to process the request, maybe because
1853 servers were out of order.
1854 Tq/Tw/-1/xx/Tt: the connection could not establish on the server. Either it
1855 refused it or it timed out after Tt-(Tq+Tw) ms.
1856 Tq/Tw/Tc/-1/Tt: the server has accepted the connection but did not return a
1857 complete response in time, or it closed its connexion
1858 unexpectedly, after Tt-(Tq+Tw+Tc) ms.
willy tarreauc1cae632005-12-17 14:12:23 +01001859
18604.2.4) Session state at disconnection
1861-------------------------------------
willy tarreauc5f73ed2005-12-18 01:26:38 +01001862TCP and HTTP logs provide a session completion indicator in the
1863<termination_state> field, just before the number of active
1864connections. It is 2-characters long in TCP, and 4-characters long in
1865HTTP, each of which has a special meaning :
1866
willy tarreau197e8ec2005-12-17 14:10:59 +01001867 - On the first character, a code reporting the first event which caused the
1868 session to terminate :
willy tarreaueedaa9f2005-12-17 14:08:03 +01001869
willy tarreauc5f73ed2005-12-18 01:26:38 +01001870 C : the TCP session was unexpectedly aborted by the client.
1871
1872 S : the TCP session was unexpectedly aborted by the server, or the
1873 server explicitly refused it.
1874
1875 P : the session was prematurely aborted by the proxy, because of a
1876 connection limit enforcement, because a DENY filter was matched,
1877 or because of a security check which detected and blocked a
1878 dangerous error in server response which might have caused
1879 information leak (eg: cacheable cookie).
1880
1881 R : a resource on the proxy has been exhausted (memory, sockets, source
1882 ports, ...). Usually, this appears during the connection phase, and
1883 system logs should contain a copy of the precise error.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001884
willy tarreauc5f73ed2005-12-18 01:26:38 +01001885 I : an internal error was identified by the proxy during a self-check.
1886 This should NEVER happen, and you are encouraged to report any log
1887 containing this, because this is a bug.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001888
willy tarreauc5f73ed2005-12-18 01:26:38 +01001889 c : the client-side time-out expired first.
1890
1891 s : the server-side time-out expired first.
1892
1893 - : normal session completion.
1894
1895 - on the second character, the TCP/HTTP session state when it was closed :
1896
1897 R : waiting for complete REQUEST from the client (HTTP only). Nothing
1898 was sent to any server.
1899
willy tarreau532bb552006-05-13 18:40:37 +02001900 Q : waiting in the QUEUE for a connection slot. This can only happen on
1901 servers which have a 'maxconn' parameter set. No connection attempt
1902 was made to any server.
1903
willy tarreauc5f73ed2005-12-18 01:26:38 +01001904 C : waiting for CONNECTION to establish on the server. The server might
1905 at most have noticed a connection attempt.
1906
1907 H : waiting for, receiving and processing server HEADERS (HTTP only).
1908
1909 D : the session was in the DATA phase.
1910
1911 L : the proxy was still transmitting LAST data to the client while the
1912 server had already finished.
1913
Willy Tarreau2272dc12006-09-03 10:19:38 +02001914 T : the request was tarpitted. It has been held open on with the client
Willy Tarreau08fa2e32006-09-03 10:47:37 +02001915 during the whole contimeout duration or untill the client closed.
Willy Tarreau2272dc12006-09-03 10:19:38 +02001916
willy tarreauc5f73ed2005-12-18 01:26:38 +01001917 - : normal session completion after end of data transfer.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001918
willy tarreau197e8ec2005-12-17 14:10:59 +01001919 - the third character tells whether the persistence cookie was provided by
willy tarreauc1cae632005-12-17 14:12:23 +01001920 the client (only in HTTP mode) :
willy tarreaueedaa9f2005-12-17 14:08:03 +01001921
willy tarreauc5f73ed2005-12-18 01:26:38 +01001922 N : the client provided NO cookie. This is usually the case on new
1923 connections.
1924
1925 I : the client provided an INVALID cookie matching no known
1926 server. This might be caused by a recent configuration change,
1927 mixed cookies between HTTP/HTTPS sites, or an attack.
1928
1929 D : the client provided a cookie designating a server which was DOWN,
1930 so either the 'persist' option was used and the client was sent to
1931 this server, or it was not set and the client was redispatched to
1932 another server.
1933
1934 V : the client provided a valid cookie, and was sent to the associated
1935 server.
1936
1937 - : does not apply (no cookie set in configuration).
willy tarreaueedaa9f2005-12-17 14:08:03 +01001938
willy tarreau197e8ec2005-12-17 14:10:59 +01001939 - the last character reports what operations were performed on the persistence
willy tarreauc1cae632005-12-17 14:12:23 +01001940 cookie returned by the server (only in HTTP mode) :
willy tarreaueedaa9f2005-12-17 14:08:03 +01001941
willy tarreauc5f73ed2005-12-18 01:26:38 +01001942 N : NO cookie was provided by the server, and none was inserted either.
1943
1944 I : no cookie was provided by the server, and the proxy INSERTED one.
1945
willy tarreau197e8ec2005-12-17 14:10:59 +01001946 P : a cookie was PROVIDED by the server and transmitted as-is.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001947
willy tarreauc5f73ed2005-12-18 01:26:38 +01001948 R : the cookie provided by the server was REWRITTEN by the proxy.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001949
willy tarreauc5f73ed2005-12-18 01:26:38 +01001950 D : the cookie provided by the server was DELETED by the proxy.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001951
willy tarreauc5f73ed2005-12-18 01:26:38 +01001952 - : does not apply (no cookie set in configuration).
willy tarreaueedaa9f2005-12-17 14:08:03 +01001953
willy tarreauc5f73ed2005-12-18 01:26:38 +01001954The combination of the two first flags give a lot of information about what was
1955happening when the session terminated. It can be helpful to detect server
1956saturation, network troubles, local system resource starvation, attacks, etc...
willy tarreaueedaa9f2005-12-17 14:08:03 +01001957
willy tarreauc5f73ed2005-12-18 01:26:38 +01001958The most common termination flags combinations are indicated here.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001959
willy tarreauc5f73ed2005-12-18 01:26:38 +01001960 Flags Reason
1961 CR The client aborted before sending a full request. Most probably the
1962 request was done by hand using a telnet client, and aborted early.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001963
willy tarreauc5f73ed2005-12-18 01:26:38 +01001964 cR The client timed out before sending a full request. This is sometimes
1965 caused by too large TCP MSS values on the client side for PPPoE
1966 networks which cannot transport full-sized packets, or by clients
1967 sending requests by hand and not typing fast enough.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001968
willy tarreauc5f73ed2005-12-18 01:26:38 +01001969 SC The server explicitly refused the connection (the proxy received a
1970 TCP RST or an ICMP in return). Under some circumstances, it can
1971 also be the network stack telling the proxy that the server is
1972 unreachable (eg: no route, or no ARP response on local network).
willy tarreau982249e2005-12-18 00:57:06 +01001973
willy tarreauc5f73ed2005-12-18 01:26:38 +01001974 sC The connection to the server did not complete during contimeout.
willy tarreau982249e2005-12-18 00:57:06 +01001975
willy tarreauc5f73ed2005-12-18 01:26:38 +01001976 PC The proxy refused to establish a connection to the server because the
1977 maxconn limit has been reached. The listener's maxconn parameter may
1978 be increased in the proxy configuration, as well as the global
1979 maxconn parameter.
willy tarreauc1cae632005-12-17 14:12:23 +01001980
willy tarreauc5f73ed2005-12-18 01:26:38 +01001981 RC A local resource has been exhausted (memory, sockets, source ports)
1982 preventing the connection to the server from establishing. The error
1983 logs will tell precisely what was missing. Anyway, this can only be
1984 solved by system tuning.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001985
willy tarreauc5f73ed2005-12-18 01:26:38 +01001986 cH The client timed out during a POST request. This is sometimes caused
1987 by too large TCP MSS values for PPPoE networks which cannot transport
1988 full-sized packets.
willy tarreauc1cae632005-12-17 14:12:23 +01001989
willy tarreau078c79a2006-05-13 12:23:58 +02001990 CH The client aborted while waiting for the server to start responding.
1991 It might be the server taking too long to respond or the client
1992 clicking the 'Stop' button too fast.
1993
1994 CQ The client aborted while its session was queued, waiting for a server
1995 with enough empty slots to accept it. It might be that either all the
1996 servers were saturated or the assigned server taking too long to
1997 respond.
1998
Willy Tarreau08fa2e32006-09-03 10:47:37 +02001999 CT The client aborted while its session was tarpitted.
2000
willy tarreau078c79a2006-05-13 12:23:58 +02002001 sQ The session spent too much time in queue and has been expired.
2002
willy tarreauc5f73ed2005-12-18 01:26:38 +01002003 SH The server aborted before sending its full headers, or it crashed.
willy tarreaueedaa9f2005-12-17 14:08:03 +01002004
willy tarreauc5f73ed2005-12-18 01:26:38 +01002005 sH The server failed to reply during the srvtimeout delay, which
2006 indicates too long transactions, probably caused by back-end
2007 saturation. The only solutions are to fix the problem on the
2008 application or to increase the 'srvtimeout' parameter to support
2009 longer delays (at the risk of the client giving up anyway).
2010
2011 PR The proxy blocked the client's request, either because of an invalid
2012 HTTP syntax, in which case it returned an HTTP 400 error to the
2013 client, or because a deny filter matched, in which case it returned
2014 an HTTP 403 error.
2015
2016 PH The proxy blocked the server's response, because it was invalid,
2017 incomplete, dangerous (cache control), or matched a security filter.
2018 In any case, an HTTP 502 error is sent to the client.
2019
Willy Tarreau2272dc12006-09-03 10:19:38 +02002020 PT The proxy blocked the client's request and has tarpitted its
2021 connection before returning it a 500 server error. Nothing was sent
2022 to the server.
2023
willy tarreauc5f73ed2005-12-18 01:26:38 +01002024 cD The client did not read any data for as long as the clitimeout delay.
2025 This is often caused by network failures on the client side.
2026
2027 CD The client unexpectedly aborted during data transfer. This is either
2028 caused by a browser crash, or by a keep-alive session between the
2029 server and the client terminated first by the client.
2030
2031 sD The server did nothing during the srvtimeout delay. This is often
2032 caused by too short timeouts on L4 equipements before the server
2033 (firewalls, load-balancers, ...).
2034
20354.2.5) Non-printable characters
willy tarreau4302f492005-12-18 01:00:37 +01002036-------------------------------
2037As of version 1.1.29, non-printable characters are not sent as-is into log
2038files, but are converted to their two-digits hexadecimal representation,
2039prefixed by the character '#'. The only characters that can now be logged
2040without being escaped are between 32 and 126 (inclusive). Obviously, the
2041escape character '#' is also encoded to avoid any ambiguity. It is the same for
2042the character '"', as well as '{', '|' and '}' when logging headers.
2043
willy tarreauc5f73ed2005-12-18 01:26:38 +010020444.2.6) Capturing HTTP headers and cookies
2045-----------------------------------------
2046Version 1.1.23 brought cookie capture, and 1.1.29 the header capture. All this
2047is performed using the 'capture' keyword.
2048
2049Cookie capture makes it easy to track a complete user session. The syntax is :
2050
2051 capture cookie <cookie_prefix> len <capture_length>
2052
2053This will enable cookie capture from both requests and responses. This way,
2054it's easy to detect when a user switches to a new session for example, because
2055the server will reassign it a new cookie.
2056
2057The FIRST cookie whose name starts with <cookie_prefix> will be captured, and
2058logged as 'NAME=value', without exceeding <capture_length> characters (64 max).
2059When the cookie name is fixed and known, it's preferable to suffix '=' to it to
2060ensure that no other cookie will be logged.
2061
2062Examples :
2063----------
2064 # capture the first cookie whose name starts with "ASPSESSION"
2065 capture cookie ASPSESSION len 32
2066
2067 # capture the first cookie whose name is exactly "vgnvisitor"
2068 capture cookie vgnvisitor= len 32
2069
2070In the logs, the field preceeding the completion indicator contains the cookie
2071value as sent by the server, preceeded by the cookie value as sent by the
2072client. Each of these field is replaced with '-' when no cookie was seen or
2073when the option is disabled.
2074
2075Header captures have a different goal. They are useful to track unique request
2076identifiers set by a previous proxy, virtual host names, user-agents, POST
2077content-length, referrers, etc. In the response, one can search for information
2078about the response length, how the server asked the cache to behave, or an
2079object location during a redirection. As for cookie captures, it is both
2080possible to include request headers and response headers at the same time. The
2081syntax is :
willy tarreau4302f492005-12-18 01:00:37 +01002082
2083 capture request header <name> len <max length>
2084 capture response header <name> len <max length>
2085
2086Note: Header names are not case-sensitive.
2087
2088Examples:
2089---------
2090 # keep the name of the virtual server
2091 capture request header Host len 20
2092 # keep the amount of data uploaded during a POST
2093 capture request header Content-Length len 10
2094
2095 # note the expected cache behaviour on the response
2096 capture response header Cache-Control len 8
2097 # note the URL location during a redirection
2098 capture response header Location len 20
2099
2100Non-existant headers are logged as empty strings, and if one header appears more
2101than once, only its last occurence will be kept. Request headers are grouped
2102within braces '{' and '}' in the same order as they were declared, and delimited
2103with a vertical bar '|' without any space. Response headers follow the same
2104representation, but are displayed after a space following the request headers
2105block. These blocks are displayed just before the HTTP request in the logs.
willy tarreauc5f73ed2005-12-18 01:26:38 +01002106
willy tarreau4302f492005-12-18 01:00:37 +01002107Example :
2108
willy tarreauc5f73ed2005-12-18 01:26:38 +01002109 Config:
2110
2111 capture request header Host len 20
2112 capture request header Content-Length len 10
2113 capture request header Referer len 20
2114 capture response header Server len 20
2115 capture response header Content-Length len 10
2116 capture response header Cache-Control len 8
2117 capture response header Via len 20
2118 capture response header Location len 20
2119
2120 Log :
2121
willy tarreau532bb552006-05-13 18:40:37 +02002122 Aug 9 20:26:09 localhost haproxy[2022]: 127.0.0.1:34014 [09/Aug/2004:20:26:09] relais-http netcache 0/0/0/162/+162 200 +350 - - ---- 0/0/0 0/0 {fr.adserver.yahoo.co||http://fr.f416.mail.} {|864|private||} "GET http://fr.adserver.yahoo.com/"
2123 Aug 9 20:30:46 localhost haproxy[2022]: 127.0.0.1:34020 [09/Aug/2004:20:30:46] relais-http netcache 0/0/0/182/+182 200 +279 - - ---- 0/0/0 0/0 {w.ods.org||} {Formilux/0.1.8|3495|||} "GET http://w.ods.org/sytadin.html HTTP/1.1"
2124 Aug 9 20:30:46 localhost haproxy[2022]: 127.0.0.1:34028 [09/Aug/2004:20:30:46] relais-http netcache 0/0/2/126/+128 200 +223 - - ---- 0/0/0 0/0 {www.infotrafic.com||http://w.ods.org/syt} {Apache/2.0.40 (Red H|9068|||} "GET http://www.infotrafic.com/images/live/cartesidf/grandes/idf_ne.png HTTP/1.1"
willy tarreauc5f73ed2005-12-18 01:26:38 +01002125
2126
21274.2.7) Examples of logs
2128-----------------------
willy tarreau532bb552006-05-13 18:40:37 +02002129- haproxy[674]: 127.0.0.1:33319 [15/Oct/2003:08:31:57] relais-http Srv1 6559/0/7/147/6723 200 243 - - ---- 1/3/5 0/0 "HEAD / HTTP/1.0"
willy tarreauc5f73ed2005-12-18 01:26:38 +01002130 => long request (6.5s) entered by hand through 'telnet'. The server replied
2131 in 147 ms, and the session ended normally ('----')
2132
willy tarreau532bb552006-05-13 18:40:37 +02002133- haproxy[674]: 127.0.0.1:33319 [15/Oct/2003:08:31:57] relais-http Srv1 6559/1230/7/147/6870 200 243 - - ---- 99/239/324 0/9 "HEAD / HTTP/1.0"
2134 => Idem, but the request was queued in the global queue behind 9 other
2135 requests, and waited there for 1230 ms.
2136
2137- haproxy[674]: 127.0.0.1:33320 [15/Oct/2003:08:32:17] relais-http Srv1 9/0/7/14/+30 200 +243 - - ---- 1/3/3 0/0 "GET /image.iso HTTP/1.0"
willy tarreauc5f73ed2005-12-18 01:26:38 +01002138 => request for a long data transfer. The 'logasap' option was specified, so
2139 the log was produced just before transfering data. The server replied in
2140 14 ms, 243 bytes of headers were sent to the client, and total time from
2141 accept to first data byte is 30 ms.
2142
willy tarreau532bb552006-05-13 18:40:37 +02002143- haproxy[674]: 127.0.0.1:33320 [15/Oct/2003:08:32:17] relais-http Srv1 9/0/7/14/30 502 243 - - PH-- 0/2/3 0/0 "GET /cgi-bin/bug.cgi? HTTP/1.0"
willy tarreauc5f73ed2005-12-18 01:26:38 +01002144 => the proxy blocked a server response either because of an 'rspdeny' or
2145 'rspideny' filter, or because it blocked sensible information which risked
2146 being cached. In this case, the response is replaced with a '502 bad
2147 gateway'.
2148
willy tarreau532bb552006-05-13 18:40:37 +02002149- haproxy[18113]: 127.0.0.1:34548 [15/Oct/2003:15:18:55] relais-http <NOSRV> -1/-1/-1/-1/8490 -1 0 - - CR-- 0/2/2 0/0 ""
willy tarreauc5f73ed2005-12-18 01:26:38 +01002150 => the client never completed its request and aborted itself ('C---') after
2151 8.5s, while the proxy was waiting for the request headers ('-R--').
2152 Nothing was sent to the server.
2153
willy tarreau532bb552006-05-13 18:40:37 +02002154- haproxy[18113]: 127.0.0.1:34549 [15/Oct/2003:15:19:06] relais-http <NOSRV> -1/-1/-1/-1/50001 408 0 - - cR-- 2/2 0/0 ""
willy tarreauc5f73ed2005-12-18 01:26:38 +01002155 => The client never completed its request, which was aborted by the time-out
2156 ('c---') after 50s, while the proxy was waiting for the request headers ('-R--').
2157 Nothing was sent to the server, but the proxy could send a 408 return code
2158 to the client.
willy tarreau4302f492005-12-18 01:00:37 +01002159
willy tarreau532bb552006-05-13 18:40:37 +02002160- haproxy[18989]: 127.0.0.1:34550 [15/Oct/2003:15:24:28] relais-tcp Srv1 0/0/5007 0 cD 0/0/0 0/0
willy tarreauc5f73ed2005-12-18 01:26:38 +01002161 => This is a 'tcplog' entry. Client-side time-out ('c----') occured after 5s.
willy tarreau4302f492005-12-18 01:00:37 +01002162
willy tarreau532bb552006-05-13 18:40:37 +02002163- haproxy[18989]: 10.0.0.1:34552 [15/Oct/2003:15:26:31] relais-http Srv1 3183/-1/-1/-1/11215 503 0 - - SC-- 115/202/205 0/0 "HEAD / HTTP/1.0"
willy tarreauc5f73ed2005-12-18 01:26:38 +01002164 => The request took 3s to complete (probably a network problem), and the
2165 connection to the server failed ('SC--') after 4 attemps of 2 seconds
2166 (config says 'retries 3'), then a 503 error code was sent to the client.
willy tarreau532bb552006-05-13 18:40:37 +02002167 There were 115 connections on this server, 202 connections on this proxy,
2168 and 205 on the global process. It is possible that the server refused the
2169 connection because of too many already established.
willy tarreau4302f492005-12-18 01:00:37 +01002170
willy tarreau4302f492005-12-18 01:00:37 +01002171
willy tarreau197e8ec2005-12-17 14:10:59 +010021724.3) HTTP header manipulation
2173-----------------------------
2174In HTTP mode, it is possible to rewrite, add or delete some of the request and
2175response headers based on regular expressions. It is also possible to block a
2176request or a response if a particular header matches a regular expression,
2177which is enough to stops most elementary protocol attacks, and to protect
2178against information leak from the internal network. But there is a limitation
2179to this : since haproxy's HTTP engine knows nothing about keep-alive, only
2180headers passed during the first request of a TCP session will be seen. All
2181subsequent headers will be considered data only and not analyzed. Furthermore,
2182haproxy doesn't touch data contents, it stops at the end of headers.
willy tarreaueedaa9f2005-12-17 14:08:03 +01002183
willy tarreau197e8ec2005-12-17 14:10:59 +01002184The syntax is :
2185 reqadd <string> to add a header to the request
2186 reqrep <search> <replace> to modify the request
2187 reqirep <search> <replace> same, but ignoring the case
2188 reqdel <search> to delete a header in the request
2189 reqidel <search> same, but ignoring the case
2190 reqallow <search> definitely allow a request if a header matches <search>
2191 reqiallow <search> same, but ignoring the case
2192 reqdeny <search> denies a request if a header matches <search>
2193 reqideny <search> same, but ignoring the case
2194 reqpass <search> ignore a header matching <search>
2195 reqipass <search> same, but ignoring the case
Willy Tarreau2272dc12006-09-03 10:19:38 +02002196 reqtarpit <search> tarpit a request matching <search>
2197 reqitarpit <search> same, but ignoring the case
willy tarreaueedaa9f2005-12-17 14:08:03 +01002198
willy tarreau197e8ec2005-12-17 14:10:59 +01002199 rspadd <string> to add a header to the response
2200 rsprep <search> <replace> to modify the response
2201 rspirep <search> <replace> same, but ignoring the case
2202 rspdel <search> to delete the response
2203 rspidel <search> same, but ignoring the case
willy tarreau982249e2005-12-18 00:57:06 +01002204 rspdeny <search> replaces a response with a HTTP 502 if a header matches <search>
2205 rspideny <search> same, but ignoring the case
willy tarreaueedaa9f2005-12-17 14:08:03 +01002206
2207
willy tarreau197e8ec2005-12-17 14:10:59 +01002208<search> is a POSIX regular expression (regex) which supports grouping through
2209parenthesis (without the backslash). Spaces and other delimiters must be
2210prefixed with a backslash ('\') to avoid confusion with a field delimiter.
2211Other characters may be prefixed with a backslash to change their meaning :
willy tarreaueedaa9f2005-12-17 14:08:03 +01002212
willy tarreau197e8ec2005-12-17 14:10:59 +01002213 \t for a tab
2214 \r for a carriage return (CR)
2215 \n for a new line (LF)
2216 \ to mark a space and differentiate it from a delimiter
2217 \# to mark a sharp and differentiate it from a comment
2218 \\ to use a backslash in a regex
2219 \\\\ to use a backslash in the text (*2 for regex, *2 for haproxy)
2220 \xXX to write the ASCII hex code XX as in the C language
willy tarreaueedaa9f2005-12-17 14:08:03 +01002221
2222
Willy Tarreau2272dc12006-09-03 10:19:38 +02002223<replace> contains the string to be used to replace the largest portion of text
willy tarreau197e8ec2005-12-17 14:10:59 +01002224matching the regex. It can make use of the special characters above, and can
2225reference a substring delimited by parenthesis in the regex, by the group
Willy Tarreau2272dc12006-09-03 10:19:38 +02002226numerical order from 0 to 9 (0 being the entire line). In this case, you would
2227write a backslash ('\') immediately followed by one digit indicating the group
2228position.
willy tarreaueedaa9f2005-12-17 14:08:03 +01002229
willy tarreau197e8ec2005-12-17 14:10:59 +01002230<string> represents the string which will systematically be added after the last
2231header line. It can also use special characters above.
willy tarreaueedaa9f2005-12-17 14:08:03 +01002232
willy tarreau197e8ec2005-12-17 14:10:59 +01002233Notes :
2234-------
2235 - the first line is considered as a header, which makes it possible to rewrite
2236 or filter HTTP requests URIs or response codes.
2237 - 'reqrep' is the equivalent of 'cliexp' in version 1.0, and 'rsprep' is the
2238 equivalent of 'srvexp' in 1.0. Those names are still supported but
2239 deprecated.
2240 - for performances reasons, the number of characters added to a request or to
2241 a response is limited to 4096 since version 1.1.5 (it was 256 before). This
2242 value is easy to modify in the code if needed (#define). If it is too short
2243 on occasional uses, it is possible to gain some space by removing some
2244 useless headers before adding new ones.
willy tarreau982249e2005-12-18 00:57:06 +01002245 - a denied request will generate an "HTTP 403 forbidden" response, while a
2246 denied response will generate an "HTTP 502 Bad gateway" response.
Willy Tarreau2272dc12006-09-03 10:19:38 +02002247 - a tarpitted request will be held open on the client side for a duration
Willy Tarreau08fa2e32006-09-03 10:47:37 +02002248 defined in the contimeout parameter, or untill the client aborts. Nothing
2249 will be sent to any server. When the timeout is reached, the proxy will
2250 reply with a 500 server error response so that the attacker does not
2251 suspect it has been tarpitted. The logs may report the 500, but the
2252 termination flags will indicate 'PT' in this case.
Willy Tarreau2272dc12006-09-03 10:19:38 +02002253
willy tarreaueedaa9f2005-12-17 14:08:03 +01002254
willy tarreau197e8ec2005-12-17 14:10:59 +01002255Examples :
2256----------
willy tarreauc5f73ed2005-12-18 01:26:38 +01002257 ###### a few examples ######
willy tarreau197e8ec2005-12-17 14:10:59 +01002258
willy tarreauc5f73ed2005-12-18 01:26:38 +01002259 # rewrite 'online.fr' instead of 'free.fr' for GET and POST requests
2260 reqrep ^(GET\ .*)(.free.fr)(.*) \1.online.fr\3
2261 reqrep ^(POST\ .*)(.free.fr)(.*) \1.online.fr\3
willy tarreau197e8ec2005-12-17 14:10:59 +01002262
willy tarreauc5f73ed2005-12-18 01:26:38 +01002263 # force proxy connections to close
2264 reqirep ^Proxy-Connection:.* Proxy-Connection:\ close
2265 # rewrite locations
2266 rspirep ^(Location:\ )([^:]*://[^/]*)(.*) \1\3
willy tarreaueedaa9f2005-12-17 14:08:03 +01002267
willy tarreauc5f73ed2005-12-18 01:26:38 +01002268 ###### A full configuration being used on production ######
willy tarreaueedaa9f2005-12-17 14:08:03 +01002269
willy tarreau197e8ec2005-12-17 14:10:59 +01002270 # Every header should end with a colon followed by one space.
willy tarreauc5f73ed2005-12-18 01:26:38 +01002271 reqideny ^[^:\ ]*[\ ]*$
willy tarreaueedaa9f2005-12-17 14:08:03 +01002272
willy tarreau197e8ec2005-12-17 14:10:59 +01002273 # block Apache chunk exploit
willy tarreauc5f73ed2005-12-18 01:26:38 +01002274 reqideny ^Transfer-Encoding:[\ ]*chunked
2275 reqideny ^Host:\ apache-
willy tarreaueedaa9f2005-12-17 14:08:03 +01002276
willy tarreau197e8ec2005-12-17 14:10:59 +01002277 # block annoying worms that fill the logs...
willy tarreauc5f73ed2005-12-18 01:26:38 +01002278 reqideny ^[^:\ ]*\ .*(\.|%2e)(\.|%2e)(%2f|%5c|/|\\\\)
2279 reqideny ^[^:\ ]*\ ([^\ ]*\ [^\ ]*\ |.*%00)
2280 reqideny ^[^:\ ]*\ .*<script
2281 reqideny ^[^:\ ]*\ .*/(root\.exe\?|cmd\.exe\?|default\.ida\?)
willy tarreau197e8ec2005-12-17 14:10:59 +01002282
Willy Tarreau2272dc12006-09-03 10:19:38 +02002283 # tarpit attacks on the login page.
2284 reqtarpit ^[^:\ ]*\ .*\.php?login=[^0-9]
2285
willy tarreau197e8ec2005-12-17 14:10:59 +01002286 # allow other syntactically valid requests, and block any other method
willy tarreauc5f73ed2005-12-18 01:26:38 +01002287 reqipass ^(GET|POST|HEAD|OPTIONS)\ /.*\ HTTP/1\.[01]$
2288 reqipass ^OPTIONS\ \\*\ HTTP/1\.[01]$
2289 reqideny ^[^:\ ]*\
willy tarreau197e8ec2005-12-17 14:10:59 +01002290
2291 # force connection:close, thus disabling HTTP keep-alive
willy tarreauc5f73ed2005-12-18 01:26:38 +01002292 option httpclose
willy tarreau197e8ec2005-12-17 14:10:59 +01002293
willy tarreauc5f73ed2005-12-18 01:26:38 +01002294 # change the server name
2295 rspidel ^Server:\
2296 rspadd Server:\ Formilux/0.1.8
willy tarreau197e8ec2005-12-17 14:10:59 +01002297
2298
willy tarreau982249e2005-12-18 00:57:06 +01002299Also, the 'forwardfor' option creates an HTTP 'X-Forwarded-For' header which
willy tarreauc1cae632005-12-17 14:12:23 +01002300contains the client's IP address. This is useful to let the final web server
Willy Tarreau7ac51f62007-03-25 16:00:04 +02002301know what the client address was (eg for statistics on domains). Starting with
2302version 1.3.8, it is possible to specify the "except" keyword followed by a
2303source IP address or network for which no header will be added. This is very
2304useful when another reverse-proxy which already adds the header runs on the
2305same machine or in a known DMZ, the most common case being the local use of
2306stunnel on the same system.
willy tarreauc1cae632005-12-17 14:12:23 +01002307
willy tarreau982249e2005-12-18 00:57:06 +01002308Last, the 'httpclose' option removes any 'Connection' header both ways, and
2309adds a 'Connection: close' header in each direction. This makes it easier to
Willy TARREAU767ba712006-03-01 22:40:50 +01002310disable HTTP keep-alive than the previous 4-rules block.
willy tarreau982249e2005-12-18 00:57:06 +01002311
willy tarreauc1cae632005-12-17 14:12:23 +01002312Example :
2313---------
2314 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01002315 mode http
2316 log global
2317 option httplog
2318 option dontlognull
Willy Tarreau7ac51f62007-03-25 16:00:04 +02002319 option forwardfor except 127.0.0.1/8
willy tarreauc5f73ed2005-12-18 01:26:38 +01002320 option httpclose
2321
Willy TARREAU767ba712006-03-01 22:40:50 +01002322Note that some HTTP servers do not necessarily close the connections when they
2323receive the 'Connection: close', and if the client does not close either, then
2324the connection will be maintained up to the time-out. This translates into high
2325number of simultaneous sessions and high global session times in the logs. To
2326workaround this, a new option 'forceclose' appeared in version 1.2.9 to enforce
2327the closing of the outgoing server channel as soon as the server begins to
2328reply and only if the request buffer is empty. Note that this should NOT be
2329used if CONNECT requests are expected between the client and the server. The
2330'forceclose' option implies the 'httpclose' option.
2331
2332Example :
2333---------
2334 listen http_proxy 0.0.0.0:80
2335 mode http
2336 log global
2337 option httplog
2338 option dontlognull
2339 option forwardfor
2340 option forceclose
2341
willy tarreau197e8ec2005-12-17 14:10:59 +01002342
23434.4) Load balancing with persistence
2344------------------------------------
willy tarreau197e8ec2005-12-17 14:10:59 +01002345Combining cookie insertion with internal load balancing allows to transparently
2346bring persistence to applications. The principle is quite simple :
2347 - assign a cookie value to each server
2348 - enable the load balancing between servers
2349 - insert a cookie into responses resulting from the balancing algorithm
2350 (indirect accesses), end ensure that no upstream proxy will cache it.
2351 - remove the cookie in the request headers so that the application never sees
2352 it.
2353
2354Example :
willy tarreaueedaa9f2005-12-17 14:08:03 +01002355---------
2356 listen application 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01002357 mode http
2358 cookie SERVERID insert nocache indirect
2359 balance roundrobin
2360 server srv1 192.168.1.1:80 cookie server01 check
2361 server srv2 192.168.1.2:80 cookie server02 check
willy tarreaueedaa9f2005-12-17 14:08:03 +01002362
willy tarreau0174f312005-12-18 01:02:42 +01002363The other solution brought by versions 1.1.30 and 1.2.3 is to reuse a cookie
2364from the server, and prefix the server's name to it. In this case, don't forget
2365to force "httpclose" mode so that you can be assured that every subsequent
2366request will have its cookie fixed.
2367
2368 listen application 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01002369 mode http
2370 cookie JSESSIONID prefix
2371 balance roundrobin
2372 server srv1 192.168.1.1:80 cookie srv1 check
2373 server srv2 192.168.1.2:80 cookie srv2 check
2374 option httpclose
willy tarreau0174f312005-12-18 01:02:42 +01002375
2376
willy tarreau982249e2005-12-18 00:57:06 +010023774.5) Protection against information leak from the servers
2378---------------------------------------------------------
2379In versions 1.1.28/1.2.1, a new option 'checkcache' was created. It carefully
2380checks 'Cache-control', 'Pragma' and 'Set-cookie' headers in server response
2381to check if there's a risk of caching a cookie on a client-side proxy. When this
2382option is enabled, the only responses which can be delivered to the client are :
2383 - all those without 'Set-Cookie' header ;
2384 - all those with a return code other than 200, 203, 206, 300, 301, 410,
2385 provided that the server has not set a 'Cache-control: public' header ;
2386 - all those that come from a POST request, provided that the server has not
2387 set a 'Cache-Control: public' header ;
2388 - those with a 'Pragma: no-cache' header
2389 - those with a 'Cache-control: private' header
2390 - those with a 'Cache-control: no-store' header
2391 - those with a 'Cache-control: max-age=0' header
2392 - those with a 'Cache-control: s-maxage=0' header
2393 - those with a 'Cache-control: no-cache' header
2394 - those with a 'Cache-control: no-cache="set-cookie"' header
2395 - those with a 'Cache-control: no-cache="set-cookie,' header
2396 (allowing other fields after set-cookie)
willy tarreaueedaa9f2005-12-17 14:08:03 +01002397
willy tarreau982249e2005-12-18 00:57:06 +01002398If a response doesn't respect these requirements, then it will be blocked just
2399as if it was from an 'rspdeny' filter, with an "HTTP 502 bad gateway". The
2400session state shows "PH--" meaning that the proxy blocked the response during
2401headers processing. Additionnaly, an alert will be sent in the logs so that
2402admins are told that there's something to be done.
2403
willy tarreauc5f73ed2005-12-18 01:26:38 +01002404
willy tarreau982249e2005-12-18 00:57:06 +010024054.6) Customizing errors
2406-----------------------
willy tarreau197e8ec2005-12-17 14:10:59 +01002407Some situations can make haproxy return an HTTP error code to the client :
2408 - invalid or too long request => HTTP 400
2409 - request not completely sent in time => HTTP 408
2410 - forbidden request (matches a deny filter) => HTTP 403
2411 - internal error in haproxy => HTTP 500
2412 - the server returned an invalid or incomplete response => HTTP 502
2413 - no server was available to handle the request => HTTP 503
2414 - the server failed to reply in time => HTTP 504
willy tarreaueedaa9f2005-12-17 14:08:03 +01002415
willy tarreau197e8ec2005-12-17 14:10:59 +01002416A succint error message taken from the RFC accompanies these return codes.
2417But depending on the clients knowledge, it may be better to return custom, user
Willy Tarreau3f49b302007-06-11 00:29:26 +02002418friendly, error pages. This is made possible in two ways, one involving a
2419redirection to a known server, and another one consisting in returning a local
2420file.
2421
24224.6.1) Relocation
2423-----------------
2424An error relocation is achieved using the 'errorloc' command :
willy tarreaueedaa9f2005-12-17 14:08:03 +01002425
willy tarreau197e8ec2005-12-17 14:10:59 +01002426 errorloc <HTTP_code> <location>
willy tarreaueedaa9f2005-12-17 14:08:03 +01002427
willy tarreau197e8ec2005-12-17 14:10:59 +01002428Instead of generating an HTTP error <HTTP_code> among those above, the proxy
2429will return a temporary redirection code (HTTP 302) towards the address
2430specified in <location>. This address may be either relative to the site or
2431absolute. Since this request will be handled by the client's browser, it's
2432mandatory that the returned address be reachable from the outside.
willy tarreaueedaa9f2005-12-17 14:08:03 +01002433
willy tarreau197e8ec2005-12-17 14:10:59 +01002434Example :
willy tarreaueedaa9f2005-12-17 14:08:03 +01002435---------
2436 listen application 0.0.0.0:80
2437 errorloc 400 /badrequest.html
2438 errorloc 403 /forbidden.html
2439 errorloc 408 /toolong.html
willy tarreauc5f73ed2005-12-18 01:26:38 +01002440 errorloc 500 http://haproxy.domain.net/bugreport.html
willy tarreaueedaa9f2005-12-17 14:08:03 +01002441 errorloc 502 http://192.168.114.58/error50x.html
2442 errorloc 503 http://192.168.114.58/error50x.html
2443 errorloc 504 http://192.168.114.58/error50x.html
2444
willy tarreauc1f47532005-12-18 01:08:26 +01002445Note: RFC2616 says that a client must reuse the same method to fetch the
2446Location returned by a 302, which causes problems with the POST method.
2447The return code 303 was designed explicitly to force the client to fetch the
2448Location URL with the GET method, but there are some browsers pre-dating
2449HTTP/1.1 which don't support it. Anyway, most browsers still behave with 302 as
willy tarreauc5f73ed2005-12-18 01:26:38 +01002450if it was a 303. In order to allow the user to chose, versions 1.1.31 and 1.2.5
2451bring two new keywords to replace 'errorloc' : 'errorloc302' and 'errorloc303'.
willy tarreauc1f47532005-12-18 01:08:26 +01002452
2453They are preffered over errorloc (which still does 302). Consider using
2454errorloc303 everytime you know that your clients support HTTP 303 responses..
2455
Willy Tarreau3f49b302007-06-11 00:29:26 +020024564.6.2) Local files
2457------------------
2458Sometimes, it is desirable to change the returned error without resorting to
2459redirections. The second method consists in loading local files during startup
2460and send them as pure HTTP content upon error. This is what the 'errorfile'
2461keyword does.
2462
2463Warning, there are traps to consider :
2464 - The files are loaded while parsing configuration, before doing a chroot().
2465 Thus, they are relative to the real filesystem. For this reason, it is
2466 recommended to pass an absolute path to those files.
2467
2468 - The contents of those files is not HTML, but real HTTP protocol with
2469 possible HTML body. So the first line and headers are mandatory. Ideally,
2470 every line in the HTTP part should end with CR-LF for maximum compatibility.
2471
2472 - The response is limited to the buffer size (BUSIZE), generally 8 or 16 kB.
2473
2474 - The response should not include references to the local server, in order to
2475 avoid infinite loops on the browser in case of local failure.
2476
2477Example :
2478---------
2479 errorfile 400 /etc/haproxy/errorfiles/400badreq.http
2480 errorfile 403 /etc/haproxy/errorfiles/403forbid.http
2481 errorfile 503 /etc/haproxy/errorfiles/503sorry.http
2482
willy tarreauc1f47532005-12-18 01:08:26 +01002483
willy tarreau982249e2005-12-18 00:57:06 +010024844.7) Modifying default values
willy tarreau197e8ec2005-12-17 14:10:59 +01002485-----------------------------
willy tarreau197e8ec2005-12-17 14:10:59 +01002486Version 1.1.22 introduced the notion of default values, which eliminates the
2487pain of often repeating common parameters between many instances, such as
2488logs, timeouts, modes, etc...
willy tarreaueedaa9f2005-12-17 14:08:03 +01002489
willy tarreau197e8ec2005-12-17 14:10:59 +01002490Default values are set in a 'defaults' section. Each of these section clears
2491all previously set default parameters, so there may be as many default
2492parameters as needed. Only the last one before a 'listen' section will be
2493used for this section. The 'defaults' section uses the same syntax as the
2494'listen' section, for the supported parameters. The 'defaults' keyword ignores
2495everything on its command line, so that fake instance names can be specified
2496there for better clarity.
willy tarreaueedaa9f2005-12-17 14:08:03 +01002497
willy tarreau982249e2005-12-18 00:57:06 +01002498In version 1.1.28/1.2.1, only those parameters can be preset in the 'default'
willy tarreau197e8ec2005-12-17 14:10:59 +01002499section :
2500 - log (the first and second one)
willy tarreaueedaa9f2005-12-17 14:08:03 +01002501 - mode { tcp, http, health }
2502 - balance { roundrobin }
willy tarreau197e8ec2005-12-17 14:10:59 +01002503 - disabled (to disable every further instances)
2504 - enabled (to enable every further instances, this is the default)
willy tarreaueedaa9f2005-12-17 14:08:03 +01002505 - contimeout, clitimeout, srvtimeout, grace, retries, maxconn
willy tarreau982249e2005-12-18 00:57:06 +01002506 - option { redispatch, transparent, keepalive, forwardfor, logasap, httpclose,
2507 checkcache, httplog, tcplog, dontlognull, persist, httpchk }
willy tarreaueedaa9f2005-12-17 14:08:03 +01002508 - redispatch, redisp, transparent, source { addr:port }
2509 - cookie, capture
2510 - errorloc
2511
willy tarreau197e8ec2005-12-17 14:10:59 +01002512As of 1.1.24, it is not possible to put certain parameters in a 'defaults'
2513section, mainly regular expressions and server configurations :
willy tarreaueedaa9f2005-12-17 14:08:03 +01002514 - dispatch, server,
willy tarreau197e8ec2005-12-17 14:10:59 +01002515 - req*, rsp*
willy tarreaueedaa9f2005-12-17 14:08:03 +01002516
willy tarreau197e8ec2005-12-17 14:10:59 +01002517Last, there's no way yet to change a boolean option from its assigned default
2518value. So if an 'option' statement is set in a 'defaults' section, the only
2519way to flush it is to redefine a new 'defaults' section without this 'option'.
willy tarreaueedaa9f2005-12-17 14:08:03 +01002520
willy tarreau197e8ec2005-12-17 14:10:59 +01002521Examples :
willy tarreaueedaa9f2005-12-17 14:08:03 +01002522----------
2523 defaults applications TCP
willy tarreauc5f73ed2005-12-18 01:26:38 +01002524 log global
2525 mode tcp
2526 balance roundrobin
2527 clitimeout 180000
2528 srvtimeout 180000
2529 contimeout 4000
2530 retries 3
2531 redispatch
willy tarreaueedaa9f2005-12-17 14:08:03 +01002532
2533 listen app_tcp1 10.0.0.1:6000-6063
willy tarreauc5f73ed2005-12-18 01:26:38 +01002534 server srv1 192.168.1.1 check port 6000 inter 10000
2535 server srv2 192.168.1.2 backup
willy tarreaueedaa9f2005-12-17 14:08:03 +01002536
2537 listen app_tcp2 10.0.0.2:6000-6063
willy tarreauc5f73ed2005-12-18 01:26:38 +01002538 server srv1 192.168.2.1 check port 6000 inter 10000
2539 server srv2 192.168.2.2 backup
willy tarreaueedaa9f2005-12-17 14:08:03 +01002540
2541 defaults applications HTTP
willy tarreauc5f73ed2005-12-18 01:26:38 +01002542 log global
2543 mode http
2544 option httplog
2545 option forwardfor
2546 option dontlognull
2547 balance roundrobin
2548 clitimeout 20000
2549 srvtimeout 20000
2550 contimeout 4000
2551 retries 3
willy tarreaueedaa9f2005-12-17 14:08:03 +01002552
2553 listen app_http1 10.0.0.1:80-81
willy tarreauc5f73ed2005-12-18 01:26:38 +01002554 cookie SERVERID postonly insert indirect
2555 capture cookie userid= len 10
2556 server srv1 192.168.1.1:+8000 cookie srv1 check port 8080 inter 1000
2557 server srv1 192.168.1.2:+8000 cookie srv2 check port 8080 inter 1000
willy tarreaueedaa9f2005-12-17 14:08:03 +01002558
2559 defaults
willy tarreauc5f73ed2005-12-18 01:26:38 +01002560 # this empty section voids all default parameters
willy tarreaueedaa9f2005-12-17 14:08:03 +01002561
willy tarreau481132e2006-05-21 21:43:10 +02002562
25634.8) Status report in HTML page
2564-------------------------------
2565Starting with 1.2.14, it is possible for HAProxy to intercept requests for a
2566particular URI and return a full report of the proxy's activity and servers
2567statistics. This is available through the 'stats' keyword, associated to any
2568such options :
2569
2570 - stats enable
2571 - stats uri <uri prefix>
2572 - stats realm <authentication realm>
2573 - stats auth <user:password>
2574 - stats scope <proxy_id> | '.'
2575
2576By default, the status report is disabled. Specifying any combination above
2577enables it for the proxy instance referencing it. The easiest solution is to
2578use "stats enable" which will enable the report with default parameters :
2579
2580 - default URI : "/haproxy?stats" (CONFIG_STATS_DEFAULT_URI)
2581 - default auth : unspecified (no authentication)
2582 - default realm : "HAProxy Statistics" (CONFIG_STATS_DEFAULT_REALM)
2583 - default scope : unspecified (access to all instances)
2584
2585The "stats uri <uri_prefix>" option allows one to intercept another URI prefix.
2586Note that any URI that BEGINS with this string will match. For instance, one
2587proxy instance might be dedicated to status page only and would reply to any
2588URI.
2589
2590Example :
2591---------
2592 # catches any URI and returns the status page.
2593 listen stats :8080
2594 mode http
2595 stats uri /
2596
2597The "stats auth <user:password>" option enables Basic authentication and adds a
2598valid user:password combination to the list of authorized accounts. The user
2599and password are passed in the configuration file as clear text, and since this
2600is HTTP Basic authentication, you should be aware that it transits as clear
2601text on the network, so you must not use any sensible account. The list is
2602unlimited in order to provide easy accesses to developpers or customers.
2603
2604The "stats realm <realm>" option defines the "realm" name which is displayed
2605in the popup box when the browser asks for a password. It's important to ensure
2606that this one is not used by the application, otherwise the browser will try to
2607use a cached one from the application. Note that any space in the realm name
2608should be escaped with a backslash ('\').
2609
2610The "stats scope <proxy_id>" option limits the scope of the status report. By
2611default, all proxy instances are listed. But under some circumstances, it would
2612be better to limit the listing to some proxies or only to the current one. This
2613is what this option does. The special proxy name "." (a single dot) references
2614the current proxy. The proxy name can be repeated multiple times, even for
2615proxies defined later in the configuration or some which do not exist. The name
2616is the one which appears after the 'listen' keyword.
2617
2618Example :
2619---------
2620 # simple application with authenticated embedded status report
2621 listen app1 192.168.1.100:80
2622 mode http
willy tarreaud4ba08d2006-05-21 21:54:14 +02002623 option httpclose
willy tarreau481132e2006-05-21 21:43:10 +02002624 balance roundrobin
2625 cookie SERVERID postonly insert indirect
2626 server srv1 192.168.1.1:8080 cookie srv1 check inter 1000
2627 server srv1 192.168.1.2:8080 cookie srv2 check inter 1000
2628 stats uri /my_stats
willy tarreaud4ba08d2006-05-21 21:54:14 +02002629 stats realm Statistics\ for\ MyApp1-2
2630 stats auth guest:guest
2631 stats auth admin:AdMiN123
2632 stats scope .
2633 stats scope app2
willy tarreau481132e2006-05-21 21:43:10 +02002634
2635 # simple application with anonymous embedded status report
2636 listen app2 192.168.2.100:80
2637 mode http
willy tarreaud4ba08d2006-05-21 21:54:14 +02002638 option httpclose
willy tarreau481132e2006-05-21 21:43:10 +02002639 balance roundrobin
2640 cookie SERVERID postonly insert indirect
2641 server srv1 192.168.2.1:8080 cookie srv1 check inter 1000
2642 server srv1 192.168.2.2:8080 cookie srv2 check inter 1000
2643 stats uri /my_stats
willy tarreaud4ba08d2006-05-21 21:54:14 +02002644 stats realm Statistics\ for\ MyApp2
2645 stats scope .
willy tarreau481132e2006-05-21 21:43:10 +02002646
2647 listen admin_page :8080
2648 mode http
2649 stats uri /my_stats
willy tarreaud4ba08d2006-05-21 21:54:14 +02002650 stats realm Global\ statistics
2651 stats auth admin:AdMiN123
willy tarreau481132e2006-05-21 21:43:10 +02002652
2653Notes :
2654-------
2655 - The 'stats' options can also be specified in the 'defaults' section, in
2656 which case it will provide the exact same configuration to all further
2657 instances (hence the usefulness of the scope "."). However, if an instance
2658 redefines any 'stats' parameter, defaults will not be used for this
2659 instance.
2660
2661 - HTTP Basic authentication is very basic and unsecure from snooping. No
2662 sensible password should be used, and be aware that there is no way to
2663 remove it from the browser so it will be sent to the whole application
2664 upon further accesses.
2665
willy tarreaud4ba08d2006-05-21 21:54:14 +02002666 - It is very important that the 'option httpclose' is specified, otherwise
2667 the proxy will not be able to detect the URI within keep-alive sessions
2668 maintained between the browser and the servers, so the stats URI will be
2669 forwarded unmodified to the server as if the option was not set.
2670
willy tarreau481132e2006-05-21 21:43:10 +02002671
Willy Tarreau726c2bf2007-05-09 01:31:45 +020026725) Access lists
2673===============
2674
2675With version 1.3.10, a new concept of access lists (acl) was born. As it was
2676not necesary to reinvent the wheel, and because even long thoughts lead to
2677unsatisfying proposals, it was finally decided that something close to what
2678Squid provides would be a good compromise between features and ease of use.
2679
2680The principle is very simple : acls are declared with a name, a test and a list
2681of valid values to check against during the test. Conditions are applied on
2682various actions, and those conditions apply a logical AND between acls. The
2683condition is then only met if all acls are true.
2684
2685It is possible to use the reserved keyword "OR" in conditions, and it is
2686possible for an acl to be specified multiple times, even with various tests, in
2687which case the first one which returns true validates the ACL.
2688
Willy Tarreauae8b7962007-06-09 23:10:04 +02002689As of 1.3.12, only the following tests have been implemented :
Willy Tarreau726c2bf2007-05-09 01:31:45 +02002690
2691 Layer 3/4 :
2692 src <ipv4_address>[/mask] ... : match IPv4 source address
2693 dst <ipv4_address>[/mask] ... : match IPv4 destination address
Willy Tarreauae8b7962007-06-09 23:10:04 +02002694 src_port <range> ... : match source port range
2695 dst_port <range> ... : match destination port range
2696 dst_conn <range> ... : match #connections on frontend
Willy Tarreau726c2bf2007-05-09 01:31:45 +02002697
2698 Layer 7 :
2699 method <HTTP method> ... : match HTTP method
2700 req_ver <1.0|1.1> ... : match HTTP request version
2701 resp_ver <1.0|1.1> ... : match HTTP response version
Willy Tarreauae8b7962007-06-09 23:10:04 +02002702 status <range> ... : match HTTP response status code in range
Willy Tarreau726c2bf2007-05-09 01:31:45 +02002703 url <string> ... : exact string match on URI
2704 url_reg <regex> ... : regex string match on URI
2705 url_beg <string> ... : true if URI begins with <string>
2706 url_end <string> ... : true if URI ends with <string>
2707 url_sub <string> ... : true if URI contains <string>
2708 url_dir <string> ... : true if URI contains <string> between slashes
2709 url_dom <string> ... : true if URI contains <string> between slashes or dots
2710
Willy Tarreauae8b7962007-06-09 23:10:04 +02002711A 'range' is one or two integers which may be prefixed by an operator.
2712The syntax is :
2713
2714 [<op>] <low>[:<high>]
2715
2716Where <op> can be :
2717 'eq' : the tested value must be equal to <low> or within <low>..<high>
2718 'le' : the tested value must be lower than or equal to <low>
2719 'lt' : the tested value must be lower than <low>
2720 'ge' : the tested value must be greater than or equal to <low>
2721 'gt' : the tested value must be greater than <low>
2722
2723When no operator is defined, 'eq' is assumed. Note that when the operator is
2724specified, it applies to all subsequent ranges of values until the end of the
2725line is reached or another operator is specified. Example :
2726
2727 acl status_error status 400:599
2728 acl saturated_frt dst_conn ge 1000
2729 acl invalid_ports src_port lt 512 ge 65535
2730
Willy Tarreau726c2bf2007-05-09 01:31:45 +02002731Other ones are coming (headers, cookies, time, auth), it's just a matter of
2732time. It is also planned to be able to read the patterns from a file, as well
2733as to ignore the case for some of them.
2734
2735The only command supporting a condition right now is the "block" command, which
2736blocks a request and returns a 403 if its condition is true (with the "if"
2737keyword), or if it is false (with the "unless" keyword).
2738
2739Example :
2740---------
2741
2742 acl options_uris url *
2743 acl meth_option method OPTIONS
2744 acl http_1.1 req_ver 1.1
2745 acl allowed_meth method GET HEAD POST OPTIONS CONNECT
2746 acl connect_meth method CONNECT
2747 acl proxy_url url_beg http://
2748
2749 # block if reserved URI "*" used with a method other than "OPTIONS"
2750 block if options_uris !meth_option
2751
2752 # block if the OPTIONS method is used with HTTP 1.0
2753 block if meth_option !http_1.1
2754
2755 # allow non-proxy url with anything but the CONNECT method
2756 block if !connect_meth !proxy_url
2757
2758 # block all unknown methods
2759 block unless allowed_meth
2760
2761Note: this documentation is very light but should permit one to start and above
2762all it should permit to work on the project without being slowed down too much
2763with the doc.
2764
2765
willy tarreau197e8ec2005-12-17 14:10:59 +01002766=========================
2767| System-specific setup |
2768=========================
willy tarreaueedaa9f2005-12-17 14:08:03 +01002769
willy tarreau197e8ec2005-12-17 14:10:59 +01002770Linux 2.4
2771=========
willy tarreaueedaa9f2005-12-17 14:08:03 +01002772
2773-- cut here --
2774#!/bin/sh
2775# set this to about 256/4M (16384 for 256M machine)
2776MAXFILES=16384
2777echo $MAXFILES > /proc/sys/fs/file-max
2778ulimit -n $MAXFILES
2779
2780if [ -e /proc/sys/net/ipv4/ip_conntrack_max ]; then
willy tarreauc5f73ed2005-12-18 01:26:38 +01002781 echo 65536 > /proc/sys/net/ipv4/ip_conntrack_max
willy tarreaueedaa9f2005-12-17 14:08:03 +01002782fi
2783
2784if [ -e /proc/sys/net/ipv4/netfilter/ip_ct_tcp_timeout_fin_wait ]; then
willy tarreauc5f73ed2005-12-18 01:26:38 +01002785 # 30 seconds for fin, 15 for time wait
2786 echo 3000 > /proc/sys/net/ipv4/netfilter/ip_ct_tcp_timeout_fin_wait
2787 echo 1500 > /proc/sys/net/ipv4/netfilter/ip_ct_tcp_timeout_time_wait
2788 echo 0 > /proc/sys/net/ipv4/netfilter/ip_ct_tcp_log_invalid_scale
2789 echo 0 > /proc/sys/net/ipv4/netfilter/ip_ct_tcp_log_out_of_window
willy tarreaueedaa9f2005-12-17 14:08:03 +01002790fi
2791
2792echo 1024 60999 > /proc/sys/net/ipv4/ip_local_port_range
2793echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout
2794echo 4096 > /proc/sys/net/ipv4/tcp_max_syn_backlog
2795echo 262144 > /proc/sys/net/ipv4/tcp_max_tw_buckets
2796echo 262144 > /proc/sys/net/ipv4/tcp_max_orphans
2797echo 300 > /proc/sys/net/ipv4/tcp_keepalive_time
2798echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle
2799echo 0 > /proc/sys/net/ipv4/tcp_timestamps
2800echo 0 > /proc/sys/net/ipv4/tcp_ecn
willy tarreauc5f73ed2005-12-18 01:26:38 +01002801echo 1 > /proc/sys/net/ipv4/tcp_sack
willy tarreaueedaa9f2005-12-17 14:08:03 +01002802echo 0 > /proc/sys/net/ipv4/tcp_dsack
2803
2804# auto-tuned on 2.4
2805#echo 262143 > /proc/sys/net/core/rmem_max
2806#echo 262143 > /proc/sys/net/core/rmem_default
2807
2808echo 16384 65536 524288 > /proc/sys/net/ipv4/tcp_rmem
2809echo 16384 349520 699040 > /proc/sys/net/ipv4/tcp_wmem
2810
2811-- cut here --
2812
willy tarreau197e8ec2005-12-17 14:10:59 +01002813
2814FreeBSD
2815=======
2816
2817A FreeBSD port of HA-Proxy is now available and maintained, thanks to
2818Clement Laforet <sheepkiller@cultdeadsheep.org>.
2819
2820For more information :
2821http://www.freebsd.org/cgi/url.cgi?ports/net/haproxy/pkg-descr
2822http://www.freebsd.org/cgi/cvsweb.cgi/ports/net/haproxy/
2823http://www.freshports.org/net/haproxy
2824
2825
2826-- end --