blob: de40f10fb8ca77f7445f074d4144795922f90331 [file] [log] [blame]
willy tarreau0174f312005-12-18 01:02:42 +01001 -------------------
Willy Tarreau94b45912006-05-31 06:40:15 +02002 HAProxy
willy tarreau0174f312005-12-18 01:02:42 +01003 Reference Manual
4 -------------------
Willy Tarreau7b4c5ae2008-04-19 21:06:14 +02005 version 1.3.15
willy tarreauc5f73ed2005-12-18 01:26:38 +01006 willy tarreau
Willy Tarreau7b4c5ae2008-04-19 21:06:14 +02007 2008/04/19
willy tarreaueedaa9f2005-12-17 14:08:03 +01008
9============
10| Abstract |
11============
12
Willy Tarreau94b45912006-05-31 06:40:15 +020013HAProxy is a TCP/HTTP reverse proxy which is particularly suited for high
willy tarreaueedaa9f2005-12-17 14:08:03 +010014availability environments. Indeed, it can :
15 - route HTTP requests depending on statically assigned cookies ;
16 - spread the load among several servers while assuring server persistence
17 through the use of HTTP cookies ;
18 - switch to backup servers in the event a main one fails ;
19 - accept connections to special ports dedicated to service monitoring ;
20 - stop accepting connections without breaking existing ones ;
21 - add/modify/delete HTTP headers both ways ;
22 - block requests matching a particular pattern ;
willy tarreau532bb552006-05-13 18:40:37 +020023 - hold clients to the right application server depending on application
24 cookies
willy tarreau481132e2006-05-21 21:43:10 +020025 - report detailed status as HTML pages to authenticated users from an URI
26 intercepted from the application.
willy tarreaueedaa9f2005-12-17 14:08:03 +010027
28It needs very little resource. Its event-driven architecture allows it to easily
29handle thousands of simultaneous connections on hundreds of instances without
30risking the system's stability.
31
32====================
33| Start parameters |
34====================
35
36There are only a few command line options :
37
38 -f <configuration file>
39 -n <high limit for the total number of simultaneous connections>
willy tarreau532bb552006-05-13 18:40:37 +020040 = 'maxconn' in 'global' section
41 -N <high limit for the per-listener number of simultaneous connections>
42 = 'maxconn' in 'listen' or 'default' sections
willy tarreaueedaa9f2005-12-17 14:08:03 +010043 -d starts in foregreound with debugging mode enabled
44 -D starts in daemon mode
willy tarreau982249e2005-12-18 00:57:06 +010045 -q disable messages on output
46 -V displays messages on output even when -q or 'quiet' are specified.
47 -c only checks config file and exits with code 0 if no error was found, or
48 exits with code 1 if a syntax error was found.
willy tarreaufe2c5c12005-12-17 14:14:34 +010049 -p <pidfile> asks the process to write down each of its children's
50 pids to this file in daemon mode.
willy tarreau34f45302006-04-15 21:37:14 +020051 -sf specifies a list of pids to send a FINISH signal to after startup.
52 -st specifies a list of pids to send a TERMINATE signal to after startup.
willy tarreaueedaa9f2005-12-17 14:08:03 +010053 -s shows statistics (only if compiled in)
54 -l shows even more statistics (implies '-s')
Willy Tarreaude99e992007-04-16 00:53:59 +020055 -dk disables use of kqueue()
56 -ds disables use of speculative epoll()
willy tarreau64a3cc32005-12-18 01:13:11 +010057 -de disables use of epoll()
58 -dp disables use of poll()
willy tarreau34f45302006-04-15 21:37:14 +020059 -db disables background mode (stays in foreground, useful for debugging)
60 -m <megs> enforces a memory usage limit to a maximum of <megs> megabytes.
willy tarreaueedaa9f2005-12-17 14:08:03 +010061
willy tarreau532bb552006-05-13 18:40:37 +020062The maximal number of connections per proxy instance is used as the default
63parameter for each instance for which the 'maxconn' paramter is not set in the
64'listen' section.
willy tarreaueedaa9f2005-12-17 14:08:03 +010065
66The maximal number of total connections limits the number of connections used by
67the whole process if the 'maxconn' parameter is not set in the 'global' section.
68
69The debugging mode has the same effect as the 'debug' option in the 'global'
70section. When the proxy runs in this mode, it dumps every connections,
71disconnections, timestamps, and HTTP headers to stdout. This should NEVER
72be used in an init script since it will prevent the system from starting up.
73
willy tarreau34f45302006-04-15 21:37:14 +020074For debugging, the '-db' option is very useful as it temporarily disables
75daemon mode and multi-process mode. The service can then be stopped by simply
76pressing Ctrl-C, without having to edit the config nor run full debug.
77
willy tarreaueedaa9f2005-12-17 14:08:03 +010078Statistics are only available if compiled in with the 'STATTIME' option. It's
willy tarreau481132e2006-05-21 21:43:10 +020079only used during code optimization phases, and will soon disappear.
willy tarreaueedaa9f2005-12-17 14:08:03 +010080
willy tarreau532bb552006-05-13 18:40:37 +020081The '-st' and '-sf' options are used for hot reconfiguration (see below).
willy tarreau34f45302006-04-15 21:37:14 +020082
willy tarreaueedaa9f2005-12-17 14:08:03 +010083======================
84| Configuration file |
85======================
86
87Structure
88=========
89
90The configuration file parser ignores empty lines, spaces, tabs. Anything
91between a sharp ('#') not following a backslash ('\'), and the end of a line
92constitutes a comment and is ignored too.
93
94The configuration file is segmented in sections. A section begins whenever
95one of these 3 keywords are encountered :
96
97 - 'global'
98 - 'listen'
99 - 'defaults'
100
101Every parameter refer to the section beginning at the last one of these 3
102keywords.
103
104
1051) Global parameters
106====================
107
108Global parameters affect the whole process behaviour. They are all set in the
109'global' section. There may be several 'global' sections if needed, but their
110parameters will only be merged. Allowed parameters in 'global' section include
111the following ones :
112
113 - log <address> <facility> [max_level]
114 - maxconn <number>
115 - uid <user id>
116 - gid <group id>
Willy Tarreau95c20ac2007-03-25 15:39:23 +0200117 - user <user name>
118 - group <group name>
willy tarreaueedaa9f2005-12-17 14:08:03 +0100119 - chroot <directory>
120 - nbproc <number>
121 - daemon
122 - debug
Willy Tarreaude99e992007-04-16 00:53:59 +0200123 - nokqueue
124 - nosepoll
willy tarreau64a3cc32005-12-18 01:13:11 +0100125 - noepoll
126 - nopoll
willy tarreaueedaa9f2005-12-17 14:08:03 +0100127 - quiet
willy tarreaufe2c5c12005-12-17 14:14:34 +0100128 - pidfile <file>
willy tarreauc5f73ed2005-12-18 01:26:38 +0100129 - ulimit-n <number>
willy tarreau598da412005-12-18 01:07:29 +0100130 - stats
Willy Tarreau1db37712007-06-03 17:16:49 +0200131 - tune.maxpollevents <number>
willy tarreaueedaa9f2005-12-17 14:08:03 +0100132
willy tarreauc5f73ed2005-12-18 01:26:38 +0100133
willy tarreaueedaa9f2005-12-17 14:08:03 +01001341.1) Event logging
135------------------
136Most events are logged : start, stop, servers going up and down, connections and
137errors. Each event generates a syslog message which can be sent to up to 2
138servers. The syntax is :
139
140 log <ip_address> <facility> [max_level]
141
142Connections are logged at level "info". Services initialization and servers
143going up are logged at level "notice", termination signals are logged at
144"warning", and definitive service termination, as well as loss of servers are
145logged at level "alert". The optional parameter <max_level> specifies above
146what level messages should be sent. Level can take one of these 8 values :
147
148 emerg, alert, crit, err, warning, notice, info, debug
149
150For backwards compatibility with versions 1.1.16 and earlier, the default level
151value is "debug" if not specified.
152
153Permitted facilities are :
154 kern, user, mail, daemon, auth, syslog, lpr, news,
155 uucp, cron, auth2, ftp, ntp, audit, alert, cron2,
156 local0, local1, local2, local3, local4, local5, local6, local7
157
158According to RFC3164, messages are truncated to 1024 bytes before being emitted.
159
160Example :
161---------
162 global
willy tarreauc5f73ed2005-12-18 01:26:38 +0100163 log 192.168.2.200 local3
164 log 127.0.0.1 local4 notice
165
willy tarreaueedaa9f2005-12-17 14:08:03 +0100166
1671.2) limiting the number of connections
168---------------------------------------
169It is possible and recommended to limit the global number of per-process
willy tarreauc5f73ed2005-12-18 01:26:38 +0100170connections using the 'maxconn' global keyword. Since one connection includes
171both a client and a server, it means that the max number of TCP sessions will
172be about the double of this number. It's important to understand this when
173trying to find best values for 'ulimit -n' before starting the proxy. To
174anticipate the number of sockets needed, all these parameters must be counted :
willy tarreaueedaa9f2005-12-17 14:08:03 +0100175
176 - 1 socket per incoming connection
177 - 1 socket per outgoing connection
178 - 1 socket per address/port/proxy tuple.
179 - 1 socket per server being health-checked
180 - 1 socket for all logs
181
182In simple configurations where each proxy only listens one one address/port,
willy tarreauc5f73ed2005-12-18 01:26:38 +0100183set the limit of file descriptors (ulimit -n) to
184(2 * maxconn + nbproxies + nbservers + 1). Starting with versions 1.1.32/1.2.6,
185it is now possible to set the limit in the configuration using the 'ulimit-n'
186global keyword, provided the proxy is started as root. This puts an end to the
187recurrent problem of ensuring that the system limits are adapted to the proxy
188values. Note that these limits are per-process.
189
190Example :
191---------
192 global
193 maxconn 32000
194 ulimit-n 65536
195
willy tarreaueedaa9f2005-12-17 14:08:03 +0100196
1971.3) Drop of priviledges
198------------------------
199In order to reduce the risk and consequences of attacks, in the event where a
200yet non-identified vulnerability would be successfully exploited, it's possible
201to lower the process priviledges and even isolate it in a riskless directory.
202
203In the 'global' section, the 'uid' parameter sets a numerical user identifier
204which the process will switch to after binding its listening sockets. The value
205'0', which normally represents the super-user, here indicates that the UID must
206not change during startup. It's the default behaviour. The 'gid' parameter does
Willy Tarreau95c20ac2007-03-25 15:39:23 +0200207the same for the group identifier. If setting an uid is not possible because of
208deployment constraints, it is possible to set a user name with the 'user'
209keyword followed by a valid user name. The same is true for the gid. It is
210possible to specify a group name after the 'group' keyword.
211
212It is particularly advised against use of generic accounts such as 'nobody'
213because it has the same consequences as using 'root' if other services use
214them.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100215
216The 'chroot' parameter makes the process isolate itself in an empty directory
217just before switching its UID. This type of isolation (chroot) can sometimes
218be worked around on certain OS (Linux, Solaris), provided that the attacker
219has gained 'root' priviledges and has the ability to use or create a directory.
220For this reason, it's capital to use a dedicated directory and not to share one
221between several services of different nature. To make isolation more resistant,
222it's recommended to use an empty directory without any right, and to change the
223UID of the process so that it cannot do anything there.
224
225Note: in the event where such a vulnerability would be exploited, it's most
226likely that first attempts would kill the process due to 'Segmentation Fault',
227'Bus Error' or 'Illegal Instruction' signals. Eventhough it's true that
228isolating the server reduces the risks of intrusion, it's sometimes useful to
229find why a process dies, via the analysis of a 'core' file, although very rare
230(the last bug of this sort was fixed in 1.1.9). For security reasons, most
231systems disable the generation of core file when a process changes its UID. So
232the two workarounds are either to start the process from a restricted user
233account, which will not be able to chroot itself, or start it as root and not
234change the UID. In both cases the core will be either in the start or the chroot
235directories. Do not forget to allow core dumps prior to start the process :
236
237# ulimit -c unlimited
238
239Example :
240---------
241
Willy Tarreau95c20ac2007-03-25 15:39:23 +0200242 # with uid/gid
willy tarreaueedaa9f2005-12-17 14:08:03 +0100243 global
willy tarreauc5f73ed2005-12-18 01:26:38 +0100244 uid 30000
245 gid 30000
246 chroot /var/chroot/haproxy
247
Willy Tarreau95c20ac2007-03-25 15:39:23 +0200248 # with user/group
249 global
250 user haproxy
251 group public
252 chroot /var/chroot/haproxy
253
willy tarreaueedaa9f2005-12-17 14:08:03 +0100254
2551.4) Startup modes
256------------------
willy tarreau34f45302006-04-15 21:37:14 +0200257The service can start in several different modes :
willy tarreaueedaa9f2005-12-17 14:08:03 +0100258 - foreground / background
259 - quiet / normal / debug
260
261The default mode is normal, foreground, which means that the program doesn't
262return once started. NEVER EVER use this mode in a system startup script, or
263the system won't boot. It needs to be started in background, so that it
264returns immediately after forking. That's accomplished by the 'daemon' option
265in the 'global' section, which is the equivalent of the '-D' command line
266argument.
267
willy tarreau34f45302006-04-15 21:37:14 +0200268The '-db' command line argument overrides the 'daemon' and 'nbproc' global
269options to make the process run in normal, foreground mode.
270
willy tarreaueedaa9f2005-12-17 14:08:03 +0100271Moreover, certain alert messages are still sent to the standard output even
272in 'daemon' mode. To make them disappear, simply add the 'quiet' option in the
273'global' section. This option has no command-line equivalent.
274
275Last, the 'debug' mode, enabled with the 'debug' option in the 'global' section,
276and which is equivalent of the '-d' option, allows deep TCP/HTTP analysis, with
277timestamped display of each connection, disconnection, and HTTP headers for both
278ways. This mode is incompatible with 'daemon' and 'quiet' modes for obvious
279reasons.
280
willy tarreauc5f73ed2005-12-18 01:26:38 +0100281
willy tarreaueedaa9f2005-12-17 14:08:03 +01002821.5) Increasing the overall processing power
283--------------------------------------------
284On multi-processor systems, it may seem to be a shame to use only one processor,
willy tarreau982249e2005-12-18 00:57:06 +0100285eventhough the load needed to saturate a recent processor is far above common
willy tarreaueedaa9f2005-12-17 14:08:03 +0100286usage. Anyway, for very specific needs, the proxy can start several processes
287between which the operating system will spread the incoming connections. The
288number of processes is controlled by the 'nbproc' parameter in the 'global'
willy tarreau4302f492005-12-18 01:00:37 +0100289section. It defaults to 1, and obviously works only in 'daemon' mode. One
290typical usage of this parameter has been to workaround the default per-process
291file-descriptor limit that Solaris imposes to user processes.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100292
293Example :
294---------
295
296 global
willy tarreauc5f73ed2005-12-18 01:26:38 +0100297 daemon
298 quiet
299 nbproc 2
willy tarreaueedaa9f2005-12-17 14:08:03 +0100300
301
willy tarreaufe2c5c12005-12-17 14:14:34 +01003021.6) Helping process management
303-------------------------------
304Haproxy now supports the notion of pidfile. If the '-p' command line argument,
305or the 'pidfile' global option is followed with a file name, this file will be
306removed, then filled with all children's pids, one per line (only in daemon
307mode). This file is NOT within the chroot, which allows to work with a readonly
308 chroot. It will be owned by the user starting the process, and will have
309permissions 0644.
310
311Example :
312---------
313
314 global
315 daemon
316 quiet
willy tarreauc5f73ed2005-12-18 01:26:38 +0100317 nbproc 2
willy tarreaufe2c5c12005-12-17 14:14:34 +0100318 pidfile /var/run/haproxy-private.pid
319
320 # to stop only those processes among others :
321 # kill $(</var/run/haproxy-private.pid)
322
willy tarreau34f45302006-04-15 21:37:14 +0200323 # to reload a new configuration with minimal service impact and without
324 # breaking existing sessions :
Willy Tarreau10806d52007-09-09 23:49:18 +0200325 # haproxy -f haproxy.cfg -p /var/run/haproxy-private.pid -sf $(</var/run/haproxy-private.pid)
willy tarreaufe2c5c12005-12-17 14:14:34 +0100326
willy tarreau64a3cc32005-12-18 01:13:11 +01003271.7) Polling mechanisms
328-----------------------
329Starting from version 1.2.5, haproxy supports the poll() and epoll() polling
330mechanisms. On systems where select() is limited by FD_SETSIZE (like Solaris),
331poll() can be an interesting alternative. Performance tests show that Solaris'
332poll() performance does not decay as fast as the numbers of sockets increase,
333making it a safe solution for high loads. However, Solaris already uses poll()
334to emulate select(), so as long as the number of sockets has no reason to go
335higher than FD_SETSIZE, poll() should not provide any better performance. On
336Linux systems with the epoll() patch (or any 2.6 version), haproxy will use
337epoll() which is extremely fast and non dependant on the number of sockets.
338Tests have shown constant performance from 1 to 20000 simultaneous sessions.
Willy Tarreaude99e992007-04-16 00:53:59 +0200339Version 1.3.9 introduced kqueue() for FreeBSD/OpenBSD, and speculative epoll()
340which consists in trying to perform I/O before queuing the events via syscalls.
willy tarreau64a3cc32005-12-18 01:13:11 +0100341
Willy Tarreau1db37712007-06-03 17:16:49 +0200342In order to optimize latency, it is now possible to limit the number of events
343returned by a single call to poll. The limit is fixed to 200 by default. If a
344smaller latency is seeked, it may be useful to reduce this value by using the
345'tune.maxpollevents' parameter in the 'global' section. Increasing it will
346slightly save CPU cycles in presence of large number of connections.
347
Willy Tarreaude99e992007-04-16 00:53:59 +0200348Haproxy will use kqueue() or speculative epoll() when available, then epoll(),
349and will fall back to poll(), then to select(). However, if for any reason you
350need to disable epoll() or poll() (eg. because of a bug or just to compare
351performance), new global options have been created for this matter : 'nosepoll',
352'nokqueue', 'noepoll' and 'nopoll'.
willy tarreau64a3cc32005-12-18 01:13:11 +0100353
354Example :
355---------
356
357 global
358 # use only select()
359 noepoll
360 nopoll
Willy Tarreau1db37712007-06-03 17:16:49 +0200361 tune.maxpollevents 100
willy tarreau64a3cc32005-12-18 01:13:11 +0100362
363Note :
364------
365For the sake of configuration file portability, these options are accepted but
366ignored if the poll() or epoll() mechanisms have not been enabled at compile
367time.
368
Willy Tarreaude99e992007-04-16 00:53:59 +0200369To make debugging easier, the '-de' runtime argument disables epoll support,
370the '-dp' argument disables poll support, '-dk' disables kqueue and '-ds'
371disables speculative epoll(). They are respectively equivalent to 'noepoll',
372'nopoll', 'nokqueue' and 'nosepoll'.
willy tarreau64a3cc32005-12-18 01:13:11 +0100373
374
willy tarreaueedaa9f2005-12-17 14:08:03 +01003752) Declaration of a listening service
376=====================================
377
378Service sections start with the 'listen' keyword :
379
380 listen <instance_name> [ <IP_address>:<port_range>[,...] ]
381
382- <instance_name> is the name of the instance. This name will be reported in
383 logs, so it is good to have it reflect the proxied service. No unicity test
384 is done on this name, and it's not mandatory for it to be unique, but highly
385 recommended.
386
387- <IP_address> is the IP address the proxy binds to. Empty address, '*' and
388 '0.0.0.0' all mean that the proxy listens to all valid addresses on the
389 system.
390
391- <port_range> is either a unique port, or a port range for which the proxy will
392 accept connections for the IP address specified above. This range can be :
393 - a numerical port (ex: '80')
394 - a dash-delimited ports range explicitly stating the lower and upper bounds
395 (ex: '2000-2100') which are included in the range.
396
397 Particular care must be taken against port ranges, because every <addr:port>
398 couple consumes one socket (=a file descriptor), so it's easy to eat lots of
399 descriptors with a simple range. The <addr:port> couple must be used only once
400 among all instances running on a same system. Please note that attaching to
401 ports lower than 1024 need particular priviledges to start the program, which
402 are independant of the 'uid' parameter.
403
404- the <IP_address>:<port_range> couple may be repeated indefinitely to require
405 the proxy to listen to other addresses and/or ports. To achieve this, simply
406 separate them with a coma.
407
408Examples :
409---------
410 listen http_proxy :80
411 listen x11_proxy 127.0.0.1:6000-6009
412 listen smtp_proxy 127.0.0.1:25,127.0.0.1:587
413 listen ldap_proxy :389,:663
414
415In the event that all addresses do not fit line width, it's preferable to
416detach secondary addresses on other lines with the 'bind' keyword. If this
417keyword is used, it's not even necessary to specify the first address on the
418'listen' line, which sometimes makes multiple configuration handling easier :
419
420 bind [ <IP_address>:<port_range>[,...] ]
421
422Examples :
423----------
424 listen http_proxy
425 bind :80,:443
willy tarreauc5f73ed2005-12-18 01:26:38 +0100426 bind 10.0.0.1:10080,10.0.0.1:10443
427
willy tarreaueedaa9f2005-12-17 14:08:03 +0100428
4292.1) Inhibiting a service
430-------------------------
431A service may be disabled for maintenance reasons, without needing to comment
432out the whole section, simply by specifying the 'disabled' keyword in the
433section to be disabled :
434
435 listen smtp_proxy 0.0.0.0:25
willy tarreauc5f73ed2005-12-18 01:26:38 +0100436 disabled
willy tarreaueedaa9f2005-12-17 14:08:03 +0100437
438Note: the 'enabled' keyword allows to enable a service which has been disabled
439 previously by a default configuration.
440
willy tarreauc5f73ed2005-12-18 01:26:38 +0100441
willy tarreaueedaa9f2005-12-17 14:08:03 +01004422.2) Modes of operation
443-----------------------
444A service can work in 3 different distinct modes :
445 - TCP
446 - HTTP
willy tarreau532bb552006-05-13 18:40:37 +0200447 - health
willy tarreaueedaa9f2005-12-17 14:08:03 +0100448
449TCP mode
450--------
451In this mode, the service relays TCP connections as soon as they're established,
452towards one or several servers. No processing is done on the stream. It's only
453an association of source(addr:port) -> destination(addr:port). To use this mode,
454you must specify 'mode tcp' in the 'listen' section. This is the default mode.
455
456Example :
457---------
458 listen smtp_proxy 0.0.0.0:25
willy tarreauc5f73ed2005-12-18 01:26:38 +0100459 mode tcp
willy tarreaueedaa9f2005-12-17 14:08:03 +0100460
461HTTP mode
462---------
463In this mode, the service relays TCP connections towards one or several servers,
464when it has enough informations to decide, which normally means that all HTTP
465headers have been read. Some of them may be scanned for a cookie or a pattern
466matching a regex. To use this mode, specify 'mode http' in the 'listen' section.
467
468Example :
469---------
470 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +0100471 mode http
willy tarreaueedaa9f2005-12-17 14:08:03 +0100472
473Health-checking mode
474--------------------
475This mode provides a way for external components to check the proxy's health.
476It is meant to be used with intelligent load-balancers which can use send/expect
477scripts to check for all of their servers' availability. This one simply accepts
willy tarreau197e8ec2005-12-17 14:10:59 +0100478the connection, returns the word 'OK' and closes it. If the 'option httpchk' is
479set, then the reply will be 'HTTP/1.0 200 OK' with no data, so that it can be
480tested from a tool which supports HTTP health-checks. To enable it, simply
willy tarreaueedaa9f2005-12-17 14:08:03 +0100481specify 'health' as the working mode :
482
483Example :
484---------
willy tarreau197e8ec2005-12-17 14:10:59 +0100485 # simple response : 'OK'
willy tarreaueedaa9f2005-12-17 14:08:03 +0100486 listen health_check 0.0.0.0:60000
willy tarreauc5f73ed2005-12-18 01:26:38 +0100487 mode health
willy tarreaueedaa9f2005-12-17 14:08:03 +0100488
willy tarreau197e8ec2005-12-17 14:10:59 +0100489 # HTTP response : 'HTTP/1.0 200 OK'
490 listen http_health_check 0.0.0.0:60001
willy tarreauc5f73ed2005-12-18 01:26:38 +0100491 mode health
492 option httpchk
493
willy tarreau532bb552006-05-13 18:40:37 +02004942.2.1 Monitoring
495----------------
willy tarreauc5f73ed2005-12-18 01:26:38 +0100496Versions 1.1.32 and 1.2.6 provide a new solution to check the proxy's
497availability without perturbating the service. The 'monitor-net' keyword was
498created to specify a network of equipments which CANNOT use the service for
499anything but health-checks. This is particularly suited to TCP proxies, because
500it prevents the proxy from relaying the monitor's connection to the remote
501server.
502
503When used with TCP, the connection is accepted then closed and nothing is
504logged. This is enough for a front-end load-balancer to detect the service as
505available.
willy tarreau197e8ec2005-12-17 14:10:59 +0100506
willy tarreauc5f73ed2005-12-18 01:26:38 +0100507When used with HTTP, the connection is accepted, nothing is logged, the
508following response is sent, then the session is closed : "HTTP/1.0 200 OK".
509This is normally enough for any front-end HTTP load-balancer to detect the
510service as available too, both with TCP and HTTP checks.
511
512Proxies using the "monitor-net" keyword can remove the "option dontlognull", as
513it will make them log empty connections from hosts outside the monitoring
514network.
515
516Example :
517---------
518
519 listen tse-proxy
520 bind :3389,:1494,:5900 # TSE, ICA and VNC at once.
521 mode tcp
522 balance roundrobin
523 server tse-farm 192.168.1.10
524 monitor-net 192.168.1.252/31 # L4 load-balancers on .252 and .253
525
willy tarreaueedaa9f2005-12-17 14:08:03 +0100526
Willy Tarreau1c47f852006-07-09 08:22:27 +0200527When the system executing the checks is located behind a proxy, the monitor-net
528keyword cannot be used because haproxy will always see the proxy's address. To
529overcome this limitation, version 1.2.15 brought the 'monitor-uri' keyword. It
530defines an URI which will not be forwarded nor logged, but for which haproxy
531will immediately send an "HTTP/1.0 200 OK" response. This makes it possible to
532check the validity of the reverse-proxy->haproxy chain with one request. It can
533be used in HTTPS checks in front of an stunnel -> haproxy combination for
534instance. Obviously, this keyword is only valid in HTTP mode, otherwise there
535is no notion of URI. Note that the method and HTTP versions are simply ignored.
536
537Example :
538---------
539
540 listen stunnel_backend :8080
541 mode http
542 balance roundrobin
543 server web1 192.168.1.10:80 check
544 server web2 192.168.1.11:80 check
545 monitor-uri /haproxy_test
546
547
willy tarreaueedaa9f2005-12-17 14:08:03 +01005482.3) Limiting the number of simultaneous connections
549----------------------------------------------------
550The 'maxconn' parameter allows a proxy to refuse connections above a certain
551amount of simultaneous ones. When the limit is reached, it simply stops
552listening, but the system may still be accepting them because of the back log
willy tarreau982249e2005-12-18 00:57:06 +0100553queue. These connections will be processed later when other ones have freed
willy tarreaueedaa9f2005-12-17 14:08:03 +0100554some slots. This provides a serialization effect which helps very fragile
willy tarreau34f45302006-04-15 21:37:14 +0200555servers resist to high loads. See further for system limitations.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100556
557Example :
558---------
559 listen tiny_server 0.0.0.0:80
560 maxconn 10
561
562
5632.4) Soft stop
564--------------
565It is possible to stop services without breaking existing connections by the
willy tarreau22739ef2006-01-20 20:43:32 +0100566sending of the SIGUSR1 signal to the process. All services are then put into
willy tarreaueedaa9f2005-12-17 14:08:03 +0100567soft-stop state, which means that they will refuse to accept new connections,
568except for those which have a non-zero value in the 'grace' parameter, in which
569case they will still accept connections for the specified amount of time, in
willy tarreau22739ef2006-01-20 20:43:32 +0100570milliseconds. This makes it possible to tell a load-balancer that the service
571is failing, while still doing the job during the time it needs to detect it.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100572
573Note: active connections are never killed. In the worst case, the user will have
574to wait for all of them to close or to time-out, or simply kill the process
willy tarreau22739ef2006-01-20 20:43:32 +0100575normally (SIGTERM). The default 'grace' value is '0'.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100576
577Example :
578---------
579 # enter soft stop after 'killall -USR1 haproxy'
580 # the service will still run 10 seconds after the signal
581 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +0100582 mode http
583 grace 10000
willy tarreaueedaa9f2005-12-17 14:08:03 +0100584
585 # this port is dedicated to a load-balancer, and must fail immediately
586 listen health_check 0.0.0.0:60000
willy tarreauc5f73ed2005-12-18 01:26:38 +0100587 mode health
588 grace 0
willy tarreaueedaa9f2005-12-17 14:08:03 +0100589
590
willy tarreau39df2dc2006-01-29 21:56:05 +0100591As of version 1.2.8, a new soft-reconfiguration mechanism has been introduced.
willy tarreau22739ef2006-01-20 20:43:32 +0100592It is now possible to "pause" all the proxies by sending a SIGTTOU signal to
593the processes. This will disable the listening socket without breaking existing
594connections. After that, sending a SIGTTIN signal to those processes enables
595the listening sockets again. This is very useful to try to load a new
596configuration or even a new version of haproxy without breaking existing
597connections. If the load succeeds, then simply send a SIGUSR1 which will make
598the previous proxies exit immediately once their sessions are closed ; and if
599the load fails, then simply send a SIGTTIN to restore the service immediately.
600Please note that the 'grace' parameter is ignored for SIGTTOU, as well as for
601SIGUSR1 when the process was in the pause mode. Please also note that it would
602be useful to save the pidfile before starting a new instance.
603
willy tarreau34f45302006-04-15 21:37:14 +0200604This mechanism fully exploited since 1.2.11 with the '-st' and '-sf' options
willy tarreau532bb552006-05-13 18:40:37 +0200605(see below).
606
6072.4.1) Hot reconfiguration
608--------------------------
609The '-st' and '-sf' command line options are used to inform previously running
610processes that a configuration is being reloaded. They will receive the SIGTTOU
611signal to ask them to temporarily stop listening to the ports so that the new
612process can grab them. If anything wrong happens, the new process will send
613them a SIGTTIN to tell them to re-listen to the ports and continue their normal
614work. Otherwise, it will either ask them to finish (-sf) their work then softly
615exit, or immediately terminate (-st), breaking existing sessions. A typical use
616of this allows a configuration reload without service interruption :
617
618 # haproxy -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid)
619
willy tarreau22739ef2006-01-20 20:43:32 +0100620
willy tarreaueedaa9f2005-12-17 14:08:03 +01006212.5) Connections expiration time
622--------------------------------
623It is possible (and recommended) to configure several time-outs on TCP
624connections. Three independant timers are adjustable with values specified
625in milliseconds. A session will be terminated if either one of these timers
626expire.
627
628 - the time we accept to wait for data from the client, or for the client to
629 accept data : 'clitimeout' :
630
willy tarreauc5f73ed2005-12-18 01:26:38 +0100631 # client time-out set to 2mn30.
632 clitimeout 150000
willy tarreaueedaa9f2005-12-17 14:08:03 +0100633
634 - the time we accept to wait for data from the server, or for the server to
635 accept data : 'srvtimeout' :
636
willy tarreauc5f73ed2005-12-18 01:26:38 +0100637 # server time-out set to 30s.
638 srvtimeout 30000
willy tarreaueedaa9f2005-12-17 14:08:03 +0100639
640 - the time we accept to wait for a connection to establish on a server :
641 'contimeout' :
642
643 # we give up if the connection does not complete within 4 seconds
willy tarreauc5f73ed2005-12-18 01:26:38 +0100644 contimeout 4000
willy tarreaueedaa9f2005-12-17 14:08:03 +0100645
646Notes :
647-------
648 - 'contimeout' and 'srvtimeout' have no sense on 'health' mode servers ;
649 - under high loads, or with a saturated or defective network, it's possible
650 that some packets get lost. Since the first TCP retransmit only happens
651 after 3 seconds, a time-out equal to, or lower than 3 seconds cannot
652 compensate for a packet loss. A 4 seconds time-out seems a reasonable
653 minimum which will considerably reduce connection failures.
Willy Tarreaubefdff12007-12-02 22:27:38 +0100654 - starting with version 1.3.14, it is possible to specify timeouts in
655 arbitrary time units among { us, ms, s, m, h, d }. For this, the integer
656 value just has to be suffixed with the unit.
willy tarreauc5f73ed2005-12-18 01:26:38 +0100657
willy tarreaueedaa9f2005-12-17 14:08:03 +01006582.6) Attempts to reconnect
659--------------------------
660After a connection failure to a server, it is possible to retry, potentially
661on another server. This is useful if health-checks are too rare and you don't
662want the clients to see the failures. The number of attempts to reconnect is
663set by the 'retries' paramter.
664
665Example :
666---------
willy tarreauc5f73ed2005-12-18 01:26:38 +0100667 # we can retry 3 times max after a failure
668 retries 3
willy tarreaueedaa9f2005-12-17 14:08:03 +0100669
willy tarreau34f45302006-04-15 21:37:14 +0200670Please note that the reconnection attempt may lead to getting the connection
671sent to a new server if the original one died between connection attempts.
672
willy tarreaueedaa9f2005-12-17 14:08:03 +0100673
6742.7) Address of the dispatch server (deprecated)
675------------------------------------------------
676The server which will be sent all new connections is defined by the 'dispatch'
677parameter, in the form <address>:<port>. It generally is dedicated to unknown
678connections and will assign them a cookie, in case of HTTP persistence mode,
679or simply is a single server in case of generic TCP proxy. This old mode is only
680provided for backwards compatibility, but doesn't allow to check remote servers
681state, and has a rather limited usage. All new setups should switch to 'balance'
682mode. The principle of the dispatcher is to be able to perform the load
683balancing itself, but work only on new clients so that the server doesn't need
684to be a big machine.
685
686Example :
687---------
willy tarreauc5f73ed2005-12-18 01:26:38 +0100688 # all new connections go there
689 dispatch 192.168.1.2:80
willy tarreaueedaa9f2005-12-17 14:08:03 +0100690
691Note :
692------
693This parameter has no sense for 'health' servers, and is incompatible with
694'balance' mode.
695
696
6972.8) Outgoing source address
698----------------------------
699It is often necessary to bind to a particular address when connecting to some
700remote hosts. This is done via the 'source' parameter which is a per-proxy
701parameter. A newer version may allow to fix different sources to reach different
702servers. The syntax is 'source <address>[:<port>]', where <address> is a valid
703local address (or '0.0.0.0' or '*' or empty to let the system choose), and
704<port> is an optional parameter allowing the user to force the source port for
705very specific needs. If the port is not specified or is '0', the system will
706choose a free port. Note that as of version 1.1.18, the servers health checks
707are also performed from the same source.
708
709Examples :
710----------
711 listen http_proxy *:80
willy tarreauc5f73ed2005-12-18 01:26:38 +0100712 # all connections take 192.168.1.200 as source address
713 source 192.168.1.200:0
willy tarreaueedaa9f2005-12-17 14:08:03 +0100714
715 listen rlogin_proxy *:513
willy tarreauc5f73ed2005-12-18 01:26:38 +0100716 # use address 192.168.1.200 and the reserved port 900 (needs to be root)
717 source 192.168.1.200:900
willy tarreaueedaa9f2005-12-17 14:08:03 +0100718
719
7202.9) Setting the cookie name
721----------------------------
722In HTTP mode, it is possible to look for a particular cookie which will contain
723a server identifier which should handle the connection. The cookie name is set
724via the 'cookie' parameter.
725
726Example :
727---------
728 listen http_proxy :80
willy tarreauc5f73ed2005-12-18 01:26:38 +0100729 mode http
730 cookie SERVERID
willy tarreaueedaa9f2005-12-17 14:08:03 +0100731
732It is possible to change the cookie behaviour to get a smarter persistence,
733depending on applications. It is notably possible to delete or modify a cookie
734emitted by a server, insert a cookie identifying the server in an HTTP response
735and even add a header to tell upstream caches not to cache this response.
736
737Examples :
738----------
739
740To remove the cookie for direct accesses (ie when the server matches the one
741which was specified in the client cookie) :
742
willy tarreauc5f73ed2005-12-18 01:26:38 +0100743 cookie SERVERID indirect
willy tarreaueedaa9f2005-12-17 14:08:03 +0100744
745To replace the cookie value with the one assigned to the server if any (no
746cookie will be created if the server does not provide one, nor if the
747configuration does not provide one). This lets the application put the cookie
748exactly on certain pages (eg: successful authentication) :
749
willy tarreauc5f73ed2005-12-18 01:26:38 +0100750 cookie SERVERID rewrite
willy tarreaueedaa9f2005-12-17 14:08:03 +0100751
752To create a new cookie and assign the server identifier to it (in this case, all
753servers should be associated with a valid cookie, since no cookie will simply
754delete the cookie from the client's browser) :
755
willy tarreauc5f73ed2005-12-18 01:26:38 +0100756 cookie SERVERID insert
willy tarreaueedaa9f2005-12-17 14:08:03 +0100757
willy tarreau0174f312005-12-18 01:02:42 +0100758To reuse an existing application cookie and prefix it with the server's
759identifier, and remove it in the request, use the 'prefix' option. This allows
760to insert a haproxy in front of an application without risking to break clients
761which does not support more than one cookie :
762
willy tarreauc5f73ed2005-12-18 01:26:38 +0100763 cookie JSESSIONID prefix
willy tarreau0174f312005-12-18 01:02:42 +0100764
willy tarreaueedaa9f2005-12-17 14:08:03 +0100765To insert a cookie and ensure that no upstream cache will store it, add the
766'nocache' option :
767
willy tarreauc5f73ed2005-12-18 01:26:38 +0100768 cookie SERVERID insert nocache
willy tarreaueedaa9f2005-12-17 14:08:03 +0100769
770To insert a cookie only after a POST request, add 'postonly' after 'insert'.
771This has the advantage that there's no risk of caching, and that all pages
772seen before the POST one can still be cached :
773
willy tarreauc5f73ed2005-12-18 01:26:38 +0100774 cookie SERVERID insert postonly
willy tarreaueedaa9f2005-12-17 14:08:03 +0100775
776Notes :
777-----------
778- it is possible to combine 'insert' with 'indirect' or 'rewrite' to adapt to
779 applications which already generate the cookie with an invalid content.
780
781- in the case where 'insert' and 'indirect' are both specified, the cookie is
willy tarreau0174f312005-12-18 01:02:42 +0100782 never transmitted to the server, since it wouldn't understand it. This is the
783 most application-transparent mode.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100784
785- it is particularly recommended to use 'nocache' in 'insert' mode if any
786 upstream HTTP/1.0 cache is susceptible to cache the result, because this may
787 lead to many clients going to the same server, or even worse, some clients
788 having their server changed while retrieving a page from the cache.
789
willy tarreau0174f312005-12-18 01:02:42 +0100790- the 'prefix' mode normally does not need 'indirect', 'nocache', nor
791 'postonly', because just as in the 'rewrite' mode, it relies on the
792 application to know when a cookie can be emitted. However, since it has to
793 fix the cookie name in every subsequent requests, you must ensure that the
794 proxy will be used without any "HTTP keep-alive". Use option "httpclose" if
795 unsure.
796
willy tarreaueedaa9f2005-12-17 14:08:03 +0100797- when the application is well known and controlled, the best method is to
798 only add the persistence cookie on a POST form because it's up to the
willy tarreau0174f312005-12-18 01:02:42 +0100799 application to select which page it wants the upstream servers to cache. In
800 this case, you would use 'insert postonly indirect'.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100801
willy tarreauc5f73ed2005-12-18 01:26:38 +0100802
willy tarreaueedaa9f2005-12-17 14:08:03 +01008032.10) Associating a cookie value with a server
804----------------------------------------------
805In HTTP mode, it's possible to associate a cookie value to each server. This
806was initially used in combination with 'dispatch' mode to handle direct accesses
807but it is now the standard way of doing the load balancing. The syntax is :
808
809 server <identifier> <address>:<port> cookie <value>
810
811- <identifier> is any name which can be used to identify the server in the logs.
812- <address>:<port> specifies where the server is bound.
813- <value> is the value to put in or to read from the cookie.
814
815Example : the 'SERVERID' cookie can be either 'server01' or 'server02'
816---------
817 listen http_proxy :80
willy tarreauc5f73ed2005-12-18 01:26:38 +0100818 mode http
819 cookie SERVERID
820 dispatch 192.168.1.100:80
821 server web1 192.168.1.1:80 cookie server01
822 server web2 192.168.1.2:80 cookie server02
willy tarreaueedaa9f2005-12-17 14:08:03 +0100823
824Warning : the syntax has changed since version 1.0 !
825---------
826
willy tarreauc5f73ed2005-12-18 01:26:38 +0100827
willy tarreau598da412005-12-18 01:07:29 +01008282.11) Application Cookies
829-------------------------
830Since 1.2.4 it is possible to catch the cookie that comes from an
831application server in order to apply "application session stickyness".
832The server's response is searched for 'appsession' cookie, the first
833'len' bytes are used for matching and it is stored for a period of
834'timeout'.
835The syntax is:
836
willy tarreau532bb552006-05-13 18:40:37 +0200837 appsession <session_cookie> len <match_length> timeout <holdtime>
willy tarreau598da412005-12-18 01:07:29 +0100838
willy tarreau532bb552006-05-13 18:40:37 +0200839- <session_cookie> is the cookie, the server uses for it's session-handling
840- <match_length> how many bytes/characters should be used for matching equal
willy tarreau598da412005-12-18 01:07:29 +0100841 sessions
willy tarreau532bb552006-05-13 18:40:37 +0200842- <holdtime> after this inactivaty time, in ms, the cookie will be deleted
willy tarreau598da412005-12-18 01:07:29 +0100843 from the sessionstore
Willy Tarreaubefdff12007-12-02 22:27:38 +0100844- starting with version 1.3.14, it is possible to specify timeouts in
845 arbitrary time units among { us, ms, s, m, h, d }. For this, the integer
846 value just has to be prefixed with the unit.
willy tarreau598da412005-12-18 01:07:29 +0100847
848The appsession is only per 'listen' section possible.
849
850Example :
851---------
willy tarreau532bb552006-05-13 18:40:37 +0200852 listen http_lb1 192.168.3.4:80
853 mode http
854 capture request header Cookie len 200
855 # Havind a ServerID cookie on the client allows him to reach
856 # the right server even after expiration of the appsession.
857 cookie ServerID insert nocache indirect
858 # Will memorize 52 bytes of the cookie 'JSESSIONID' and keep them
859 # for 3 hours. It will match it in the cookie and the URL field.
Willy Tarreaubefdff12007-12-02 22:27:38 +0100860 appsession JSESSIONID len 52 timeout 3h
willy tarreau532bb552006-05-13 18:40:37 +0200861 server first1 10.3.9.2:10805 check inter 3000 cookie first
862 server secon1 10.3.9.3:10805 check inter 3000 cookie secon
863 server first1 10.3.9.4:10805 check inter 3000 cookie first
864 server secon2 10.3.9.5:10805 check inter 3000 cookie secon
865 option httpchk GET /test.jsp
willy tarreau598da412005-12-18 01:07:29 +0100866
willy tarreauc5f73ed2005-12-18 01:26:38 +0100867
willy tarreaueedaa9f2005-12-17 14:08:03 +01008683) Autonomous load balancer
869===========================
870
871The proxy can perform the load-balancing itself, both in TCP and in HTTP modes.
872This is the most interesting mode which obsoletes the old 'dispatch' mode
873described above. It has advantages such as server health monitoring, multiple
874port binding and port mapping. To use this mode, the 'balance' keyword is used,
willy tarreau34f45302006-04-15 21:37:14 +0200875followed by the selected algorithm. Up to version 1.2.11, only 'roundrobin' was
876available, which is also the default value if unspecified. Starting with
Willy Tarreau2fcb5002007-05-08 13:35:26 +0200877version 1.2.12, a new 'source' keyword appeared. A new 'uri' keyword was added
878in version 1.3.10. In this mode, there will be no dispatch address, but the
879proxy needs at least one server.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100880
881Example : same as the last one, with internal load balancer
882---------
883
884 listen http_proxy :80
willy tarreauc5f73ed2005-12-18 01:26:38 +0100885 mode http
886 cookie SERVERID
887 balance roundrobin
888 server web1 192.168.1.1:80 cookie server01
889 server web2 192.168.1.2:80 cookie server02
willy tarreaueedaa9f2005-12-17 14:08:03 +0100890
891
892Since version 1.1.22, it is possible to automatically determine on which port
893the server will get the connection, depending on the port the client connected
894to. Indeed, there now are 4 possible combinations for the server's <port> field:
895
896 - unspecified or '0' :
897 the connection will be sent to the same port as the one on which the proxy
898 received the client connection itself.
899
900 - numerical value (the only one supported in versions earlier than 1.1.22) :
901 the connection will always be sent to the specified port.
902
903 - '+' followed by a numerical value :
904 the connection will be sent to the same port as the one on which the proxy
905 received the connection, plus this value.
906
907 - '-' followed by a numerical value :
908 the connection will be sent to the same port as the one on which the proxy
909 received the connection, minus this value.
910
911Examples :
912----------
913
914# same as previous example
915
916 listen http_proxy :80
willy tarreauc5f73ed2005-12-18 01:26:38 +0100917 mode http
918 cookie SERVERID
919 balance roundrobin
920 server web1 192.168.1.1 cookie server01
921 server web2 192.168.1.2 cookie server02
willy tarreaueedaa9f2005-12-17 14:08:03 +0100922
923# simultaneous relaying of ports 80, 81 and 8080-8089
924
925 listen http_proxy :80,:81,:8080-8089
willy tarreauc5f73ed2005-12-18 01:26:38 +0100926 mode http
927 cookie SERVERID
928 balance roundrobin
929 server web1 192.168.1.1 cookie server01
930 server web2 192.168.1.2 cookie server02
willy tarreaueedaa9f2005-12-17 14:08:03 +0100931
932# relaying of TCP ports 25, 389 and 663 to ports 1025, 1389 and 1663
933
934 listen http_proxy :25,:389,:663
willy tarreauc5f73ed2005-12-18 01:26:38 +0100935 mode tcp
936 balance roundrobin
937 server srv1 192.168.1.1:+1000
938 server srv2 192.168.1.2:+1000
willy tarreaueedaa9f2005-12-17 14:08:03 +0100939
willy tarreau34f45302006-04-15 21:37:14 +0200940As previously stated, version 1.2.12 brought the 'source' keyword. When this
941keyword is used, the client's IP address is hashed and evenly distributed among
942the available servers so that a same source IP will always go to the same
943server as long as there are no change in the number of available servers. This
944can be used for instance to bind HTTP and HTTPS to the same server. It can also
945be used to improve stickyness when one part of the client population does not
946accept cookies. In this case, only those ones will be perturbated should a
947server fail.
948
949NOTE: It is important to consider the fact that many clients surf the net
950 through proxy farms which assign different IP addresses for each
951 request. Others use dialup connections with a different IP at each
952 connection. Thus, the 'source' parameter should be used with extreme
953 care.
954
955Examples :
956----------
957
958# make a same IP go to the same server whatever the service
959
960 listen http_proxy
961 bind :80,:443
962 mode http
963 balance source
964 server web1 192.168.1.1
965 server web2 192.168.1.2
966
967# try to improve client-server binding by using both source IP and cookie :
968
969 listen http_proxy :80
970 mode http
971 cookie SERVERID
972 balance source
973 server web1 192.168.1.1 cookie server01
974 server web2 192.168.1.2 cookie server02
975
Willy Tarreau2fcb5002007-05-08 13:35:26 +0200976As indicated above, the 'uri' keyword was introduced in version 1.3.10. It is
977useful when load-balancing between reverse proxy-caches, because it will hash
978the URI and use the hash result to select a server, thus optimizing the hit
979rate on the caches, because the same URI will always reach the same cache. This
980keyword is only allowed in HTTP mode.
981
982Example :
983---------
984
985# Always send a given URI to the same server
986
987 listen http_proxy
988 bind :3128
989 mode http
990 balance uri
991 server squid1 192.168.1.1
992 server squid2 192.168.1.2
993
Willy Tarreau01732802007-11-01 22:48:15 +0100994Version 1.3.14 introduced the "balance url_param" method. It consists in
995relying on a parameter passed in the URL to perform a hash. This is mostly
996useful for applications which do not have strict persistence requirements,
997but for which it still provides a performance boost due to local caching.
998Some of these applications may not be able to use a cookie for whatever reason,
999but may be able to look for a parameter passed in the URL. If the parameter is
1000missing from the URL, then the 'round robin' method applies.
1001
matt.farnsworth@nokia.com1c2ab962008-04-14 20:47:37 +02001002A modifier may be added to specify that parameters in POST requests may be
1003found in the messsage body if the URL lacks a '?' separator character.
1004A wait limit may also be applied, if no limit is requested then
1005the default value is 48 octets, the minimum is 3. HAProxy may wait, until 48
1006octets are received. If Content-Length is missing, or zero it need not
1007wait for more data then the client promissed to send. When Content-Length is
1008present, and more than <max_wait>; then waiting is limited and it is assumed this
1009will be enough data to search for the presence of a parameter. If
1010Transfer-Encoding: chunked is used (unlikely), then the length of the first chunk
1011is the maximum number of bytes to wait for.
1012
1013balance url_param <param> [check_post [<max_wait>]]
1014
1015Caveats for using the check_post extension:
1016
1017 - all POST requests are eligable for consideration, because there is
1018 no way to determine if the parameters will be found in the body or
1019 entity which may contain binary data. Therefore another method may be
1020 required to restrict consideration of POST requests that have no URL
1021 parameters in the body. (see acl reqideny http_end)
1022
1023Limitations on inspecting the entity body of a POST:
1024
1025 - Content-Encoding is not supported, the parameter search will probably fail;
1026 and load balancing will fall back to Round Robin.
1027
1028 - Expect: 100-continue is not supported, load balancing will fall back to
1029 Round Robin.
1030
1031 - Transfer-Encoding(RFC2616 3.6.1) is only supported in the first chunk. If
1032 the entire parameter value is not present in the first chunk, the selection
1033 of server is undefined (actually, defined by how little actually appeared in
1034 the first chunk).
1035
1036 - This feature does not support generation of a 100, 411 or 501 response.
1037
1038 - In some cases, requesting check_post MAY attempt to scan the entire contents
1039 of a message body. Scaning normally terminates when linear white space or
1040 control characters are found, indicating the end of what might be a URL parameter
1041 list. This is probably not a concern with SGML type message bodies.
1042
1043
Willy Tarreau01732802007-11-01 22:48:15 +01001044Example :
1045---------
1046
1047# Hash the "basket_id" argument from the URL to determine the server
1048
1049 listen http_proxy
1050 bind :3128
1051 mode http
1052 balance url_param basket_id
1053 server ebiz1 192.168.1.1
1054 server ebiz2 192.168.1.2
1055
willy tarreaueedaa9f2005-12-17 14:08:03 +01001056
willy tarreau197e8ec2005-12-17 14:10:59 +010010573.1) Server monitoring
1058----------------------
willy tarreaueedaa9f2005-12-17 14:08:03 +01001059It is possible to check the servers status by trying to establish TCP
1060connections or even sending HTTP requests to them. A server which fails to
1061reply to health checks as expected will not be used by the load balancing
1062algorithms. To enable monitoring, add the 'check' keyword on a server line.
1063It is possible to specify the interval between tests (in milliseconds) with
1064the 'inter' parameter, the number of failures supported before declaring that
1065the server has fallen down with the 'fall' parameter, and the number of valid
1066checks needed for the server to fully get up with the 'rise' parameter. Since
1067version 1.1.22, it is also possible to send checks to a different port
1068(mandatory when none is specified) with the 'port' parameter. The default
1069values are the following ones :
1070
1071 - inter : 2000
1072 - rise : 2
1073 - fall : 3
1074 - port : default server port
Willy Tarreau2ea3abb2007-03-25 16:45:16 +02001075 - addr : specific address for the test (default = address server)
1076
willy tarreaueedaa9f2005-12-17 14:08:03 +01001077The default mode consists in establishing TCP connections only. But in certain
1078types of application failures, it is often that the server continues to accept
1079connections because the system does it itself while the application is running
1080an endless loop, or is completely stuck. So in version 1.1.16 were introduced
1081HTTP health checks which only performed simple lightweight requests and analysed
1082the response. Now, as of version 1.1.23, it is possible to change the HTTP
1083method, the URI, and the HTTP version string (which even allows to send headers
1084with a dirty trick). To enable HTTP health-checks, use 'option httpchk'.
1085
1086By default, requests use the 'OPTIONS' method because it's very light and easy
1087to filter from logs, and does it on '/'. Only HTTP responses 2xx and 3xx are
1088considered valid ones, and only if they come before the time to send a new
1089request is reached ('inter' parameter). If some servers block this type of
1090request, 3 other forms help to forge a request :
1091
1092 - option httpchk -> OPTIONS / HTTP/1.0
1093 - option httpchk URI -> OPTIONS <URI> HTTP/1.0
1094 - option httpchk METH URI -> <METH> <URI> HTTP/1.0
1095 - option httpchk METH URI VER -> <METH> <URI> <VER>
1096
Willy Tarreauf3c69202006-07-09 16:42:34 +02001097Some people are using HAProxy to relay various TCP-based protocols such as
1098HTTPS, SMTP or LDAP, with the most common one being HTTPS. One problem commonly
1099encountered in data centers is the need to forward the traffic to far remote
1100servers while providing server fail-over. Often, TCP-only checks are not enough
1101because intermediate firewalls, load balancers or proxies might acknowledge the
1102connection before it reaches the real server. The only solution to this problem
1103is to send application-level health checks. Since the demand for HTTPS checks
1104is high, it has been implemented in 1.2.15 based on SSLv3 Client Hello packets.
1105To enable it, use 'option ssl-hello-chk'. It will send SSL CLIENT HELLO packets
1106to the servers, announcing support for most common cipher suites. If the server
1107responds what looks like a SERVER HELLO or an ALERT (refuses the ciphers) then
1108the response is considered as valid. Note that Apache does not generate a log
1109when it receives only an HELLO message, which makes this type of message
1110perfectly suit this need.
1111
Willy Tarreau23677902007-05-08 23:50:35 +02001112Version 1.3.10 introduced the SMTP health check. By default, it sends
1113"HELO localhost" to the servers, and waits for the 250 message. Note that it
1114can also send a specific request :
1115
1116 - option smtpchk -> sends "HELO localhost"
1117 - option smtpchk EHLO mail.mydomain.com -> sends this ESMTP greeting
1118
willy tarreaueedaa9f2005-12-17 14:08:03 +01001119See examples below.
1120
1121Since version 1.1.17, it is possible to specify backup servers. These servers
1122are only sollicited when no other server is available. This may only be useful
1123to serve a maintenance page, or define one active and one backup server (seldom
1124used in TCP mode). To make a server a backup one, simply add the 'backup' option
1125on its line. These servers also support cookies, so if a cookie is specified for
1126a backup server, clients assigned to this server will stick to it even when the
1127other ones come back. Conversely, if no cookie is assigned to such a server,
1128the clients will get their cookies removed (empty cookie = removal), and will
1129be balanced against other servers once they come back. Please note that there
Willy TARREAU3481c462006-03-01 22:37:57 +01001130is no load-balancing among backup servers by default. If there are several
1131backup servers, the second one will only be used when the first one dies, and
1132so on. To force load-balancing between backup servers, specify the 'allbackups'
1133option.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001134
Willy Tarreau2ea3abb2007-03-25 16:45:16 +02001135Since version 1.1.22, it is possible to send health checks to a different port
1136than the service. It is mainly needed in setups where the server does not have
1137any predefined port, for instance when the port is deduced from the listening
1138port. For this, use the 'port' parameter followed by the port number which must
1139respond to health checks. It is also possible to send health checks to a
1140different address than the service. It makes it easier to use a dedicated check
1141daemon on the servers, for instance, check return contents and stop several
1142farms at once in the event of an error anywhere.
1143
willy tarreaueedaa9f2005-12-17 14:08:03 +01001144Since version 1.1.17, it is also possible to visually check the status of all
1145servers at once. For this, you just have to send a SIGHUP signal to the proxy.
1146The servers status will be dumped into the logs at the 'notice' level, as well
1147as on <stderr> if not closed. For this reason, it's always a good idea to have
1148one local log server at the 'notice' level.
1149
willy tarreau982249e2005-12-18 00:57:06 +01001150Since version 1.1.28 and 1.2.1, if an instance loses all its servers, an
willy tarreau0174f312005-12-18 01:02:42 +01001151emergency message will be sent in the logs to inform the administator that an
willy tarreau982249e2005-12-18 00:57:06 +01001152immediate action must be taken.
1153
willy tarreau0174f312005-12-18 01:02:42 +01001154Since version 1.1.30 and 1.2.3, several servers can share the same cookie
1155value. This is particularly useful in backup mode, to select alternate paths
1156for a given server for example, to provide soft-stop, or to direct the clients
1157to a temporary page during an application restart. The principle is that when
1158a server is dead, the proxy will first look for another server which shares the
1159same cookie value for every client which presents the cookie. If there is no
1160standard server for this cookie, it will then look for a backup server which
1161shares the same name. Please consult the architecture guide for more information.
willy tarreau982249e2005-12-18 00:57:06 +01001162
willy tarreaueedaa9f2005-12-17 14:08:03 +01001163Examples :
1164----------
1165# same setup as in paragraph 3) with TCP monitoring
1166 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001167 mode http
1168 cookie SERVERID
1169 balance roundrobin
1170 server web1 192.168.1.1:80 cookie server01 check
1171 server web2 192.168.1.2:80 cookie server02 check inter 500 rise 1 fall 2
willy tarreaueedaa9f2005-12-17 14:08:03 +01001172
1173# same with HTTP monitoring via 'OPTIONS / HTTP/1.0'
1174 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001175 mode http
1176 cookie SERVERID
1177 balance roundrobin
1178 option httpchk
1179 server web1 192.168.1.1:80 cookie server01 check
1180 server web2 192.168.1.2:80 cookie server02 check inter 500 rise 1 fall 2
willy tarreaueedaa9f2005-12-17 14:08:03 +01001181
1182# same with HTTP monitoring via 'OPTIONS /index.html HTTP/1.0'
1183 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001184 mode http
1185 cookie SERVERID
1186 balance roundrobin
1187 option httpchk /index.html
1188 server web1 192.168.1.1:80 cookie server01 check
1189 server web2 192.168.1.2:80 cookie server02 check inter 500 rise 1 fall 2
willy tarreaueedaa9f2005-12-17 14:08:03 +01001190
1191# same with HTTP monitoring via 'HEAD /index.jsp? HTTP/1.1\r\nHost: www'
1192 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001193 mode http
1194 cookie SERVERID
1195 balance roundrobin
1196 option httpchk HEAD /index.jsp? HTTP/1.1\r\nHost:\ www
1197 server web1 192.168.1.1:80 cookie server01 check
1198 server web2 192.168.1.2:80 cookie server02 check inter 500 rise 1 fall 2
willy tarreaueedaa9f2005-12-17 14:08:03 +01001199
willy tarreau0174f312005-12-18 01:02:42 +01001200# Load-balancing with 'prefixed cookie' persistence, and soft-stop using an
1201# alternate port 81 on the server for health-checks.
1202 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001203 mode http
1204 cookie JSESSIONID prefix
1205 balance roundrobin
1206 option httpchk HEAD /index.jsp? HTTP/1.1\r\nHost:\ www
1207 server web1-norm 192.168.1.1:80 cookie s1 check port 81
1208 server web2-norm 192.168.1.2:80 cookie s2 check port 81
1209 server web1-stop 192.168.1.1:80 cookie s1 check port 80 backup
1210 server web2-stop 192.168.1.2:80 cookie s2 check port 80 backup
willy tarreau0174f312005-12-18 01:02:42 +01001211
willy tarreaueedaa9f2005-12-17 14:08:03 +01001212# automatic insertion of a cookie in the server's response, and automatic
1213# deletion of the cookie in the client request, while asking upstream caches
1214# not to cache replies.
1215 listen web_appl 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001216 mode http
1217 cookie SERVERID insert nocache indirect
1218 balance roundrobin
1219 server web1 192.168.1.1:80 cookie server01 check
1220 server web2 192.168.1.2:80 cookie server02 check
willy tarreaueedaa9f2005-12-17 14:08:03 +01001221
1222# same with off-site application backup and local error pages server
1223 listen web_appl 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001224 mode http
1225 cookie SERVERID insert nocache indirect
1226 balance roundrobin
1227 server web1 192.168.1.1:80 cookie server01 check
1228 server web2 192.168.1.2:80 cookie server02 check
1229 server web-backup 192.168.2.1:80 cookie server03 check backup
1230 server web-excuse 192.168.3.1:80 check backup
willy tarreaueedaa9f2005-12-17 14:08:03 +01001231
willy tarreauc5f73ed2005-12-18 01:26:38 +01001232# SMTP+TLS relaying with health-checks and backup servers
willy tarreaueedaa9f2005-12-17 14:08:03 +01001233
1234 listen http_proxy :25,:587
willy tarreauc5f73ed2005-12-18 01:26:38 +01001235 mode tcp
1236 balance roundrobin
1237 server srv1 192.168.1.1 check port 25 inter 30000 rise 1 fall 2
1238 server srv2 192.168.1.2 backup
willy tarreaueedaa9f2005-12-17 14:08:03 +01001239
Willy Tarreauf3c69202006-07-09 16:42:34 +02001240# HTTPS relaying with health-checks and backup servers
1241
1242 listen http_proxy :443
1243 mode tcp
1244 option ssl-hello-chk
1245 balance roundrobin
1246 server srv1 192.168.1.1 check inter 30000 rise 1 fall 2
1247 server srv2 192.168.1.2 backup
1248
Willy TARREAU3481c462006-03-01 22:37:57 +01001249# Load-balancing using a backup pool (requires haproxy 1.2.9)
1250 listen http_proxy 0.0.0.0:80
1251 mode http
1252 balance roundrobin
1253 option httpchk
1254 server inst1 192.168.1.1:80 cookie s1 check
1255 server inst2 192.168.1.2:80 cookie s2 check
1256 server inst3 192.168.1.3:80 cookie s3 check
1257 server back1 192.168.1.10:80 check backup
1258 server back2 192.168.1.11:80 check backup
1259 option allbackups # all backups will be used
1260
willy tarreaueedaa9f2005-12-17 14:08:03 +01001261
12623.2) Redistribute connections in case of failure
1263------------------------------------------------
1264In HTTP mode, if a server designated by a cookie does not respond, the clients
1265may definitely stick to it because they cannot flush the cookie, so they will
1266not be able to access the service anymore. Specifying 'redispatch' will allow
1267the proxy to break their persistence and redistribute them to working servers.
1268
1269Example :
1270---------
1271 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001272 mode http
1273 cookie SERVERID
1274 dispatch 192.168.1.100:80
1275 server web1 192.168.1.1:80 cookie server01
1276 server web2 192.168.1.2:80 cookie server02
1277 redispatch # send back to dispatch in case of connection failure
willy tarreaueedaa9f2005-12-17 14:08:03 +01001278
1279Up to, and including version 1.1.16, this parameter only applied to connection
1280failures. Since version 1.1.17, it also applies to servers which have been
1281detected as failed by the health check mechanism. Indeed, a server may be broken
1282but still accepting connections, which would not solve every case. But it is
1283possible to conserve the old behaviour, that is, make a client insist on trying
1284to connect to a server even if it is said to be down, by setting the 'persist'
1285option :
1286
1287 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001288 mode http
1289 option persist
1290 cookie SERVERID
1291 dispatch 192.168.1.100:80
1292 server web1 192.168.1.1:80 cookie server01
1293 server web2 192.168.1.2:80 cookie server02
1294 redispatch # send back to dispatch in case of connection failure
willy tarreaueedaa9f2005-12-17 14:08:03 +01001295
1296
willy tarreau34f45302006-04-15 21:37:14 +020012973.3) Assigning different weights to servers
1298-------------------------------------------
1299Sometimes you will need to bring new servers to increase your server farm's
1300capacity, but the new server will be either smaller (emergency use of anything
1301that fits) or bigger (when investing in new hardware). For this reason, it
1302might be wise to be able to send more clients to biggest servers. Till version
13031.2.11, it was necessary to replicate the same server multiple times in the
1304configuration. Starting with 1.2.12, the 'weight' option is available. HAProxy
1305then computes the most homogenous possible map of servers based on their
willy tarreau532bb552006-05-13 18:40:37 +02001306weights so that the load gets distributed as smoothly as possible among them.
1307The weight, between 1 and 256, should reflect one server's capacity relative to
1308others. Weight 1 represents the lowest frequency and 256 the highest. This way,
1309if a server fails, the remaining capacities are still respected.
willy tarreau34f45302006-04-15 21:37:14 +02001310
1311Example :
1312---------
1313# fair distribution among two opterons and one old pentium3
1314
1315 listen web_appl 0.0.0.0:80
1316 mode http
1317 cookie SERVERID insert nocache indirect
1318 balance roundrobin
1319 server pentium3-800 192.168.1.1:80 cookie server01 weight 8 check
1320 server opteron-2.0G 192.168.1.2:80 cookie server02 weight 20 check
1321 server opteron-2.4G 192.168.1.3:80 cookie server03 weight 24 check
1322 server web-backup1 192.168.2.1:80 cookie server04 check backup
1323 server web-excuse 192.168.3.1:80 check backup
1324
1325Notes :
1326-------
1327 - if unspecified, the default weight is 1
1328
1329 - the weight does not impact health checks, so it is cleaner to use weights
1330 than replicating the same server several times
1331
1332 - weights also work on backup servers if the 'allbackups' option is used
1333
1334 - the weights also apply to the source address load balancing
1335 ('balance source').
1336
1337 - whatever the weights, the first server will always be assigned first. This
1338 is helpful for troubleshooting.
1339
1340 - for the purists, the map calculation algorithm gives precedence to first
1341 server, so the map is the most uniform when servers are declared in
1342 ascending order relative to their weights.
1343
willy tarreau532bb552006-05-13 18:40:37 +02001344The load distribution will follow exactly this sequence :
1345
1346 Request| 1 1 1 1
1347 number | 1 2 3 4 5 6 7 8 9 0 1 2 3
1348 --------+---------------------------
1349 p3-800 | X . . . . . . X . . . . .
1350 opt-20 | . X . X . X . . . X . X .
1351 opt-24 | . . X . X . X . X . X . X
1352
1353
13543.4) Limiting the number of concurrent sessions on each server
1355--------------------------------------------------------------
1356Some pre-forked servers such as Apache suffer from too many concurrent
1357sessions, because it's very expensive to run hundreds or thousands of
1358processes on one system. One solution is to increase the number of servers
1359and load-balance between them, but it is a problem when the only goal is
1360to resist to short surges.
1361
1362To solve this problem, a new feature was implemented in HAProxy 1.2.13.
1363It's a per-server 'maxconn', associated with a per-server and a per-proxy
1364queue. This transforms haproxy into a request buffer between the thousands of
1365clients and the few servers. On many circumstances, lowering the maxconn value
1366will increase the server's performance and decrease the overall response times
1367because the servers will be less congested.
1368
1369When a request tries to reach any server, the first non-saturated server is
1370used, respective to the load balancing algorithm. If all servers are saturated,
1371then the request gets queued into the instance's global queue. It will be
1372dequeued once a server will have freed a session and all previously queued
1373requests have been processed.
1374
1375If a request references a particular server (eg: source hashing, or persistence
1376cookie), and if this server is full, then the request will be queued into the
1377server's dedicated queue. This queue has higher priority than the global queue,
1378so it's easier for already registered users to enter the site than for new
1379users.
1380
1381For this, the logs have been enhanced to show the number of sessions per
1382server, the request's position in the queue and the time spent in the queue.
1383This helps doing capacity planning. See the 'logs' section below for more info.
1384
1385Example :
1386---------
1387 # be nice with P3 which only has 256 MB of RAM.
1388 listen web_appl 0.0.0.0:80
1389 maxconn 10000
1390 mode http
1391 cookie SERVERID insert nocache indirect
1392 balance roundrobin
1393 server pentium3-800 192.168.1.1:80 cookie s1 weight 8 maxconn 100 check
1394 server opteron-2.0G 192.168.1.2:80 cookie s2 weight 20 maxconn 300 check
1395 server opteron-2.4G 192.168.1.3:80 cookie s3 weight 24 maxconn 300 check
1396 server web-backup1 192.168.2.1:80 cookie s4 check maxconn 200 backup
1397 server web-excuse 192.168.3.1:80 check backup
1398
willy tarreauf76e6ca2006-05-21 21:09:55 +02001399
1400This was so much efficient at reducing the server's response time that some
1401users wanted to use low values to improve their server's performance. However,
1402they were not able anymore to handle very large loads because it was not
1403possible anymore to saturate the servers. For this reason, version 1.2.14 has
1404brought dynamic limitation with the addition of the parameter 'minconn'. When
1405this parameter is set along with maxconn, it will enable dynamic limitation
1406based on the instance's load. The maximum number of concurrent sessions on a
1407server will be proportionnal to the number of sessions on the instance relative
1408to its maxconn. A minimum of <minconn> will be allowed whatever the load. This
1409will ensure that servers will perform at their best level under normal loads,
1410while still handling surges when needed. The dynamic limit is computed like
1411this :
1412
1413 srv.dyn_limit = max(srv.minconn, srv.maxconn * inst.sess / inst.maxconn)
1414
1415Example :
1416---------
1417 # be nice with P3 which only has 256 MB of RAM.
1418 listen web_appl 0.0.0.0:80
1419 maxconn 10000
1420 mode http
1421 cookie SERVERID insert nocache indirect
1422 balance roundrobin
1423 server pentium3-800 192.168.1.1:80 cookie s1 weight 8 minconn 10 maxconn 100 check
1424 server opteron-2.0G 192.168.1.2:80 cookie s2 weight 20 minconn 30 maxconn 300 check
1425 server opteron-2.4G 192.168.1.3:80 cookie s3 weight 24 minconn 30 maxconn 300 check
1426 server web-backup1 192.168.2.1:80 cookie s4 check maxconn 200 backup
1427 server web-excuse 192.168.3.1:80 check backup
1428
1429In the example above, the server 'pentium3-800' will receive at most 100
1430simultaneous sessions when the proxy instance will reach 10000 sessions, and
1431will receive only 10 simultaneous sessions when the proxy will be under 1000
1432sessions.
1433
Elijah Epifanovacafc5f2007-10-25 20:15:38 +02001434It is possible to limit server queue length in order to rebalance excess
1435sessions between less busy application servers IF session affinity isn't
1436hard functional requirement (for example it just gives huge performance boost
1437by keeping server-local caches hot and compact). 'maxqueue' option sets a
1438queue limit on a server, as in example below:
1439
1440... (just the same as in example above)
1441 server pentium3-800 192.168.1.1:80 cookie s1 weight 8 minconn 10 maxconn 100 check maxqueue 50
1442 server opteron-2.0G 192.168.1.2:80 cookie s2 weight 20 minconn 30 maxconn 300 check maxqueue 200
1443 server opteron-2.4G 192.168.1.3:80 cookie s3 weight 24 minconn 30 maxconn 300 check
1444
1445Absence of 'maxqueue' option means unlimited queue. When queue gets filled
1446up to 'maxqueue' client session is moved from server-local queue to a global
1447one.
1448
willy tarreau532bb552006-05-13 18:40:37 +02001449Notes :
1450-------
1451 - The requests will not stay indefinitely in the queue, they follow the
1452 'contimeout' parameter, and if a request cannot be dequeued within this
1453 timeout because the server is saturated or because the queue is filled,
1454 the session will expire with a 503 error.
1455
willy tarreauf76e6ca2006-05-21 21:09:55 +02001456 - if only <minconn> is specified, it has the same effect as <maxconn>
1457
willy tarreau532bb552006-05-13 18:40:37 +02001458 - setting too low values for maxconn might improve performance but might also
1459 allow slow users to block access to the server for other users.
1460
willy tarreau34f45302006-04-15 21:37:14 +02001461
willy tarreaue0bdd622006-05-21 20:51:54 +020014623.5) Dropping aborted requests
1463------------------------------
1464In presence of very high loads, the servers will take some time to respond. The
1465per-proxy's connection queue will inflate, and the response time will increase
1466respective to the size of the queue times the average per-session response
1467time. When clients will wait for more than a few seconds, they will often hit
1468the 'STOP' button on their browser, leaving a useless request in the queue, and
1469slowing down other users.
1470
1471As there is no way to distinguish between a full STOP and a simple
1472shutdown(SHUT_WR) on the client side, HTTP agents should be conservative and
1473consider that the client might only have closed its output channel while
1474waiting for the response. However, this introduces risks of congestion when
1475lots of users do the same, and is completely useless nowadays because probably
1476no client at all will close the session while waiting for the response. Some
1477HTTP agents support this (Squid, Apache, HAProxy), and others do not (TUX, most
1478hardware-based load balancers). So the probability for a closed input channel
1479to represent a user hitting the 'STOP' button is close to 100%, and it is very
1480tempting to be able to abort the session early without polluting the servers.
1481
1482For this reason, a new option "abortonclose" was introduced in version 1.2.14.
1483By default (without the option) the behaviour is HTTP-compliant. But when the
1484option is specified, a session with an incoming channel closed will be aborted
1485if it's still possible, which means that it's either waiting for a connect() to
1486establish or it is queued waiting for a connection slot. This considerably
1487reduces the queue size and the load on saturated servers when users are tempted
1488to click on STOP, which in turn reduces the response time for other users.
1489
1490Example :
1491---------
1492 listen web_appl 0.0.0.0:80
1493 maxconn 10000
1494 mode http
1495 cookie SERVERID insert nocache indirect
1496 balance roundrobin
1497 server web1 192.168.1.1:80 cookie s1 weight 10 maxconn 100 check
1498 server web2 192.168.1.2:80 cookie s2 weight 10 maxconn 100 check
1499 server web3 192.168.1.3:80 cookie s3 weight 10 maxconn 100 check
1500 server bck1 192.168.2.1:80 cookie s4 check maxconn 200 backup
1501 option abortonclose
1502
1503
willy tarreaueedaa9f2005-12-17 14:08:03 +010015044) Additionnal features
1505=======================
1506
willy tarreau481132e2006-05-21 21:43:10 +02001507Other features are available. They are transparent mode, event logging, header
1508rewriting/filtering, and the status as an HTML page.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001509
willy tarreauc5f73ed2005-12-18 01:26:38 +01001510
willy tarreau0174f312005-12-18 01:02:42 +010015114.1) Network features
willy tarreaueedaa9f2005-12-17 14:08:03 +01001512---------------------
willy tarreau0174f312005-12-18 01:02:42 +010015134.1.1) Transparent mode
1514-----------------------
willy tarreaueedaa9f2005-12-17 14:08:03 +01001515In HTTP mode, the 'transparent' keyword allows to intercept sessions which are
1516routed through the system hosting the proxy. This mode was implemented as a
1517replacement for the 'dispatch' mode, since connections without cookie will be
1518sent to the original address while known cookies will be sent to the servers.
1519This mode implies that the system can redirect sessions to a local port.
1520
1521Example :
1522---------
1523 listen http_proxy 0.0.0.0:65000
willy tarreauc5f73ed2005-12-18 01:26:38 +01001524 mode http
1525 transparent
1526 cookie SERVERID
1527 server server01 192.168.1.1:80
1528 server server02 192.168.1.2:80
willy tarreaueedaa9f2005-12-17 14:08:03 +01001529
1530 # iptables -t nat -A PREROUTING -i eth0 -p tcp -d 192.168.1.100 \
1531 --dport 80 -j REDIRECT --to-ports 65000
1532
1533Note :
1534------
1535If the port is left unspecified on the server, the port the client connected to
1536will be used. This allows to relay a full port range without using transparent
1537mode nor thousands of file descriptors, provided that the system can redirect
1538sessions to local ports.
1539
1540Example :
1541---------
1542 # redirect all ports to local port 65000, then forward to the server on the
1543 # original port.
1544 listen http_proxy 0.0.0.0:65000
willy tarreauc5f73ed2005-12-18 01:26:38 +01001545 mode tcp
1546 server server01 192.168.1.1 check port 60000
1547 server server02 192.168.1.2 check port 60000
willy tarreaueedaa9f2005-12-17 14:08:03 +01001548
1549 # iptables -t nat -A PREROUTING -i eth0 -p tcp -d 192.168.1.100 \
1550 -j REDIRECT --to-ports 65000
1551
willy tarreau0174f312005-12-18 01:02:42 +010015524.1.2) Per-server source address binding
1553----------------------------------------
1554As of versions 1.1.30 and 1.2.3, it is possible to specify a particular source
1555to reach each server. This is useful when reaching backup servers from a
1556different LAN, or to use an alternate path to reach the same server. It is also
1557usable to provide source load-balancing for outgoing connections. Obviously,
1558the same source address is used to send health-checks.
1559
1560Example :
1561---------
1562 # use a particular source to reach both servers
1563 listen http_proxy 0.0.0.0:65000
willy tarreauc5f73ed2005-12-18 01:26:38 +01001564 mode http
1565 balance roundrobin
1566 server server01 192.168.1.1:80 source 192.168.2.13
1567 server server02 192.168.1.2:80 source 192.168.2.13
willy tarreau0174f312005-12-18 01:02:42 +01001568
1569Example :
1570---------
1571 # use a particular source to reach each servers
1572 listen http_proxy 0.0.0.0:65000
willy tarreauc5f73ed2005-12-18 01:26:38 +01001573 mode http
1574 balance roundrobin
1575 server server01 192.168.1.1:80 source 192.168.1.1
1576 server server02 192.168.2.1:80 source 192.168.2.1
willy tarreau0174f312005-12-18 01:02:42 +01001577
1578Example :
1579---------
1580 # provide source load-balancing to reach the same proxy through 2 WAN links
1581 listen http_proxy 0.0.0.0:65000
willy tarreauc5f73ed2005-12-18 01:26:38 +01001582 mode http
1583 balance roundrobin
1584 server remote-proxy-way1 192.168.1.1:3128 source 192.168.2.1
1585 server remote-proxy-way2 192.168.1.1:3128 source 192.168.3.1
willy tarreau0174f312005-12-18 01:02:42 +01001586
1587Example :
1588---------
1589 # force a TCP connection to bind to a specific port
1590 listen http_proxy 0.0.0.0:2000
willy tarreauc5f73ed2005-12-18 01:26:38 +01001591 mode tcp
1592 balance roundrobin
1593 server srv1 192.168.1.1:80 source 192.168.2.1:20
1594 server srv2 192.168.1.2:80 source 192.168.2.1:20
willy tarreau0174f312005-12-18 01:02:42 +01001595
willy tarreaub952e1d2005-12-18 01:31:20 +010015964.1.3) TCP keep-alive
1597---------------------
1598With version 1.2.7, it becomes possible to enable TCP keep-alives on both the
1599client and server sides. This makes it possible to prevent long sessions from
1600expiring on external layer 4 components such as firewalls and load-balancers.
1601It also allows the system to terminate dead sessions when no timeout has been
1602set (not recommanded). The proxy cannot set the keep-alive probes intervals nor
1603maximal count, consult your operating system manual for this. There are 3
1604options to enable TCP keep-alive :
1605
1606 option tcpka # enables keep-alive both on client and server side
1607 option clitcpka # enables keep-alive only on client side
1608 option srvtcpka # enables keep-alive only on server side
1609
Alexandre Cassen87ea5482007-10-11 20:48:58 +020016104.1.4) TCP lingering
1611--------------------
1612It is possible to disable the system's lingering of data unacked by the client
1613at the end of a session. This is sometimes required when haproxy is used as a
1614front-end with lots of unreliable clients, and you observe thousands of sockets
1615in the FIN_WAIT state on the machine. This may be used in a frontend to affect
1616the client-side connection, as well as in a backend for the server-side
1617connection :
1618
1619 option nolinger # disables data lingering
1620
willy tarreaueedaa9f2005-12-17 14:08:03 +01001621
16224.2) Event logging
1623------------------
willy tarreauc5f73ed2005-12-18 01:26:38 +01001624
1625HAProxy's strength certainly lies in its precise logs. It probably provides the
1626finest level of information available for such a product, which is very
1627important for troubleshooting complex environments. Standard log information
1628include client ports, TCP/HTTP state timers, precise session state at
1629termination and precise termination cause, information about decisions to
1630direct trafic to a server, and of course the ability to capture arbitrary
1631headers.
1632
1633In order to improve administrators reactivity, it offers a great transparency
1634about encountered problems, both internal and external, and it is possible to
1635send logs to different sources at the same time with different level filters :
1636
1637 - global process-level logs (system errors, start/stop, etc..)
1638 - per-listener system and internal errors (lack of resource, bugs, ...)
1639 - per-listener external troubles (servers up/down, max connections)
1640 - per-listener activity (client connections), either at the establishment or
1641 at the termination.
1642
1643The ability to distribute different levels of logs to different log servers
1644allow several production teams to interact and to fix their problems as soon
1645as possible. For example, the system team might monitor system-wide errors,
1646while the application team might be monitoring the up/down for their servers in
1647real time, and the security team might analyze the activity logs with one hour
1648delay.
1649
willy tarreauc1cae632005-12-17 14:12:23 +010016504.2.1) Log levels
1651-----------------
willy tarreau197e8ec2005-12-17 14:10:59 +01001652TCP and HTTP connections can be logged with informations such as date, time,
1653source IP address, destination address, connection duration, response times,
1654HTTP request, the HTTP return code, number of bytes transmitted, the conditions
1655in which the session ended, and even exchanged cookies values, to track a
1656particular user's problems for example. All messages are sent to up to two
1657syslog servers. Consult section 1.1 for more info about log facilities. The
1658syntax follows :
willy tarreaueedaa9f2005-12-17 14:08:03 +01001659
willy tarreau197e8ec2005-12-17 14:10:59 +01001660 log <address_1> <facility_1> [max_level_1]
1661 log <address_2> <facility_2> [max_level_2]
1662or
willy tarreaueedaa9f2005-12-17 14:08:03 +01001663 log global
1664
willy tarreau197e8ec2005-12-17 14:10:59 +01001665Note :
1666------
1667The particular syntax 'log global' means that the same log configuration as the
1668'global' section will be used.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001669
willy tarreau197e8ec2005-12-17 14:10:59 +01001670Example :
willy tarreaueedaa9f2005-12-17 14:08:03 +01001671---------
1672 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001673 mode http
1674 log 192.168.2.200 local3
1675 log 192.168.2.201 local4
willy tarreaueedaa9f2005-12-17 14:08:03 +01001676
willy tarreauc1cae632005-12-17 14:12:23 +010016774.2.2) Log format
1678-----------------
1679By default, connections are logged at the TCP level, as soon as the session
1680establishes between the client and the proxy. By enabling the 'tcplog' option,
1681the proxy will wait until the session ends to generate an enhanced log
1682containing more information such as session duration and its state during the
willy tarreau532bb552006-05-13 18:40:37 +02001683disconnection. The number of remaining session after disconnection is also
1684indicated (for the server, the listener, and the process).
willy tarreauc1cae632005-12-17 14:12:23 +01001685
willy tarreauc5f73ed2005-12-18 01:26:38 +01001686Example of TCP logging :
1687------------------------
willy tarreau982249e2005-12-18 00:57:06 +01001688 listen relais-tcp 0.0.0.0:8000
willy tarreauc5f73ed2005-12-18 01:26:38 +01001689 mode tcp
1690 option tcplog
1691 log 192.168.2.200 local3
willy tarreau982249e2005-12-18 00:57:06 +01001692
willy tarreau532bb552006-05-13 18:40:37 +02001693>>> haproxy[18989]: 127.0.0.1:34550 [15/Oct/2003:15:24:28] relais-tcp Srv1 0/0/5007 0 -- 1/1/1 0/0
willy tarreauc5f73ed2005-12-18 01:26:38 +01001694
willy tarreau532bb552006-05-13 18:40:37 +02001695 Field Format Example
willy tarreauc5f73ed2005-12-18 01:26:38 +01001696
willy tarreau532bb552006-05-13 18:40:37 +02001697 1 process_name '[' pid ']:' haproxy[18989]:
1698 2 client_ip ':' client_port 127.0.0.1:34550
1699 3 '[' date ']' [15/Oct/2003:15:24:28]
1700 4 listener_name relais-tcp
1701 5 server_name Srv1
1702 6 queue_time '/' connect_time '/' total_time 0/0/5007
1703 7 bytes_read 0
1704 8 termination_state --
1705 9 srv_conn '/' listener_conn '/' process_conn 1/1/1
1706 10 position in srv_queue / listener_queue 0/0
1707
willy tarreau982249e2005-12-18 00:57:06 +01001708
willy tarreauc1cae632005-12-17 14:12:23 +01001709Another option, 'httplog', provides more detailed information about HTTP
1710contents, such as the request and some cookies. In the event where an external
1711component would establish frequent connections to check the service, logs may be
1712full of useless lines. So it is possible not to log any session which didn't
1713transfer any data, by the setting of the 'dontlognull' option. This only has
1714effect on sessions which are established then closed.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001715
willy tarreauc5f73ed2005-12-18 01:26:38 +01001716Example of HTTP logging :
1717-------------------------
willy tarreaueedaa9f2005-12-17 14:08:03 +01001718 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001719 mode http
1720 option httplog
1721 option dontlognull
1722 log 192.168.2.200 local3
1723
Krzysztof Piotr Oledzki25b501a2008-01-06 16:36:16 +01001724>>> haproxy[674]: 127.0.0.1:33319 [15/Oct/2003:08:31:57] relais-http Srv1 9/0/7/147/723 200 243 - - ---- 34/34/15/8/3 0/0 "HEAD / HTTP/1.0"
willy tarreaueedaa9f2005-12-17 14:08:03 +01001725
willy tarreauc5f73ed2005-12-18 01:26:38 +01001726More complete example
Krzysztof Piotr Oledzki25b501a2008-01-06 16:36:16 +01001727 haproxy[18989]: 10.0.0.1:34552 [15/Oct/2003:15:26:31] relais-http Srv1 3183/-1/-1/-1/11215 503 0 - - SC-- 205/202/150/137/+4 0/0 {w.ods.org|Mozilla} {} "HEAD / HTTP/1.0"
willy tarreauc5f73ed2005-12-18 01:26:38 +01001728
Krzysztof Piotr Oledzki25b501a2008-01-06 16:36:16 +01001729 Field Format Example
willy tarreauc5f73ed2005-12-18 01:26:38 +01001730
Krzysztof Piotr Oledzki25b501a2008-01-06 16:36:16 +01001731 1 process_name '[' pid ']:' haproxy[18989]:
1732 2 client_ip ':' client_port 10.0.0.1:34552
1733 3 '[' date ']' [15/Oct/2003:15:26:31]
1734 4 listener_name relais-http
1735 5 server_name Srv1
1736 6 Tq '/' Tw '/' Tc '/' Tr '/' Tt 3183/-1/-1/-1/11215
1737 7 HTTP_return_code 503
1738 8 bytes_read 0
1739 9 captured_request_cookie -
1740 10 captured_response_cookie -
1741 11 termination_state SC--
1742 12 actconn '/' feconn '/' beconn '/' srv_conn '/' retries 205/202/150/137/+4
1743 13 position in srv_queue / listener_queue 0/0
1744 14 '{' captured_request_headers '}' {w.ods.org|Mozilla}
1745 15 '{' captured_response_headers '}' {}
1746 16 '"' HTTP_request '"' "HEAD / HTTP/1.0"
willy tarreauc5f73ed2005-12-18 01:26:38 +01001747
1748Note for log parsers: the URI is ALWAYS the end of the line starting with the
1749 first double quote '"'.
willy tarreau982249e2005-12-18 00:57:06 +01001750
Krzysztof Piotr Oledzki25b501a2008-01-06 16:36:16 +01001751The retries count may have additional '+' sign means that the connection had been
1752redispatched from one server to another shortly before retries limit (retries 4
1753in above example) was depleted.
1754
willy tarreau982249e2005-12-18 00:57:06 +01001755The problem when logging at end of connection is that you have no clue about
1756what is happening during very long sessions. To workaround this problem, a
1757new option 'logasap' has been introduced in 1.1.28/1.2.1. When specified, the
1758proxy will log as soon as possible, just before data transfer begins. This means
1759that in case of TCP, it will still log the connection status to the server, and
1760in case of HTTP, it will log just after processing the server headers. In this
1761case, the number of bytes reported is the number of header bytes sent to the
1762client.
1763
1764In order to avoid confusion with normal logs, the total time field and the
Krzysztof Piotr Oledzki25b501a2008-01-06 16:36:16 +01001765number of bytes are prefixed with a '+' sign which means that real numbers are
willy tarreau982249e2005-12-18 00:57:06 +01001766certainly bigger.
1767
1768Example :
1769---------
1770
1771 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001772 mode http
1773 option httplog
1774 option dontlognull
1775 option logasap
1776 log 192.168.2.200 local3
willy tarreau982249e2005-12-18 00:57:06 +01001777
Krzysztof Piotr Oledzki25b501a2008-01-06 16:36:16 +01001778>>> haproxy[674]: 127.0.0.1:33320 [15/Oct/2003:08:32:17] relais-http Srv1 9/10/7/14/+30 200 +243 - - ---- 3/1/1/1/0 1/0 "GET /image.iso HTTP/1.0"
willy tarreau982249e2005-12-18 00:57:06 +01001779
willy tarreauc1cae632005-12-17 14:12:23 +010017804.2.3) Timing events
1781--------------------
1782Timers provide a great help in trouble shooting network problems. All values
1783are reported in milliseconds (ms). In HTTP mode, four control points are
willy tarreau532bb552006-05-13 18:40:37 +02001784reported under the form 'Tq/Tw/Tc/Tr/Tt' :
willy tarreauc1cae632005-12-17 14:12:23 +01001785
1786 - Tq: total time to get the client request.
1787 It's the time elapsed between the moment the client connection was accepted
1788 and the moment the proxy received the last HTTP header. The value '-1'
1789 indicates that the end of headers (empty line) has never been seen.
1790
willy tarreau532bb552006-05-13 18:40:37 +02001791 - Tw: total time spent in the queues waiting for a connection slot. It
1792 accounts for listener's queue as well as the server's queue, and depends
1793 on the queue size, and the time needed for the server to complete previous
1794 sessions. The value '-1' means that the request was killed before reaching
1795 the queue.
1796
willy tarreauc1cae632005-12-17 14:12:23 +01001797 - Tc: total time to establish the TCP connection to the server.
1798 It's the time elapsed between the moment the proxy sent the connection
1799 request, and the moment it was acknowledged, or between the TCP SYN packet
1800 and the matching SYN/ACK in return. The value '-1' means that the
1801 connection never established.
1802
1803 - Tr: server response time. It's the time elapsed between the moment the
1804 TCP connection was established to the server and the moment it send its
1805 complete response header. It purely shows its request processing time,
1806 without the network overhead due to the data transmission. The value '-1'
1807 means that the last the response header (empty line) was never seen.
1808
1809 - Tt: total session duration time, between the moment the proxy accepted it
willy tarreau982249e2005-12-18 00:57:06 +01001810 and the moment both ends were closed. The exception is when the 'logasap'
willy tarreau532bb552006-05-13 18:40:37 +02001811 option is specified. In this case, it only equals (Tq+Tw+Tc+Tr), and is
willy tarreau982249e2005-12-18 00:57:06 +01001812 prefixed with a '+' sign. From this field, we can deduce Td, the data
1813 transmission time, by substracting other timers when valid :
willy tarreauc1cae632005-12-17 14:12:23 +01001814
willy tarreau532bb552006-05-13 18:40:37 +02001815 Td = Tt - (Tq + Tw + Tc + Tr)
willy tarreauc1cae632005-12-17 14:12:23 +01001816
1817 Timers with '-1' values have to be excluded from this equation.
1818
willy tarreau532bb552006-05-13 18:40:37 +02001819In TCP mode ('option tcplog'), only Tw, Tc and Tt are reported.
willy tarreauc1cae632005-12-17 14:12:23 +01001820
1821These timers provide precious indications on trouble causes. Since the TCP
1822protocol defines retransmit delays of 3, 6, 12... seconds, we know for sure
1823that timers close to multiples of 3s are nearly always related to packets lost
1824due to network problems (wires or negociation). Moreover, if <Tt> is close to
1825a timeout value specified in the configuration, it often means that a session
1826has been aborted on time-out.
1827
1828Most common cases :
1829
1830 - If Tq is close to 3000, a packet has probably been lost between the client
1831 and the proxy.
1832 - If Tc is close to 3000, a packet has probably been lost between the server
1833 and the proxy during the server connection phase. This one should always be
1834 very low (less than a few tens).
1835 - If Tr is nearly always lower than 3000 except some rare values which seem to
1836 be the average majored by 3000, there are probably some packets lost between
1837 the proxy and the server.
1838 - If Tt is often slightly higher than a time-out, it's often because the
1839 client and the server use HTTP keep-alive and the session is maintained
1840 after the response ends. Se further for how to disable HTTP keep-alive.
1841
1842Other cases ('xx' means any value to be ignored) :
willy tarreau532bb552006-05-13 18:40:37 +02001843 -1/xx/xx/xx/Tt: the client was not able to send its complete request in time,
1844 or that it aborted it too early.
1845 Tq/-1/xx/xx/Tt: it was not possible to process the request, maybe because
1846 servers were out of order.
1847 Tq/Tw/-1/xx/Tt: the connection could not establish on the server. Either it
1848 refused it or it timed out after Tt-(Tq+Tw) ms.
1849 Tq/Tw/Tc/-1/Tt: the server has accepted the connection but did not return a
1850 complete response in time, or it closed its connexion
1851 unexpectedly, after Tt-(Tq+Tw+Tc) ms.
willy tarreauc1cae632005-12-17 14:12:23 +01001852
18534.2.4) Session state at disconnection
1854-------------------------------------
willy tarreauc5f73ed2005-12-18 01:26:38 +01001855TCP and HTTP logs provide a session completion indicator in the
1856<termination_state> field, just before the number of active
1857connections. It is 2-characters long in TCP, and 4-characters long in
1858HTTP, each of which has a special meaning :
1859
willy tarreau197e8ec2005-12-17 14:10:59 +01001860 - On the first character, a code reporting the first event which caused the
1861 session to terminate :
willy tarreaueedaa9f2005-12-17 14:08:03 +01001862
willy tarreauc5f73ed2005-12-18 01:26:38 +01001863 C : the TCP session was unexpectedly aborted by the client.
1864
1865 S : the TCP session was unexpectedly aborted by the server, or the
1866 server explicitly refused it.
1867
1868 P : the session was prematurely aborted by the proxy, because of a
1869 connection limit enforcement, because a DENY filter was matched,
1870 or because of a security check which detected and blocked a
1871 dangerous error in server response which might have caused
1872 information leak (eg: cacheable cookie).
1873
1874 R : a resource on the proxy has been exhausted (memory, sockets, source
1875 ports, ...). Usually, this appears during the connection phase, and
1876 system logs should contain a copy of the precise error.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001877
willy tarreauc5f73ed2005-12-18 01:26:38 +01001878 I : an internal error was identified by the proxy during a self-check.
1879 This should NEVER happen, and you are encouraged to report any log
1880 containing this, because this is a bug.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001881
willy tarreauc5f73ed2005-12-18 01:26:38 +01001882 c : the client-side time-out expired first.
1883
1884 s : the server-side time-out expired first.
1885
1886 - : normal session completion.
1887
1888 - on the second character, the TCP/HTTP session state when it was closed :
1889
1890 R : waiting for complete REQUEST from the client (HTTP only). Nothing
1891 was sent to any server.
1892
willy tarreau532bb552006-05-13 18:40:37 +02001893 Q : waiting in the QUEUE for a connection slot. This can only happen on
1894 servers which have a 'maxconn' parameter set. No connection attempt
1895 was made to any server.
1896
willy tarreauc5f73ed2005-12-18 01:26:38 +01001897 C : waiting for CONNECTION to establish on the server. The server might
1898 at most have noticed a connection attempt.
1899
1900 H : waiting for, receiving and processing server HEADERS (HTTP only).
1901
1902 D : the session was in the DATA phase.
1903
1904 L : the proxy was still transmitting LAST data to the client while the
1905 server had already finished.
1906
Willy Tarreau2272dc12006-09-03 10:19:38 +02001907 T : the request was tarpitted. It has been held open on with the client
Willy Tarreau08fa2e32006-09-03 10:47:37 +02001908 during the whole contimeout duration or untill the client closed.
Willy Tarreau2272dc12006-09-03 10:19:38 +02001909
willy tarreauc5f73ed2005-12-18 01:26:38 +01001910 - : normal session completion after end of data transfer.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001911
willy tarreau197e8ec2005-12-17 14:10:59 +01001912 - the third character tells whether the persistence cookie was provided by
willy tarreauc1cae632005-12-17 14:12:23 +01001913 the client (only in HTTP mode) :
willy tarreaueedaa9f2005-12-17 14:08:03 +01001914
willy tarreauc5f73ed2005-12-18 01:26:38 +01001915 N : the client provided NO cookie. This is usually the case on new
1916 connections.
1917
1918 I : the client provided an INVALID cookie matching no known
1919 server. This might be caused by a recent configuration change,
1920 mixed cookies between HTTP/HTTPS sites, or an attack.
1921
1922 D : the client provided a cookie designating a server which was DOWN,
1923 so either the 'persist' option was used and the client was sent to
1924 this server, or it was not set and the client was redispatched to
1925 another server.
1926
1927 V : the client provided a valid cookie, and was sent to the associated
1928 server.
1929
1930 - : does not apply (no cookie set in configuration).
willy tarreaueedaa9f2005-12-17 14:08:03 +01001931
willy tarreau197e8ec2005-12-17 14:10:59 +01001932 - the last character reports what operations were performed on the persistence
willy tarreauc1cae632005-12-17 14:12:23 +01001933 cookie returned by the server (only in HTTP mode) :
willy tarreaueedaa9f2005-12-17 14:08:03 +01001934
willy tarreauc5f73ed2005-12-18 01:26:38 +01001935 N : NO cookie was provided by the server, and none was inserted either.
1936
1937 I : no cookie was provided by the server, and the proxy INSERTED one.
1938
willy tarreau197e8ec2005-12-17 14:10:59 +01001939 P : a cookie was PROVIDED by the server and transmitted as-is.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001940
willy tarreauc5f73ed2005-12-18 01:26:38 +01001941 R : the cookie provided by the server was REWRITTEN by the proxy.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001942
willy tarreauc5f73ed2005-12-18 01:26:38 +01001943 D : the cookie provided by the server was DELETED by the proxy.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001944
willy tarreauc5f73ed2005-12-18 01:26:38 +01001945 - : does not apply (no cookie set in configuration).
willy tarreaueedaa9f2005-12-17 14:08:03 +01001946
willy tarreauc5f73ed2005-12-18 01:26:38 +01001947The combination of the two first flags give a lot of information about what was
1948happening when the session terminated. It can be helpful to detect server
1949saturation, network troubles, local system resource starvation, attacks, etc...
willy tarreaueedaa9f2005-12-17 14:08:03 +01001950
willy tarreauc5f73ed2005-12-18 01:26:38 +01001951The most common termination flags combinations are indicated here.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001952
willy tarreauc5f73ed2005-12-18 01:26:38 +01001953 Flags Reason
1954 CR The client aborted before sending a full request. Most probably the
1955 request was done by hand using a telnet client, and aborted early.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001956
willy tarreauc5f73ed2005-12-18 01:26:38 +01001957 cR The client timed out before sending a full request. This is sometimes
1958 caused by too large TCP MSS values on the client side for PPPoE
1959 networks which cannot transport full-sized packets, or by clients
1960 sending requests by hand and not typing fast enough.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001961
willy tarreauc5f73ed2005-12-18 01:26:38 +01001962 SC The server explicitly refused the connection (the proxy received a
1963 TCP RST or an ICMP in return). Under some circumstances, it can
1964 also be the network stack telling the proxy that the server is
1965 unreachable (eg: no route, or no ARP response on local network).
willy tarreau982249e2005-12-18 00:57:06 +01001966
willy tarreauc5f73ed2005-12-18 01:26:38 +01001967 sC The connection to the server did not complete during contimeout.
willy tarreau982249e2005-12-18 00:57:06 +01001968
willy tarreauc5f73ed2005-12-18 01:26:38 +01001969 PC The proxy refused to establish a connection to the server because the
1970 maxconn limit has been reached. The listener's maxconn parameter may
1971 be increased in the proxy configuration, as well as the global
1972 maxconn parameter.
willy tarreauc1cae632005-12-17 14:12:23 +01001973
willy tarreauc5f73ed2005-12-18 01:26:38 +01001974 RC A local resource has been exhausted (memory, sockets, source ports)
1975 preventing the connection to the server from establishing. The error
1976 logs will tell precisely what was missing. Anyway, this can only be
1977 solved by system tuning.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001978
willy tarreauc5f73ed2005-12-18 01:26:38 +01001979 cH The client timed out during a POST request. This is sometimes caused
1980 by too large TCP MSS values for PPPoE networks which cannot transport
1981 full-sized packets.
willy tarreauc1cae632005-12-17 14:12:23 +01001982
willy tarreau078c79a2006-05-13 12:23:58 +02001983 CH The client aborted while waiting for the server to start responding.
1984 It might be the server taking too long to respond or the client
1985 clicking the 'Stop' button too fast.
1986
1987 CQ The client aborted while its session was queued, waiting for a server
1988 with enough empty slots to accept it. It might be that either all the
1989 servers were saturated or the assigned server taking too long to
1990 respond.
1991
Willy Tarreau08fa2e32006-09-03 10:47:37 +02001992 CT The client aborted while its session was tarpitted.
1993
willy tarreau078c79a2006-05-13 12:23:58 +02001994 sQ The session spent too much time in queue and has been expired.
1995
willy tarreauc5f73ed2005-12-18 01:26:38 +01001996 SH The server aborted before sending its full headers, or it crashed.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001997
willy tarreauc5f73ed2005-12-18 01:26:38 +01001998 sH The server failed to reply during the srvtimeout delay, which
1999 indicates too long transactions, probably caused by back-end
2000 saturation. The only solutions are to fix the problem on the
2001 application or to increase the 'srvtimeout' parameter to support
2002 longer delays (at the risk of the client giving up anyway).
2003
2004 PR The proxy blocked the client's request, either because of an invalid
2005 HTTP syntax, in which case it returned an HTTP 400 error to the
2006 client, or because a deny filter matched, in which case it returned
2007 an HTTP 403 error.
2008
2009 PH The proxy blocked the server's response, because it was invalid,
2010 incomplete, dangerous (cache control), or matched a security filter.
2011 In any case, an HTTP 502 error is sent to the client.
2012
Willy Tarreau2272dc12006-09-03 10:19:38 +02002013 PT The proxy blocked the client's request and has tarpitted its
2014 connection before returning it a 500 server error. Nothing was sent
2015 to the server.
2016
willy tarreauc5f73ed2005-12-18 01:26:38 +01002017 cD The client did not read any data for as long as the clitimeout delay.
2018 This is often caused by network failures on the client side.
2019
2020 CD The client unexpectedly aborted during data transfer. This is either
2021 caused by a browser crash, or by a keep-alive session between the
2022 server and the client terminated first by the client.
2023
2024 sD The server did nothing during the srvtimeout delay. This is often
2025 caused by too short timeouts on L4 equipements before the server
2026 (firewalls, load-balancers, ...).
2027
20284.2.5) Non-printable characters
willy tarreau4302f492005-12-18 01:00:37 +01002029-------------------------------
2030As of version 1.1.29, non-printable characters are not sent as-is into log
2031files, but are converted to their two-digits hexadecimal representation,
2032prefixed by the character '#'. The only characters that can now be logged
2033without being escaped are between 32 and 126 (inclusive). Obviously, the
2034escape character '#' is also encoded to avoid any ambiguity. It is the same for
2035the character '"', as well as '{', '|' and '}' when logging headers.
2036
willy tarreauc5f73ed2005-12-18 01:26:38 +010020374.2.6) Capturing HTTP headers and cookies
2038-----------------------------------------
2039Version 1.1.23 brought cookie capture, and 1.1.29 the header capture. All this
2040is performed using the 'capture' keyword.
2041
2042Cookie capture makes it easy to track a complete user session. The syntax is :
2043
2044 capture cookie <cookie_prefix> len <capture_length>
2045
2046This will enable cookie capture from both requests and responses. This way,
2047it's easy to detect when a user switches to a new session for example, because
2048the server will reassign it a new cookie.
2049
2050The FIRST cookie whose name starts with <cookie_prefix> will be captured, and
2051logged as 'NAME=value', without exceeding <capture_length> characters (64 max).
2052When the cookie name is fixed and known, it's preferable to suffix '=' to it to
2053ensure that no other cookie will be logged.
2054
2055Examples :
2056----------
2057 # capture the first cookie whose name starts with "ASPSESSION"
2058 capture cookie ASPSESSION len 32
2059
2060 # capture the first cookie whose name is exactly "vgnvisitor"
2061 capture cookie vgnvisitor= len 32
2062
2063In the logs, the field preceeding the completion indicator contains the cookie
2064value as sent by the server, preceeded by the cookie value as sent by the
2065client. Each of these field is replaced with '-' when no cookie was seen or
2066when the option is disabled.
2067
2068Header captures have a different goal. They are useful to track unique request
2069identifiers set by a previous proxy, virtual host names, user-agents, POST
2070content-length, referrers, etc. In the response, one can search for information
2071about the response length, how the server asked the cache to behave, or an
2072object location during a redirection. As for cookie captures, it is both
2073possible to include request headers and response headers at the same time. The
2074syntax is :
willy tarreau4302f492005-12-18 01:00:37 +01002075
2076 capture request header <name> len <max length>
2077 capture response header <name> len <max length>
2078
2079Note: Header names are not case-sensitive.
2080
2081Examples:
2082---------
2083 # keep the name of the virtual server
2084 capture request header Host len 20
2085 # keep the amount of data uploaded during a POST
2086 capture request header Content-Length len 10
2087
2088 # note the expected cache behaviour on the response
2089 capture response header Cache-Control len 8
2090 # note the URL location during a redirection
2091 capture response header Location len 20
2092
2093Non-existant headers are logged as empty strings, and if one header appears more
2094than once, only its last occurence will be kept. Request headers are grouped
2095within braces '{' and '}' in the same order as they were declared, and delimited
2096with a vertical bar '|' without any space. Response headers follow the same
2097representation, but are displayed after a space following the request headers
2098block. These blocks are displayed just before the HTTP request in the logs.
willy tarreauc5f73ed2005-12-18 01:26:38 +01002099
willy tarreau4302f492005-12-18 01:00:37 +01002100Example :
2101
willy tarreauc5f73ed2005-12-18 01:26:38 +01002102 Config:
2103
2104 capture request header Host len 20
2105 capture request header Content-Length len 10
2106 capture request header Referer len 20
2107 capture response header Server len 20
2108 capture response header Content-Length len 10
2109 capture response header Cache-Control len 8
2110 capture response header Via len 20
2111 capture response header Location len 20
2112
2113 Log :
2114
willy tarreau532bb552006-05-13 18:40:37 +02002115 Aug 9 20:26:09 localhost haproxy[2022]: 127.0.0.1:34014 [09/Aug/2004:20:26:09] relais-http netcache 0/0/0/162/+162 200 +350 - - ---- 0/0/0 0/0 {fr.adserver.yahoo.co||http://fr.f416.mail.} {|864|private||} "GET http://fr.adserver.yahoo.com/"
2116 Aug 9 20:30:46 localhost haproxy[2022]: 127.0.0.1:34020 [09/Aug/2004:20:30:46] relais-http netcache 0/0/0/182/+182 200 +279 - - ---- 0/0/0 0/0 {w.ods.org||} {Formilux/0.1.8|3495|||} "GET http://w.ods.org/sytadin.html HTTP/1.1"
2117 Aug 9 20:30:46 localhost haproxy[2022]: 127.0.0.1:34028 [09/Aug/2004:20:30:46] relais-http netcache 0/0/2/126/+128 200 +223 - - ---- 0/0/0 0/0 {www.infotrafic.com||http://w.ods.org/syt} {Apache/2.0.40 (Red H|9068|||} "GET http://www.infotrafic.com/images/live/cartesidf/grandes/idf_ne.png HTTP/1.1"
willy tarreauc5f73ed2005-12-18 01:26:38 +01002118
2119
21204.2.7) Examples of logs
2121-----------------------
willy tarreau532bb552006-05-13 18:40:37 +02002122- haproxy[674]: 127.0.0.1:33319 [15/Oct/2003:08:31:57] relais-http Srv1 6559/0/7/147/6723 200 243 - - ---- 1/3/5 0/0 "HEAD / HTTP/1.0"
willy tarreauc5f73ed2005-12-18 01:26:38 +01002123 => long request (6.5s) entered by hand through 'telnet'. The server replied
2124 in 147 ms, and the session ended normally ('----')
2125
willy tarreau532bb552006-05-13 18:40:37 +02002126- haproxy[674]: 127.0.0.1:33319 [15/Oct/2003:08:31:57] relais-http Srv1 6559/1230/7/147/6870 200 243 - - ---- 99/239/324 0/9 "HEAD / HTTP/1.0"
2127 => Idem, but the request was queued in the global queue behind 9 other
2128 requests, and waited there for 1230 ms.
2129
2130- haproxy[674]: 127.0.0.1:33320 [15/Oct/2003:08:32:17] relais-http Srv1 9/0/7/14/+30 200 +243 - - ---- 1/3/3 0/0 "GET /image.iso HTTP/1.0"
willy tarreauc5f73ed2005-12-18 01:26:38 +01002131 => request for a long data transfer. The 'logasap' option was specified, so
2132 the log was produced just before transfering data. The server replied in
2133 14 ms, 243 bytes of headers were sent to the client, and total time from
2134 accept to first data byte is 30 ms.
2135
willy tarreau532bb552006-05-13 18:40:37 +02002136- haproxy[674]: 127.0.0.1:33320 [15/Oct/2003:08:32:17] relais-http Srv1 9/0/7/14/30 502 243 - - PH-- 0/2/3 0/0 "GET /cgi-bin/bug.cgi? HTTP/1.0"
willy tarreauc5f73ed2005-12-18 01:26:38 +01002137 => the proxy blocked a server response either because of an 'rspdeny' or
2138 'rspideny' filter, or because it blocked sensible information which risked
2139 being cached. In this case, the response is replaced with a '502 bad
2140 gateway'.
2141
willy tarreau532bb552006-05-13 18:40:37 +02002142- haproxy[18113]: 127.0.0.1:34548 [15/Oct/2003:15:18:55] relais-http <NOSRV> -1/-1/-1/-1/8490 -1 0 - - CR-- 0/2/2 0/0 ""
willy tarreauc5f73ed2005-12-18 01:26:38 +01002143 => the client never completed its request and aborted itself ('C---') after
2144 8.5s, while the proxy was waiting for the request headers ('-R--').
2145 Nothing was sent to the server.
2146
willy tarreau532bb552006-05-13 18:40:37 +02002147- haproxy[18113]: 127.0.0.1:34549 [15/Oct/2003:15:19:06] relais-http <NOSRV> -1/-1/-1/-1/50001 408 0 - - cR-- 2/2 0/0 ""
willy tarreauc5f73ed2005-12-18 01:26:38 +01002148 => The client never completed its request, which was aborted by the time-out
2149 ('c---') after 50s, while the proxy was waiting for the request headers ('-R--').
2150 Nothing was sent to the server, but the proxy could send a 408 return code
2151 to the client.
willy tarreau4302f492005-12-18 01:00:37 +01002152
willy tarreau532bb552006-05-13 18:40:37 +02002153- haproxy[18989]: 127.0.0.1:34550 [15/Oct/2003:15:24:28] relais-tcp Srv1 0/0/5007 0 cD 0/0/0 0/0
willy tarreauc5f73ed2005-12-18 01:26:38 +01002154 => This is a 'tcplog' entry. Client-side time-out ('c----') occured after 5s.
willy tarreau4302f492005-12-18 01:00:37 +01002155
willy tarreau532bb552006-05-13 18:40:37 +02002156- haproxy[18989]: 10.0.0.1:34552 [15/Oct/2003:15:26:31] relais-http Srv1 3183/-1/-1/-1/11215 503 0 - - SC-- 115/202/205 0/0 "HEAD / HTTP/1.0"
willy tarreauc5f73ed2005-12-18 01:26:38 +01002157 => The request took 3s to complete (probably a network problem), and the
2158 connection to the server failed ('SC--') after 4 attemps of 2 seconds
2159 (config says 'retries 3'), then a 503 error code was sent to the client.
willy tarreau532bb552006-05-13 18:40:37 +02002160 There were 115 connections on this server, 202 connections on this proxy,
2161 and 205 on the global process. It is possible that the server refused the
2162 connection because of too many already established.
willy tarreau4302f492005-12-18 01:00:37 +01002163
willy tarreau4302f492005-12-18 01:00:37 +01002164
willy tarreau197e8ec2005-12-17 14:10:59 +010021654.3) HTTP header manipulation
2166-----------------------------
2167In HTTP mode, it is possible to rewrite, add or delete some of the request and
2168response headers based on regular expressions. It is also possible to block a
2169request or a response if a particular header matches a regular expression,
2170which is enough to stops most elementary protocol attacks, and to protect
2171against information leak from the internal network. But there is a limitation
2172to this : since haproxy's HTTP engine knows nothing about keep-alive, only
2173headers passed during the first request of a TCP session will be seen. All
2174subsequent headers will be considered data only and not analyzed. Furthermore,
2175haproxy doesn't touch data contents, it stops at the end of headers.
willy tarreaueedaa9f2005-12-17 14:08:03 +01002176
willy tarreau197e8ec2005-12-17 14:10:59 +01002177The syntax is :
2178 reqadd <string> to add a header to the request
2179 reqrep <search> <replace> to modify the request
2180 reqirep <search> <replace> same, but ignoring the case
2181 reqdel <search> to delete a header in the request
2182 reqidel <search> same, but ignoring the case
2183 reqallow <search> definitely allow a request if a header matches <search>
2184 reqiallow <search> same, but ignoring the case
2185 reqdeny <search> denies a request if a header matches <search>
2186 reqideny <search> same, but ignoring the case
2187 reqpass <search> ignore a header matching <search>
2188 reqipass <search> same, but ignoring the case
Willy Tarreau2272dc12006-09-03 10:19:38 +02002189 reqtarpit <search> tarpit a request matching <search>
2190 reqitarpit <search> same, but ignoring the case
willy tarreaueedaa9f2005-12-17 14:08:03 +01002191
willy tarreau197e8ec2005-12-17 14:10:59 +01002192 rspadd <string> to add a header to the response
2193 rsprep <search> <replace> to modify the response
2194 rspirep <search> <replace> same, but ignoring the case
2195 rspdel <search> to delete the response
2196 rspidel <search> same, but ignoring the case
willy tarreau982249e2005-12-18 00:57:06 +01002197 rspdeny <search> replaces a response with a HTTP 502 if a header matches <search>
2198 rspideny <search> same, but ignoring the case
willy tarreaueedaa9f2005-12-17 14:08:03 +01002199
2200
willy tarreau197e8ec2005-12-17 14:10:59 +01002201<search> is a POSIX regular expression (regex) which supports grouping through
2202parenthesis (without the backslash). Spaces and other delimiters must be
2203prefixed with a backslash ('\') to avoid confusion with a field delimiter.
2204Other characters may be prefixed with a backslash to change their meaning :
willy tarreaueedaa9f2005-12-17 14:08:03 +01002205
willy tarreau197e8ec2005-12-17 14:10:59 +01002206 \t for a tab
2207 \r for a carriage return (CR)
2208 \n for a new line (LF)
2209 \ to mark a space and differentiate it from a delimiter
2210 \# to mark a sharp and differentiate it from a comment
2211 \\ to use a backslash in a regex
2212 \\\\ to use a backslash in the text (*2 for regex, *2 for haproxy)
2213 \xXX to write the ASCII hex code XX as in the C language
willy tarreaueedaa9f2005-12-17 14:08:03 +01002214
2215
Willy Tarreau2272dc12006-09-03 10:19:38 +02002216<replace> contains the string to be used to replace the largest portion of text
willy tarreau197e8ec2005-12-17 14:10:59 +01002217matching the regex. It can make use of the special characters above, and can
2218reference a substring delimited by parenthesis in the regex, by the group
Willy Tarreau2272dc12006-09-03 10:19:38 +02002219numerical order from 0 to 9 (0 being the entire line). In this case, you would
2220write a backslash ('\') immediately followed by one digit indicating the group
2221position.
willy tarreaueedaa9f2005-12-17 14:08:03 +01002222
willy tarreau197e8ec2005-12-17 14:10:59 +01002223<string> represents the string which will systematically be added after the last
2224header line. It can also use special characters above.
willy tarreaueedaa9f2005-12-17 14:08:03 +01002225
willy tarreau197e8ec2005-12-17 14:10:59 +01002226Notes :
2227-------
2228 - the first line is considered as a header, which makes it possible to rewrite
2229 or filter HTTP requests URIs or response codes.
2230 - 'reqrep' is the equivalent of 'cliexp' in version 1.0, and 'rsprep' is the
2231 equivalent of 'srvexp' in 1.0. Those names are still supported but
2232 deprecated.
2233 - for performances reasons, the number of characters added to a request or to
2234 a response is limited to 4096 since version 1.1.5 (it was 256 before). This
2235 value is easy to modify in the code if needed (#define). If it is too short
2236 on occasional uses, it is possible to gain some space by removing some
2237 useless headers before adding new ones.
willy tarreau982249e2005-12-18 00:57:06 +01002238 - a denied request will generate an "HTTP 403 forbidden" response, while a
2239 denied response will generate an "HTTP 502 Bad gateway" response.
Willy Tarreau2272dc12006-09-03 10:19:38 +02002240 - a tarpitted request will be held open on the client side for a duration
Willy Tarreau08fa2e32006-09-03 10:47:37 +02002241 defined in the contimeout parameter, or untill the client aborts. Nothing
2242 will be sent to any server. When the timeout is reached, the proxy will
2243 reply with a 500 server error response so that the attacker does not
2244 suspect it has been tarpitted. The logs may report the 500, but the
2245 termination flags will indicate 'PT' in this case.
Willy Tarreau2272dc12006-09-03 10:19:38 +02002246
willy tarreaueedaa9f2005-12-17 14:08:03 +01002247
willy tarreau197e8ec2005-12-17 14:10:59 +01002248Examples :
2249----------
willy tarreauc5f73ed2005-12-18 01:26:38 +01002250 ###### a few examples ######
willy tarreau197e8ec2005-12-17 14:10:59 +01002251
willy tarreauc5f73ed2005-12-18 01:26:38 +01002252 # rewrite 'online.fr' instead of 'free.fr' for GET and POST requests
2253 reqrep ^(GET\ .*)(.free.fr)(.*) \1.online.fr\3
2254 reqrep ^(POST\ .*)(.free.fr)(.*) \1.online.fr\3
willy tarreau197e8ec2005-12-17 14:10:59 +01002255
willy tarreauc5f73ed2005-12-18 01:26:38 +01002256 # force proxy connections to close
2257 reqirep ^Proxy-Connection:.* Proxy-Connection:\ close
2258 # rewrite locations
2259 rspirep ^(Location:\ )([^:]*://[^/]*)(.*) \1\3
willy tarreaueedaa9f2005-12-17 14:08:03 +01002260
willy tarreauc5f73ed2005-12-18 01:26:38 +01002261 ###### A full configuration being used on production ######
willy tarreaueedaa9f2005-12-17 14:08:03 +01002262
willy tarreau197e8ec2005-12-17 14:10:59 +01002263 # Every header should end with a colon followed by one space.
willy tarreauc5f73ed2005-12-18 01:26:38 +01002264 reqideny ^[^:\ ]*[\ ]*$
willy tarreaueedaa9f2005-12-17 14:08:03 +01002265
willy tarreau197e8ec2005-12-17 14:10:59 +01002266 # block Apache chunk exploit
willy tarreauc5f73ed2005-12-18 01:26:38 +01002267 reqideny ^Transfer-Encoding:[\ ]*chunked
2268 reqideny ^Host:\ apache-
willy tarreaueedaa9f2005-12-17 14:08:03 +01002269
willy tarreau197e8ec2005-12-17 14:10:59 +01002270 # block annoying worms that fill the logs...
willy tarreauc5f73ed2005-12-18 01:26:38 +01002271 reqideny ^[^:\ ]*\ .*(\.|%2e)(\.|%2e)(%2f|%5c|/|\\\\)
2272 reqideny ^[^:\ ]*\ ([^\ ]*\ [^\ ]*\ |.*%00)
2273 reqideny ^[^:\ ]*\ .*<script
2274 reqideny ^[^:\ ]*\ .*/(root\.exe\?|cmd\.exe\?|default\.ida\?)
willy tarreau197e8ec2005-12-17 14:10:59 +01002275
Willy Tarreau2272dc12006-09-03 10:19:38 +02002276 # tarpit attacks on the login page.
2277 reqtarpit ^[^:\ ]*\ .*\.php?login=[^0-9]
2278
willy tarreau197e8ec2005-12-17 14:10:59 +01002279 # allow other syntactically valid requests, and block any other method
willy tarreauc5f73ed2005-12-18 01:26:38 +01002280 reqipass ^(GET|POST|HEAD|OPTIONS)\ /.*\ HTTP/1\.[01]$
2281 reqipass ^OPTIONS\ \\*\ HTTP/1\.[01]$
2282 reqideny ^[^:\ ]*\
willy tarreau197e8ec2005-12-17 14:10:59 +01002283
2284 # force connection:close, thus disabling HTTP keep-alive
willy tarreauc5f73ed2005-12-18 01:26:38 +01002285 option httpclose
willy tarreau197e8ec2005-12-17 14:10:59 +01002286
willy tarreauc5f73ed2005-12-18 01:26:38 +01002287 # change the server name
2288 rspidel ^Server:\
2289 rspadd Server:\ Formilux/0.1.8
willy tarreau197e8ec2005-12-17 14:10:59 +01002290
2291
willy tarreau982249e2005-12-18 00:57:06 +01002292Also, the 'forwardfor' option creates an HTTP 'X-Forwarded-For' header which
willy tarreauc1cae632005-12-17 14:12:23 +01002293contains the client's IP address. This is useful to let the final web server
Willy Tarreau7ac51f62007-03-25 16:00:04 +02002294know what the client address was (eg for statistics on domains). Starting with
2295version 1.3.8, it is possible to specify the "except" keyword followed by a
2296source IP address or network for which no header will be added. This is very
2297useful when another reverse-proxy which already adds the header runs on the
2298same machine or in a known DMZ, the most common case being the local use of
2299stunnel on the same system.
willy tarreauc1cae632005-12-17 14:12:23 +01002300
willy tarreau982249e2005-12-18 00:57:06 +01002301Last, the 'httpclose' option removes any 'Connection' header both ways, and
2302adds a 'Connection: close' header in each direction. This makes it easier to
Willy TARREAU767ba712006-03-01 22:40:50 +01002303disable HTTP keep-alive than the previous 4-rules block.
willy tarreau982249e2005-12-18 00:57:06 +01002304
willy tarreauc1cae632005-12-17 14:12:23 +01002305Example :
2306---------
2307 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01002308 mode http
2309 log global
2310 option httplog
2311 option dontlognull
Willy Tarreau7ac51f62007-03-25 16:00:04 +02002312 option forwardfor except 127.0.0.1/8
willy tarreauc5f73ed2005-12-18 01:26:38 +01002313 option httpclose
2314
Willy TARREAU767ba712006-03-01 22:40:50 +01002315Note that some HTTP servers do not necessarily close the connections when they
2316receive the 'Connection: close', and if the client does not close either, then
2317the connection will be maintained up to the time-out. This translates into high
2318number of simultaneous sessions and high global session times in the logs. To
2319workaround this, a new option 'forceclose' appeared in version 1.2.9 to enforce
2320the closing of the outgoing server channel as soon as the server begins to
2321reply and only if the request buffer is empty. Note that this should NOT be
2322used if CONNECT requests are expected between the client and the server. The
2323'forceclose' option implies the 'httpclose' option.
2324
2325Example :
2326---------
2327 listen http_proxy 0.0.0.0:80
2328 mode http
2329 log global
2330 option httplog
2331 option dontlognull
2332 option forwardfor
2333 option forceclose
2334
willy tarreau197e8ec2005-12-17 14:10:59 +01002335
23364.4) Load balancing with persistence
2337------------------------------------
willy tarreau197e8ec2005-12-17 14:10:59 +01002338Combining cookie insertion with internal load balancing allows to transparently
2339bring persistence to applications. The principle is quite simple :
2340 - assign a cookie value to each server
2341 - enable the load balancing between servers
2342 - insert a cookie into responses resulting from the balancing algorithm
2343 (indirect accesses), end ensure that no upstream proxy will cache it.
2344 - remove the cookie in the request headers so that the application never sees
2345 it.
2346
2347Example :
willy tarreaueedaa9f2005-12-17 14:08:03 +01002348---------
2349 listen application 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01002350 mode http
2351 cookie SERVERID insert nocache indirect
2352 balance roundrobin
2353 server srv1 192.168.1.1:80 cookie server01 check
2354 server srv2 192.168.1.2:80 cookie server02 check
willy tarreaueedaa9f2005-12-17 14:08:03 +01002355
willy tarreau0174f312005-12-18 01:02:42 +01002356The other solution brought by versions 1.1.30 and 1.2.3 is to reuse a cookie
2357from the server, and prefix the server's name to it. In this case, don't forget
2358to force "httpclose" mode so that you can be assured that every subsequent
2359request will have its cookie fixed.
2360
2361 listen application 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01002362 mode http
2363 cookie JSESSIONID prefix
2364 balance roundrobin
2365 server srv1 192.168.1.1:80 cookie srv1 check
2366 server srv2 192.168.1.2:80 cookie srv2 check
2367 option httpclose
willy tarreau0174f312005-12-18 01:02:42 +01002368
2369
willy tarreau982249e2005-12-18 00:57:06 +010023704.5) Protection against information leak from the servers
2371---------------------------------------------------------
2372In versions 1.1.28/1.2.1, a new option 'checkcache' was created. It carefully
2373checks 'Cache-control', 'Pragma' and 'Set-cookie' headers in server response
2374to check if there's a risk of caching a cookie on a client-side proxy. When this
2375option is enabled, the only responses which can be delivered to the client are :
2376 - all those without 'Set-Cookie' header ;
2377 - all those with a return code other than 200, 203, 206, 300, 301, 410,
2378 provided that the server has not set a 'Cache-control: public' header ;
2379 - all those that come from a POST request, provided that the server has not
2380 set a 'Cache-Control: public' header ;
2381 - those with a 'Pragma: no-cache' header
2382 - those with a 'Cache-control: private' header
2383 - those with a 'Cache-control: no-store' header
2384 - those with a 'Cache-control: max-age=0' header
2385 - those with a 'Cache-control: s-maxage=0' header
2386 - those with a 'Cache-control: no-cache' header
2387 - those with a 'Cache-control: no-cache="set-cookie"' header
2388 - those with a 'Cache-control: no-cache="set-cookie,' header
2389 (allowing other fields after set-cookie)
willy tarreaueedaa9f2005-12-17 14:08:03 +01002390
willy tarreau982249e2005-12-18 00:57:06 +01002391If a response doesn't respect these requirements, then it will be blocked just
2392as if it was from an 'rspdeny' filter, with an "HTTP 502 bad gateway". The
2393session state shows "PH--" meaning that the proxy blocked the response during
2394headers processing. Additionnaly, an alert will be sent in the logs so that
2395admins are told that there's something to be done.
2396
willy tarreauc5f73ed2005-12-18 01:26:38 +01002397
willy tarreau982249e2005-12-18 00:57:06 +010023984.6) Customizing errors
2399-----------------------
willy tarreau197e8ec2005-12-17 14:10:59 +01002400Some situations can make haproxy return an HTTP error code to the client :
2401 - invalid or too long request => HTTP 400
2402 - request not completely sent in time => HTTP 408
2403 - forbidden request (matches a deny filter) => HTTP 403
2404 - internal error in haproxy => HTTP 500
2405 - the server returned an invalid or incomplete response => HTTP 502
2406 - no server was available to handle the request => HTTP 503
2407 - the server failed to reply in time => HTTP 504
willy tarreaueedaa9f2005-12-17 14:08:03 +01002408
willy tarreau197e8ec2005-12-17 14:10:59 +01002409A succint error message taken from the RFC accompanies these return codes.
2410But depending on the clients knowledge, it may be better to return custom, user
Willy Tarreau3f49b302007-06-11 00:29:26 +02002411friendly, error pages. This is made possible in two ways, one involving a
2412redirection to a known server, and another one consisting in returning a local
2413file.
2414
24154.6.1) Relocation
2416-----------------
2417An error relocation is achieved using the 'errorloc' command :
willy tarreaueedaa9f2005-12-17 14:08:03 +01002418
willy tarreau197e8ec2005-12-17 14:10:59 +01002419 errorloc <HTTP_code> <location>
willy tarreaueedaa9f2005-12-17 14:08:03 +01002420
willy tarreau197e8ec2005-12-17 14:10:59 +01002421Instead of generating an HTTP error <HTTP_code> among those above, the proxy
2422will return a temporary redirection code (HTTP 302) towards the address
2423specified in <location>. This address may be either relative to the site or
2424absolute. Since this request will be handled by the client's browser, it's
2425mandatory that the returned address be reachable from the outside.
willy tarreaueedaa9f2005-12-17 14:08:03 +01002426
willy tarreau197e8ec2005-12-17 14:10:59 +01002427Example :
willy tarreaueedaa9f2005-12-17 14:08:03 +01002428---------
2429 listen application 0.0.0.0:80
2430 errorloc 400 /badrequest.html
2431 errorloc 403 /forbidden.html
2432 errorloc 408 /toolong.html
willy tarreauc5f73ed2005-12-18 01:26:38 +01002433 errorloc 500 http://haproxy.domain.net/bugreport.html
willy tarreaueedaa9f2005-12-17 14:08:03 +01002434 errorloc 502 http://192.168.114.58/error50x.html
2435 errorloc 503 http://192.168.114.58/error50x.html
2436 errorloc 504 http://192.168.114.58/error50x.html
2437
willy tarreauc1f47532005-12-18 01:08:26 +01002438Note: RFC2616 says that a client must reuse the same method to fetch the
2439Location returned by a 302, which causes problems with the POST method.
2440The return code 303 was designed explicitly to force the client to fetch the
2441Location URL with the GET method, but there are some browsers pre-dating
2442HTTP/1.1 which don't support it. Anyway, most browsers still behave with 302 as
willy tarreauc5f73ed2005-12-18 01:26:38 +01002443if it was a 303. In order to allow the user to chose, versions 1.1.31 and 1.2.5
2444bring two new keywords to replace 'errorloc' : 'errorloc302' and 'errorloc303'.
willy tarreauc1f47532005-12-18 01:08:26 +01002445
2446They are preffered over errorloc (which still does 302). Consider using
2447errorloc303 everytime you know that your clients support HTTP 303 responses..
2448
Willy Tarreau3f49b302007-06-11 00:29:26 +020024494.6.2) Local files
2450------------------
2451Sometimes, it is desirable to change the returned error without resorting to
2452redirections. The second method consists in loading local files during startup
2453and send them as pure HTTP content upon error. This is what the 'errorfile'
2454keyword does.
2455
2456Warning, there are traps to consider :
2457 - The files are loaded while parsing configuration, before doing a chroot().
2458 Thus, they are relative to the real filesystem. For this reason, it is
2459 recommended to pass an absolute path to those files.
2460
2461 - The contents of those files is not HTML, but real HTTP protocol with
2462 possible HTML body. So the first line and headers are mandatory. Ideally,
2463 every line in the HTTP part should end with CR-LF for maximum compatibility.
2464
2465 - The response is limited to the buffer size (BUSIZE), generally 8 or 16 kB.
2466
2467 - The response should not include references to the local server, in order to
2468 avoid infinite loops on the browser in case of local failure.
2469
2470Example :
2471---------
2472 errorfile 400 /etc/haproxy/errorfiles/400badreq.http
2473 errorfile 403 /etc/haproxy/errorfiles/403forbid.http
2474 errorfile 503 /etc/haproxy/errorfiles/503sorry.http
2475
willy tarreauc1f47532005-12-18 01:08:26 +01002476
willy tarreau982249e2005-12-18 00:57:06 +010024774.7) Modifying default values
willy tarreau197e8ec2005-12-17 14:10:59 +01002478-----------------------------
willy tarreau197e8ec2005-12-17 14:10:59 +01002479Version 1.1.22 introduced the notion of default values, which eliminates the
2480pain of often repeating common parameters between many instances, such as
2481logs, timeouts, modes, etc...
willy tarreaueedaa9f2005-12-17 14:08:03 +01002482
willy tarreau197e8ec2005-12-17 14:10:59 +01002483Default values are set in a 'defaults' section. Each of these section clears
2484all previously set default parameters, so there may be as many default
2485parameters as needed. Only the last one before a 'listen' section will be
2486used for this section. The 'defaults' section uses the same syntax as the
2487'listen' section, for the supported parameters. The 'defaults' keyword ignores
2488everything on its command line, so that fake instance names can be specified
2489there for better clarity.
willy tarreaueedaa9f2005-12-17 14:08:03 +01002490
willy tarreau982249e2005-12-18 00:57:06 +01002491In version 1.1.28/1.2.1, only those parameters can be preset in the 'default'
willy tarreau197e8ec2005-12-17 14:10:59 +01002492section :
2493 - log (the first and second one)
willy tarreaueedaa9f2005-12-17 14:08:03 +01002494 - mode { tcp, http, health }
2495 - balance { roundrobin }
willy tarreau197e8ec2005-12-17 14:10:59 +01002496 - disabled (to disable every further instances)
2497 - enabled (to enable every further instances, this is the default)
willy tarreaueedaa9f2005-12-17 14:08:03 +01002498 - contimeout, clitimeout, srvtimeout, grace, retries, maxconn
willy tarreau982249e2005-12-18 00:57:06 +01002499 - option { redispatch, transparent, keepalive, forwardfor, logasap, httpclose,
2500 checkcache, httplog, tcplog, dontlognull, persist, httpchk }
willy tarreaueedaa9f2005-12-17 14:08:03 +01002501 - redispatch, redisp, transparent, source { addr:port }
2502 - cookie, capture
2503 - errorloc
2504
willy tarreau197e8ec2005-12-17 14:10:59 +01002505As of 1.1.24, it is not possible to put certain parameters in a 'defaults'
2506section, mainly regular expressions and server configurations :
willy tarreaueedaa9f2005-12-17 14:08:03 +01002507 - dispatch, server,
willy tarreau197e8ec2005-12-17 14:10:59 +01002508 - req*, rsp*
willy tarreaueedaa9f2005-12-17 14:08:03 +01002509
willy tarreau197e8ec2005-12-17 14:10:59 +01002510Last, there's no way yet to change a boolean option from its assigned default
2511value. So if an 'option' statement is set in a 'defaults' section, the only
2512way to flush it is to redefine a new 'defaults' section without this 'option'.
willy tarreaueedaa9f2005-12-17 14:08:03 +01002513
willy tarreau197e8ec2005-12-17 14:10:59 +01002514Examples :
willy tarreaueedaa9f2005-12-17 14:08:03 +01002515----------
2516 defaults applications TCP
willy tarreauc5f73ed2005-12-18 01:26:38 +01002517 log global
2518 mode tcp
2519 balance roundrobin
2520 clitimeout 180000
2521 srvtimeout 180000
2522 contimeout 4000
2523 retries 3
2524 redispatch
willy tarreaueedaa9f2005-12-17 14:08:03 +01002525
2526 listen app_tcp1 10.0.0.1:6000-6063
willy tarreauc5f73ed2005-12-18 01:26:38 +01002527 server srv1 192.168.1.1 check port 6000 inter 10000
2528 server srv2 192.168.1.2 backup
willy tarreaueedaa9f2005-12-17 14:08:03 +01002529
2530 listen app_tcp2 10.0.0.2:6000-6063
willy tarreauc5f73ed2005-12-18 01:26:38 +01002531 server srv1 192.168.2.1 check port 6000 inter 10000
2532 server srv2 192.168.2.2 backup
willy tarreaueedaa9f2005-12-17 14:08:03 +01002533
2534 defaults applications HTTP
willy tarreauc5f73ed2005-12-18 01:26:38 +01002535 log global
2536 mode http
2537 option httplog
2538 option forwardfor
2539 option dontlognull
2540 balance roundrobin
2541 clitimeout 20000
2542 srvtimeout 20000
2543 contimeout 4000
2544 retries 3
willy tarreaueedaa9f2005-12-17 14:08:03 +01002545
2546 listen app_http1 10.0.0.1:80-81
willy tarreauc5f73ed2005-12-18 01:26:38 +01002547 cookie SERVERID postonly insert indirect
2548 capture cookie userid= len 10
2549 server srv1 192.168.1.1:+8000 cookie srv1 check port 8080 inter 1000
2550 server srv1 192.168.1.2:+8000 cookie srv2 check port 8080 inter 1000
willy tarreaueedaa9f2005-12-17 14:08:03 +01002551
2552 defaults
willy tarreauc5f73ed2005-12-18 01:26:38 +01002553 # this empty section voids all default parameters
willy tarreaueedaa9f2005-12-17 14:08:03 +01002554
willy tarreau481132e2006-05-21 21:43:10 +02002555
25564.8) Status report in HTML page
2557-------------------------------
2558Starting with 1.2.14, it is possible for HAProxy to intercept requests for a
2559particular URI and return a full report of the proxy's activity and servers
2560statistics. This is available through the 'stats' keyword, associated to any
2561such options :
2562
2563 - stats enable
2564 - stats uri <uri prefix>
2565 - stats realm <authentication realm>
2566 - stats auth <user:password>
2567 - stats scope <proxy_id> | '.'
2568
2569By default, the status report is disabled. Specifying any combination above
2570enables it for the proxy instance referencing it. The easiest solution is to
2571use "stats enable" which will enable the report with default parameters :
2572
2573 - default URI : "/haproxy?stats" (CONFIG_STATS_DEFAULT_URI)
2574 - default auth : unspecified (no authentication)
2575 - default realm : "HAProxy Statistics" (CONFIG_STATS_DEFAULT_REALM)
2576 - default scope : unspecified (access to all instances)
2577
2578The "stats uri <uri_prefix>" option allows one to intercept another URI prefix.
2579Note that any URI that BEGINS with this string will match. For instance, one
2580proxy instance might be dedicated to status page only and would reply to any
2581URI.
2582
2583Example :
2584---------
2585 # catches any URI and returns the status page.
2586 listen stats :8080
2587 mode http
2588 stats uri /
2589
2590The "stats auth <user:password>" option enables Basic authentication and adds a
2591valid user:password combination to the list of authorized accounts. The user
2592and password are passed in the configuration file as clear text, and since this
2593is HTTP Basic authentication, you should be aware that it transits as clear
2594text on the network, so you must not use any sensible account. The list is
2595unlimited in order to provide easy accesses to developpers or customers.
2596
2597The "stats realm <realm>" option defines the "realm" name which is displayed
2598in the popup box when the browser asks for a password. It's important to ensure
2599that this one is not used by the application, otherwise the browser will try to
2600use a cached one from the application. Note that any space in the realm name
2601should be escaped with a backslash ('\').
2602
2603The "stats scope <proxy_id>" option limits the scope of the status report. By
2604default, all proxy instances are listed. But under some circumstances, it would
2605be better to limit the listing to some proxies or only to the current one. This
2606is what this option does. The special proxy name "." (a single dot) references
2607the current proxy. The proxy name can be repeated multiple times, even for
2608proxies defined later in the configuration or some which do not exist. The name
2609is the one which appears after the 'listen' keyword.
2610
2611Example :
2612---------
2613 # simple application with authenticated embedded status report
2614 listen app1 192.168.1.100:80
2615 mode http
willy tarreaud4ba08d2006-05-21 21:54:14 +02002616 option httpclose
willy tarreau481132e2006-05-21 21:43:10 +02002617 balance roundrobin
2618 cookie SERVERID postonly insert indirect
2619 server srv1 192.168.1.1:8080 cookie srv1 check inter 1000
2620 server srv1 192.168.1.2:8080 cookie srv2 check inter 1000
2621 stats uri /my_stats
willy tarreaud4ba08d2006-05-21 21:54:14 +02002622 stats realm Statistics\ for\ MyApp1-2
2623 stats auth guest:guest
2624 stats auth admin:AdMiN123
2625 stats scope .
2626 stats scope app2
willy tarreau481132e2006-05-21 21:43:10 +02002627
2628 # simple application with anonymous embedded status report
2629 listen app2 192.168.2.100:80
2630 mode http
willy tarreaud4ba08d2006-05-21 21:54:14 +02002631 option httpclose
willy tarreau481132e2006-05-21 21:43:10 +02002632 balance roundrobin
2633 cookie SERVERID postonly insert indirect
2634 server srv1 192.168.2.1:8080 cookie srv1 check inter 1000
2635 server srv1 192.168.2.2:8080 cookie srv2 check inter 1000
2636 stats uri /my_stats
willy tarreaud4ba08d2006-05-21 21:54:14 +02002637 stats realm Statistics\ for\ MyApp2
2638 stats scope .
willy tarreau481132e2006-05-21 21:43:10 +02002639
2640 listen admin_page :8080
2641 mode http
2642 stats uri /my_stats
willy tarreaud4ba08d2006-05-21 21:54:14 +02002643 stats realm Global\ statistics
2644 stats auth admin:AdMiN123
willy tarreau481132e2006-05-21 21:43:10 +02002645
2646Notes :
2647-------
2648 - The 'stats' options can also be specified in the 'defaults' section, in
2649 which case it will provide the exact same configuration to all further
2650 instances (hence the usefulness of the scope "."). However, if an instance
2651 redefines any 'stats' parameter, defaults will not be used for this
2652 instance.
2653
2654 - HTTP Basic authentication is very basic and unsecure from snooping. No
2655 sensible password should be used, and be aware that there is no way to
2656 remove it from the browser so it will be sent to the whole application
2657 upon further accesses.
2658
willy tarreaud4ba08d2006-05-21 21:54:14 +02002659 - It is very important that the 'option httpclose' is specified, otherwise
2660 the proxy will not be able to detect the URI within keep-alive sessions
2661 maintained between the browser and the servers, so the stats URI will be
2662 forwarded unmodified to the server as if the option was not set.
2663
willy tarreau481132e2006-05-21 21:43:10 +02002664
Willy Tarreau726c2bf2007-05-09 01:31:45 +020026655) Access lists
2666===============
2667
2668With version 1.3.10, a new concept of access lists (acl) was born. As it was
2669not necesary to reinvent the wheel, and because even long thoughts lead to
2670unsatisfying proposals, it was finally decided that something close to what
2671Squid provides would be a good compromise between features and ease of use.
2672
2673The principle is very simple : acls are declared with a name, a test and a list
2674of valid values to check against during the test. Conditions are applied on
2675various actions, and those conditions apply a logical AND between acls. The
2676condition is then only met if all acls are true.
2677
2678It is possible to use the reserved keyword "OR" in conditions, and it is
2679possible for an acl to be specified multiple times, even with various tests, in
2680which case the first one which returns true validates the ACL.
2681
Willy Tarreauae8b7962007-06-09 23:10:04 +02002682As of 1.3.12, only the following tests have been implemented :
Willy Tarreau726c2bf2007-05-09 01:31:45 +02002683
2684 Layer 3/4 :
2685 src <ipv4_address>[/mask] ... : match IPv4 source address
2686 dst <ipv4_address>[/mask] ... : match IPv4 destination address
Willy Tarreauae8b7962007-06-09 23:10:04 +02002687 src_port <range> ... : match source port range
2688 dst_port <range> ... : match destination port range
2689 dst_conn <range> ... : match #connections on frontend
Willy Tarreau726c2bf2007-05-09 01:31:45 +02002690
2691 Layer 7 :
2692 method <HTTP method> ... : match HTTP method
2693 req_ver <1.0|1.1> ... : match HTTP request version
2694 resp_ver <1.0|1.1> ... : match HTTP response version
Willy Tarreauae8b7962007-06-09 23:10:04 +02002695 status <range> ... : match HTTP response status code in range
Willy Tarreau726c2bf2007-05-09 01:31:45 +02002696 url <string> ... : exact string match on URI
2697 url_reg <regex> ... : regex string match on URI
2698 url_beg <string> ... : true if URI begins with <string>
2699 url_end <string> ... : true if URI ends with <string>
2700 url_sub <string> ... : true if URI contains <string>
2701 url_dir <string> ... : true if URI contains <string> between slashes
2702 url_dom <string> ... : true if URI contains <string> between slashes or dots
2703
Willy Tarreauae8b7962007-06-09 23:10:04 +02002704A 'range' is one or two integers which may be prefixed by an operator.
2705The syntax is :
2706
2707 [<op>] <low>[:<high>]
2708
2709Where <op> can be :
2710 'eq' : the tested value must be equal to <low> or within <low>..<high>
2711 'le' : the tested value must be lower than or equal to <low>
2712 'lt' : the tested value must be lower than <low>
2713 'ge' : the tested value must be greater than or equal to <low>
2714 'gt' : the tested value must be greater than <low>
2715
2716When no operator is defined, 'eq' is assumed. Note that when the operator is
2717specified, it applies to all subsequent ranges of values until the end of the
2718line is reached or another operator is specified. Example :
2719
2720 acl status_error status 400:599
2721 acl saturated_frt dst_conn ge 1000
2722 acl invalid_ports src_port lt 512 ge 65535
2723
Willy Tarreau726c2bf2007-05-09 01:31:45 +02002724Other ones are coming (headers, cookies, time, auth), it's just a matter of
2725time. It is also planned to be able to read the patterns from a file, as well
2726as to ignore the case for some of them.
2727
2728The only command supporting a condition right now is the "block" command, which
2729blocks a request and returns a 403 if its condition is true (with the "if"
2730keyword), or if it is false (with the "unless" keyword).
2731
2732Example :
2733---------
2734
2735 acl options_uris url *
2736 acl meth_option method OPTIONS
2737 acl http_1.1 req_ver 1.1
2738 acl allowed_meth method GET HEAD POST OPTIONS CONNECT
2739 acl connect_meth method CONNECT
2740 acl proxy_url url_beg http://
2741
2742 # block if reserved URI "*" used with a method other than "OPTIONS"
2743 block if options_uris !meth_option
2744
2745 # block if the OPTIONS method is used with HTTP 1.0
2746 block if meth_option !http_1.1
2747
2748 # allow non-proxy url with anything but the CONNECT method
2749 block if !connect_meth !proxy_url
2750
2751 # block all unknown methods
2752 block unless allowed_meth
2753
2754Note: this documentation is very light but should permit one to start and above
2755all it should permit to work on the project without being slowed down too much
2756with the doc.
2757
2758
willy tarreau197e8ec2005-12-17 14:10:59 +01002759=========================
2760| System-specific setup |
2761=========================
willy tarreaueedaa9f2005-12-17 14:08:03 +01002762
willy tarreau197e8ec2005-12-17 14:10:59 +01002763Linux 2.4
2764=========
willy tarreaueedaa9f2005-12-17 14:08:03 +01002765
2766-- cut here --
2767#!/bin/sh
2768# set this to about 256/4M (16384 for 256M machine)
2769MAXFILES=16384
2770echo $MAXFILES > /proc/sys/fs/file-max
2771ulimit -n $MAXFILES
2772
2773if [ -e /proc/sys/net/ipv4/ip_conntrack_max ]; then
willy tarreauc5f73ed2005-12-18 01:26:38 +01002774 echo 65536 > /proc/sys/net/ipv4/ip_conntrack_max
willy tarreaueedaa9f2005-12-17 14:08:03 +01002775fi
2776
2777if [ -e /proc/sys/net/ipv4/netfilter/ip_ct_tcp_timeout_fin_wait ]; then
willy tarreauc5f73ed2005-12-18 01:26:38 +01002778 # 30 seconds for fin, 15 for time wait
2779 echo 3000 > /proc/sys/net/ipv4/netfilter/ip_ct_tcp_timeout_fin_wait
2780 echo 1500 > /proc/sys/net/ipv4/netfilter/ip_ct_tcp_timeout_time_wait
2781 echo 0 > /proc/sys/net/ipv4/netfilter/ip_ct_tcp_log_invalid_scale
2782 echo 0 > /proc/sys/net/ipv4/netfilter/ip_ct_tcp_log_out_of_window
willy tarreaueedaa9f2005-12-17 14:08:03 +01002783fi
2784
2785echo 1024 60999 > /proc/sys/net/ipv4/ip_local_port_range
2786echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout
2787echo 4096 > /proc/sys/net/ipv4/tcp_max_syn_backlog
2788echo 262144 > /proc/sys/net/ipv4/tcp_max_tw_buckets
2789echo 262144 > /proc/sys/net/ipv4/tcp_max_orphans
2790echo 300 > /proc/sys/net/ipv4/tcp_keepalive_time
2791echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle
2792echo 0 > /proc/sys/net/ipv4/tcp_timestamps
2793echo 0 > /proc/sys/net/ipv4/tcp_ecn
willy tarreauc5f73ed2005-12-18 01:26:38 +01002794echo 1 > /proc/sys/net/ipv4/tcp_sack
willy tarreaueedaa9f2005-12-17 14:08:03 +01002795echo 0 > /proc/sys/net/ipv4/tcp_dsack
2796
2797# auto-tuned on 2.4
2798#echo 262143 > /proc/sys/net/core/rmem_max
2799#echo 262143 > /proc/sys/net/core/rmem_default
2800
2801echo 16384 65536 524288 > /proc/sys/net/ipv4/tcp_rmem
2802echo 16384 349520 699040 > /proc/sys/net/ipv4/tcp_wmem
2803
2804-- cut here --
2805
willy tarreau197e8ec2005-12-17 14:10:59 +01002806
2807FreeBSD
2808=======
2809
2810A FreeBSD port of HA-Proxy is now available and maintained, thanks to
2811Clement Laforet <sheepkiller@cultdeadsheep.org>.
2812
2813For more information :
2814http://www.freebsd.org/cgi/url.cgi?ports/net/haproxy/pkg-descr
2815http://www.freebsd.org/cgi/cvsweb.cgi/ports/net/haproxy/
2816http://www.freshports.org/net/haproxy
2817
2818
2819-- end --