blob: 4c673d3510e6653cc114b282682c86b8230ceb35 [file] [log] [blame]
willy tarreau0174f312005-12-18 01:02:42 +01001 -------------------
Willy Tarreau94b45912006-05-31 06:40:15 +02002 HAProxy
willy tarreau0174f312005-12-18 01:02:42 +01003 Reference Manual
4 -------------------
Willy Tarreau7b4c5ae2008-04-19 21:06:14 +02005 version 1.3.15
willy tarreauc5f73ed2005-12-18 01:26:38 +01006 willy tarreau
Willy Tarreau7b4c5ae2008-04-19 21:06:14 +02007 2008/04/19
willy tarreaueedaa9f2005-12-17 14:08:03 +01008
Willy Tarreaua080eca2009-10-14 20:30:15 +02009
10 !!!! NOTE: THIS DOCUMENT IS OUTDATED !!!!
11
12 Please use "configuration.txt" from the same directory, or download
13 an up-to-date version from the following location :
14
Willy Tarreaube6008f2009-10-14 22:22:03 +020015 http://haproxy.1wt.eu/download/1.4/doc/configuration.txt
Willy Tarreaua080eca2009-10-14 20:30:15 +020016
17
willy tarreaueedaa9f2005-12-17 14:08:03 +010018============
19| Abstract |
20============
21
Willy Tarreau94b45912006-05-31 06:40:15 +020022HAProxy is a TCP/HTTP reverse proxy which is particularly suited for high
willy tarreaueedaa9f2005-12-17 14:08:03 +010023availability environments. Indeed, it can :
24 - route HTTP requests depending on statically assigned cookies ;
25 - spread the load among several servers while assuring server persistence
26 through the use of HTTP cookies ;
27 - switch to backup servers in the event a main one fails ;
28 - accept connections to special ports dedicated to service monitoring ;
29 - stop accepting connections without breaking existing ones ;
30 - add/modify/delete HTTP headers both ways ;
31 - block requests matching a particular pattern ;
willy tarreau532bb552006-05-13 18:40:37 +020032 - hold clients to the right application server depending on application
33 cookies
willy tarreau481132e2006-05-21 21:43:10 +020034 - report detailed status as HTML pages to authenticated users from an URI
35 intercepted from the application.
willy tarreaueedaa9f2005-12-17 14:08:03 +010036
37It needs very little resource. Its event-driven architecture allows it to easily
38handle thousands of simultaneous connections on hundreds of instances without
39risking the system's stability.
40
41====================
42| Start parameters |
43====================
44
45There are only a few command line options :
46
47 -f <configuration file>
48 -n <high limit for the total number of simultaneous connections>
willy tarreau532bb552006-05-13 18:40:37 +020049 = 'maxconn' in 'global' section
50 -N <high limit for the per-listener number of simultaneous connections>
51 = 'maxconn' in 'listen' or 'default' sections
willy tarreaueedaa9f2005-12-17 14:08:03 +010052 -d starts in foregreound with debugging mode enabled
53 -D starts in daemon mode
Marc-Antoine Perennou992709b2013-02-12 10:53:52 +010054 -Ds starts in systemd daemon mode
willy tarreau982249e2005-12-18 00:57:06 +010055 -q disable messages on output
56 -V displays messages on output even when -q or 'quiet' are specified.
57 -c only checks config file and exits with code 0 if no error was found, or
58 exits with code 1 if a syntax error was found.
willy tarreaufe2c5c12005-12-17 14:14:34 +010059 -p <pidfile> asks the process to write down each of its children's
60 pids to this file in daemon mode.
willy tarreau34f45302006-04-15 21:37:14 +020061 -sf specifies a list of pids to send a FINISH signal to after startup.
62 -st specifies a list of pids to send a TERMINATE signal to after startup.
willy tarreaueedaa9f2005-12-17 14:08:03 +010063 -s shows statistics (only if compiled in)
64 -l shows even more statistics (implies '-s')
Willy Tarreaude99e992007-04-16 00:53:59 +020065 -dk disables use of kqueue()
willy tarreau64a3cc32005-12-18 01:13:11 +010066 -de disables use of epoll()
67 -dp disables use of poll()
willy tarreau34f45302006-04-15 21:37:14 +020068 -db disables background mode (stays in foreground, useful for debugging)
69 -m <megs> enforces a memory usage limit to a maximum of <megs> megabytes.
willy tarreaueedaa9f2005-12-17 14:08:03 +010070
willy tarreau532bb552006-05-13 18:40:37 +020071The maximal number of connections per proxy instance is used as the default
72parameter for each instance for which the 'maxconn' paramter is not set in the
73'listen' section.
willy tarreaueedaa9f2005-12-17 14:08:03 +010074
75The maximal number of total connections limits the number of connections used by
76the whole process if the 'maxconn' parameter is not set in the 'global' section.
77
78The debugging mode has the same effect as the 'debug' option in the 'global'
79section. When the proxy runs in this mode, it dumps every connections,
80disconnections, timestamps, and HTTP headers to stdout. This should NEVER
81be used in an init script since it will prevent the system from starting up.
82
willy tarreau34f45302006-04-15 21:37:14 +020083For debugging, the '-db' option is very useful as it temporarily disables
84daemon mode and multi-process mode. The service can then be stopped by simply
85pressing Ctrl-C, without having to edit the config nor run full debug.
86
willy tarreaueedaa9f2005-12-17 14:08:03 +010087Statistics are only available if compiled in with the 'STATTIME' option. It's
willy tarreau481132e2006-05-21 21:43:10 +020088only used during code optimization phases, and will soon disappear.
willy tarreaueedaa9f2005-12-17 14:08:03 +010089
willy tarreau532bb552006-05-13 18:40:37 +020090The '-st' and '-sf' options are used for hot reconfiguration (see below).
willy tarreau34f45302006-04-15 21:37:14 +020091
willy tarreaueedaa9f2005-12-17 14:08:03 +010092======================
93| Configuration file |
94======================
95
96Structure
97=========
98
99The configuration file parser ignores empty lines, spaces, tabs. Anything
100between a sharp ('#') not following a backslash ('\'), and the end of a line
101constitutes a comment and is ignored too.
102
103The configuration file is segmented in sections. A section begins whenever
104one of these 3 keywords are encountered :
105
106 - 'global'
107 - 'listen'
108 - 'defaults'
109
110Every parameter refer to the section beginning at the last one of these 3
111keywords.
112
113
1141) Global parameters
115====================
116
117Global parameters affect the whole process behaviour. They are all set in the
118'global' section. There may be several 'global' sections if needed, but their
119parameters will only be merged. Allowed parameters in 'global' section include
120the following ones :
121
122 - log <address> <facility> [max_level]
123 - maxconn <number>
124 - uid <user id>
125 - gid <group id>
Willy Tarreau95c20ac2007-03-25 15:39:23 +0200126 - user <user name>
127 - group <group name>
willy tarreaueedaa9f2005-12-17 14:08:03 +0100128 - chroot <directory>
129 - nbproc <number>
130 - daemon
131 - debug
Willy Tarreaude99e992007-04-16 00:53:59 +0200132 - nokqueue
willy tarreau64a3cc32005-12-18 01:13:11 +0100133 - noepoll
134 - nopoll
willy tarreaueedaa9f2005-12-17 14:08:03 +0100135 - quiet
willy tarreaufe2c5c12005-12-17 14:14:34 +0100136 - pidfile <file>
willy tarreauc5f73ed2005-12-18 01:26:38 +0100137 - ulimit-n <number>
willy tarreau598da412005-12-18 01:07:29 +0100138 - stats
Willy Tarreau1db37712007-06-03 17:16:49 +0200139 - tune.maxpollevents <number>
willy tarreaueedaa9f2005-12-17 14:08:03 +0100140
willy tarreauc5f73ed2005-12-18 01:26:38 +0100141
willy tarreaueedaa9f2005-12-17 14:08:03 +01001421.1) Event logging
143------------------
144Most events are logged : start, stop, servers going up and down, connections and
145errors. Each event generates a syslog message which can be sent to up to 2
146servers. The syntax is :
147
148 log <ip_address> <facility> [max_level]
149
150Connections are logged at level "info". Services initialization and servers
151going up are logged at level "notice", termination signals are logged at
152"warning", and definitive service termination, as well as loss of servers are
153logged at level "alert". The optional parameter <max_level> specifies above
154what level messages should be sent. Level can take one of these 8 values :
155
156 emerg, alert, crit, err, warning, notice, info, debug
157
158For backwards compatibility with versions 1.1.16 and earlier, the default level
159value is "debug" if not specified.
160
161Permitted facilities are :
162 kern, user, mail, daemon, auth, syslog, lpr, news,
163 uucp, cron, auth2, ftp, ntp, audit, alert, cron2,
164 local0, local1, local2, local3, local4, local5, local6, local7
165
166According to RFC3164, messages are truncated to 1024 bytes before being emitted.
167
168Example :
169---------
170 global
willy tarreauc5f73ed2005-12-18 01:26:38 +0100171 log 192.168.2.200 local3
172 log 127.0.0.1 local4 notice
173
willy tarreaueedaa9f2005-12-17 14:08:03 +0100174
1751.2) limiting the number of connections
176---------------------------------------
177It is possible and recommended to limit the global number of per-process
willy tarreauc5f73ed2005-12-18 01:26:38 +0100178connections using the 'maxconn' global keyword. Since one connection includes
179both a client and a server, it means that the max number of TCP sessions will
180be about the double of this number. It's important to understand this when
181trying to find best values for 'ulimit -n' before starting the proxy. To
182anticipate the number of sockets needed, all these parameters must be counted :
willy tarreaueedaa9f2005-12-17 14:08:03 +0100183
184 - 1 socket per incoming connection
185 - 1 socket per outgoing connection
186 - 1 socket per address/port/proxy tuple.
187 - 1 socket per server being health-checked
188 - 1 socket for all logs
189
190In simple configurations where each proxy only listens one one address/port,
willy tarreauc5f73ed2005-12-18 01:26:38 +0100191set the limit of file descriptors (ulimit -n) to
192(2 * maxconn + nbproxies + nbservers + 1). Starting with versions 1.1.32/1.2.6,
193it is now possible to set the limit in the configuration using the 'ulimit-n'
194global keyword, provided the proxy is started as root. This puts an end to the
195recurrent problem of ensuring that the system limits are adapted to the proxy
196values. Note that these limits are per-process.
197
198Example :
199---------
200 global
201 maxconn 32000
202 ulimit-n 65536
203
willy tarreaueedaa9f2005-12-17 14:08:03 +0100204
2051.3) Drop of priviledges
206------------------------
207In order to reduce the risk and consequences of attacks, in the event where a
208yet non-identified vulnerability would be successfully exploited, it's possible
209to lower the process priviledges and even isolate it in a riskless directory.
210
211In the 'global' section, the 'uid' parameter sets a numerical user identifier
212which the process will switch to after binding its listening sockets. The value
213'0', which normally represents the super-user, here indicates that the UID must
214not change during startup. It's the default behaviour. The 'gid' parameter does
Willy Tarreau95c20ac2007-03-25 15:39:23 +0200215the same for the group identifier. If setting an uid is not possible because of
216deployment constraints, it is possible to set a user name with the 'user'
217keyword followed by a valid user name. The same is true for the gid. It is
218possible to specify a group name after the 'group' keyword.
219
220It is particularly advised against use of generic accounts such as 'nobody'
221because it has the same consequences as using 'root' if other services use
222them.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100223
224The 'chroot' parameter makes the process isolate itself in an empty directory
225just before switching its UID. This type of isolation (chroot) can sometimes
226be worked around on certain OS (Linux, Solaris), provided that the attacker
227has gained 'root' priviledges and has the ability to use or create a directory.
228For this reason, it's capital to use a dedicated directory and not to share one
229between several services of different nature. To make isolation more resistant,
230it's recommended to use an empty directory without any right, and to change the
231UID of the process so that it cannot do anything there.
232
233Note: in the event where such a vulnerability would be exploited, it's most
234likely that first attempts would kill the process due to 'Segmentation Fault',
235'Bus Error' or 'Illegal Instruction' signals. Eventhough it's true that
236isolating the server reduces the risks of intrusion, it's sometimes useful to
237find why a process dies, via the analysis of a 'core' file, although very rare
238(the last bug of this sort was fixed in 1.1.9). For security reasons, most
239systems disable the generation of core file when a process changes its UID. So
240the two workarounds are either to start the process from a restricted user
241account, which will not be able to chroot itself, or start it as root and not
242change the UID. In both cases the core will be either in the start or the chroot
243directories. Do not forget to allow core dumps prior to start the process :
244
245# ulimit -c unlimited
246
247Example :
248---------
249
Willy Tarreau95c20ac2007-03-25 15:39:23 +0200250 # with uid/gid
willy tarreaueedaa9f2005-12-17 14:08:03 +0100251 global
willy tarreauc5f73ed2005-12-18 01:26:38 +0100252 uid 30000
253 gid 30000
254 chroot /var/chroot/haproxy
255
Willy Tarreau95c20ac2007-03-25 15:39:23 +0200256 # with user/group
257 global
258 user haproxy
259 group public
260 chroot /var/chroot/haproxy
261
willy tarreaueedaa9f2005-12-17 14:08:03 +0100262
2631.4) Startup modes
264------------------
willy tarreau34f45302006-04-15 21:37:14 +0200265The service can start in several different modes :
willy tarreaueedaa9f2005-12-17 14:08:03 +0100266 - foreground / background
267 - quiet / normal / debug
268
269The default mode is normal, foreground, which means that the program doesn't
270return once started. NEVER EVER use this mode in a system startup script, or
271the system won't boot. It needs to be started in background, so that it
272returns immediately after forking. That's accomplished by the 'daemon' option
273in the 'global' section, which is the equivalent of the '-D' command line
274argument.
275
willy tarreau34f45302006-04-15 21:37:14 +0200276The '-db' command line argument overrides the 'daemon' and 'nbproc' global
277options to make the process run in normal, foreground mode.
278
willy tarreaueedaa9f2005-12-17 14:08:03 +0100279Moreover, certain alert messages are still sent to the standard output even
280in 'daemon' mode. To make them disappear, simply add the 'quiet' option in the
281'global' section. This option has no command-line equivalent.
282
283Last, the 'debug' mode, enabled with the 'debug' option in the 'global' section,
284and which is equivalent of the '-d' option, allows deep TCP/HTTP analysis, with
285timestamped display of each connection, disconnection, and HTTP headers for both
286ways. This mode is incompatible with 'daemon' and 'quiet' modes for obvious
287reasons.
288
willy tarreauc5f73ed2005-12-18 01:26:38 +0100289
willy tarreaueedaa9f2005-12-17 14:08:03 +01002901.5) Increasing the overall processing power
291--------------------------------------------
292On multi-processor systems, it may seem to be a shame to use only one processor,
willy tarreau982249e2005-12-18 00:57:06 +0100293eventhough the load needed to saturate a recent processor is far above common
willy tarreaueedaa9f2005-12-17 14:08:03 +0100294usage. Anyway, for very specific needs, the proxy can start several processes
295between which the operating system will spread the incoming connections. The
296number of processes is controlled by the 'nbproc' parameter in the 'global'
willy tarreau4302f492005-12-18 01:00:37 +0100297section. It defaults to 1, and obviously works only in 'daemon' mode. One
298typical usage of this parameter has been to workaround the default per-process
299file-descriptor limit that Solaris imposes to user processes.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100300
301Example :
302---------
303
304 global
willy tarreauc5f73ed2005-12-18 01:26:38 +0100305 daemon
306 quiet
307 nbproc 2
willy tarreaueedaa9f2005-12-17 14:08:03 +0100308
309
willy tarreaufe2c5c12005-12-17 14:14:34 +01003101.6) Helping process management
311-------------------------------
312Haproxy now supports the notion of pidfile. If the '-p' command line argument,
313or the 'pidfile' global option is followed with a file name, this file will be
314removed, then filled with all children's pids, one per line (only in daemon
315mode). This file is NOT within the chroot, which allows to work with a readonly
316 chroot. It will be owned by the user starting the process, and will have
317permissions 0644.
318
319Example :
320---------
321
322 global
323 daemon
324 quiet
willy tarreauc5f73ed2005-12-18 01:26:38 +0100325 nbproc 2
willy tarreaufe2c5c12005-12-17 14:14:34 +0100326 pidfile /var/run/haproxy-private.pid
327
328 # to stop only those processes among others :
329 # kill $(</var/run/haproxy-private.pid)
330
willy tarreau34f45302006-04-15 21:37:14 +0200331 # to reload a new configuration with minimal service impact and without
332 # breaking existing sessions :
Willy Tarreau10806d52007-09-09 23:49:18 +0200333 # haproxy -f haproxy.cfg -p /var/run/haproxy-private.pid -sf $(</var/run/haproxy-private.pid)
willy tarreaufe2c5c12005-12-17 14:14:34 +0100334
willy tarreau64a3cc32005-12-18 01:13:11 +01003351.7) Polling mechanisms
336-----------------------
337Starting from version 1.2.5, haproxy supports the poll() and epoll() polling
338mechanisms. On systems where select() is limited by FD_SETSIZE (like Solaris),
339poll() can be an interesting alternative. Performance tests show that Solaris'
340poll() performance does not decay as fast as the numbers of sockets increase,
341making it a safe solution for high loads. However, Solaris already uses poll()
342to emulate select(), so as long as the number of sockets has no reason to go
343higher than FD_SETSIZE, poll() should not provide any better performance. On
344Linux systems with the epoll() patch (or any 2.6 version), haproxy will use
345epoll() which is extremely fast and non dependant on the number of sockets.
346Tests have shown constant performance from 1 to 20000 simultaneous sessions.
Willy Tarreaude99e992007-04-16 00:53:59 +0200347Version 1.3.9 introduced kqueue() for FreeBSD/OpenBSD, and speculative epoll()
348which consists in trying to perform I/O before queuing the events via syscalls.
willy tarreau64a3cc32005-12-18 01:13:11 +0100349
Willy Tarreau1db37712007-06-03 17:16:49 +0200350In order to optimize latency, it is now possible to limit the number of events
351returned by a single call to poll. The limit is fixed to 200 by default. If a
352smaller latency is seeked, it may be useful to reduce this value by using the
353'tune.maxpollevents' parameter in the 'global' section. Increasing it will
354slightly save CPU cycles in presence of large number of connections.
355
Willy Tarreaude99e992007-04-16 00:53:59 +0200356Haproxy will use kqueue() or speculative epoll() when available, then epoll(),
357and will fall back to poll(), then to select(). However, if for any reason you
358need to disable epoll() or poll() (eg. because of a bug or just to compare
Willy Tarreaue9f49e72012-11-11 17:42:00 +0100359performance), new global options have been created for this matter : 'nopoll',
360'nokqueue', and 'noepoll'.
willy tarreau64a3cc32005-12-18 01:13:11 +0100361
362Example :
363---------
364
365 global
366 # use only select()
367 noepoll
368 nopoll
Willy Tarreau1db37712007-06-03 17:16:49 +0200369 tune.maxpollevents 100
willy tarreau64a3cc32005-12-18 01:13:11 +0100370
371Note :
372------
373For the sake of configuration file portability, these options are accepted but
374ignored if the poll() or epoll() mechanisms have not been enabled at compile
375time.
376
Willy Tarreaude99e992007-04-16 00:53:59 +0200377To make debugging easier, the '-de' runtime argument disables epoll support,
378the '-dp' argument disables poll support, '-dk' disables kqueue and '-ds'
379disables speculative epoll(). They are respectively equivalent to 'noepoll',
Willy Tarreaue9f49e72012-11-11 17:42:00 +0100380'nopoll', and 'nokqueue'.
willy tarreau64a3cc32005-12-18 01:13:11 +0100381
382
willy tarreaueedaa9f2005-12-17 14:08:03 +01003832) Declaration of a listening service
384=====================================
385
386Service sections start with the 'listen' keyword :
387
388 listen <instance_name> [ <IP_address>:<port_range>[,...] ]
389
390- <instance_name> is the name of the instance. This name will be reported in
391 logs, so it is good to have it reflect the proxied service. No unicity test
392 is done on this name, and it's not mandatory for it to be unique, but highly
393 recommended.
394
395- <IP_address> is the IP address the proxy binds to. Empty address, '*' and
396 '0.0.0.0' all mean that the proxy listens to all valid addresses on the
397 system.
398
399- <port_range> is either a unique port, or a port range for which the proxy will
400 accept connections for the IP address specified above. This range can be :
401 - a numerical port (ex: '80')
402 - a dash-delimited ports range explicitly stating the lower and upper bounds
403 (ex: '2000-2100') which are included in the range.
404
405 Particular care must be taken against port ranges, because every <addr:port>
406 couple consumes one socket (=a file descriptor), so it's easy to eat lots of
407 descriptors with a simple range. The <addr:port> couple must be used only once
408 among all instances running on a same system. Please note that attaching to
409 ports lower than 1024 need particular priviledges to start the program, which
Jamie Gloudon801a0a32012-08-25 00:18:33 -0400410 are independent of the 'uid' parameter.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100411
412- the <IP_address>:<port_range> couple may be repeated indefinitely to require
413 the proxy to listen to other addresses and/or ports. To achieve this, simply
414 separate them with a coma.
415
416Examples :
417---------
418 listen http_proxy :80
419 listen x11_proxy 127.0.0.1:6000-6009
420 listen smtp_proxy 127.0.0.1:25,127.0.0.1:587
421 listen ldap_proxy :389,:663
422
423In the event that all addresses do not fit line width, it's preferable to
424detach secondary addresses on other lines with the 'bind' keyword. If this
425keyword is used, it's not even necessary to specify the first address on the
426'listen' line, which sometimes makes multiple configuration handling easier :
427
428 bind [ <IP_address>:<port_range>[,...] ]
429
430Examples :
431----------
432 listen http_proxy
433 bind :80,:443
willy tarreauc5f73ed2005-12-18 01:26:38 +0100434 bind 10.0.0.1:10080,10.0.0.1:10443
435
willy tarreaueedaa9f2005-12-17 14:08:03 +0100436
4372.1) Inhibiting a service
438-------------------------
439A service may be disabled for maintenance reasons, without needing to comment
440out the whole section, simply by specifying the 'disabled' keyword in the
441section to be disabled :
442
443 listen smtp_proxy 0.0.0.0:25
willy tarreauc5f73ed2005-12-18 01:26:38 +0100444 disabled
willy tarreaueedaa9f2005-12-17 14:08:03 +0100445
446Note: the 'enabled' keyword allows to enable a service which has been disabled
447 previously by a default configuration.
448
willy tarreauc5f73ed2005-12-18 01:26:38 +0100449
willy tarreaueedaa9f2005-12-17 14:08:03 +01004502.2) Modes of operation
451-----------------------
452A service can work in 3 different distinct modes :
453 - TCP
454 - HTTP
willy tarreau532bb552006-05-13 18:40:37 +0200455 - health
willy tarreaueedaa9f2005-12-17 14:08:03 +0100456
457TCP mode
458--------
459In this mode, the service relays TCP connections as soon as they're established,
460towards one or several servers. No processing is done on the stream. It's only
461an association of source(addr:port) -> destination(addr:port). To use this mode,
462you must specify 'mode tcp' in the 'listen' section. This is the default mode.
463
464Example :
465---------
466 listen smtp_proxy 0.0.0.0:25
willy tarreauc5f73ed2005-12-18 01:26:38 +0100467 mode tcp
willy tarreaueedaa9f2005-12-17 14:08:03 +0100468
469HTTP mode
470---------
471In this mode, the service relays TCP connections towards one or several servers,
472when it has enough informations to decide, which normally means that all HTTP
473headers have been read. Some of them may be scanned for a cookie or a pattern
474matching a regex. To use this mode, specify 'mode http' in the 'listen' section.
475
476Example :
477---------
478 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +0100479 mode http
willy tarreaueedaa9f2005-12-17 14:08:03 +0100480
481Health-checking mode
482--------------------
483This mode provides a way for external components to check the proxy's health.
484It is meant to be used with intelligent load-balancers which can use send/expect
485scripts to check for all of their servers' availability. This one simply accepts
willy tarreau197e8ec2005-12-17 14:10:59 +0100486the connection, returns the word 'OK' and closes it. If the 'option httpchk' is
487set, then the reply will be 'HTTP/1.0 200 OK' with no data, so that it can be
488tested from a tool which supports HTTP health-checks. To enable it, simply
willy tarreaueedaa9f2005-12-17 14:08:03 +0100489specify 'health' as the working mode :
490
491Example :
492---------
willy tarreau197e8ec2005-12-17 14:10:59 +0100493 # simple response : 'OK'
willy tarreaueedaa9f2005-12-17 14:08:03 +0100494 listen health_check 0.0.0.0:60000
willy tarreauc5f73ed2005-12-18 01:26:38 +0100495 mode health
willy tarreaueedaa9f2005-12-17 14:08:03 +0100496
willy tarreau197e8ec2005-12-17 14:10:59 +0100497 # HTTP response : 'HTTP/1.0 200 OK'
498 listen http_health_check 0.0.0.0:60001
willy tarreauc5f73ed2005-12-18 01:26:38 +0100499 mode health
500 option httpchk
501
willy tarreau532bb552006-05-13 18:40:37 +02005022.2.1 Monitoring
503----------------
willy tarreauc5f73ed2005-12-18 01:26:38 +0100504Versions 1.1.32 and 1.2.6 provide a new solution to check the proxy's
505availability without perturbating the service. The 'monitor-net' keyword was
506created to specify a network of equipments which CANNOT use the service for
507anything but health-checks. This is particularly suited to TCP proxies, because
508it prevents the proxy from relaying the monitor's connection to the remote
509server.
510
511When used with TCP, the connection is accepted then closed and nothing is
512logged. This is enough for a front-end load-balancer to detect the service as
513available.
willy tarreau197e8ec2005-12-17 14:10:59 +0100514
willy tarreauc5f73ed2005-12-18 01:26:38 +0100515When used with HTTP, the connection is accepted, nothing is logged, the
516following response is sent, then the session is closed : "HTTP/1.0 200 OK".
517This is normally enough for any front-end HTTP load-balancer to detect the
518service as available too, both with TCP and HTTP checks.
519
520Proxies using the "monitor-net" keyword can remove the "option dontlognull", as
521it will make them log empty connections from hosts outside the monitoring
522network.
523
524Example :
525---------
526
527 listen tse-proxy
528 bind :3389,:1494,:5900 # TSE, ICA and VNC at once.
529 mode tcp
530 balance roundrobin
531 server tse-farm 192.168.1.10
532 monitor-net 192.168.1.252/31 # L4 load-balancers on .252 and .253
533
willy tarreaueedaa9f2005-12-17 14:08:03 +0100534
Willy Tarreau1c47f852006-07-09 08:22:27 +0200535When the system executing the checks is located behind a proxy, the monitor-net
536keyword cannot be used because haproxy will always see the proxy's address. To
537overcome this limitation, version 1.2.15 brought the 'monitor-uri' keyword. It
538defines an URI which will not be forwarded nor logged, but for which haproxy
539will immediately send an "HTTP/1.0 200 OK" response. This makes it possible to
540check the validity of the reverse-proxy->haproxy chain with one request. It can
541be used in HTTPS checks in front of an stunnel -> haproxy combination for
542instance. Obviously, this keyword is only valid in HTTP mode, otherwise there
543is no notion of URI. Note that the method and HTTP versions are simply ignored.
544
545Example :
546---------
547
548 listen stunnel_backend :8080
549 mode http
550 balance roundrobin
551 server web1 192.168.1.10:80 check
552 server web2 192.168.1.11:80 check
553 monitor-uri /haproxy_test
554
555
willy tarreaueedaa9f2005-12-17 14:08:03 +01005562.3) Limiting the number of simultaneous connections
557----------------------------------------------------
558The 'maxconn' parameter allows a proxy to refuse connections above a certain
559amount of simultaneous ones. When the limit is reached, it simply stops
560listening, but the system may still be accepting them because of the back log
willy tarreau982249e2005-12-18 00:57:06 +0100561queue. These connections will be processed later when other ones have freed
willy tarreaueedaa9f2005-12-17 14:08:03 +0100562some slots. This provides a serialization effect which helps very fragile
willy tarreau34f45302006-04-15 21:37:14 +0200563servers resist to high loads. See further for system limitations.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100564
565Example :
566---------
567 listen tiny_server 0.0.0.0:80
568 maxconn 10
569
570
5712.4) Soft stop
572--------------
573It is possible to stop services without breaking existing connections by the
willy tarreau22739ef2006-01-20 20:43:32 +0100574sending of the SIGUSR1 signal to the process. All services are then put into
willy tarreaueedaa9f2005-12-17 14:08:03 +0100575soft-stop state, which means that they will refuse to accept new connections,
576except for those which have a non-zero value in the 'grace' parameter, in which
577case they will still accept connections for the specified amount of time, in
willy tarreau22739ef2006-01-20 20:43:32 +0100578milliseconds. This makes it possible to tell a load-balancer that the service
579is failing, while still doing the job during the time it needs to detect it.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100580
581Note: active connections are never killed. In the worst case, the user will have
582to wait for all of them to close or to time-out, or simply kill the process
willy tarreau22739ef2006-01-20 20:43:32 +0100583normally (SIGTERM). The default 'grace' value is '0'.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100584
585Example :
586---------
587 # enter soft stop after 'killall -USR1 haproxy'
588 # the service will still run 10 seconds after the signal
589 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +0100590 mode http
591 grace 10000
willy tarreaueedaa9f2005-12-17 14:08:03 +0100592
593 # this port is dedicated to a load-balancer, and must fail immediately
594 listen health_check 0.0.0.0:60000
willy tarreauc5f73ed2005-12-18 01:26:38 +0100595 mode health
596 grace 0
willy tarreaueedaa9f2005-12-17 14:08:03 +0100597
598
willy tarreau39df2dc2006-01-29 21:56:05 +0100599As of version 1.2.8, a new soft-reconfiguration mechanism has been introduced.
willy tarreau22739ef2006-01-20 20:43:32 +0100600It is now possible to "pause" all the proxies by sending a SIGTTOU signal to
601the processes. This will disable the listening socket without breaking existing
602connections. After that, sending a SIGTTIN signal to those processes enables
603the listening sockets again. This is very useful to try to load a new
604configuration or even a new version of haproxy without breaking existing
605connections. If the load succeeds, then simply send a SIGUSR1 which will make
606the previous proxies exit immediately once their sessions are closed ; and if
607the load fails, then simply send a SIGTTIN to restore the service immediately.
608Please note that the 'grace' parameter is ignored for SIGTTOU, as well as for
609SIGUSR1 when the process was in the pause mode. Please also note that it would
610be useful to save the pidfile before starting a new instance.
611
willy tarreau34f45302006-04-15 21:37:14 +0200612This mechanism fully exploited since 1.2.11 with the '-st' and '-sf' options
willy tarreau532bb552006-05-13 18:40:37 +0200613(see below).
614
6152.4.1) Hot reconfiguration
616--------------------------
617The '-st' and '-sf' command line options are used to inform previously running
618processes that a configuration is being reloaded. They will receive the SIGTTOU
619signal to ask them to temporarily stop listening to the ports so that the new
620process can grab them. If anything wrong happens, the new process will send
621them a SIGTTIN to tell them to re-listen to the ports and continue their normal
622work. Otherwise, it will either ask them to finish (-sf) their work then softly
623exit, or immediately terminate (-st), breaking existing sessions. A typical use
624of this allows a configuration reload without service interruption :
625
626 # haproxy -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid)
627
willy tarreau22739ef2006-01-20 20:43:32 +0100628
willy tarreaueedaa9f2005-12-17 14:08:03 +01006292.5) Connections expiration time
630--------------------------------
631It is possible (and recommended) to configure several time-outs on TCP
Jamie Gloudon801a0a32012-08-25 00:18:33 -0400632connections. Three independent timers are adjustable with values specified
willy tarreaueedaa9f2005-12-17 14:08:03 +0100633in milliseconds. A session will be terminated if either one of these timers
634expire.
635
636 - the time we accept to wait for data from the client, or for the client to
637 accept data : 'clitimeout' :
638
willy tarreauc5f73ed2005-12-18 01:26:38 +0100639 # client time-out set to 2mn30.
640 clitimeout 150000
willy tarreaueedaa9f2005-12-17 14:08:03 +0100641
642 - the time we accept to wait for data from the server, or for the server to
643 accept data : 'srvtimeout' :
644
willy tarreauc5f73ed2005-12-18 01:26:38 +0100645 # server time-out set to 30s.
646 srvtimeout 30000
willy tarreaueedaa9f2005-12-17 14:08:03 +0100647
648 - the time we accept to wait for a connection to establish on a server :
649 'contimeout' :
650
651 # we give up if the connection does not complete within 4 seconds
willy tarreauc5f73ed2005-12-18 01:26:38 +0100652 contimeout 4000
willy tarreaueedaa9f2005-12-17 14:08:03 +0100653
654Notes :
655-------
656 - 'contimeout' and 'srvtimeout' have no sense on 'health' mode servers ;
657 - under high loads, or with a saturated or defective network, it's possible
658 that some packets get lost. Since the first TCP retransmit only happens
659 after 3 seconds, a time-out equal to, or lower than 3 seconds cannot
660 compensate for a packet loss. A 4 seconds time-out seems a reasonable
661 minimum which will considerably reduce connection failures.
Willy Tarreaubefdff12007-12-02 22:27:38 +0100662 - starting with version 1.3.14, it is possible to specify timeouts in
663 arbitrary time units among { us, ms, s, m, h, d }. For this, the integer
664 value just has to be suffixed with the unit.
willy tarreauc5f73ed2005-12-18 01:26:38 +0100665
willy tarreaueedaa9f2005-12-17 14:08:03 +01006662.6) Attempts to reconnect
667--------------------------
668After a connection failure to a server, it is possible to retry, potentially
669on another server. This is useful if health-checks are too rare and you don't
670want the clients to see the failures. The number of attempts to reconnect is
671set by the 'retries' paramter.
672
673Example :
674---------
willy tarreauc5f73ed2005-12-18 01:26:38 +0100675 # we can retry 3 times max after a failure
676 retries 3
willy tarreaueedaa9f2005-12-17 14:08:03 +0100677
willy tarreau34f45302006-04-15 21:37:14 +0200678Please note that the reconnection attempt may lead to getting the connection
679sent to a new server if the original one died between connection attempts.
680
willy tarreaueedaa9f2005-12-17 14:08:03 +0100681
6822.7) Address of the dispatch server (deprecated)
683------------------------------------------------
684The server which will be sent all new connections is defined by the 'dispatch'
685parameter, in the form <address>:<port>. It generally is dedicated to unknown
686connections and will assign them a cookie, in case of HTTP persistence mode,
687or simply is a single server in case of generic TCP proxy. This old mode is only
688provided for backwards compatibility, but doesn't allow to check remote servers
689state, and has a rather limited usage. All new setups should switch to 'balance'
690mode. The principle of the dispatcher is to be able to perform the load
691balancing itself, but work only on new clients so that the server doesn't need
692to be a big machine.
693
694Example :
695---------
willy tarreauc5f73ed2005-12-18 01:26:38 +0100696 # all new connections go there
697 dispatch 192.168.1.2:80
willy tarreaueedaa9f2005-12-17 14:08:03 +0100698
699Note :
700------
701This parameter has no sense for 'health' servers, and is incompatible with
702'balance' mode.
703
704
7052.8) Outgoing source address
706----------------------------
707It is often necessary to bind to a particular address when connecting to some
708remote hosts. This is done via the 'source' parameter which is a per-proxy
709parameter. A newer version may allow to fix different sources to reach different
710servers. The syntax is 'source <address>[:<port>]', where <address> is a valid
711local address (or '0.0.0.0' or '*' or empty to let the system choose), and
712<port> is an optional parameter allowing the user to force the source port for
713very specific needs. If the port is not specified or is '0', the system will
714choose a free port. Note that as of version 1.1.18, the servers health checks
715are also performed from the same source.
716
717Examples :
718----------
719 listen http_proxy *:80
willy tarreauc5f73ed2005-12-18 01:26:38 +0100720 # all connections take 192.168.1.200 as source address
721 source 192.168.1.200:0
willy tarreaueedaa9f2005-12-17 14:08:03 +0100722
723 listen rlogin_proxy *:513
willy tarreauc5f73ed2005-12-18 01:26:38 +0100724 # use address 192.168.1.200 and the reserved port 900 (needs to be root)
725 source 192.168.1.200:900
willy tarreaueedaa9f2005-12-17 14:08:03 +0100726
727
7282.9) Setting the cookie name
729----------------------------
730In HTTP mode, it is possible to look for a particular cookie which will contain
731a server identifier which should handle the connection. The cookie name is set
732via the 'cookie' parameter.
733
734Example :
735---------
736 listen http_proxy :80
willy tarreauc5f73ed2005-12-18 01:26:38 +0100737 mode http
738 cookie SERVERID
willy tarreaueedaa9f2005-12-17 14:08:03 +0100739
740It is possible to change the cookie behaviour to get a smarter persistence,
741depending on applications. It is notably possible to delete or modify a cookie
742emitted by a server, insert a cookie identifying the server in an HTTP response
743and even add a header to tell upstream caches not to cache this response.
744
745Examples :
746----------
747
748To remove the cookie for direct accesses (ie when the server matches the one
749which was specified in the client cookie) :
750
willy tarreauc5f73ed2005-12-18 01:26:38 +0100751 cookie SERVERID indirect
willy tarreaueedaa9f2005-12-17 14:08:03 +0100752
753To replace the cookie value with the one assigned to the server if any (no
754cookie will be created if the server does not provide one, nor if the
755configuration does not provide one). This lets the application put the cookie
756exactly on certain pages (eg: successful authentication) :
757
willy tarreauc5f73ed2005-12-18 01:26:38 +0100758 cookie SERVERID rewrite
willy tarreaueedaa9f2005-12-17 14:08:03 +0100759
760To create a new cookie and assign the server identifier to it (in this case, all
761servers should be associated with a valid cookie, since no cookie will simply
762delete the cookie from the client's browser) :
763
willy tarreauc5f73ed2005-12-18 01:26:38 +0100764 cookie SERVERID insert
willy tarreaueedaa9f2005-12-17 14:08:03 +0100765
willy tarreau0174f312005-12-18 01:02:42 +0100766To reuse an existing application cookie and prefix it with the server's
767identifier, and remove it in the request, use the 'prefix' option. This allows
768to insert a haproxy in front of an application without risking to break clients
769which does not support more than one cookie :
770
willy tarreauc5f73ed2005-12-18 01:26:38 +0100771 cookie JSESSIONID prefix
willy tarreau0174f312005-12-18 01:02:42 +0100772
willy tarreaueedaa9f2005-12-17 14:08:03 +0100773To insert a cookie and ensure that no upstream cache will store it, add the
774'nocache' option :
775
willy tarreauc5f73ed2005-12-18 01:26:38 +0100776 cookie SERVERID insert nocache
willy tarreaueedaa9f2005-12-17 14:08:03 +0100777
778To insert a cookie only after a POST request, add 'postonly' after 'insert'.
779This has the advantage that there's no risk of caching, and that all pages
780seen before the POST one can still be cached :
781
willy tarreauc5f73ed2005-12-18 01:26:38 +0100782 cookie SERVERID insert postonly
willy tarreaueedaa9f2005-12-17 14:08:03 +0100783
784Notes :
785-----------
786- it is possible to combine 'insert' with 'indirect' or 'rewrite' to adapt to
787 applications which already generate the cookie with an invalid content.
788
789- in the case where 'insert' and 'indirect' are both specified, the cookie is
willy tarreau0174f312005-12-18 01:02:42 +0100790 never transmitted to the server, since it wouldn't understand it. This is the
791 most application-transparent mode.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100792
793- it is particularly recommended to use 'nocache' in 'insert' mode if any
794 upstream HTTP/1.0 cache is susceptible to cache the result, because this may
795 lead to many clients going to the same server, or even worse, some clients
796 having their server changed while retrieving a page from the cache.
797
willy tarreau0174f312005-12-18 01:02:42 +0100798- the 'prefix' mode normally does not need 'indirect', 'nocache', nor
799 'postonly', because just as in the 'rewrite' mode, it relies on the
800 application to know when a cookie can be emitted. However, since it has to
801 fix the cookie name in every subsequent requests, you must ensure that the
802 proxy will be used without any "HTTP keep-alive". Use option "httpclose" if
803 unsure.
804
willy tarreaueedaa9f2005-12-17 14:08:03 +0100805- when the application is well known and controlled, the best method is to
806 only add the persistence cookie on a POST form because it's up to the
willy tarreau0174f312005-12-18 01:02:42 +0100807 application to select which page it wants the upstream servers to cache. In
808 this case, you would use 'insert postonly indirect'.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100809
willy tarreauc5f73ed2005-12-18 01:26:38 +0100810
willy tarreaueedaa9f2005-12-17 14:08:03 +01008112.10) Associating a cookie value with a server
812----------------------------------------------
813In HTTP mode, it's possible to associate a cookie value to each server. This
814was initially used in combination with 'dispatch' mode to handle direct accesses
815but it is now the standard way of doing the load balancing. The syntax is :
816
817 server <identifier> <address>:<port> cookie <value>
818
819- <identifier> is any name which can be used to identify the server in the logs.
820- <address>:<port> specifies where the server is bound.
821- <value> is the value to put in or to read from the cookie.
822
823Example : the 'SERVERID' cookie can be either 'server01' or 'server02'
824---------
825 listen http_proxy :80
willy tarreauc5f73ed2005-12-18 01:26:38 +0100826 mode http
827 cookie SERVERID
828 dispatch 192.168.1.100:80
829 server web1 192.168.1.1:80 cookie server01
830 server web2 192.168.1.2:80 cookie server02
willy tarreaueedaa9f2005-12-17 14:08:03 +0100831
832Warning : the syntax has changed since version 1.0 !
833---------
834
willy tarreauc5f73ed2005-12-18 01:26:38 +0100835
willy tarreau598da412005-12-18 01:07:29 +01008362.11) Application Cookies
837-------------------------
838Since 1.2.4 it is possible to catch the cookie that comes from an
839application server in order to apply "application session stickyness".
840The server's response is searched for 'appsession' cookie, the first
841'len' bytes are used for matching and it is stored for a period of
842'timeout'.
843The syntax is:
844
willy tarreau532bb552006-05-13 18:40:37 +0200845 appsession <session_cookie> len <match_length> timeout <holdtime>
willy tarreau598da412005-12-18 01:07:29 +0100846
willy tarreau532bb552006-05-13 18:40:37 +0200847- <session_cookie> is the cookie, the server uses for it's session-handling
848- <match_length> how many bytes/characters should be used for matching equal
willy tarreau598da412005-12-18 01:07:29 +0100849 sessions
willy tarreau532bb552006-05-13 18:40:37 +0200850- <holdtime> after this inactivaty time, in ms, the cookie will be deleted
willy tarreau598da412005-12-18 01:07:29 +0100851 from the sessionstore
Willy Tarreaubefdff12007-12-02 22:27:38 +0100852- starting with version 1.3.14, it is possible to specify timeouts in
853 arbitrary time units among { us, ms, s, m, h, d }. For this, the integer
854 value just has to be prefixed with the unit.
willy tarreau598da412005-12-18 01:07:29 +0100855
856The appsession is only per 'listen' section possible.
857
858Example :
859---------
willy tarreau532bb552006-05-13 18:40:37 +0200860 listen http_lb1 192.168.3.4:80
861 mode http
862 capture request header Cookie len 200
863 # Havind a ServerID cookie on the client allows him to reach
864 # the right server even after expiration of the appsession.
865 cookie ServerID insert nocache indirect
866 # Will memorize 52 bytes of the cookie 'JSESSIONID' and keep them
867 # for 3 hours. It will match it in the cookie and the URL field.
Willy Tarreaubefdff12007-12-02 22:27:38 +0100868 appsession JSESSIONID len 52 timeout 3h
willy tarreau532bb552006-05-13 18:40:37 +0200869 server first1 10.3.9.2:10805 check inter 3000 cookie first
870 server secon1 10.3.9.3:10805 check inter 3000 cookie secon
871 server first1 10.3.9.4:10805 check inter 3000 cookie first
872 server secon2 10.3.9.5:10805 check inter 3000 cookie secon
873 option httpchk GET /test.jsp
willy tarreau598da412005-12-18 01:07:29 +0100874
willy tarreauc5f73ed2005-12-18 01:26:38 +0100875
willy tarreaueedaa9f2005-12-17 14:08:03 +01008763) Autonomous load balancer
877===========================
878
879The proxy can perform the load-balancing itself, both in TCP and in HTTP modes.
880This is the most interesting mode which obsoletes the old 'dispatch' mode
881described above. It has advantages such as server health monitoring, multiple
882port binding and port mapping. To use this mode, the 'balance' keyword is used,
willy tarreau34f45302006-04-15 21:37:14 +0200883followed by the selected algorithm. Up to version 1.2.11, only 'roundrobin' was
884available, which is also the default value if unspecified. Starting with
Willy Tarreau2fcb5002007-05-08 13:35:26 +0200885version 1.2.12, a new 'source' keyword appeared. A new 'uri' keyword was added
886in version 1.3.10. In this mode, there will be no dispatch address, but the
887proxy needs at least one server.
willy tarreaueedaa9f2005-12-17 14:08:03 +0100888
889Example : same as the last one, with internal load balancer
890---------
891
892 listen http_proxy :80
willy tarreauc5f73ed2005-12-18 01:26:38 +0100893 mode http
894 cookie SERVERID
895 balance roundrobin
896 server web1 192.168.1.1:80 cookie server01
897 server web2 192.168.1.2:80 cookie server02
willy tarreaueedaa9f2005-12-17 14:08:03 +0100898
899
900Since version 1.1.22, it is possible to automatically determine on which port
901the server will get the connection, depending on the port the client connected
902to. Indeed, there now are 4 possible combinations for the server's <port> field:
903
904 - unspecified or '0' :
905 the connection will be sent to the same port as the one on which the proxy
906 received the client connection itself.
907
908 - numerical value (the only one supported in versions earlier than 1.1.22) :
909 the connection will always be sent to the specified port.
910
911 - '+' followed by a numerical value :
912 the connection will be sent to the same port as the one on which the proxy
913 received the connection, plus this value.
914
915 - '-' followed by a numerical value :
916 the connection will be sent to the same port as the one on which the proxy
917 received the connection, minus this value.
918
919Examples :
920----------
921
922# same as previous example
923
924 listen http_proxy :80
willy tarreauc5f73ed2005-12-18 01:26:38 +0100925 mode http
926 cookie SERVERID
927 balance roundrobin
928 server web1 192.168.1.1 cookie server01
929 server web2 192.168.1.2 cookie server02
willy tarreaueedaa9f2005-12-17 14:08:03 +0100930
931# simultaneous relaying of ports 80, 81 and 8080-8089
932
933 listen http_proxy :80,:81,:8080-8089
willy tarreauc5f73ed2005-12-18 01:26:38 +0100934 mode http
935 cookie SERVERID
936 balance roundrobin
937 server web1 192.168.1.1 cookie server01
938 server web2 192.168.1.2 cookie server02
willy tarreaueedaa9f2005-12-17 14:08:03 +0100939
940# relaying of TCP ports 25, 389 and 663 to ports 1025, 1389 and 1663
941
942 listen http_proxy :25,:389,:663
willy tarreauc5f73ed2005-12-18 01:26:38 +0100943 mode tcp
944 balance roundrobin
945 server srv1 192.168.1.1:+1000
946 server srv2 192.168.1.2:+1000
willy tarreaueedaa9f2005-12-17 14:08:03 +0100947
willy tarreau34f45302006-04-15 21:37:14 +0200948As previously stated, version 1.2.12 brought the 'source' keyword. When this
949keyword is used, the client's IP address is hashed and evenly distributed among
950the available servers so that a same source IP will always go to the same
951server as long as there are no change in the number of available servers. This
952can be used for instance to bind HTTP and HTTPS to the same server. It can also
953be used to improve stickyness when one part of the client population does not
954accept cookies. In this case, only those ones will be perturbated should a
955server fail.
956
957NOTE: It is important to consider the fact that many clients surf the net
958 through proxy farms which assign different IP addresses for each
959 request. Others use dialup connections with a different IP at each
960 connection. Thus, the 'source' parameter should be used with extreme
961 care.
962
963Examples :
964----------
965
966# make a same IP go to the same server whatever the service
967
968 listen http_proxy
969 bind :80,:443
970 mode http
971 balance source
972 server web1 192.168.1.1
973 server web2 192.168.1.2
974
975# try to improve client-server binding by using both source IP and cookie :
976
977 listen http_proxy :80
978 mode http
979 cookie SERVERID
980 balance source
981 server web1 192.168.1.1 cookie server01
982 server web2 192.168.1.2 cookie server02
983
Willy Tarreau2fcb5002007-05-08 13:35:26 +0200984As indicated above, the 'uri' keyword was introduced in version 1.3.10. It is
985useful when load-balancing between reverse proxy-caches, because it will hash
986the URI and use the hash result to select a server, thus optimizing the hit
987rate on the caches, because the same URI will always reach the same cache. This
988keyword is only allowed in HTTP mode.
989
990Example :
991---------
992
993# Always send a given URI to the same server
994
995 listen http_proxy
996 bind :3128
997 mode http
998 balance uri
999 server squid1 192.168.1.1
1000 server squid2 192.168.1.2
1001
Willy Tarreau01732802007-11-01 22:48:15 +01001002Version 1.3.14 introduced the "balance url_param" method. It consists in
1003relying on a parameter passed in the URL to perform a hash. This is mostly
1004useful for applications which do not have strict persistence requirements,
1005but for which it still provides a performance boost due to local caching.
1006Some of these applications may not be able to use a cookie for whatever reason,
1007but may be able to look for a parameter passed in the URL. If the parameter is
1008missing from the URL, then the 'round robin' method applies.
1009
matt.farnsworth@nokia.com1c2ab962008-04-14 20:47:37 +02001010A modifier may be added to specify that parameters in POST requests may be
1011found in the messsage body if the URL lacks a '?' separator character.
1012A wait limit may also be applied, if no limit is requested then
1013the default value is 48 octets, the minimum is 3. HAProxy may wait, until 48
1014octets are received. If Content-Length is missing, or zero it need not
1015wait for more data then the client promissed to send. When Content-Length is
1016present, and more than <max_wait>; then waiting is limited and it is assumed this
1017will be enough data to search for the presence of a parameter. If
1018Transfer-Encoding: chunked is used (unlikely), then the length of the first chunk
1019is the maximum number of bytes to wait for.
1020
1021balance url_param <param> [check_post [<max_wait>]]
1022
1023Caveats for using the check_post extension:
1024
1025 - all POST requests are eligable for consideration, because there is
1026 no way to determine if the parameters will be found in the body or
1027 entity which may contain binary data. Therefore another method may be
1028 required to restrict consideration of POST requests that have no URL
1029 parameters in the body. (see acl reqideny http_end)
1030
1031Limitations on inspecting the entity body of a POST:
1032
1033 - Content-Encoding is not supported, the parameter search will probably fail;
1034 and load balancing will fall back to Round Robin.
1035
1036 - Expect: 100-continue is not supported, load balancing will fall back to
1037 Round Robin.
1038
1039 - Transfer-Encoding(RFC2616 3.6.1) is only supported in the first chunk. If
1040 the entire parameter value is not present in the first chunk, the selection
1041 of server is undefined (actually, defined by how little actually appeared in
1042 the first chunk).
1043
1044 - This feature does not support generation of a 100, 411 or 501 response.
1045
1046 - In some cases, requesting check_post MAY attempt to scan the entire contents
1047 of a message body. Scaning normally terminates when linear white space or
1048 control characters are found, indicating the end of what might be a URL parameter
1049 list. This is probably not a concern with SGML type message bodies.
1050
1051
Willy Tarreau01732802007-11-01 22:48:15 +01001052Example :
1053---------
1054
1055# Hash the "basket_id" argument from the URL to determine the server
1056
1057 listen http_proxy
1058 bind :3128
1059 mode http
1060 balance url_param basket_id
1061 server ebiz1 192.168.1.1
1062 server ebiz2 192.168.1.2
1063
willy tarreaueedaa9f2005-12-17 14:08:03 +01001064
willy tarreau197e8ec2005-12-17 14:10:59 +010010653.1) Server monitoring
1066----------------------
willy tarreaueedaa9f2005-12-17 14:08:03 +01001067It is possible to check the servers status by trying to establish TCP
1068connections or even sending HTTP requests to them. A server which fails to
1069reply to health checks as expected will not be used by the load balancing
1070algorithms. To enable monitoring, add the 'check' keyword on a server line.
1071It is possible to specify the interval between tests (in milliseconds) with
1072the 'inter' parameter, the number of failures supported before declaring that
1073the server has fallen down with the 'fall' parameter, and the number of valid
1074checks needed for the server to fully get up with the 'rise' parameter. Since
1075version 1.1.22, it is also possible to send checks to a different port
1076(mandatory when none is specified) with the 'port' parameter. The default
1077values are the following ones :
1078
1079 - inter : 2000
1080 - rise : 2
1081 - fall : 3
1082 - port : default server port
Willy Tarreau2ea3abb2007-03-25 16:45:16 +02001083 - addr : specific address for the test (default = address server)
1084
willy tarreaueedaa9f2005-12-17 14:08:03 +01001085The default mode consists in establishing TCP connections only. But in certain
1086types of application failures, it is often that the server continues to accept
1087connections because the system does it itself while the application is running
1088an endless loop, or is completely stuck. So in version 1.1.16 were introduced
1089HTTP health checks which only performed simple lightweight requests and analysed
1090the response. Now, as of version 1.1.23, it is possible to change the HTTP
1091method, the URI, and the HTTP version string (which even allows to send headers
1092with a dirty trick). To enable HTTP health-checks, use 'option httpchk'.
1093
1094By default, requests use the 'OPTIONS' method because it's very light and easy
1095to filter from logs, and does it on '/'. Only HTTP responses 2xx and 3xx are
1096considered valid ones, and only if they come before the time to send a new
1097request is reached ('inter' parameter). If some servers block this type of
1098request, 3 other forms help to forge a request :
1099
1100 - option httpchk -> OPTIONS / HTTP/1.0
1101 - option httpchk URI -> OPTIONS <URI> HTTP/1.0
1102 - option httpchk METH URI -> <METH> <URI> HTTP/1.0
1103 - option httpchk METH URI VER -> <METH> <URI> <VER>
1104
Willy Tarreauf3c69202006-07-09 16:42:34 +02001105Some people are using HAProxy to relay various TCP-based protocols such as
1106HTTPS, SMTP or LDAP, with the most common one being HTTPS. One problem commonly
1107encountered in data centers is the need to forward the traffic to far remote
1108servers while providing server fail-over. Often, TCP-only checks are not enough
1109because intermediate firewalls, load balancers or proxies might acknowledge the
1110connection before it reaches the real server. The only solution to this problem
1111is to send application-level health checks. Since the demand for HTTPS checks
1112is high, it has been implemented in 1.2.15 based on SSLv3 Client Hello packets.
1113To enable it, use 'option ssl-hello-chk'. It will send SSL CLIENT HELLO packets
1114to the servers, announcing support for most common cipher suites. If the server
1115responds what looks like a SERVER HELLO or an ALERT (refuses the ciphers) then
1116the response is considered as valid. Note that Apache does not generate a log
1117when it receives only an HELLO message, which makes this type of message
1118perfectly suit this need.
1119
Willy Tarreau23677902007-05-08 23:50:35 +02001120Version 1.3.10 introduced the SMTP health check. By default, it sends
1121"HELO localhost" to the servers, and waits for the 250 message. Note that it
1122can also send a specific request :
1123
1124 - option smtpchk -> sends "HELO localhost"
1125 - option smtpchk EHLO mail.mydomain.com -> sends this ESMTP greeting
1126
willy tarreaueedaa9f2005-12-17 14:08:03 +01001127See examples below.
1128
1129Since version 1.1.17, it is possible to specify backup servers. These servers
1130are only sollicited when no other server is available. This may only be useful
1131to serve a maintenance page, or define one active and one backup server (seldom
1132used in TCP mode). To make a server a backup one, simply add the 'backup' option
1133on its line. These servers also support cookies, so if a cookie is specified for
1134a backup server, clients assigned to this server will stick to it even when the
1135other ones come back. Conversely, if no cookie is assigned to such a server,
1136the clients will get their cookies removed (empty cookie = removal), and will
1137be balanced against other servers once they come back. Please note that there
Willy TARREAU3481c462006-03-01 22:37:57 +01001138is no load-balancing among backup servers by default. If there are several
1139backup servers, the second one will only be used when the first one dies, and
1140so on. To force load-balancing between backup servers, specify the 'allbackups'
1141option.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001142
Willy Tarreau2ea3abb2007-03-25 16:45:16 +02001143Since version 1.1.22, it is possible to send health checks to a different port
1144than the service. It is mainly needed in setups where the server does not have
1145any predefined port, for instance when the port is deduced from the listening
1146port. For this, use the 'port' parameter followed by the port number which must
1147respond to health checks. It is also possible to send health checks to a
1148different address than the service. It makes it easier to use a dedicated check
1149daemon on the servers, for instance, check return contents and stop several
1150farms at once in the event of an error anywhere.
1151
willy tarreaueedaa9f2005-12-17 14:08:03 +01001152Since version 1.1.17, it is also possible to visually check the status of all
1153servers at once. For this, you just have to send a SIGHUP signal to the proxy.
1154The servers status will be dumped into the logs at the 'notice' level, as well
1155as on <stderr> if not closed. For this reason, it's always a good idea to have
1156one local log server at the 'notice' level.
1157
willy tarreau982249e2005-12-18 00:57:06 +01001158Since version 1.1.28 and 1.2.1, if an instance loses all its servers, an
willy tarreau0174f312005-12-18 01:02:42 +01001159emergency message will be sent in the logs to inform the administator that an
willy tarreau982249e2005-12-18 00:57:06 +01001160immediate action must be taken.
1161
willy tarreau0174f312005-12-18 01:02:42 +01001162Since version 1.1.30 and 1.2.3, several servers can share the same cookie
1163value. This is particularly useful in backup mode, to select alternate paths
1164for a given server for example, to provide soft-stop, or to direct the clients
1165to a temporary page during an application restart. The principle is that when
1166a server is dead, the proxy will first look for another server which shares the
1167same cookie value for every client which presents the cookie. If there is no
1168standard server for this cookie, it will then look for a backup server which
1169shares the same name. Please consult the architecture guide for more information.
willy tarreau982249e2005-12-18 00:57:06 +01001170
willy tarreaueedaa9f2005-12-17 14:08:03 +01001171Examples :
1172----------
1173# same setup as in paragraph 3) with TCP monitoring
1174 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001175 mode http
1176 cookie SERVERID
1177 balance roundrobin
1178 server web1 192.168.1.1:80 cookie server01 check
1179 server web2 192.168.1.2:80 cookie server02 check inter 500 rise 1 fall 2
willy tarreaueedaa9f2005-12-17 14:08:03 +01001180
1181# same with HTTP monitoring via 'OPTIONS / HTTP/1.0'
1182 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001183 mode http
1184 cookie SERVERID
1185 balance roundrobin
1186 option httpchk
1187 server web1 192.168.1.1:80 cookie server01 check
1188 server web2 192.168.1.2:80 cookie server02 check inter 500 rise 1 fall 2
willy tarreaueedaa9f2005-12-17 14:08:03 +01001189
1190# same with HTTP monitoring via 'OPTIONS /index.html HTTP/1.0'
1191 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001192 mode http
1193 cookie SERVERID
1194 balance roundrobin
1195 option httpchk /index.html
1196 server web1 192.168.1.1:80 cookie server01 check
1197 server web2 192.168.1.2:80 cookie server02 check inter 500 rise 1 fall 2
willy tarreaueedaa9f2005-12-17 14:08:03 +01001198
1199# same with HTTP monitoring via 'HEAD /index.jsp? HTTP/1.1\r\nHost: www'
1200 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001201 mode http
1202 cookie SERVERID
1203 balance roundrobin
1204 option httpchk HEAD /index.jsp? HTTP/1.1\r\nHost:\ www
1205 server web1 192.168.1.1:80 cookie server01 check
1206 server web2 192.168.1.2:80 cookie server02 check inter 500 rise 1 fall 2
willy tarreaueedaa9f2005-12-17 14:08:03 +01001207
willy tarreau0174f312005-12-18 01:02:42 +01001208# Load-balancing with 'prefixed cookie' persistence, and soft-stop using an
1209# alternate port 81 on the server for health-checks.
1210 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001211 mode http
1212 cookie JSESSIONID prefix
1213 balance roundrobin
1214 option httpchk HEAD /index.jsp? HTTP/1.1\r\nHost:\ www
1215 server web1-norm 192.168.1.1:80 cookie s1 check port 81
1216 server web2-norm 192.168.1.2:80 cookie s2 check port 81
1217 server web1-stop 192.168.1.1:80 cookie s1 check port 80 backup
1218 server web2-stop 192.168.1.2:80 cookie s2 check port 80 backup
willy tarreau0174f312005-12-18 01:02:42 +01001219
willy tarreaueedaa9f2005-12-17 14:08:03 +01001220# automatic insertion of a cookie in the server's response, and automatic
1221# deletion of the cookie in the client request, while asking upstream caches
1222# not to cache replies.
1223 listen web_appl 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001224 mode http
1225 cookie SERVERID insert nocache indirect
1226 balance roundrobin
1227 server web1 192.168.1.1:80 cookie server01 check
1228 server web2 192.168.1.2:80 cookie server02 check
willy tarreaueedaa9f2005-12-17 14:08:03 +01001229
1230# same with off-site application backup and local error pages server
1231 listen web_appl 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001232 mode http
1233 cookie SERVERID insert nocache indirect
1234 balance roundrobin
1235 server web1 192.168.1.1:80 cookie server01 check
1236 server web2 192.168.1.2:80 cookie server02 check
1237 server web-backup 192.168.2.1:80 cookie server03 check backup
1238 server web-excuse 192.168.3.1:80 check backup
willy tarreaueedaa9f2005-12-17 14:08:03 +01001239
willy tarreauc5f73ed2005-12-18 01:26:38 +01001240# SMTP+TLS relaying with health-checks and backup servers
willy tarreaueedaa9f2005-12-17 14:08:03 +01001241
1242 listen http_proxy :25,:587
willy tarreauc5f73ed2005-12-18 01:26:38 +01001243 mode tcp
1244 balance roundrobin
1245 server srv1 192.168.1.1 check port 25 inter 30000 rise 1 fall 2
1246 server srv2 192.168.1.2 backup
willy tarreaueedaa9f2005-12-17 14:08:03 +01001247
Willy Tarreauf3c69202006-07-09 16:42:34 +02001248# HTTPS relaying with health-checks and backup servers
1249
1250 listen http_proxy :443
1251 mode tcp
1252 option ssl-hello-chk
1253 balance roundrobin
1254 server srv1 192.168.1.1 check inter 30000 rise 1 fall 2
1255 server srv2 192.168.1.2 backup
1256
Willy TARREAU3481c462006-03-01 22:37:57 +01001257# Load-balancing using a backup pool (requires haproxy 1.2.9)
1258 listen http_proxy 0.0.0.0:80
1259 mode http
1260 balance roundrobin
1261 option httpchk
1262 server inst1 192.168.1.1:80 cookie s1 check
1263 server inst2 192.168.1.2:80 cookie s2 check
1264 server inst3 192.168.1.3:80 cookie s3 check
1265 server back1 192.168.1.10:80 check backup
1266 server back2 192.168.1.11:80 check backup
1267 option allbackups # all backups will be used
1268
willy tarreaueedaa9f2005-12-17 14:08:03 +01001269
12703.2) Redistribute connections in case of failure
1271------------------------------------------------
1272In HTTP mode, if a server designated by a cookie does not respond, the clients
1273may definitely stick to it because they cannot flush the cookie, so they will
1274not be able to access the service anymore. Specifying 'redispatch' will allow
1275the proxy to break their persistence and redistribute them to working servers.
1276
1277Example :
1278---------
1279 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001280 mode http
1281 cookie SERVERID
1282 dispatch 192.168.1.100:80
1283 server web1 192.168.1.1:80 cookie server01
1284 server web2 192.168.1.2:80 cookie server02
1285 redispatch # send back to dispatch in case of connection failure
willy tarreaueedaa9f2005-12-17 14:08:03 +01001286
1287Up to, and including version 1.1.16, this parameter only applied to connection
1288failures. Since version 1.1.17, it also applies to servers which have been
1289detected as failed by the health check mechanism. Indeed, a server may be broken
1290but still accepting connections, which would not solve every case. But it is
1291possible to conserve the old behaviour, that is, make a client insist on trying
1292to connect to a server even if it is said to be down, by setting the 'persist'
1293option :
1294
1295 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001296 mode http
1297 option persist
1298 cookie SERVERID
1299 dispatch 192.168.1.100:80
1300 server web1 192.168.1.1:80 cookie server01
1301 server web2 192.168.1.2:80 cookie server02
1302 redispatch # send back to dispatch in case of connection failure
willy tarreaueedaa9f2005-12-17 14:08:03 +01001303
1304
willy tarreau34f45302006-04-15 21:37:14 +020013053.3) Assigning different weights to servers
1306-------------------------------------------
1307Sometimes you will need to bring new servers to increase your server farm's
1308capacity, but the new server will be either smaller (emergency use of anything
1309that fits) or bigger (when investing in new hardware). For this reason, it
1310might be wise to be able to send more clients to biggest servers. Till version
13111.2.11, it was necessary to replicate the same server multiple times in the
1312configuration. Starting with 1.2.12, the 'weight' option is available. HAProxy
1313then computes the most homogenous possible map of servers based on their
willy tarreau532bb552006-05-13 18:40:37 +02001314weights so that the load gets distributed as smoothly as possible among them.
1315The weight, between 1 and 256, should reflect one server's capacity relative to
1316others. Weight 1 represents the lowest frequency and 256 the highest. This way,
1317if a server fails, the remaining capacities are still respected.
willy tarreau34f45302006-04-15 21:37:14 +02001318
1319Example :
1320---------
1321# fair distribution among two opterons and one old pentium3
1322
1323 listen web_appl 0.0.0.0:80
1324 mode http
1325 cookie SERVERID insert nocache indirect
1326 balance roundrobin
1327 server pentium3-800 192.168.1.1:80 cookie server01 weight 8 check
1328 server opteron-2.0G 192.168.1.2:80 cookie server02 weight 20 check
1329 server opteron-2.4G 192.168.1.3:80 cookie server03 weight 24 check
1330 server web-backup1 192.168.2.1:80 cookie server04 check backup
1331 server web-excuse 192.168.3.1:80 check backup
1332
1333Notes :
1334-------
1335 - if unspecified, the default weight is 1
1336
1337 - the weight does not impact health checks, so it is cleaner to use weights
1338 than replicating the same server several times
1339
1340 - weights also work on backup servers if the 'allbackups' option is used
1341
1342 - the weights also apply to the source address load balancing
1343 ('balance source').
1344
1345 - whatever the weights, the first server will always be assigned first. This
1346 is helpful for troubleshooting.
1347
1348 - for the purists, the map calculation algorithm gives precedence to first
1349 server, so the map is the most uniform when servers are declared in
1350 ascending order relative to their weights.
1351
willy tarreau532bb552006-05-13 18:40:37 +02001352The load distribution will follow exactly this sequence :
1353
1354 Request| 1 1 1 1
1355 number | 1 2 3 4 5 6 7 8 9 0 1 2 3
1356 --------+---------------------------
1357 p3-800 | X . . . . . . X . . . . .
1358 opt-20 | . X . X . X . . . X . X .
1359 opt-24 | . . X . X . X . X . X . X
1360
1361
13623.4) Limiting the number of concurrent sessions on each server
1363--------------------------------------------------------------
1364Some pre-forked servers such as Apache suffer from too many concurrent
1365sessions, because it's very expensive to run hundreds or thousands of
1366processes on one system. One solution is to increase the number of servers
1367and load-balance between them, but it is a problem when the only goal is
1368to resist to short surges.
1369
1370To solve this problem, a new feature was implemented in HAProxy 1.2.13.
1371It's a per-server 'maxconn', associated with a per-server and a per-proxy
1372queue. This transforms haproxy into a request buffer between the thousands of
1373clients and the few servers. On many circumstances, lowering the maxconn value
1374will increase the server's performance and decrease the overall response times
1375because the servers will be less congested.
1376
1377When a request tries to reach any server, the first non-saturated server is
1378used, respective to the load balancing algorithm. If all servers are saturated,
1379then the request gets queued into the instance's global queue. It will be
1380dequeued once a server will have freed a session and all previously queued
1381requests have been processed.
1382
1383If a request references a particular server (eg: source hashing, or persistence
1384cookie), and if this server is full, then the request will be queued into the
1385server's dedicated queue. This queue has higher priority than the global queue,
1386so it's easier for already registered users to enter the site than for new
1387users.
1388
1389For this, the logs have been enhanced to show the number of sessions per
1390server, the request's position in the queue and the time spent in the queue.
1391This helps doing capacity planning. See the 'logs' section below for more info.
1392
1393Example :
1394---------
1395 # be nice with P3 which only has 256 MB of RAM.
1396 listen web_appl 0.0.0.0:80
1397 maxconn 10000
1398 mode http
1399 cookie SERVERID insert nocache indirect
1400 balance roundrobin
1401 server pentium3-800 192.168.1.1:80 cookie s1 weight 8 maxconn 100 check
1402 server opteron-2.0G 192.168.1.2:80 cookie s2 weight 20 maxconn 300 check
1403 server opteron-2.4G 192.168.1.3:80 cookie s3 weight 24 maxconn 300 check
1404 server web-backup1 192.168.2.1:80 cookie s4 check maxconn 200 backup
1405 server web-excuse 192.168.3.1:80 check backup
1406
willy tarreauf76e6ca2006-05-21 21:09:55 +02001407
1408This was so much efficient at reducing the server's response time that some
1409users wanted to use low values to improve their server's performance. However,
1410they were not able anymore to handle very large loads because it was not
1411possible anymore to saturate the servers. For this reason, version 1.2.14 has
1412brought dynamic limitation with the addition of the parameter 'minconn'. When
1413this parameter is set along with maxconn, it will enable dynamic limitation
1414based on the instance's load. The maximum number of concurrent sessions on a
1415server will be proportionnal to the number of sessions on the instance relative
1416to its maxconn. A minimum of <minconn> will be allowed whatever the load. This
1417will ensure that servers will perform at their best level under normal loads,
1418while still handling surges when needed. The dynamic limit is computed like
1419this :
1420
1421 srv.dyn_limit = max(srv.minconn, srv.maxconn * inst.sess / inst.maxconn)
1422
1423Example :
1424---------
1425 # be nice with P3 which only has 256 MB of RAM.
1426 listen web_appl 0.0.0.0:80
1427 maxconn 10000
1428 mode http
1429 cookie SERVERID insert nocache indirect
1430 balance roundrobin
1431 server pentium3-800 192.168.1.1:80 cookie s1 weight 8 minconn 10 maxconn 100 check
1432 server opteron-2.0G 192.168.1.2:80 cookie s2 weight 20 minconn 30 maxconn 300 check
1433 server opteron-2.4G 192.168.1.3:80 cookie s3 weight 24 minconn 30 maxconn 300 check
1434 server web-backup1 192.168.2.1:80 cookie s4 check maxconn 200 backup
1435 server web-excuse 192.168.3.1:80 check backup
1436
1437In the example above, the server 'pentium3-800' will receive at most 100
1438simultaneous sessions when the proxy instance will reach 10000 sessions, and
1439will receive only 10 simultaneous sessions when the proxy will be under 1000
1440sessions.
1441
Elijah Epifanovacafc5f2007-10-25 20:15:38 +02001442It is possible to limit server queue length in order to rebalance excess
1443sessions between less busy application servers IF session affinity isn't
1444hard functional requirement (for example it just gives huge performance boost
1445by keeping server-local caches hot and compact). 'maxqueue' option sets a
1446queue limit on a server, as in example below:
1447
1448... (just the same as in example above)
1449 server pentium3-800 192.168.1.1:80 cookie s1 weight 8 minconn 10 maxconn 100 check maxqueue 50
1450 server opteron-2.0G 192.168.1.2:80 cookie s2 weight 20 minconn 30 maxconn 300 check maxqueue 200
1451 server opteron-2.4G 192.168.1.3:80 cookie s3 weight 24 minconn 30 maxconn 300 check
1452
1453Absence of 'maxqueue' option means unlimited queue. When queue gets filled
1454up to 'maxqueue' client session is moved from server-local queue to a global
1455one.
1456
willy tarreau532bb552006-05-13 18:40:37 +02001457Notes :
1458-------
1459 - The requests will not stay indefinitely in the queue, they follow the
1460 'contimeout' parameter, and if a request cannot be dequeued within this
1461 timeout because the server is saturated or because the queue is filled,
1462 the session will expire with a 503 error.
1463
willy tarreauf76e6ca2006-05-21 21:09:55 +02001464 - if only <minconn> is specified, it has the same effect as <maxconn>
1465
willy tarreau532bb552006-05-13 18:40:37 +02001466 - setting too low values for maxconn might improve performance but might also
1467 allow slow users to block access to the server for other users.
1468
willy tarreau34f45302006-04-15 21:37:14 +02001469
willy tarreaue0bdd622006-05-21 20:51:54 +020014703.5) Dropping aborted requests
1471------------------------------
1472In presence of very high loads, the servers will take some time to respond. The
1473per-proxy's connection queue will inflate, and the response time will increase
1474respective to the size of the queue times the average per-session response
1475time. When clients will wait for more than a few seconds, they will often hit
1476the 'STOP' button on their browser, leaving a useless request in the queue, and
1477slowing down other users.
1478
1479As there is no way to distinguish between a full STOP and a simple
1480shutdown(SHUT_WR) on the client side, HTTP agents should be conservative and
1481consider that the client might only have closed its output channel while
1482waiting for the response. However, this introduces risks of congestion when
1483lots of users do the same, and is completely useless nowadays because probably
1484no client at all will close the session while waiting for the response. Some
1485HTTP agents support this (Squid, Apache, HAProxy), and others do not (TUX, most
1486hardware-based load balancers). So the probability for a closed input channel
1487to represent a user hitting the 'STOP' button is close to 100%, and it is very
1488tempting to be able to abort the session early without polluting the servers.
1489
1490For this reason, a new option "abortonclose" was introduced in version 1.2.14.
1491By default (without the option) the behaviour is HTTP-compliant. But when the
1492option is specified, a session with an incoming channel closed will be aborted
1493if it's still possible, which means that it's either waiting for a connect() to
1494establish or it is queued waiting for a connection slot. This considerably
1495reduces the queue size and the load on saturated servers when users are tempted
1496to click on STOP, which in turn reduces the response time for other users.
1497
1498Example :
1499---------
1500 listen web_appl 0.0.0.0:80
1501 maxconn 10000
1502 mode http
1503 cookie SERVERID insert nocache indirect
1504 balance roundrobin
1505 server web1 192.168.1.1:80 cookie s1 weight 10 maxconn 100 check
1506 server web2 192.168.1.2:80 cookie s2 weight 10 maxconn 100 check
1507 server web3 192.168.1.3:80 cookie s3 weight 10 maxconn 100 check
1508 server bck1 192.168.2.1:80 cookie s4 check maxconn 200 backup
1509 option abortonclose
1510
1511
willy tarreaueedaa9f2005-12-17 14:08:03 +010015124) Additionnal features
1513=======================
1514
willy tarreau481132e2006-05-21 21:43:10 +02001515Other features are available. They are transparent mode, event logging, header
1516rewriting/filtering, and the status as an HTML page.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001517
willy tarreauc5f73ed2005-12-18 01:26:38 +01001518
willy tarreau0174f312005-12-18 01:02:42 +010015194.1) Network features
willy tarreaueedaa9f2005-12-17 14:08:03 +01001520---------------------
willy tarreau0174f312005-12-18 01:02:42 +010015214.1.1) Transparent mode
1522-----------------------
willy tarreaueedaa9f2005-12-17 14:08:03 +01001523In HTTP mode, the 'transparent' keyword allows to intercept sessions which are
1524routed through the system hosting the proxy. This mode was implemented as a
1525replacement for the 'dispatch' mode, since connections without cookie will be
1526sent to the original address while known cookies will be sent to the servers.
1527This mode implies that the system can redirect sessions to a local port.
1528
1529Example :
1530---------
1531 listen http_proxy 0.0.0.0:65000
willy tarreauc5f73ed2005-12-18 01:26:38 +01001532 mode http
1533 transparent
1534 cookie SERVERID
1535 server server01 192.168.1.1:80
1536 server server02 192.168.1.2:80
willy tarreaueedaa9f2005-12-17 14:08:03 +01001537
1538 # iptables -t nat -A PREROUTING -i eth0 -p tcp -d 192.168.1.100 \
1539 --dport 80 -j REDIRECT --to-ports 65000
1540
1541Note :
1542------
1543If the port is left unspecified on the server, the port the client connected to
1544will be used. This allows to relay a full port range without using transparent
1545mode nor thousands of file descriptors, provided that the system can redirect
1546sessions to local ports.
1547
1548Example :
1549---------
1550 # redirect all ports to local port 65000, then forward to the server on the
1551 # original port.
1552 listen http_proxy 0.0.0.0:65000
willy tarreauc5f73ed2005-12-18 01:26:38 +01001553 mode tcp
1554 server server01 192.168.1.1 check port 60000
1555 server server02 192.168.1.2 check port 60000
willy tarreaueedaa9f2005-12-17 14:08:03 +01001556
1557 # iptables -t nat -A PREROUTING -i eth0 -p tcp -d 192.168.1.100 \
1558 -j REDIRECT --to-ports 65000
1559
willy tarreau0174f312005-12-18 01:02:42 +010015604.1.2) Per-server source address binding
1561----------------------------------------
1562As of versions 1.1.30 and 1.2.3, it is possible to specify a particular source
1563to reach each server. This is useful when reaching backup servers from a
1564different LAN, or to use an alternate path to reach the same server. It is also
1565usable to provide source load-balancing for outgoing connections. Obviously,
1566the same source address is used to send health-checks.
1567
1568Example :
1569---------
1570 # use a particular source to reach both servers
1571 listen http_proxy 0.0.0.0:65000
willy tarreauc5f73ed2005-12-18 01:26:38 +01001572 mode http
1573 balance roundrobin
1574 server server01 192.168.1.1:80 source 192.168.2.13
1575 server server02 192.168.1.2:80 source 192.168.2.13
willy tarreau0174f312005-12-18 01:02:42 +01001576
1577Example :
1578---------
1579 # use a particular source to reach each servers
1580 listen http_proxy 0.0.0.0:65000
willy tarreauc5f73ed2005-12-18 01:26:38 +01001581 mode http
1582 balance roundrobin
1583 server server01 192.168.1.1:80 source 192.168.1.1
1584 server server02 192.168.2.1:80 source 192.168.2.1
willy tarreau0174f312005-12-18 01:02:42 +01001585
1586Example :
1587---------
1588 # provide source load-balancing to reach the same proxy through 2 WAN links
1589 listen http_proxy 0.0.0.0:65000
willy tarreauc5f73ed2005-12-18 01:26:38 +01001590 mode http
1591 balance roundrobin
1592 server remote-proxy-way1 192.168.1.1:3128 source 192.168.2.1
1593 server remote-proxy-way2 192.168.1.1:3128 source 192.168.3.1
willy tarreau0174f312005-12-18 01:02:42 +01001594
1595Example :
1596---------
1597 # force a TCP connection to bind to a specific port
1598 listen http_proxy 0.0.0.0:2000
willy tarreauc5f73ed2005-12-18 01:26:38 +01001599 mode tcp
1600 balance roundrobin
1601 server srv1 192.168.1.1:80 source 192.168.2.1:20
1602 server srv2 192.168.1.2:80 source 192.168.2.1:20
willy tarreau0174f312005-12-18 01:02:42 +01001603
willy tarreaub952e1d2005-12-18 01:31:20 +010016044.1.3) TCP keep-alive
1605---------------------
1606With version 1.2.7, it becomes possible to enable TCP keep-alives on both the
1607client and server sides. This makes it possible to prevent long sessions from
1608expiring on external layer 4 components such as firewalls and load-balancers.
1609It also allows the system to terminate dead sessions when no timeout has been
1610set (not recommanded). The proxy cannot set the keep-alive probes intervals nor
1611maximal count, consult your operating system manual for this. There are 3
1612options to enable TCP keep-alive :
1613
1614 option tcpka # enables keep-alive both on client and server side
1615 option clitcpka # enables keep-alive only on client side
1616 option srvtcpka # enables keep-alive only on server side
1617
Alexandre Cassen87ea5482007-10-11 20:48:58 +020016184.1.4) TCP lingering
1619--------------------
1620It is possible to disable the system's lingering of data unacked by the client
1621at the end of a session. This is sometimes required when haproxy is used as a
1622front-end with lots of unreliable clients, and you observe thousands of sockets
1623in the FIN_WAIT state on the machine. This may be used in a frontend to affect
1624the client-side connection, as well as in a backend for the server-side
1625connection :
1626
1627 option nolinger # disables data lingering
1628
willy tarreaueedaa9f2005-12-17 14:08:03 +01001629
16304.2) Event logging
1631------------------
willy tarreauc5f73ed2005-12-18 01:26:38 +01001632
1633HAProxy's strength certainly lies in its precise logs. It probably provides the
1634finest level of information available for such a product, which is very
1635important for troubleshooting complex environments. Standard log information
1636include client ports, TCP/HTTP state timers, precise session state at
1637termination and precise termination cause, information about decisions to
1638direct trafic to a server, and of course the ability to capture arbitrary
1639headers.
1640
1641In order to improve administrators reactivity, it offers a great transparency
1642about encountered problems, both internal and external, and it is possible to
1643send logs to different sources at the same time with different level filters :
1644
1645 - global process-level logs (system errors, start/stop, etc..)
1646 - per-listener system and internal errors (lack of resource, bugs, ...)
1647 - per-listener external troubles (servers up/down, max connections)
1648 - per-listener activity (client connections), either at the establishment or
1649 at the termination.
1650
1651The ability to distribute different levels of logs to different log servers
1652allow several production teams to interact and to fix their problems as soon
1653as possible. For example, the system team might monitor system-wide errors,
1654while the application team might be monitoring the up/down for their servers in
1655real time, and the security team might analyze the activity logs with one hour
1656delay.
1657
willy tarreauc1cae632005-12-17 14:12:23 +010016584.2.1) Log levels
1659-----------------
willy tarreau197e8ec2005-12-17 14:10:59 +01001660TCP and HTTP connections can be logged with informations such as date, time,
1661source IP address, destination address, connection duration, response times,
1662HTTP request, the HTTP return code, number of bytes transmitted, the conditions
1663in which the session ended, and even exchanged cookies values, to track a
1664particular user's problems for example. All messages are sent to up to two
1665syslog servers. Consult section 1.1 for more info about log facilities. The
1666syntax follows :
willy tarreaueedaa9f2005-12-17 14:08:03 +01001667
willy tarreau197e8ec2005-12-17 14:10:59 +01001668 log <address_1> <facility_1> [max_level_1]
1669 log <address_2> <facility_2> [max_level_2]
1670or
willy tarreaueedaa9f2005-12-17 14:08:03 +01001671 log global
1672
willy tarreau197e8ec2005-12-17 14:10:59 +01001673Note :
1674------
1675The particular syntax 'log global' means that the same log configuration as the
1676'global' section will be used.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001677
willy tarreau197e8ec2005-12-17 14:10:59 +01001678Example :
willy tarreaueedaa9f2005-12-17 14:08:03 +01001679---------
1680 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001681 mode http
1682 log 192.168.2.200 local3
1683 log 192.168.2.201 local4
willy tarreaueedaa9f2005-12-17 14:08:03 +01001684
willy tarreauc1cae632005-12-17 14:12:23 +010016854.2.2) Log format
1686-----------------
1687By default, connections are logged at the TCP level, as soon as the session
1688establishes between the client and the proxy. By enabling the 'tcplog' option,
1689the proxy will wait until the session ends to generate an enhanced log
1690containing more information such as session duration and its state during the
willy tarreau532bb552006-05-13 18:40:37 +02001691disconnection. The number of remaining session after disconnection is also
1692indicated (for the server, the listener, and the process).
willy tarreauc1cae632005-12-17 14:12:23 +01001693
willy tarreauc5f73ed2005-12-18 01:26:38 +01001694Example of TCP logging :
1695------------------------
willy tarreau982249e2005-12-18 00:57:06 +01001696 listen relais-tcp 0.0.0.0:8000
willy tarreauc5f73ed2005-12-18 01:26:38 +01001697 mode tcp
1698 option tcplog
1699 log 192.168.2.200 local3
willy tarreau982249e2005-12-18 00:57:06 +01001700
willy tarreau532bb552006-05-13 18:40:37 +02001701>>> haproxy[18989]: 127.0.0.1:34550 [15/Oct/2003:15:24:28] relais-tcp Srv1 0/0/5007 0 -- 1/1/1 0/0
willy tarreauc5f73ed2005-12-18 01:26:38 +01001702
willy tarreau532bb552006-05-13 18:40:37 +02001703 Field Format Example
willy tarreauc5f73ed2005-12-18 01:26:38 +01001704
willy tarreau532bb552006-05-13 18:40:37 +02001705 1 process_name '[' pid ']:' haproxy[18989]:
1706 2 client_ip ':' client_port 127.0.0.1:34550
1707 3 '[' date ']' [15/Oct/2003:15:24:28]
1708 4 listener_name relais-tcp
1709 5 server_name Srv1
1710 6 queue_time '/' connect_time '/' total_time 0/0/5007
1711 7 bytes_read 0
1712 8 termination_state --
1713 9 srv_conn '/' listener_conn '/' process_conn 1/1/1
1714 10 position in srv_queue / listener_queue 0/0
1715
willy tarreau982249e2005-12-18 00:57:06 +01001716
willy tarreauc1cae632005-12-17 14:12:23 +01001717Another option, 'httplog', provides more detailed information about HTTP
1718contents, such as the request and some cookies. In the event where an external
1719component would establish frequent connections to check the service, logs may be
1720full of useless lines. So it is possible not to log any session which didn't
1721transfer any data, by the setting of the 'dontlognull' option. This only has
1722effect on sessions which are established then closed.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001723
willy tarreauc5f73ed2005-12-18 01:26:38 +01001724Example of HTTP logging :
1725-------------------------
willy tarreaueedaa9f2005-12-17 14:08:03 +01001726 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001727 mode http
1728 option httplog
1729 option dontlognull
1730 log 192.168.2.200 local3
1731
Krzysztof Piotr Oledzki25b501a2008-01-06 16:36:16 +01001732>>> haproxy[674]: 127.0.0.1:33319 [15/Oct/2003:08:31:57] relais-http Srv1 9/0/7/147/723 200 243 - - ---- 34/34/15/8/3 0/0 "HEAD / HTTP/1.0"
willy tarreaueedaa9f2005-12-17 14:08:03 +01001733
willy tarreauc5f73ed2005-12-18 01:26:38 +01001734More complete example
Krzysztof Piotr Oledzki25b501a2008-01-06 16:36:16 +01001735 haproxy[18989]: 10.0.0.1:34552 [15/Oct/2003:15:26:31] relais-http Srv1 3183/-1/-1/-1/11215 503 0 - - SC-- 205/202/150/137/+4 0/0 {w.ods.org|Mozilla} {} "HEAD / HTTP/1.0"
willy tarreauc5f73ed2005-12-18 01:26:38 +01001736
Krzysztof Piotr Oledzki25b501a2008-01-06 16:36:16 +01001737 Field Format Example
willy tarreauc5f73ed2005-12-18 01:26:38 +01001738
Krzysztof Piotr Oledzki25b501a2008-01-06 16:36:16 +01001739 1 process_name '[' pid ']:' haproxy[18989]:
1740 2 client_ip ':' client_port 10.0.0.1:34552
1741 3 '[' date ']' [15/Oct/2003:15:26:31]
1742 4 listener_name relais-http
1743 5 server_name Srv1
1744 6 Tq '/' Tw '/' Tc '/' Tr '/' Tt 3183/-1/-1/-1/11215
1745 7 HTTP_return_code 503
1746 8 bytes_read 0
1747 9 captured_request_cookie -
1748 10 captured_response_cookie -
1749 11 termination_state SC--
1750 12 actconn '/' feconn '/' beconn '/' srv_conn '/' retries 205/202/150/137/+4
1751 13 position in srv_queue / listener_queue 0/0
1752 14 '{' captured_request_headers '}' {w.ods.org|Mozilla}
1753 15 '{' captured_response_headers '}' {}
1754 16 '"' HTTP_request '"' "HEAD / HTTP/1.0"
willy tarreauc5f73ed2005-12-18 01:26:38 +01001755
1756Note for log parsers: the URI is ALWAYS the end of the line starting with the
1757 first double quote '"'.
willy tarreau982249e2005-12-18 00:57:06 +01001758
Krzysztof Piotr Oledzki25b501a2008-01-06 16:36:16 +01001759The retries count may have additional '+' sign means that the connection had been
1760redispatched from one server to another shortly before retries limit (retries 4
1761in above example) was depleted.
1762
willy tarreau982249e2005-12-18 00:57:06 +01001763The problem when logging at end of connection is that you have no clue about
1764what is happening during very long sessions. To workaround this problem, a
1765new option 'logasap' has been introduced in 1.1.28/1.2.1. When specified, the
1766proxy will log as soon as possible, just before data transfer begins. This means
1767that in case of TCP, it will still log the connection status to the server, and
1768in case of HTTP, it will log just after processing the server headers. In this
1769case, the number of bytes reported is the number of header bytes sent to the
1770client.
1771
1772In order to avoid confusion with normal logs, the total time field and the
Krzysztof Piotr Oledzki25b501a2008-01-06 16:36:16 +01001773number of bytes are prefixed with a '+' sign which means that real numbers are
willy tarreau982249e2005-12-18 00:57:06 +01001774certainly bigger.
1775
1776Example :
1777---------
1778
1779 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01001780 mode http
1781 option httplog
1782 option dontlognull
1783 option logasap
1784 log 192.168.2.200 local3
willy tarreau982249e2005-12-18 00:57:06 +01001785
Krzysztof Piotr Oledzki25b501a2008-01-06 16:36:16 +01001786>>> haproxy[674]: 127.0.0.1:33320 [15/Oct/2003:08:32:17] relais-http Srv1 9/10/7/14/+30 200 +243 - - ---- 3/1/1/1/0 1/0 "GET /image.iso HTTP/1.0"
willy tarreau982249e2005-12-18 00:57:06 +01001787
willy tarreauc1cae632005-12-17 14:12:23 +010017884.2.3) Timing events
1789--------------------
1790Timers provide a great help in trouble shooting network problems. All values
1791are reported in milliseconds (ms). In HTTP mode, four control points are
willy tarreau532bb552006-05-13 18:40:37 +02001792reported under the form 'Tq/Tw/Tc/Tr/Tt' :
willy tarreauc1cae632005-12-17 14:12:23 +01001793
1794 - Tq: total time to get the client request.
1795 It's the time elapsed between the moment the client connection was accepted
1796 and the moment the proxy received the last HTTP header. The value '-1'
1797 indicates that the end of headers (empty line) has never been seen.
1798
willy tarreau532bb552006-05-13 18:40:37 +02001799 - Tw: total time spent in the queues waiting for a connection slot. It
1800 accounts for listener's queue as well as the server's queue, and depends
1801 on the queue size, and the time needed for the server to complete previous
1802 sessions. The value '-1' means that the request was killed before reaching
1803 the queue.
1804
willy tarreauc1cae632005-12-17 14:12:23 +01001805 - Tc: total time to establish the TCP connection to the server.
1806 It's the time elapsed between the moment the proxy sent the connection
1807 request, and the moment it was acknowledged, or between the TCP SYN packet
1808 and the matching SYN/ACK in return. The value '-1' means that the
1809 connection never established.
1810
1811 - Tr: server response time. It's the time elapsed between the moment the
1812 TCP connection was established to the server and the moment it send its
1813 complete response header. It purely shows its request processing time,
1814 without the network overhead due to the data transmission. The value '-1'
1815 means that the last the response header (empty line) was never seen.
1816
1817 - Tt: total session duration time, between the moment the proxy accepted it
willy tarreau982249e2005-12-18 00:57:06 +01001818 and the moment both ends were closed. The exception is when the 'logasap'
willy tarreau532bb552006-05-13 18:40:37 +02001819 option is specified. In this case, it only equals (Tq+Tw+Tc+Tr), and is
willy tarreau982249e2005-12-18 00:57:06 +01001820 prefixed with a '+' sign. From this field, we can deduce Td, the data
1821 transmission time, by substracting other timers when valid :
willy tarreauc1cae632005-12-17 14:12:23 +01001822
willy tarreau532bb552006-05-13 18:40:37 +02001823 Td = Tt - (Tq + Tw + Tc + Tr)
willy tarreauc1cae632005-12-17 14:12:23 +01001824
1825 Timers with '-1' values have to be excluded from this equation.
1826
willy tarreau532bb552006-05-13 18:40:37 +02001827In TCP mode ('option tcplog'), only Tw, Tc and Tt are reported.
willy tarreauc1cae632005-12-17 14:12:23 +01001828
1829These timers provide precious indications on trouble causes. Since the TCP
1830protocol defines retransmit delays of 3, 6, 12... seconds, we know for sure
1831that timers close to multiples of 3s are nearly always related to packets lost
1832due to network problems (wires or negociation). Moreover, if <Tt> is close to
1833a timeout value specified in the configuration, it often means that a session
1834has been aborted on time-out.
1835
1836Most common cases :
1837
1838 - If Tq is close to 3000, a packet has probably been lost between the client
1839 and the proxy.
1840 - If Tc is close to 3000, a packet has probably been lost between the server
1841 and the proxy during the server connection phase. This one should always be
1842 very low (less than a few tens).
1843 - If Tr is nearly always lower than 3000 except some rare values which seem to
1844 be the average majored by 3000, there are probably some packets lost between
1845 the proxy and the server.
1846 - If Tt is often slightly higher than a time-out, it's often because the
1847 client and the server use HTTP keep-alive and the session is maintained
1848 after the response ends. Se further for how to disable HTTP keep-alive.
1849
1850Other cases ('xx' means any value to be ignored) :
willy tarreau532bb552006-05-13 18:40:37 +02001851 -1/xx/xx/xx/Tt: the client was not able to send its complete request in time,
1852 or that it aborted it too early.
1853 Tq/-1/xx/xx/Tt: it was not possible to process the request, maybe because
1854 servers were out of order.
1855 Tq/Tw/-1/xx/Tt: the connection could not establish on the server. Either it
1856 refused it or it timed out after Tt-(Tq+Tw) ms.
1857 Tq/Tw/Tc/-1/Tt: the server has accepted the connection but did not return a
1858 complete response in time, or it closed its connexion
1859 unexpectedly, after Tt-(Tq+Tw+Tc) ms.
willy tarreauc1cae632005-12-17 14:12:23 +01001860
18614.2.4) Session state at disconnection
1862-------------------------------------
willy tarreauc5f73ed2005-12-18 01:26:38 +01001863TCP and HTTP logs provide a session completion indicator in the
1864<termination_state> field, just before the number of active
1865connections. It is 2-characters long in TCP, and 4-characters long in
1866HTTP, each of which has a special meaning :
1867
willy tarreau197e8ec2005-12-17 14:10:59 +01001868 - On the first character, a code reporting the first event which caused the
1869 session to terminate :
willy tarreaueedaa9f2005-12-17 14:08:03 +01001870
willy tarreauc5f73ed2005-12-18 01:26:38 +01001871 C : the TCP session was unexpectedly aborted by the client.
1872
1873 S : the TCP session was unexpectedly aborted by the server, or the
1874 server explicitly refused it.
1875
1876 P : the session was prematurely aborted by the proxy, because of a
1877 connection limit enforcement, because a DENY filter was matched,
1878 or because of a security check which detected and blocked a
1879 dangerous error in server response which might have caused
1880 information leak (eg: cacheable cookie).
1881
1882 R : a resource on the proxy has been exhausted (memory, sockets, source
1883 ports, ...). Usually, this appears during the connection phase, and
1884 system logs should contain a copy of the precise error.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001885
willy tarreauc5f73ed2005-12-18 01:26:38 +01001886 I : an internal error was identified by the proxy during a self-check.
1887 This should NEVER happen, and you are encouraged to report any log
1888 containing this, because this is a bug.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001889
willy tarreauc5f73ed2005-12-18 01:26:38 +01001890 c : the client-side time-out expired first.
1891
1892 s : the server-side time-out expired first.
1893
1894 - : normal session completion.
1895
1896 - on the second character, the TCP/HTTP session state when it was closed :
1897
1898 R : waiting for complete REQUEST from the client (HTTP only). Nothing
1899 was sent to any server.
1900
willy tarreau532bb552006-05-13 18:40:37 +02001901 Q : waiting in the QUEUE for a connection slot. This can only happen on
1902 servers which have a 'maxconn' parameter set. No connection attempt
1903 was made to any server.
1904
willy tarreauc5f73ed2005-12-18 01:26:38 +01001905 C : waiting for CONNECTION to establish on the server. The server might
1906 at most have noticed a connection attempt.
1907
1908 H : waiting for, receiving and processing server HEADERS (HTTP only).
1909
1910 D : the session was in the DATA phase.
1911
1912 L : the proxy was still transmitting LAST data to the client while the
1913 server had already finished.
1914
Willy Tarreau2272dc12006-09-03 10:19:38 +02001915 T : the request was tarpitted. It has been held open on with the client
Willy Tarreau08fa2e32006-09-03 10:47:37 +02001916 during the whole contimeout duration or untill the client closed.
Willy Tarreau2272dc12006-09-03 10:19:38 +02001917
willy tarreauc5f73ed2005-12-18 01:26:38 +01001918 - : normal session completion after end of data transfer.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001919
willy tarreau197e8ec2005-12-17 14:10:59 +01001920 - the third character tells whether the persistence cookie was provided by
willy tarreauc1cae632005-12-17 14:12:23 +01001921 the client (only in HTTP mode) :
willy tarreaueedaa9f2005-12-17 14:08:03 +01001922
willy tarreauc5f73ed2005-12-18 01:26:38 +01001923 N : the client provided NO cookie. This is usually the case on new
1924 connections.
1925
1926 I : the client provided an INVALID cookie matching no known
1927 server. This might be caused by a recent configuration change,
1928 mixed cookies between HTTP/HTTPS sites, or an attack.
1929
1930 D : the client provided a cookie designating a server which was DOWN,
1931 so either the 'persist' option was used and the client was sent to
1932 this server, or it was not set and the client was redispatched to
1933 another server.
1934
1935 V : the client provided a valid cookie, and was sent to the associated
1936 server.
1937
1938 - : does not apply (no cookie set in configuration).
willy tarreaueedaa9f2005-12-17 14:08:03 +01001939
willy tarreau197e8ec2005-12-17 14:10:59 +01001940 - the last character reports what operations were performed on the persistence
willy tarreauc1cae632005-12-17 14:12:23 +01001941 cookie returned by the server (only in HTTP mode) :
willy tarreaueedaa9f2005-12-17 14:08:03 +01001942
willy tarreauc5f73ed2005-12-18 01:26:38 +01001943 N : NO cookie was provided by the server, and none was inserted either.
1944
1945 I : no cookie was provided by the server, and the proxy INSERTED one.
1946
willy tarreau197e8ec2005-12-17 14:10:59 +01001947 P : a cookie was PROVIDED by the server and transmitted as-is.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001948
willy tarreauc5f73ed2005-12-18 01:26:38 +01001949 R : the cookie provided by the server was REWRITTEN by the proxy.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001950
willy tarreauc5f73ed2005-12-18 01:26:38 +01001951 D : the cookie provided by the server was DELETED by the proxy.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001952
willy tarreauc5f73ed2005-12-18 01:26:38 +01001953 - : does not apply (no cookie set in configuration).
willy tarreaueedaa9f2005-12-17 14:08:03 +01001954
willy tarreauc5f73ed2005-12-18 01:26:38 +01001955The combination of the two first flags give a lot of information about what was
1956happening when the session terminated. It can be helpful to detect server
1957saturation, network troubles, local system resource starvation, attacks, etc...
willy tarreaueedaa9f2005-12-17 14:08:03 +01001958
willy tarreauc5f73ed2005-12-18 01:26:38 +01001959The most common termination flags combinations are indicated here.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001960
willy tarreauc5f73ed2005-12-18 01:26:38 +01001961 Flags Reason
1962 CR The client aborted before sending a full request. Most probably the
1963 request was done by hand using a telnet client, and aborted early.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001964
willy tarreauc5f73ed2005-12-18 01:26:38 +01001965 cR The client timed out before sending a full request. This is sometimes
1966 caused by too large TCP MSS values on the client side for PPPoE
1967 networks which cannot transport full-sized packets, or by clients
1968 sending requests by hand and not typing fast enough.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001969
willy tarreauc5f73ed2005-12-18 01:26:38 +01001970 SC The server explicitly refused the connection (the proxy received a
1971 TCP RST or an ICMP in return). Under some circumstances, it can
1972 also be the network stack telling the proxy that the server is
1973 unreachable (eg: no route, or no ARP response on local network).
willy tarreau982249e2005-12-18 00:57:06 +01001974
willy tarreauc5f73ed2005-12-18 01:26:38 +01001975 sC The connection to the server did not complete during contimeout.
willy tarreau982249e2005-12-18 00:57:06 +01001976
willy tarreauc5f73ed2005-12-18 01:26:38 +01001977 PC The proxy refused to establish a connection to the server because the
1978 maxconn limit has been reached. The listener's maxconn parameter may
1979 be increased in the proxy configuration, as well as the global
1980 maxconn parameter.
willy tarreauc1cae632005-12-17 14:12:23 +01001981
willy tarreauc5f73ed2005-12-18 01:26:38 +01001982 RC A local resource has been exhausted (memory, sockets, source ports)
1983 preventing the connection to the server from establishing. The error
1984 logs will tell precisely what was missing. Anyway, this can only be
1985 solved by system tuning.
willy tarreaueedaa9f2005-12-17 14:08:03 +01001986
willy tarreauc5f73ed2005-12-18 01:26:38 +01001987 cH The client timed out during a POST request. This is sometimes caused
1988 by too large TCP MSS values for PPPoE networks which cannot transport
1989 full-sized packets.
willy tarreauc1cae632005-12-17 14:12:23 +01001990
willy tarreau078c79a2006-05-13 12:23:58 +02001991 CH The client aborted while waiting for the server to start responding.
1992 It might be the server taking too long to respond or the client
1993 clicking the 'Stop' button too fast.
1994
1995 CQ The client aborted while its session was queued, waiting for a server
1996 with enough empty slots to accept it. It might be that either all the
1997 servers were saturated or the assigned server taking too long to
1998 respond.
1999
Willy Tarreau08fa2e32006-09-03 10:47:37 +02002000 CT The client aborted while its session was tarpitted.
2001
willy tarreau078c79a2006-05-13 12:23:58 +02002002 sQ The session spent too much time in queue and has been expired.
2003
willy tarreauc5f73ed2005-12-18 01:26:38 +01002004 SH The server aborted before sending its full headers, or it crashed.
willy tarreaueedaa9f2005-12-17 14:08:03 +01002005
willy tarreauc5f73ed2005-12-18 01:26:38 +01002006 sH The server failed to reply during the srvtimeout delay, which
2007 indicates too long transactions, probably caused by back-end
2008 saturation. The only solutions are to fix the problem on the
2009 application or to increase the 'srvtimeout' parameter to support
2010 longer delays (at the risk of the client giving up anyway).
2011
2012 PR The proxy blocked the client's request, either because of an invalid
2013 HTTP syntax, in which case it returned an HTTP 400 error to the
2014 client, or because a deny filter matched, in which case it returned
2015 an HTTP 403 error.
2016
2017 PH The proxy blocked the server's response, because it was invalid,
2018 incomplete, dangerous (cache control), or matched a security filter.
2019 In any case, an HTTP 502 error is sent to the client.
2020
Willy Tarreau2272dc12006-09-03 10:19:38 +02002021 PT The proxy blocked the client's request and has tarpitted its
2022 connection before returning it a 500 server error. Nothing was sent
2023 to the server.
2024
willy tarreauc5f73ed2005-12-18 01:26:38 +01002025 cD The client did not read any data for as long as the clitimeout delay.
2026 This is often caused by network failures on the client side.
2027
2028 CD The client unexpectedly aborted during data transfer. This is either
2029 caused by a browser crash, or by a keep-alive session between the
2030 server and the client terminated first by the client.
2031
2032 sD The server did nothing during the srvtimeout delay. This is often
2033 caused by too short timeouts on L4 equipements before the server
2034 (firewalls, load-balancers, ...).
2035
20364.2.5) Non-printable characters
willy tarreau4302f492005-12-18 01:00:37 +01002037-------------------------------
2038As of version 1.1.29, non-printable characters are not sent as-is into log
2039files, but are converted to their two-digits hexadecimal representation,
2040prefixed by the character '#'. The only characters that can now be logged
2041without being escaped are between 32 and 126 (inclusive). Obviously, the
2042escape character '#' is also encoded to avoid any ambiguity. It is the same for
2043the character '"', as well as '{', '|' and '}' when logging headers.
2044
willy tarreauc5f73ed2005-12-18 01:26:38 +010020454.2.6) Capturing HTTP headers and cookies
2046-----------------------------------------
2047Version 1.1.23 brought cookie capture, and 1.1.29 the header capture. All this
2048is performed using the 'capture' keyword.
2049
2050Cookie capture makes it easy to track a complete user session. The syntax is :
2051
2052 capture cookie <cookie_prefix> len <capture_length>
2053
2054This will enable cookie capture from both requests and responses. This way,
2055it's easy to detect when a user switches to a new session for example, because
2056the server will reassign it a new cookie.
2057
2058The FIRST cookie whose name starts with <cookie_prefix> will be captured, and
2059logged as 'NAME=value', without exceeding <capture_length> characters (64 max).
2060When the cookie name is fixed and known, it's preferable to suffix '=' to it to
2061ensure that no other cookie will be logged.
2062
2063Examples :
2064----------
2065 # capture the first cookie whose name starts with "ASPSESSION"
2066 capture cookie ASPSESSION len 32
2067
2068 # capture the first cookie whose name is exactly "vgnvisitor"
2069 capture cookie vgnvisitor= len 32
2070
2071In the logs, the field preceeding the completion indicator contains the cookie
2072value as sent by the server, preceeded by the cookie value as sent by the
2073client. Each of these field is replaced with '-' when no cookie was seen or
2074when the option is disabled.
2075
2076Header captures have a different goal. They are useful to track unique request
2077identifiers set by a previous proxy, virtual host names, user-agents, POST
2078content-length, referrers, etc. In the response, one can search for information
2079about the response length, how the server asked the cache to behave, or an
2080object location during a redirection. As for cookie captures, it is both
2081possible to include request headers and response headers at the same time. The
2082syntax is :
willy tarreau4302f492005-12-18 01:00:37 +01002083
2084 capture request header <name> len <max length>
2085 capture response header <name> len <max length>
2086
2087Note: Header names are not case-sensitive.
2088
2089Examples:
2090---------
2091 # keep the name of the virtual server
2092 capture request header Host len 20
2093 # keep the amount of data uploaded during a POST
2094 capture request header Content-Length len 10
2095
2096 # note the expected cache behaviour on the response
2097 capture response header Cache-Control len 8
2098 # note the URL location during a redirection
2099 capture response header Location len 20
2100
2101Non-existant headers are logged as empty strings, and if one header appears more
2102than once, only its last occurence will be kept. Request headers are grouped
2103within braces '{' and '}' in the same order as they were declared, and delimited
2104with a vertical bar '|' without any space. Response headers follow the same
2105representation, but are displayed after a space following the request headers
2106block. These blocks are displayed just before the HTTP request in the logs.
willy tarreauc5f73ed2005-12-18 01:26:38 +01002107
willy tarreau4302f492005-12-18 01:00:37 +01002108Example :
2109
willy tarreauc5f73ed2005-12-18 01:26:38 +01002110 Config:
2111
2112 capture request header Host len 20
2113 capture request header Content-Length len 10
2114 capture request header Referer len 20
2115 capture response header Server len 20
2116 capture response header Content-Length len 10
2117 capture response header Cache-Control len 8
2118 capture response header Via len 20
2119 capture response header Location len 20
2120
2121 Log :
2122
willy tarreau532bb552006-05-13 18:40:37 +02002123 Aug 9 20:26:09 localhost haproxy[2022]: 127.0.0.1:34014 [09/Aug/2004:20:26:09] relais-http netcache 0/0/0/162/+162 200 +350 - - ---- 0/0/0 0/0 {fr.adserver.yahoo.co||http://fr.f416.mail.} {|864|private||} "GET http://fr.adserver.yahoo.com/"
2124 Aug 9 20:30:46 localhost haproxy[2022]: 127.0.0.1:34020 [09/Aug/2004:20:30:46] relais-http netcache 0/0/0/182/+182 200 +279 - - ---- 0/0/0 0/0 {w.ods.org||} {Formilux/0.1.8|3495|||} "GET http://w.ods.org/sytadin.html HTTP/1.1"
2125 Aug 9 20:30:46 localhost haproxy[2022]: 127.0.0.1:34028 [09/Aug/2004:20:30:46] relais-http netcache 0/0/2/126/+128 200 +223 - - ---- 0/0/0 0/0 {www.infotrafic.com||http://w.ods.org/syt} {Apache/2.0.40 (Red H|9068|||} "GET http://www.infotrafic.com/images/live/cartesidf/grandes/idf_ne.png HTTP/1.1"
willy tarreauc5f73ed2005-12-18 01:26:38 +01002126
2127
21284.2.7) Examples of logs
2129-----------------------
willy tarreau532bb552006-05-13 18:40:37 +02002130- haproxy[674]: 127.0.0.1:33319 [15/Oct/2003:08:31:57] relais-http Srv1 6559/0/7/147/6723 200 243 - - ---- 1/3/5 0/0 "HEAD / HTTP/1.0"
willy tarreauc5f73ed2005-12-18 01:26:38 +01002131 => long request (6.5s) entered by hand through 'telnet'. The server replied
2132 in 147 ms, and the session ended normally ('----')
2133
willy tarreau532bb552006-05-13 18:40:37 +02002134- haproxy[674]: 127.0.0.1:33319 [15/Oct/2003:08:31:57] relais-http Srv1 6559/1230/7/147/6870 200 243 - - ---- 99/239/324 0/9 "HEAD / HTTP/1.0"
2135 => Idem, but the request was queued in the global queue behind 9 other
2136 requests, and waited there for 1230 ms.
2137
2138- haproxy[674]: 127.0.0.1:33320 [15/Oct/2003:08:32:17] relais-http Srv1 9/0/7/14/+30 200 +243 - - ---- 1/3/3 0/0 "GET /image.iso HTTP/1.0"
willy tarreauc5f73ed2005-12-18 01:26:38 +01002139 => request for a long data transfer. The 'logasap' option was specified, so
2140 the log was produced just before transfering data. The server replied in
2141 14 ms, 243 bytes of headers were sent to the client, and total time from
2142 accept to first data byte is 30 ms.
2143
willy tarreau532bb552006-05-13 18:40:37 +02002144- haproxy[674]: 127.0.0.1:33320 [15/Oct/2003:08:32:17] relais-http Srv1 9/0/7/14/30 502 243 - - PH-- 0/2/3 0/0 "GET /cgi-bin/bug.cgi? HTTP/1.0"
willy tarreauc5f73ed2005-12-18 01:26:38 +01002145 => the proxy blocked a server response either because of an 'rspdeny' or
2146 'rspideny' filter, or because it blocked sensible information which risked
2147 being cached. In this case, the response is replaced with a '502 bad
2148 gateway'.
2149
willy tarreau532bb552006-05-13 18:40:37 +02002150- haproxy[18113]: 127.0.0.1:34548 [15/Oct/2003:15:18:55] relais-http <NOSRV> -1/-1/-1/-1/8490 -1 0 - - CR-- 0/2/2 0/0 ""
willy tarreauc5f73ed2005-12-18 01:26:38 +01002151 => the client never completed its request and aborted itself ('C---') after
2152 8.5s, while the proxy was waiting for the request headers ('-R--').
2153 Nothing was sent to the server.
2154
willy tarreau532bb552006-05-13 18:40:37 +02002155- haproxy[18113]: 127.0.0.1:34549 [15/Oct/2003:15:19:06] relais-http <NOSRV> -1/-1/-1/-1/50001 408 0 - - cR-- 2/2 0/0 ""
willy tarreauc5f73ed2005-12-18 01:26:38 +01002156 => The client never completed its request, which was aborted by the time-out
2157 ('c---') after 50s, while the proxy was waiting for the request headers ('-R--').
2158 Nothing was sent to the server, but the proxy could send a 408 return code
2159 to the client.
willy tarreau4302f492005-12-18 01:00:37 +01002160
willy tarreau532bb552006-05-13 18:40:37 +02002161- haproxy[18989]: 127.0.0.1:34550 [15/Oct/2003:15:24:28] relais-tcp Srv1 0/0/5007 0 cD 0/0/0 0/0
willy tarreauc5f73ed2005-12-18 01:26:38 +01002162 => This is a 'tcplog' entry. Client-side time-out ('c----') occured after 5s.
willy tarreau4302f492005-12-18 01:00:37 +01002163
willy tarreau532bb552006-05-13 18:40:37 +02002164- haproxy[18989]: 10.0.0.1:34552 [15/Oct/2003:15:26:31] relais-http Srv1 3183/-1/-1/-1/11215 503 0 - - SC-- 115/202/205 0/0 "HEAD / HTTP/1.0"
willy tarreauc5f73ed2005-12-18 01:26:38 +01002165 => The request took 3s to complete (probably a network problem), and the
2166 connection to the server failed ('SC--') after 4 attemps of 2 seconds
2167 (config says 'retries 3'), then a 503 error code was sent to the client.
willy tarreau532bb552006-05-13 18:40:37 +02002168 There were 115 connections on this server, 202 connections on this proxy,
2169 and 205 on the global process. It is possible that the server refused the
2170 connection because of too many already established.
willy tarreau4302f492005-12-18 01:00:37 +01002171
willy tarreau4302f492005-12-18 01:00:37 +01002172
willy tarreau197e8ec2005-12-17 14:10:59 +010021734.3) HTTP header manipulation
2174-----------------------------
2175In HTTP mode, it is possible to rewrite, add or delete some of the request and
2176response headers based on regular expressions. It is also possible to block a
2177request or a response if a particular header matches a regular expression,
2178which is enough to stops most elementary protocol attacks, and to protect
2179against information leak from the internal network. But there is a limitation
2180to this : since haproxy's HTTP engine knows nothing about keep-alive, only
2181headers passed during the first request of a TCP session will be seen. All
2182subsequent headers will be considered data only and not analyzed. Furthermore,
2183haproxy doesn't touch data contents, it stops at the end of headers.
willy tarreaueedaa9f2005-12-17 14:08:03 +01002184
willy tarreau197e8ec2005-12-17 14:10:59 +01002185The syntax is :
2186 reqadd <string> to add a header to the request
2187 reqrep <search> <replace> to modify the request
2188 reqirep <search> <replace> same, but ignoring the case
2189 reqdel <search> to delete a header in the request
2190 reqidel <search> same, but ignoring the case
2191 reqallow <search> definitely allow a request if a header matches <search>
2192 reqiallow <search> same, but ignoring the case
2193 reqdeny <search> denies a request if a header matches <search>
2194 reqideny <search> same, but ignoring the case
2195 reqpass <search> ignore a header matching <search>
2196 reqipass <search> same, but ignoring the case
Willy Tarreau2272dc12006-09-03 10:19:38 +02002197 reqtarpit <search> tarpit a request matching <search>
2198 reqitarpit <search> same, but ignoring the case
willy tarreaueedaa9f2005-12-17 14:08:03 +01002199
willy tarreau197e8ec2005-12-17 14:10:59 +01002200 rspadd <string> to add a header to the response
2201 rsprep <search> <replace> to modify the response
2202 rspirep <search> <replace> same, but ignoring the case
2203 rspdel <search> to delete the response
2204 rspidel <search> same, but ignoring the case
willy tarreau982249e2005-12-18 00:57:06 +01002205 rspdeny <search> replaces a response with a HTTP 502 if a header matches <search>
2206 rspideny <search> same, but ignoring the case
willy tarreaueedaa9f2005-12-17 14:08:03 +01002207
2208
willy tarreau197e8ec2005-12-17 14:10:59 +01002209<search> is a POSIX regular expression (regex) which supports grouping through
2210parenthesis (without the backslash). Spaces and other delimiters must be
2211prefixed with a backslash ('\') to avoid confusion with a field delimiter.
2212Other characters may be prefixed with a backslash to change their meaning :
willy tarreaueedaa9f2005-12-17 14:08:03 +01002213
willy tarreau197e8ec2005-12-17 14:10:59 +01002214 \t for a tab
2215 \r for a carriage return (CR)
2216 \n for a new line (LF)
2217 \ to mark a space and differentiate it from a delimiter
2218 \# to mark a sharp and differentiate it from a comment
2219 \\ to use a backslash in a regex
2220 \\\\ to use a backslash in the text (*2 for regex, *2 for haproxy)
2221 \xXX to write the ASCII hex code XX as in the C language
willy tarreaueedaa9f2005-12-17 14:08:03 +01002222
2223
Willy Tarreau2272dc12006-09-03 10:19:38 +02002224<replace> contains the string to be used to replace the largest portion of text
willy tarreau197e8ec2005-12-17 14:10:59 +01002225matching the regex. It can make use of the special characters above, and can
2226reference a substring delimited by parenthesis in the regex, by the group
Willy Tarreau2272dc12006-09-03 10:19:38 +02002227numerical order from 0 to 9 (0 being the entire line). In this case, you would
2228write a backslash ('\') immediately followed by one digit indicating the group
2229position.
willy tarreaueedaa9f2005-12-17 14:08:03 +01002230
willy tarreau197e8ec2005-12-17 14:10:59 +01002231<string> represents the string which will systematically be added after the last
2232header line. It can also use special characters above.
willy tarreaueedaa9f2005-12-17 14:08:03 +01002233
willy tarreau197e8ec2005-12-17 14:10:59 +01002234Notes :
2235-------
2236 - the first line is considered as a header, which makes it possible to rewrite
2237 or filter HTTP requests URIs or response codes.
2238 - 'reqrep' is the equivalent of 'cliexp' in version 1.0, and 'rsprep' is the
2239 equivalent of 'srvexp' in 1.0. Those names are still supported but
2240 deprecated.
2241 - for performances reasons, the number of characters added to a request or to
2242 a response is limited to 4096 since version 1.1.5 (it was 256 before). This
2243 value is easy to modify in the code if needed (#define). If it is too short
2244 on occasional uses, it is possible to gain some space by removing some
2245 useless headers before adding new ones.
willy tarreau982249e2005-12-18 00:57:06 +01002246 - a denied request will generate an "HTTP 403 forbidden" response, while a
2247 denied response will generate an "HTTP 502 Bad gateway" response.
Willy Tarreau2272dc12006-09-03 10:19:38 +02002248 - a tarpitted request will be held open on the client side for a duration
Willy Tarreau08fa2e32006-09-03 10:47:37 +02002249 defined in the contimeout parameter, or untill the client aborts. Nothing
2250 will be sent to any server. When the timeout is reached, the proxy will
2251 reply with a 500 server error response so that the attacker does not
2252 suspect it has been tarpitted. The logs may report the 500, but the
2253 termination flags will indicate 'PT' in this case.
Willy Tarreau2272dc12006-09-03 10:19:38 +02002254
willy tarreaueedaa9f2005-12-17 14:08:03 +01002255
willy tarreau197e8ec2005-12-17 14:10:59 +01002256Examples :
2257----------
willy tarreauc5f73ed2005-12-18 01:26:38 +01002258 ###### a few examples ######
willy tarreau197e8ec2005-12-17 14:10:59 +01002259
willy tarreauc5f73ed2005-12-18 01:26:38 +01002260 # rewrite 'online.fr' instead of 'free.fr' for GET and POST requests
2261 reqrep ^(GET\ .*)(.free.fr)(.*) \1.online.fr\3
2262 reqrep ^(POST\ .*)(.free.fr)(.*) \1.online.fr\3
willy tarreau197e8ec2005-12-17 14:10:59 +01002263
willy tarreauc5f73ed2005-12-18 01:26:38 +01002264 # force proxy connections to close
2265 reqirep ^Proxy-Connection:.* Proxy-Connection:\ close
2266 # rewrite locations
2267 rspirep ^(Location:\ )([^:]*://[^/]*)(.*) \1\3
willy tarreaueedaa9f2005-12-17 14:08:03 +01002268
willy tarreauc5f73ed2005-12-18 01:26:38 +01002269 ###### A full configuration being used on production ######
willy tarreaueedaa9f2005-12-17 14:08:03 +01002270
willy tarreau197e8ec2005-12-17 14:10:59 +01002271 # Every header should end with a colon followed by one space.
willy tarreauc5f73ed2005-12-18 01:26:38 +01002272 reqideny ^[^:\ ]*[\ ]*$
willy tarreaueedaa9f2005-12-17 14:08:03 +01002273
willy tarreau197e8ec2005-12-17 14:10:59 +01002274 # block Apache chunk exploit
willy tarreauc5f73ed2005-12-18 01:26:38 +01002275 reqideny ^Transfer-Encoding:[\ ]*chunked
2276 reqideny ^Host:\ apache-
willy tarreaueedaa9f2005-12-17 14:08:03 +01002277
willy tarreau197e8ec2005-12-17 14:10:59 +01002278 # block annoying worms that fill the logs...
willy tarreauc5f73ed2005-12-18 01:26:38 +01002279 reqideny ^[^:\ ]*\ .*(\.|%2e)(\.|%2e)(%2f|%5c|/|\\\\)
2280 reqideny ^[^:\ ]*\ ([^\ ]*\ [^\ ]*\ |.*%00)
2281 reqideny ^[^:\ ]*\ .*<script
2282 reqideny ^[^:\ ]*\ .*/(root\.exe\?|cmd\.exe\?|default\.ida\?)
willy tarreau197e8ec2005-12-17 14:10:59 +01002283
Willy Tarreau2272dc12006-09-03 10:19:38 +02002284 # tarpit attacks on the login page.
2285 reqtarpit ^[^:\ ]*\ .*\.php?login=[^0-9]
2286
willy tarreau197e8ec2005-12-17 14:10:59 +01002287 # allow other syntactically valid requests, and block any other method
willy tarreauc5f73ed2005-12-18 01:26:38 +01002288 reqipass ^(GET|POST|HEAD|OPTIONS)\ /.*\ HTTP/1\.[01]$
2289 reqipass ^OPTIONS\ \\*\ HTTP/1\.[01]$
2290 reqideny ^[^:\ ]*\
willy tarreau197e8ec2005-12-17 14:10:59 +01002291
2292 # force connection:close, thus disabling HTTP keep-alive
willy tarreauc5f73ed2005-12-18 01:26:38 +01002293 option httpclose
willy tarreau197e8ec2005-12-17 14:10:59 +01002294
willy tarreauc5f73ed2005-12-18 01:26:38 +01002295 # change the server name
2296 rspidel ^Server:\
2297 rspadd Server:\ Formilux/0.1.8
willy tarreau197e8ec2005-12-17 14:10:59 +01002298
2299
willy tarreau982249e2005-12-18 00:57:06 +01002300Also, the 'forwardfor' option creates an HTTP 'X-Forwarded-For' header which
willy tarreauc1cae632005-12-17 14:12:23 +01002301contains the client's IP address. This is useful to let the final web server
Willy Tarreau7ac51f62007-03-25 16:00:04 +02002302know what the client address was (eg for statistics on domains). Starting with
2303version 1.3.8, it is possible to specify the "except" keyword followed by a
2304source IP address or network for which no header will be added. This is very
2305useful when another reverse-proxy which already adds the header runs on the
2306same machine or in a known DMZ, the most common case being the local use of
2307stunnel on the same system.
willy tarreauc1cae632005-12-17 14:12:23 +01002308
willy tarreau982249e2005-12-18 00:57:06 +01002309Last, the 'httpclose' option removes any 'Connection' header both ways, and
2310adds a 'Connection: close' header in each direction. This makes it easier to
Willy TARREAU767ba712006-03-01 22:40:50 +01002311disable HTTP keep-alive than the previous 4-rules block.
willy tarreau982249e2005-12-18 00:57:06 +01002312
willy tarreauc1cae632005-12-17 14:12:23 +01002313Example :
2314---------
2315 listen http_proxy 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01002316 mode http
2317 log global
2318 option httplog
2319 option dontlognull
Willy Tarreau7ac51f62007-03-25 16:00:04 +02002320 option forwardfor except 127.0.0.1/8
willy tarreauc5f73ed2005-12-18 01:26:38 +01002321 option httpclose
2322
Willy TARREAU767ba712006-03-01 22:40:50 +01002323Note that some HTTP servers do not necessarily close the connections when they
2324receive the 'Connection: close', and if the client does not close either, then
2325the connection will be maintained up to the time-out. This translates into high
2326number of simultaneous sessions and high global session times in the logs. To
2327workaround this, a new option 'forceclose' appeared in version 1.2.9 to enforce
2328the closing of the outgoing server channel as soon as the server begins to
2329reply and only if the request buffer is empty. Note that this should NOT be
2330used if CONNECT requests are expected between the client and the server. The
2331'forceclose' option implies the 'httpclose' option.
2332
2333Example :
2334---------
2335 listen http_proxy 0.0.0.0:80
2336 mode http
2337 log global
2338 option httplog
2339 option dontlognull
2340 option forwardfor
2341 option forceclose
2342
willy tarreau197e8ec2005-12-17 14:10:59 +01002343
23444.4) Load balancing with persistence
2345------------------------------------
willy tarreau197e8ec2005-12-17 14:10:59 +01002346Combining cookie insertion with internal load balancing allows to transparently
2347bring persistence to applications. The principle is quite simple :
2348 - assign a cookie value to each server
2349 - enable the load balancing between servers
2350 - insert a cookie into responses resulting from the balancing algorithm
2351 (indirect accesses), end ensure that no upstream proxy will cache it.
2352 - remove the cookie in the request headers so that the application never sees
2353 it.
2354
2355Example :
willy tarreaueedaa9f2005-12-17 14:08:03 +01002356---------
2357 listen application 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01002358 mode http
2359 cookie SERVERID insert nocache indirect
2360 balance roundrobin
2361 server srv1 192.168.1.1:80 cookie server01 check
2362 server srv2 192.168.1.2:80 cookie server02 check
willy tarreaueedaa9f2005-12-17 14:08:03 +01002363
willy tarreau0174f312005-12-18 01:02:42 +01002364The other solution brought by versions 1.1.30 and 1.2.3 is to reuse a cookie
2365from the server, and prefix the server's name to it. In this case, don't forget
2366to force "httpclose" mode so that you can be assured that every subsequent
2367request will have its cookie fixed.
2368
2369 listen application 0.0.0.0:80
willy tarreauc5f73ed2005-12-18 01:26:38 +01002370 mode http
2371 cookie JSESSIONID prefix
2372 balance roundrobin
2373 server srv1 192.168.1.1:80 cookie srv1 check
2374 server srv2 192.168.1.2:80 cookie srv2 check
2375 option httpclose
willy tarreau0174f312005-12-18 01:02:42 +01002376
2377
willy tarreau982249e2005-12-18 00:57:06 +010023784.5) Protection against information leak from the servers
2379---------------------------------------------------------
2380In versions 1.1.28/1.2.1, a new option 'checkcache' was created. It carefully
2381checks 'Cache-control', 'Pragma' and 'Set-cookie' headers in server response
2382to check if there's a risk of caching a cookie on a client-side proxy. When this
2383option is enabled, the only responses which can be delivered to the client are :
2384 - all those without 'Set-Cookie' header ;
2385 - all those with a return code other than 200, 203, 206, 300, 301, 410,
2386 provided that the server has not set a 'Cache-control: public' header ;
2387 - all those that come from a POST request, provided that the server has not
2388 set a 'Cache-Control: public' header ;
2389 - those with a 'Pragma: no-cache' header
2390 - those with a 'Cache-control: private' header
2391 - those with a 'Cache-control: no-store' header
2392 - those with a 'Cache-control: max-age=0' header
2393 - those with a 'Cache-control: s-maxage=0' header
2394 - those with a 'Cache-control: no-cache' header
2395 - those with a 'Cache-control: no-cache="set-cookie"' header
2396 - those with a 'Cache-control: no-cache="set-cookie,' header
2397 (allowing other fields after set-cookie)
willy tarreaueedaa9f2005-12-17 14:08:03 +01002398
willy tarreau982249e2005-12-18 00:57:06 +01002399If a response doesn't respect these requirements, then it will be blocked just
2400as if it was from an 'rspdeny' filter, with an "HTTP 502 bad gateway". The
2401session state shows "PH--" meaning that the proxy blocked the response during
2402headers processing. Additionnaly, an alert will be sent in the logs so that
2403admins are told that there's something to be done.
2404
willy tarreauc5f73ed2005-12-18 01:26:38 +01002405
willy tarreau982249e2005-12-18 00:57:06 +010024064.6) Customizing errors
2407-----------------------
willy tarreau197e8ec2005-12-17 14:10:59 +01002408Some situations can make haproxy return an HTTP error code to the client :
2409 - invalid or too long request => HTTP 400
2410 - request not completely sent in time => HTTP 408
2411 - forbidden request (matches a deny filter) => HTTP 403
2412 - internal error in haproxy => HTTP 500
2413 - the server returned an invalid or incomplete response => HTTP 502
2414 - no server was available to handle the request => HTTP 503
2415 - the server failed to reply in time => HTTP 504
willy tarreaueedaa9f2005-12-17 14:08:03 +01002416
willy tarreau197e8ec2005-12-17 14:10:59 +01002417A succint error message taken from the RFC accompanies these return codes.
2418But depending on the clients knowledge, it may be better to return custom, user
Willy Tarreau3f49b302007-06-11 00:29:26 +02002419friendly, error pages. This is made possible in two ways, one involving a
2420redirection to a known server, and another one consisting in returning a local
2421file.
2422
24234.6.1) Relocation
2424-----------------
2425An error relocation is achieved using the 'errorloc' command :
willy tarreaueedaa9f2005-12-17 14:08:03 +01002426
willy tarreau197e8ec2005-12-17 14:10:59 +01002427 errorloc <HTTP_code> <location>
willy tarreaueedaa9f2005-12-17 14:08:03 +01002428
willy tarreau197e8ec2005-12-17 14:10:59 +01002429Instead of generating an HTTP error <HTTP_code> among those above, the proxy
2430will return a temporary redirection code (HTTP 302) towards the address
2431specified in <location>. This address may be either relative to the site or
2432absolute. Since this request will be handled by the client's browser, it's
2433mandatory that the returned address be reachable from the outside.
willy tarreaueedaa9f2005-12-17 14:08:03 +01002434
willy tarreau197e8ec2005-12-17 14:10:59 +01002435Example :
willy tarreaueedaa9f2005-12-17 14:08:03 +01002436---------
2437 listen application 0.0.0.0:80
2438 errorloc 400 /badrequest.html
2439 errorloc 403 /forbidden.html
2440 errorloc 408 /toolong.html
willy tarreauc5f73ed2005-12-18 01:26:38 +01002441 errorloc 500 http://haproxy.domain.net/bugreport.html
willy tarreaueedaa9f2005-12-17 14:08:03 +01002442 errorloc 502 http://192.168.114.58/error50x.html
2443 errorloc 503 http://192.168.114.58/error50x.html
2444 errorloc 504 http://192.168.114.58/error50x.html
2445
willy tarreauc1f47532005-12-18 01:08:26 +01002446Note: RFC2616 says that a client must reuse the same method to fetch the
2447Location returned by a 302, which causes problems with the POST method.
2448The return code 303 was designed explicitly to force the client to fetch the
2449Location URL with the GET method, but there are some browsers pre-dating
2450HTTP/1.1 which don't support it. Anyway, most browsers still behave with 302 as
willy tarreauc5f73ed2005-12-18 01:26:38 +01002451if it was a 303. In order to allow the user to chose, versions 1.1.31 and 1.2.5
2452bring two new keywords to replace 'errorloc' : 'errorloc302' and 'errorloc303'.
willy tarreauc1f47532005-12-18 01:08:26 +01002453
2454They are preffered over errorloc (which still does 302). Consider using
2455errorloc303 everytime you know that your clients support HTTP 303 responses..
2456
Willy Tarreau3f49b302007-06-11 00:29:26 +020024574.6.2) Local files
2458------------------
2459Sometimes, it is desirable to change the returned error without resorting to
2460redirections. The second method consists in loading local files during startup
2461and send them as pure HTTP content upon error. This is what the 'errorfile'
2462keyword does.
2463
2464Warning, there are traps to consider :
2465 - The files are loaded while parsing configuration, before doing a chroot().
2466 Thus, they are relative to the real filesystem. For this reason, it is
2467 recommended to pass an absolute path to those files.
2468
2469 - The contents of those files is not HTML, but real HTTP protocol with
2470 possible HTML body. So the first line and headers are mandatory. Ideally,
2471 every line in the HTTP part should end with CR-LF for maximum compatibility.
2472
2473 - The response is limited to the buffer size (BUSIZE), generally 8 or 16 kB.
2474
2475 - The response should not include references to the local server, in order to
2476 avoid infinite loops on the browser in case of local failure.
2477
2478Example :
2479---------
2480 errorfile 400 /etc/haproxy/errorfiles/400badreq.http
2481 errorfile 403 /etc/haproxy/errorfiles/403forbid.http
2482 errorfile 503 /etc/haproxy/errorfiles/503sorry.http
2483
willy tarreauc1f47532005-12-18 01:08:26 +01002484
willy tarreau982249e2005-12-18 00:57:06 +010024854.7) Modifying default values
willy tarreau197e8ec2005-12-17 14:10:59 +01002486-----------------------------
willy tarreau197e8ec2005-12-17 14:10:59 +01002487Version 1.1.22 introduced the notion of default values, which eliminates the
2488pain of often repeating common parameters between many instances, such as
2489logs, timeouts, modes, etc...
willy tarreaueedaa9f2005-12-17 14:08:03 +01002490
willy tarreau197e8ec2005-12-17 14:10:59 +01002491Default values are set in a 'defaults' section. Each of these section clears
2492all previously set default parameters, so there may be as many default
2493parameters as needed. Only the last one before a 'listen' section will be
2494used for this section. The 'defaults' section uses the same syntax as the
2495'listen' section, for the supported parameters. The 'defaults' keyword ignores
2496everything on its command line, so that fake instance names can be specified
2497there for better clarity.
willy tarreaueedaa9f2005-12-17 14:08:03 +01002498
willy tarreau982249e2005-12-18 00:57:06 +01002499In version 1.1.28/1.2.1, only those parameters can be preset in the 'default'
willy tarreau197e8ec2005-12-17 14:10:59 +01002500section :
2501 - log (the first and second one)
willy tarreaueedaa9f2005-12-17 14:08:03 +01002502 - mode { tcp, http, health }
2503 - balance { roundrobin }
willy tarreau197e8ec2005-12-17 14:10:59 +01002504 - disabled (to disable every further instances)
2505 - enabled (to enable every further instances, this is the default)
willy tarreaueedaa9f2005-12-17 14:08:03 +01002506 - contimeout, clitimeout, srvtimeout, grace, retries, maxconn
willy tarreau982249e2005-12-18 00:57:06 +01002507 - option { redispatch, transparent, keepalive, forwardfor, logasap, httpclose,
2508 checkcache, httplog, tcplog, dontlognull, persist, httpchk }
willy tarreaueedaa9f2005-12-17 14:08:03 +01002509 - redispatch, redisp, transparent, source { addr:port }
2510 - cookie, capture
2511 - errorloc
2512
willy tarreau197e8ec2005-12-17 14:10:59 +01002513As of 1.1.24, it is not possible to put certain parameters in a 'defaults'
2514section, mainly regular expressions and server configurations :
willy tarreaueedaa9f2005-12-17 14:08:03 +01002515 - dispatch, server,
willy tarreau197e8ec2005-12-17 14:10:59 +01002516 - req*, rsp*
willy tarreaueedaa9f2005-12-17 14:08:03 +01002517
willy tarreau197e8ec2005-12-17 14:10:59 +01002518Last, there's no way yet to change a boolean option from its assigned default
2519value. So if an 'option' statement is set in a 'defaults' section, the only
2520way to flush it is to redefine a new 'defaults' section without this 'option'.
willy tarreaueedaa9f2005-12-17 14:08:03 +01002521
willy tarreau197e8ec2005-12-17 14:10:59 +01002522Examples :
willy tarreaueedaa9f2005-12-17 14:08:03 +01002523----------
2524 defaults applications TCP
willy tarreauc5f73ed2005-12-18 01:26:38 +01002525 log global
2526 mode tcp
2527 balance roundrobin
2528 clitimeout 180000
2529 srvtimeout 180000
2530 contimeout 4000
2531 retries 3
2532 redispatch
willy tarreaueedaa9f2005-12-17 14:08:03 +01002533
2534 listen app_tcp1 10.0.0.1:6000-6063
willy tarreauc5f73ed2005-12-18 01:26:38 +01002535 server srv1 192.168.1.1 check port 6000 inter 10000
2536 server srv2 192.168.1.2 backup
willy tarreaueedaa9f2005-12-17 14:08:03 +01002537
2538 listen app_tcp2 10.0.0.2:6000-6063
willy tarreauc5f73ed2005-12-18 01:26:38 +01002539 server srv1 192.168.2.1 check port 6000 inter 10000
2540 server srv2 192.168.2.2 backup
willy tarreaueedaa9f2005-12-17 14:08:03 +01002541
2542 defaults applications HTTP
willy tarreauc5f73ed2005-12-18 01:26:38 +01002543 log global
2544 mode http
2545 option httplog
2546 option forwardfor
2547 option dontlognull
2548 balance roundrobin
2549 clitimeout 20000
2550 srvtimeout 20000
2551 contimeout 4000
2552 retries 3
willy tarreaueedaa9f2005-12-17 14:08:03 +01002553
2554 listen app_http1 10.0.0.1:80-81
willy tarreauc5f73ed2005-12-18 01:26:38 +01002555 cookie SERVERID postonly insert indirect
2556 capture cookie userid= len 10
2557 server srv1 192.168.1.1:+8000 cookie srv1 check port 8080 inter 1000
2558 server srv1 192.168.1.2:+8000 cookie srv2 check port 8080 inter 1000
willy tarreaueedaa9f2005-12-17 14:08:03 +01002559
2560 defaults
willy tarreauc5f73ed2005-12-18 01:26:38 +01002561 # this empty section voids all default parameters
willy tarreaueedaa9f2005-12-17 14:08:03 +01002562
willy tarreau481132e2006-05-21 21:43:10 +02002563
25644.8) Status report in HTML page
2565-------------------------------
2566Starting with 1.2.14, it is possible for HAProxy to intercept requests for a
2567particular URI and return a full report of the proxy's activity and servers
2568statistics. This is available through the 'stats' keyword, associated to any
2569such options :
2570
2571 - stats enable
2572 - stats uri <uri prefix>
2573 - stats realm <authentication realm>
2574 - stats auth <user:password>
2575 - stats scope <proxy_id> | '.'
2576
2577By default, the status report is disabled. Specifying any combination above
2578enables it for the proxy instance referencing it. The easiest solution is to
2579use "stats enable" which will enable the report with default parameters :
2580
2581 - default URI : "/haproxy?stats" (CONFIG_STATS_DEFAULT_URI)
2582 - default auth : unspecified (no authentication)
2583 - default realm : "HAProxy Statistics" (CONFIG_STATS_DEFAULT_REALM)
2584 - default scope : unspecified (access to all instances)
2585
2586The "stats uri <uri_prefix>" option allows one to intercept another URI prefix.
2587Note that any URI that BEGINS with this string will match. For instance, one
2588proxy instance might be dedicated to status page only and would reply to any
2589URI.
2590
2591Example :
2592---------
2593 # catches any URI and returns the status page.
2594 listen stats :8080
2595 mode http
2596 stats uri /
2597
2598The "stats auth <user:password>" option enables Basic authentication and adds a
2599valid user:password combination to the list of authorized accounts. The user
2600and password are passed in the configuration file as clear text, and since this
2601is HTTP Basic authentication, you should be aware that it transits as clear
2602text on the network, so you must not use any sensible account. The list is
2603unlimited in order to provide easy accesses to developpers or customers.
2604
2605The "stats realm <realm>" option defines the "realm" name which is displayed
2606in the popup box when the browser asks for a password. It's important to ensure
2607that this one is not used by the application, otherwise the browser will try to
2608use a cached one from the application. Note that any space in the realm name
2609should be escaped with a backslash ('\').
2610
2611The "stats scope <proxy_id>" option limits the scope of the status report. By
2612default, all proxy instances are listed. But under some circumstances, it would
2613be better to limit the listing to some proxies or only to the current one. This
2614is what this option does. The special proxy name "." (a single dot) references
2615the current proxy. The proxy name can be repeated multiple times, even for
2616proxies defined later in the configuration or some which do not exist. The name
2617is the one which appears after the 'listen' keyword.
2618
2619Example :
2620---------
2621 # simple application with authenticated embedded status report
2622 listen app1 192.168.1.100:80
2623 mode http
willy tarreaud4ba08d2006-05-21 21:54:14 +02002624 option httpclose
willy tarreau481132e2006-05-21 21:43:10 +02002625 balance roundrobin
2626 cookie SERVERID postonly insert indirect
2627 server srv1 192.168.1.1:8080 cookie srv1 check inter 1000
2628 server srv1 192.168.1.2:8080 cookie srv2 check inter 1000
2629 stats uri /my_stats
willy tarreaud4ba08d2006-05-21 21:54:14 +02002630 stats realm Statistics\ for\ MyApp1-2
2631 stats auth guest:guest
2632 stats auth admin:AdMiN123
2633 stats scope .
2634 stats scope app2
willy tarreau481132e2006-05-21 21:43:10 +02002635
2636 # simple application with anonymous embedded status report
2637 listen app2 192.168.2.100:80
2638 mode http
willy tarreaud4ba08d2006-05-21 21:54:14 +02002639 option httpclose
willy tarreau481132e2006-05-21 21:43:10 +02002640 balance roundrobin
2641 cookie SERVERID postonly insert indirect
2642 server srv1 192.168.2.1:8080 cookie srv1 check inter 1000
2643 server srv1 192.168.2.2:8080 cookie srv2 check inter 1000
2644 stats uri /my_stats
willy tarreaud4ba08d2006-05-21 21:54:14 +02002645 stats realm Statistics\ for\ MyApp2
2646 stats scope .
willy tarreau481132e2006-05-21 21:43:10 +02002647
2648 listen admin_page :8080
2649 mode http
2650 stats uri /my_stats
willy tarreaud4ba08d2006-05-21 21:54:14 +02002651 stats realm Global\ statistics
2652 stats auth admin:AdMiN123
willy tarreau481132e2006-05-21 21:43:10 +02002653
2654Notes :
2655-------
2656 - The 'stats' options can also be specified in the 'defaults' section, in
2657 which case it will provide the exact same configuration to all further
2658 instances (hence the usefulness of the scope "."). However, if an instance
2659 redefines any 'stats' parameter, defaults will not be used for this
2660 instance.
2661
2662 - HTTP Basic authentication is very basic and unsecure from snooping. No
2663 sensible password should be used, and be aware that there is no way to
2664 remove it from the browser so it will be sent to the whole application
2665 upon further accesses.
2666
willy tarreaud4ba08d2006-05-21 21:54:14 +02002667 - It is very important that the 'option httpclose' is specified, otherwise
2668 the proxy will not be able to detect the URI within keep-alive sessions
2669 maintained between the browser and the servers, so the stats URI will be
2670 forwarded unmodified to the server as if the option was not set.
2671
willy tarreau481132e2006-05-21 21:43:10 +02002672
Willy Tarreau726c2bf2007-05-09 01:31:45 +020026735) Access lists
2674===============
2675
2676With version 1.3.10, a new concept of access lists (acl) was born. As it was
2677not necesary to reinvent the wheel, and because even long thoughts lead to
2678unsatisfying proposals, it was finally decided that something close to what
2679Squid provides would be a good compromise between features and ease of use.
2680
2681The principle is very simple : acls are declared with a name, a test and a list
2682of valid values to check against during the test. Conditions are applied on
2683various actions, and those conditions apply a logical AND between acls. The
2684condition is then only met if all acls are true.
2685
2686It is possible to use the reserved keyword "OR" in conditions, and it is
2687possible for an acl to be specified multiple times, even with various tests, in
2688which case the first one which returns true validates the ACL.
2689
Willy Tarreauae8b7962007-06-09 23:10:04 +02002690As of 1.3.12, only the following tests have been implemented :
Willy Tarreau726c2bf2007-05-09 01:31:45 +02002691
2692 Layer 3/4 :
2693 src <ipv4_address>[/mask] ... : match IPv4 source address
2694 dst <ipv4_address>[/mask] ... : match IPv4 destination address
Willy Tarreauae8b7962007-06-09 23:10:04 +02002695 src_port <range> ... : match source port range
2696 dst_port <range> ... : match destination port range
2697 dst_conn <range> ... : match #connections on frontend
Willy Tarreau726c2bf2007-05-09 01:31:45 +02002698
2699 Layer 7 :
2700 method <HTTP method> ... : match HTTP method
2701 req_ver <1.0|1.1> ... : match HTTP request version
2702 resp_ver <1.0|1.1> ... : match HTTP response version
Willy Tarreauae8b7962007-06-09 23:10:04 +02002703 status <range> ... : match HTTP response status code in range
Willy Tarreau726c2bf2007-05-09 01:31:45 +02002704 url <string> ... : exact string match on URI
2705 url_reg <regex> ... : regex string match on URI
2706 url_beg <string> ... : true if URI begins with <string>
2707 url_end <string> ... : true if URI ends with <string>
2708 url_sub <string> ... : true if URI contains <string>
2709 url_dir <string> ... : true if URI contains <string> between slashes
2710 url_dom <string> ... : true if URI contains <string> between slashes or dots
2711
Willy Tarreauae8b7962007-06-09 23:10:04 +02002712A 'range' is one or two integers which may be prefixed by an operator.
2713The syntax is :
2714
2715 [<op>] <low>[:<high>]
2716
2717Where <op> can be :
2718 'eq' : the tested value must be equal to <low> or within <low>..<high>
2719 'le' : the tested value must be lower than or equal to <low>
2720 'lt' : the tested value must be lower than <low>
2721 'ge' : the tested value must be greater than or equal to <low>
2722 'gt' : the tested value must be greater than <low>
2723
2724When no operator is defined, 'eq' is assumed. Note that when the operator is
2725specified, it applies to all subsequent ranges of values until the end of the
2726line is reached or another operator is specified. Example :
2727
2728 acl status_error status 400:599
2729 acl saturated_frt dst_conn ge 1000
2730 acl invalid_ports src_port lt 512 ge 65535
2731
Willy Tarreau726c2bf2007-05-09 01:31:45 +02002732Other ones are coming (headers, cookies, time, auth), it's just a matter of
2733time. It is also planned to be able to read the patterns from a file, as well
2734as to ignore the case for some of them.
2735
2736The only command supporting a condition right now is the "block" command, which
2737blocks a request and returns a 403 if its condition is true (with the "if"
2738keyword), or if it is false (with the "unless" keyword).
2739
2740Example :
2741---------
2742
2743 acl options_uris url *
2744 acl meth_option method OPTIONS
2745 acl http_1.1 req_ver 1.1
2746 acl allowed_meth method GET HEAD POST OPTIONS CONNECT
2747 acl connect_meth method CONNECT
2748 acl proxy_url url_beg http://
2749
2750 # block if reserved URI "*" used with a method other than "OPTIONS"
2751 block if options_uris !meth_option
2752
2753 # block if the OPTIONS method is used with HTTP 1.0
2754 block if meth_option !http_1.1
2755
2756 # allow non-proxy url with anything but the CONNECT method
2757 block if !connect_meth !proxy_url
2758
2759 # block all unknown methods
2760 block unless allowed_meth
2761
2762Note: this documentation is very light but should permit one to start and above
2763all it should permit to work on the project without being slowed down too much
2764with the doc.
2765
2766
willy tarreau197e8ec2005-12-17 14:10:59 +01002767=========================
2768| System-specific setup |
2769=========================
willy tarreaueedaa9f2005-12-17 14:08:03 +01002770
willy tarreau197e8ec2005-12-17 14:10:59 +01002771Linux 2.4
2772=========
willy tarreaueedaa9f2005-12-17 14:08:03 +01002773
2774-- cut here --
2775#!/bin/sh
2776# set this to about 256/4M (16384 for 256M machine)
2777MAXFILES=16384
2778echo $MAXFILES > /proc/sys/fs/file-max
2779ulimit -n $MAXFILES
2780
2781if [ -e /proc/sys/net/ipv4/ip_conntrack_max ]; then
willy tarreauc5f73ed2005-12-18 01:26:38 +01002782 echo 65536 > /proc/sys/net/ipv4/ip_conntrack_max
willy tarreaueedaa9f2005-12-17 14:08:03 +01002783fi
2784
2785if [ -e /proc/sys/net/ipv4/netfilter/ip_ct_tcp_timeout_fin_wait ]; then
willy tarreauc5f73ed2005-12-18 01:26:38 +01002786 # 30 seconds for fin, 15 for time wait
2787 echo 3000 > /proc/sys/net/ipv4/netfilter/ip_ct_tcp_timeout_fin_wait
2788 echo 1500 > /proc/sys/net/ipv4/netfilter/ip_ct_tcp_timeout_time_wait
2789 echo 0 > /proc/sys/net/ipv4/netfilter/ip_ct_tcp_log_invalid_scale
2790 echo 0 > /proc/sys/net/ipv4/netfilter/ip_ct_tcp_log_out_of_window
willy tarreaueedaa9f2005-12-17 14:08:03 +01002791fi
2792
2793echo 1024 60999 > /proc/sys/net/ipv4/ip_local_port_range
2794echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout
2795echo 4096 > /proc/sys/net/ipv4/tcp_max_syn_backlog
2796echo 262144 > /proc/sys/net/ipv4/tcp_max_tw_buckets
2797echo 262144 > /proc/sys/net/ipv4/tcp_max_orphans
2798echo 300 > /proc/sys/net/ipv4/tcp_keepalive_time
2799echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle
2800echo 0 > /proc/sys/net/ipv4/tcp_timestamps
2801echo 0 > /proc/sys/net/ipv4/tcp_ecn
willy tarreauc5f73ed2005-12-18 01:26:38 +01002802echo 1 > /proc/sys/net/ipv4/tcp_sack
willy tarreaueedaa9f2005-12-17 14:08:03 +01002803echo 0 > /proc/sys/net/ipv4/tcp_dsack
2804
2805# auto-tuned on 2.4
2806#echo 262143 > /proc/sys/net/core/rmem_max
2807#echo 262143 > /proc/sys/net/core/rmem_default
2808
2809echo 16384 65536 524288 > /proc/sys/net/ipv4/tcp_rmem
2810echo 16384 349520 699040 > /proc/sys/net/ipv4/tcp_wmem
2811
2812-- cut here --
2813
willy tarreau197e8ec2005-12-17 14:10:59 +01002814
2815FreeBSD
2816=======
2817
2818A FreeBSD port of HA-Proxy is now available and maintained, thanks to
2819Clement Laforet <sheepkiller@cultdeadsheep.org>.
2820
2821For more information :
2822http://www.freebsd.org/cgi/url.cgi?ports/net/haproxy/pkg-descr
2823http://www.freebsd.org/cgi/cvsweb.cgi/ports/net/haproxy/
2824http://www.freshports.org/net/haproxy
2825
2826
2827-- end --