DOC: fix some spelling issues over multiple files
This is from the output of codespell and may be backported.
(cherry picked from commit cdbcca9995a4974c6998720e158d6f1d4d9a25c0)
Signed-off-by: Willy Tarreau <w@1wt.eu>
(cherry picked from commit dab4ba60421f3a9ea984df3ccd79604bc7224ead)
Signed-off-by: Willy Tarreau <w@1wt.eu>
(cherry picked from commit 12bbe3a4d8331579170f3f067373f88b6eeb869f)
[wt: dropped non-existing parts]
Signed-off-by: Willy Tarreau <w@1wt.eu>
(cherry picked from commit 59bf7fc483cb09f4a378009b1eb035ccfd8f07fa)
[wt: dropped non-existing parts]
Signed-off-by: Willy Tarreau <w@1wt.eu>
diff --git a/doc/configuration.txt b/doc/configuration.txt
index 78bed42..1cb14bf 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -816,7 +816,7 @@
See "option external-check".
gid <number>
- Changes the process' group ID to <number>. It is recommended that the group
+ Changes the process's group ID to <number>. It is recommended that the group
ID is dedicated to HAProxy or to a small set of similar daemons. HAProxy must
be started with a user belonging to this group, or with superuser privileges.
Note that if haproxy is started from a user having supplementary groups, it
@@ -1243,7 +1243,7 @@
possible to change this value with "stats maxconn".
uid <number>
- Changes the process' user ID to <number>. It is recommended that the user ID
+ Changes the process's user ID to <number>. It is recommended that the user ID
is dedicated to HAProxy or to a small set of similar daemons. HAProxy must
be started with superuser privileges in order to be able to switch to another
one. See also "gid" and "user".
@@ -13003,7 +13003,7 @@
HAProxy allows using a host name on the server line to retrieve its IP address
using name servers. By default, HAProxy resolves the name when parsing the
-configuration file, at startup and cache the result for the process' life.
+configuration file, at startup and cache the result for the process's life.
This is not sufficient in some cases, such as in Amazon where a server's IP
can change after a reboot or an ELB Virtual IP can change based on current
workload.
@@ -18246,7 +18246,7 @@
external attacks.
PC The proxy refused to establish a connection to the server because the
- process' socket limit has been reached while attempting to connect.
+ process's socket limit has been reached while attempting to connect.
The global "maxconn" parameter may be increased in the configuration
so that it does not happen anymore. This status is very rare and
might happen when the global "ulimit-n" parameter is forced by hand.
diff --git a/doc/internals/acl.txt b/doc/internals/acl.txt
index 320381a..0379331 100644
--- a/doc/internals/acl.txt
+++ b/doc/internals/acl.txt
@@ -25,7 +25,7 @@
found in HTTP headers, which can appear multiple times. The acl_test is kept
intact between calls and even holds a context so that the fetch function knows
where to start from for subsequent calls. The match function may also use the
-context eventhough it was not designed for that purpose.
+context even though it was not designed for that purpose.
An ACL is defined only by its name and can be a series of ACL expressions. The
ACL is deemed true when any of its expressions is true. They are evaluated in
@@ -35,7 +35,7 @@
- an ACL is a series of tests to perform on a stream, any of which is enough
to validate the result.
-
+
- each test is defined by an expression associating a keyword and a series of
patterns.
@@ -59,7 +59,7 @@
We could then represent a rule by the following BNF :
- rule = if-cond
+ rule = if-cond
| unless-cond
if-cond (struct acl_cond with ->pol = ACL_COND_IF)
diff --git a/doc/internals/buffer-api.txt b/doc/internals/buffer-api.txt
index abb5e9f..90113a1 100644
--- a/doc/internals/buffer-api.txt
+++ b/doc/internals/buffer-api.txt
@@ -9,7 +9,7 @@
defragmentation, and are used to carry intermediary representations between the
various internal layers. They support wrapping at the end, and they carry their
own size information so that in theory it would be possible to use different
-buffer sizes in parallel eventhough this is not currently implemented.
+buffer sizes in parallel even though this is not currently implemented.
The format of this structure has evolved over time, to reach a point where it
is convenient and versatile enough to have permitted to make several internal
diff --git a/doc/internals/htx-api.txt b/doc/internals/htx-api.txt
new file mode 100644
index 0000000..f1e38d1
--- /dev/null
+++ b/doc/internals/htx-api.txt
@@ -0,0 +1,498 @@
+ -----------------------------------------------
+ HTX API
+ Version 1.0
+ ( Last update: 2019-06-20 )
+ -----------------------------------------------
+ Author : Christopher Faulet
+ Contact : cfaulet at haproxy dot com
+
+1. Background
+
+Historically, HAProxy stored HTTP messages in a raw fashion in buffers, keeping
+parsing information separately in a "struct http_msg" owned by the stream. It was
+optimized to the data transfer, but not so much for rewrites. It was also HTTP/1
+centered. While it was the only HTTP version supported, it was not a
+problem. But with the rise of HTTP/2, it starts to be hard to still use this
+representation.
+
+At the first age of the HTTP/2 in HAProxy, H2 messages were converted into
+H1. This was terribly unefficient because it required two parsing passes, a
+first one in H2 and a second one in H1, with a conversion in the middle. And of
+course, the same was also true in the opposite direction. outgoing H1 messages
+had to be converted back in H2 to be sent. Even worse, because the H2->H1
+conversion, only client H2 connections were supported.
+
+So, to address all these problems, we decided to replace the old raw
+representation by a version-agnostic and self-structured internal HTTP
+representation, the HTX. As an additional benefit, with this new representation,
+the message parsing and its processing are now separated, making all the HTTP
+analysis simpler and cleaner. The parsing of HTTP messages is now handled by
+the multiplexers (h1 or h2).
+
+
+2. The HTX message
+
+The HTX is a structure containing useful information about an HTTP message
+followed by a contiguous array with some parts of the message. These parts are
+called blocks. A block is composed of metadata (htx_blk) and an associated
+payload. Blocks' metadata are stored starting from the end of the array while
+their payload are stored at the beginning. Blocks' metadata are often simply
+called blocks. it is a misuse of language that's simplify explanations.
+
+Internally, this structure is "hidden" in a buffer. This way, there are few
+changes into intermediate layers (stream-interface and channels). They still
+manipulate buffers. Only the multiplexer and the stream have to know how data
+are really stored. From the HTX perspective, a buffer is just a memory
+area. When an HTX message is stored in a buffer, this one appears as full.
+
+ * General view of an HTX message :
+
+
+ buffer->area
+ |
+ |<------------ buffer->size == buffer->data ----------------------|
+ | |
+ | |<------------- Blocks array (htx->size) ------------------>|
+ V | |
+ +-----+-----------------+-------------------------+---------------+
+ | HTX | PAYLOADS ==> | | <== HTX_BLKs |
+ +-----+-----------------+-------------------------+---------------+
+ | | | |
+ |<-payloads part->|<----- free space ------>|<-blocks part->|
+ (htx->data)
+
+
+The blocks part remains linear and sorted. You may think about it as an array
+with negative indexes. But, instead of using negative indexes, we use positive
+positions to identify a block. This position is then converted to an address
+relatively to the beginning of the blocks array.
+
+ tail head
+ | |
+ V V
+ .....--+----+-----------------------+------+------+
+ | Bn | ... | B1 | B0 |
+ .....--+----+-----------------------+------+------+
+ ^ ^ ^
+ Addr of the block Addr of the block Addr of the block
+ at the position N at the position 1 at the position 0
+
+
+In the HTX structure, 3 "special" positions are stored:
+
+ - tail : Position of the newest inserted block
+ - head : Position of the oldest inserted block
+ - first : Position of the first block to (re)start the analyse
+
+The blocks part never wrap. If we have no space to allocate a new block and if
+there is a hole at the beginning of the blocks part (so at the end of the blocks
+array), we move back all blocks.
+
+
+ tail head tail head
+ | | | |
+ V V V V
+ ...+--------------+---------+ blocks ...----------+--------------+
+ | X== HTX_BLKS | | defrag | <== HTX_BLKS |
+ ...+--------------+---------+ =====> ...----------+--------------+
+
+
+The payloads part is a raw space that may wrap. You never access to a block's
+payload directly. Instead you get a block to retrieve the address of its
+payload.
+
+
+ +------------------------( B0.addr )--------------------------+
+ | +-------------------( B1.addr )----------------------+ |
+ | | +-----------( B2.addr )----------------+ | |
+ V V V | | |
+ +-----+----+-------+----+--------+-------------+-------+----+----+----+
+ | HTX | P0 | P1 | P2 | ...==> | | <=... | B2 | B1 | B0 |
+ +-----+----+-------+----+--------+-------------+-------+----+----+----+
+
+
+Because the payloads part may wrap, there are 2 usable free spaces:
+
+ - The free space in front of the blocks part. This one is used if and only if
+ the other one was not used yet.
+
+ - The free space at the beginning of the message. Once this one is used, the
+ other one is never used again, until a message defragmentation.
+
+
+ * Linear payloads part :
+
+
+ head_addr end_addr tail_addr
+ | | |
+ V V V
+ +-----+--------------------+-------------+--------------------+-------...
+ | HTX | | PAYLOADS | | HTX_BLKs
+ +-----+--------------------+-------------+--------------------+-------...
+ |<-- free space 2 -->| |<-- free space 1 -->|
+ (used if the other is too small) (used in priority)
+
+
+ * Wrapping payloads part :
+
+
+ head_addr end_addr tail_addr
+ | | |
+ V V V
+ +-----+----+----------------+--------+----------------+-------+-------...
+ | HTX | | PAYLOADS part2 | | PAYLOADS part1 | | HTX_BLKs
+ +-----+----+----------------+--------+----------------+-------+-------...
+ |<-->| |<------>| |<----->|
+ unusable free space unusable
+ free space free space
+
+
+Finally, when the usable free space is not enough to store a new block, unsuable
+parts may be get back with a full defragmentation. The payloads part is then
+realigned at the beginning of the blocks array and the free space becomes
+continuous again.
+
+
+3. The HTX blocks
+
+An HTX block can be as well a start-line as a header, a body part or a
+trailer. For all these types of block, a payload is attached to the block. It
+can also be a marker, the end-of-headers, end-of-trailers or end-of-message. For
+these blocks, there is no payload but it counts for a byte. It is important to
+not skip it when data are forwarded.
+
+As already said, a block is composed of metadata and a payload. Metadata are
+stored in the blocks part and are composed of 2 fields :
+
+ - info : It a 32 bits field containing the block's type on 4 bits followed
+ by the payload length. See below for details.
+
+ - addr : The payload's address, if any, relatively to the beginning the
+ array used to store part of the HTTP message itself.
+
+
+ * Block's info representation :
+
+ 0b 0000 0000 0000 0000 0000 0000 0000 0000
+ ---- ------------------------ ---------
+ type value (1 MB max) name length (header/trailer - 256B max)
+ ----------------------------------
+ data length (256 MB max)
+ (body, method, path, version, status, reason)
+
+
+Supported types are :
+
+ - 0000 (0) : The request start-line
+ - 0001 (1) : The response start-line
+ - 0010 (2) : A header block
+ - 0011 (3) : The end-of-headers marker
+ - 0100 (4) : A data block
+ - 0101 (5) : A trailer block
+ - 0110 (6) : The end-of-trailers marker
+ - 0111 (7) : The end-of-message marker
+ - 1111 (15) : An unused block
+
+Other types are unused for now and reserved for futur extensions.
+
+An HTX message is typically composed of following blocks, in this order :
+
+ - a start-line
+ - zero or more header blocks
+ - an end-of-headers marker
+ - zero or more data blocks
+ - zero or more trailer blocks (optional)
+ - an end-of-trailers marker (optional but always set if there is at least
+ one trailer block)
+ - an end-of-message marker.
+
+Only one HTTP request at a time can be stored in an HTX message. For HTTP
+response, it is more complicated. Only one "final" response can be stored in an
+HTX message. It is a response with status-code 101 or greater or equal to
+200. But it may be preceded by several 1xx informational responses. Such
+responses are part of the same HTX message, so there is no end-of-message marker
+for them.
+
+
+3.1. The start-line
+
+Every HTX message starts with a start-line. Its payload is a "struct htx_sl". In
+addition to the parts of the HTTP start-line, this structure contains some
+information about the represented HTTP message, mainly in the form of flags
+(HTX_SL_F_*). For instance, if an HTTP message contains the header
+"conten-length", then the flag HTX_SL_F_CLEN is set.
+
+Each HTTP message has its own start-line. So an HTX request has one and only one
+start-line because it must contain only one HTTP request at a time. But an HTX
+response may have more than one start-line if the final HTTP response is
+precedeed by some 1xx informational responses.
+
+In HTTP/2, there is no start-line. So the H2 multiplexer must create one when it
+converts an H2 message to HTX :
+
+ - For the request, it uses the pseudo headers ":method", ":path" or
+ ":authority" depending on the method and the hardcoded version "HTTP/2.0".
+
+ - For the response, it used the hardcoded version "HTTP/2.0", the
+ pseudo-header ":status" and an empty reason.
+
+
+3.2. The headers and trailers
+
+HTX Headers and trailers are quite similar. Different types are used to simplify
+headers processing. But from the HTX point of view, there is no real difference,
+except their position in the HTX message. The header blocks always follow an HTX
+start-line while trailer blocks come after the data. If there is no data, they
+follow the end-of-headers marker.
+
+Headers and trailers are the only blocks containing a Key/Value payload. The
+corresponding end-of marker must always be placed after each group to mark, as
+it name suggests, the end.
+
+In HTTP/1, trailers are only present on chunked messages. But chunked messages
+do not always have trailers. In this case, the end-of-trailers block may or may
+not be present. Multiplexers must be able to handle both situations. In HTTP/2,
+trailers are only present if a HEADERS frame is sent after DATA frames.
+
+
+3.3. The data
+
+The payload body of an HTTP message is stored as DATA blocks in the HTX
+message. For HTTP/1 messages, it is the message body without the chunks
+formatting, if any. For HTTP/2, it is the payload of DATA frames.
+
+The DATA blocks are the only HTX blocks that may be partially processed (copied
+or removed). All other types of block must be entierly processed. This means
+DATA blocks can be resized.
+
+
+3.4. The end-of markers
+
+These blocks are used to delimit parts of an HTX message. It exists three
+markers:
+
+ - end-of-headers (EOH)
+ - end-of-trailers (EOT)
+ - end-of-message (EOM)
+
+EOH and EOM are always present in an HTX message. EOT is optional.
+
+
+4. The HTX API
+
+
+4.1. Get/set HTX message from/to the underlying buffer
+
+The first thing to do to process an HTX message is to get it from the underlying
+buffer. There are 2 functions to do so, the second one relying on the first:
+
+ - htxbuf() returns an HTX message from a buffer. It does not modify the
+ buffer. It only initialize the HTX message if the buffer is empty.
+
+ - htx_from_buf() uses htxbuf(). But it also updates the underlying buffer so
+ that it appears as full.
+
+Both functions return a "zero-sized" HTX message if the buffer is null. This
+way, you are sure to always have a valid HTX message. The first function is the
+default function to use. The second one is only useful when some content will be
+added. For instance, it used by the HTX analyzers when HAproxy generates a
+response. This way, the buffer is in a right state and you don't need to take
+care of it anymore outside the possible error paths.
+
+Once the processing done, if the HTX message has been modified, the underlying
+buffer must be also updated, except you uses htx_from_buf() and you only add
+data. For all other cases, the function htx_to_buf() must be called.
+
+Finally, the function htx_reset() may be called at any time to reset an HTX
+message. And the function buf_room_for_htx_data() may be called to know if a raw
+buffer is full from the HTX perspective. It is used during conversion from/to
+the HTX.
+
+
+4.2. Helpers to deal with free space in an HTX message
+
+Once you have an HTX message, following functions may help you to process it :
+
+ - htx_used_space() and htx_meta_space() return, respectively, the total
+ space used in an HTX message and the space used by block's metadata only.
+
+ - htx_free_space() and htx_free_data_space() return, respectively, the total
+ free space in an HTX message and the free space available for the payload
+ if a new HTX block is stored (so it is the total free space minus the size
+ of an HTX block).
+
+ - htx_is_empty() and htx_is_not_empty() are boolean functions to know if an
+ HTX message is empty or not.
+
+ - htx_get_max_blksz() returns the maximum size available for the payload,
+ not exceeding a maximum, metadata included.
+
+ - htx_almost_full() should be used to know if an HTX message uses at least
+ 3/4 of its capacity.
+
+
+4.3. HTX Blocks manipulations
+
+Once you know how much space is available in an HTX message, the next step is to
+add HTX blocks. First of all the function htx_nbblks() returns the number of
+blocks allocated in an HTX message. Then, there is an add function per block's
+type:
+
+ - htx_add_stline() adds a start-line. The type (request or response) and the
+ flags of the start-line must be provided, as well as its three parts
+ (method,uri,version or version,status-code,reason).
+
+ - htx_add_header() and htx_add_trailers() are similar. The name and the
+ value must be provided. The inserted HTX block is returned on success or
+ NULL if an error occurred.
+
+ - htx_add_endof() must be used to add any end-of marker. The block's type
+ (EOH, EOT or EOM) must be specified. The inserted HTX block is returned on
+ success or NULL if an error occurred.
+
+ - htx_add_all_headers() and htx_add_all_trailers() add, respectively, a list
+ of headers and a list of trailers, followed by the appropriate end-of
+ marker. On success, this marker is returned. Otherwise, NULL is
+ returned. Note there is no rollback on the HTX message when an error
+ occurred. Some headers or trailers may have been added. So it is the
+ caller responsibility to take care of that.
+
+ - htx_add_data() must be used to add a DATA block. Unlike previous
+ functions, this one returns the number of bytes copied or 0 if nothing was
+ copied. If possible, the data are appended to the last DATA block, if
+ any. Only a part of the payload may be copied because this function will
+ try to limit the message defragmentation and the wrapping of blocks as far
+ as possible. If you really need to add all data or nothing, the function
+ htx_add_data_atonce() must be used instead. Because it tries to insert all
+ the payload, this function returns the inserted block on success.
+ Otherwise it returns NULL.
+
+When an HTX block is added, it is always the last one (the tail). But, if you
+need to add a block at a specific place, it is not really handy. 2 functions may
+help you (others could be added) :
+
+ - htx_add_last_data() adds a DATA block just after all other DATA blocks and
+ before any trailers and EOT or EOM markers. It relies on
+ htx_add_data_atonce(), so a defragmentation may be performed.
+
+ - htx_move_blk_before() moves a specific block just after another one. Both
+ blocks must already be in the HTX message and the block to move must
+ always be placed after the "pivot".
+
+Once added, there are three functions to update the block's payload :
+
+ - htx_replace_stline() updates a start-line. The HTX block must be passed as
+ argument. Only string parts of the start-line are updated by this
+ function. On success, it returns the new start-line. So it is pretty easy
+ to update its flags. NULL is returned if an error occurred.
+
+ - htx_replace_header() fully replaces a header (its name and its value) by a
+ new one. The HTX block must be passed a argument, as well as its new name
+ and its new value. The new header can be smaller or larger than the old
+ one. This function returns the new HTX block on success, or NULL is an
+ error occurred.
+
+ - htx_replace_blk_value() replaces a part of a block's payload or its
+ totality. It works for HEADERS, TRAILERS or DATA blocks. The HTX block
+ must be provided with the part to remove and the new one. The new part can
+ be smaller or larger than the old one. This function returns the new HTX
+ block on success, or NULL is an error occurred.
+
+Finally, You may remove a block using the function htx_remove_blk(). This
+function returns the block following the one removed or NULL if it is the tail
+block.
+
+
+4.4. The HTX start-line
+
+Unlike other HTX blocks, the start-line is a bit special because its payload is
+a structure followed by its three parts :
+
+ +--------+-------+-------+-------+
+ | HTX_SL | PART1 | PART2 | PART3 |
+ +--------+-------+-------+-------+
+
+Some macros and functions may help to manipulate these parts :
+
+ - HTX_SL_P{N}_LEN() and HTX_SL_P{N}_PTR() are macros to get the length of a
+ part and a pointer on it. {N} should be 1, 2 or 3.
+
+ - HTX_SL_REQ_MLEN(), HTX_SL_REQ_ULEN(), HTX_SL_REQ_VLEN(),
+ HTX_SL_REQ_MPTR(), HTX_SL_REQ_UPTR() and HTX_SL_REQ_VPTR() are macros to
+ get info about a request start-line. These macros only wrap HTX_SL_P*
+ ones.
+
+ - HTX_SL_RES_VLEN(), HTX_SL_RES_CLEN(), HTX_SL_RES_RLEN(),
+ HTX_SL_RES_VPTR(), HTX_SL_RES_CPTR() and HTX_SL_RES_RPTR() are macros to
+ get info about a response start-line. These macros only wrap HTX_SL_P*
+ ones.
+
+ - htx_sl_p1(), htx_sl_p2() and htx_sl_p2() are functions to get the ist
+ corresponding to the right part of a start-line.
+
+ - htx_sl_req_meth(), htx_sl_req_uri() and htx_sl_req_vsn() get the ist
+ corresponding to the right part of a request start-line.
+
+ - htx_sl_res_vsn(), htx_sl_res_code() and htx_sl_res_reason() get the ist
+ corresponding to the right part of a response start-line.
+
+
+4.5. Iterate on the HTX message
+
+To iterate on an HTX message, the first thing to do is to get the HTX block to
+start the loop. There are three special blocks in an HTX message that may be
+good candidates to start a loop :
+
+ * the head block. It is the oldest inserted block. Multiplexers always start
+ to consume an HTX message from this block. The function htx_get_head()
+ returns its position and htx_get_head_blk() returns the blocks itself. In
+ addition, the function htx_get_head_type() returns its block's type.
+
+ * the tail block. It is the newest inserted block. The function htx_get_tail()
+ returns its position and htx_get_tail_blk() returns the blocks itself. In
+ addition, the function htx_get_tail_type() returns its block's type.
+
+ * the first block. It is the block where to (re)start the analyse. It is used
+ as start point by HTX analyzers. The function htx_get_first() returns its
+ position and htx_get_first_blk() returns the blocks itself. In addition, the
+ function htx_get_first_type() returns its block's type.
+
+For all these functions, if the HTX message is empty, -1 is returned for the
+block's position, NULL instead of a block and HTX_BLK_UNUSED for its type.
+
+Then to iterate on blocks, you may move foreword or backward :
+
+ * htx_get_prev() and htx_get_next() return, respectively, the position of the
+ previous block or the next block, given a specific position. Or -1 if an edge
+ is reached.
+
+ * htx_get_prev_blk() and htx_get_next_blk() return, respectively, the previous
+ block or the next one, given a specific block. Or NULL if an edge is
+ reached.
+
+
+4.6. Advanced functions
+
+Some more advanced functions may be used to do complex processing on the HTX
+message. These functions are used by HTX analyzers or by multiplexers.
+
+ * htx_truncate() removes all blocks after the one containing a specific offset
+ relatively to the head block of the HTX message. If the offset is inside a
+ DATA block, it is truncated. For all other blocks, the removal starts to the
+ next block.
+
+ * htx_drain() tries to remove a specific amount of bytes of payload. If the
+ last block is a DATA block, it may be truncated if necessary. All other
+ block are removed at once or kept. This function returns a mixed value, with
+ the first block not removed, or NULL if everything was removed, and the
+ amount of data drained.
+
+ * htx_xfer_blks() transfers HTX blocks from an HTX message to another,
+ stopping on the first block of a specified type or when a specific amount of
+ bytes, including meta-data, was moved. If the last block is a DATA block, it
+ may be partially moved. All other block are transferred at once or
+ kept. This function returns a mixed value, with the last block moved, or
+ NULL if nothing was moved, and the amount of data transferred. When HEADERS
+ or TRAILERS blocks must be transferred, this function transfers all of
+ them. Otherwise, if it is not possible, it triggers an error. It is the
+ caller responsibility to transfer all headers or trailers at once.
diff --git a/doc/management.txt b/doc/management.txt
index bc6d0f8..c64571d 100644
--- a/doc/management.txt
+++ b/doc/management.txt
@@ -1859,7 +1859,7 @@
of reports of abnormal behaviours. A typical example would be a properly
running process never sleeping and eating 100% of the CPU. The output fields
will be made of one line per metric, and per-thread counters on the same
- line. These counters are 32-bit and will wrap during the process' life, which
+ line. These counters are 32-bit and will wrap during the process's life, which
is not a problem since calls to this command will typically be performed
twice. The fields are purposely not documented so that their exact meaning is
verified in the code where the counters are fed. These values are also reset