DOC: add a diagram to explain how circular buffers work
Also add some thoughts about the existing and new design.
Note: an earlier design used the names "head" and "tail" for both sides
of the buffer, but it appears awkward as these words may be understood
in two forms (feed by head, output by tail, or make the newcomers wait
at the tail of the queue). Also there were already a few functions in the
code making use of either terminology. So better avoid this terminology
and use "input" and "output" instead.
diff --git a/doc/internals/buffer-operations.txt b/doc/internals/buffer-operations.txt
new file mode 100644
index 0000000..1692f21
--- /dev/null
+++ b/doc/internals/buffer-operations.txt
@@ -0,0 +1,128 @@
+2012/02/27 - Operations on haproxy buffers - w@1wt.eu
+
+
+1) Definitions
+--------------
+
+A buffer is a unidirectional storage between two stream interfaces which are
+most often composed of a socket file descriptor. This storage is fixed sized
+and circular, which means that once data reach the end of the buffer, it loops
+back at the beginning of the buffer :
+
+
+ Representation of a non-wrapping buffer
+ ---------------------------------------
+
+
+ beginning end
+ | -------- length --------> |
+ V V
+ +-------------------------------------------+
+ | <--------------- size ----------------> |
+ +-------------------------------------------+
+
+
+ Representation of a wrapping buffer
+ -----------------------------------
+
+ end beginning
+ +------> | | -------------+
+ | V V |
+ | +-------------------------------------------+ |
+ | | <--------------- size ----------------> | |
+ | +-------------------------------------------+ |
+ | |
+ +--------------------- length -----------------------+
+
+
+Buffers are read by two entities :
+ - stream interfaces
+ - analysers
+
+Buffers are filled by two entities :
+ - stream interfaces
+ - hijackers
+
+A stream interface writes at the input of a buffer and reads at its output. An
+analyser has to parse incoming buffer contents, so it reads the input. It does
+not really write the output though it may change the buffer's contents at the
+input, possibly causing data moves. A hijacker it able to write at the output
+of a buffer. Hijackers are not used anymore at the moment though error outputs
+still work the same way.
+
+Buffers are referenced in the session. Each session has two buffers which
+interconnect the two stream interfaces. One buffer is called the request
+buffer, it sees traffic flowing from the client to the server. The other buffer
+is the response buffer, it sees traffic flowing from the server to the client.
+
+By convention, sessions are represented as 2 buffers one on top of the other,
+and with 2 stream interfaces connected to the two buffers. The client connects
+to the left stream interface (which then acts as a server), and the right
+stream interface (which acts as a client) connects to the server. The data
+circulate clockwise, so the upper buffer is the request buffer and the lower
+buffer is the response buffer :
+
+ ,------------------------.
+ ,-----> | request buffer | ------.
+ from ,--./ `------------------------' \,--. to
+ client ( ) ( ) server
+ `--' ,------------------------. /`--'
+ ^------- | response buffer | <-----'
+ `------------------------'
+
+2) Operations
+-------------
+
+Socket-based stream interfaces write to buffers directly from the I/O layer
+without relying on any specific function.
+
+Function-based stream interfaces do use a number of non-uniform functions to
+read from the buffer's output and to write to the buffer's input. More suited
+names could be :
+
+ int buffer_output_peek_at(buf, ofs, ptr, size);
+ int buffer_output_peek(buf, ptr, size);
+ int buffer_output_read(buf, ptr, size);
+ int buffer_output_skip(buf, size);
+ int buffer_input_write(buf, ptr, size);
+
+Right now some stream interfaces use the following functions which also happen
+to automatically schedule the response for automatic forward :
+
+ buffer_put_block() [peers]
+ buffer_put_chunk() -> buffer_put_block()
+ buffer_feed_chunk() -> buffer_put_chunk() -> buffer_put_block() [dumpstats]
+ buffer_feed() -> buffer_put_string() -> buffer_put_block() [dumpstats]
+
+
+The following stream-interface oriented functions are not used :
+
+ buffer_get_char()
+ buffer_write_chunk()
+
+
+Analysers read data from the buffers' input, and may sometimes write data
+there too (or trim data). More suited names could be :
+
+ int buffer_input_peek_at(buf, ofs, ptr, size);
+ int buffer_input_truncate_at(buf, ofs);
+ int buffer_input_peek(buf, ptr, size);
+ int buffer_input_read(buf, ptr, size);
+ int buffer_input_skip(buf, size);
+ int buffer_input_cut(buf, size);
+ int buffer_input_truncate(buf);
+
+
+Functions that are available and need to be renamed :
+ - buffer_skip : buffer_output_skip
+ - buffer_ignore : buffer_input_skip ? => not exactly, more like
+ buffer_output_skip() without affecting sendmax !
+ - buffer_cut_tail : deletes all pending data after sendmax.
+ -> buffer_input_truncate(). Used by si_retnclose() only.
+ - buffer_contig_data : buffer_output_contig_data
+ - buffer_pending : buffer_input_pending_data
+ - buffer_contig_space : buffer_input_contig_space
+
+
+It looks like buf->lr could be removed and be stored in the HTTP message struct
+since it's only used at the HTTP level.
diff --git a/doc/internals/buffer-ops.fig b/doc/internals/buffer-ops.fig
new file mode 100644
index 0000000..5cca1b6
--- /dev/null
+++ b/doc/internals/buffer-ops.fig
@@ -0,0 +1,152 @@
+#FIG 3.2
+Portrait
+Center
+Metric
+A4
+100.00
+Single
+-2
+1200 2
+5 1 0 1 0 7 50 -1 -1 0.000 0 1 1 1 3133.105 2868.088 2385 3465 3150 3825 3915 3420
+ 0 0 1.00 60.00 120.00
+ 0 0 1.00 60.00 120.00
+5 1 0 1 0 7 50 -1 -1 0.000 0 0 1 1 3134.312 2832.717 2340 3420 3150 1845 3960 3375
+ 0 0 1.00 60.00 120.00
+ 0 0 1.00 60.00 120.00
+5 1 1 1 0 7 50 -1 -1 3.000 0 0 0 0 3150.000 2848.393 2115 3510 3150 1620 4185 3510
+5 1 0 1 0 7 50 -1 -1 0.000 0 1 1 1 3133.105 6423.088 2385 7020 3150 7380 3915 6975
+ 0 0 1.00 60.00 120.00
+ 0 0 1.00 60.00 120.00
+5 1 0 1 0 7 50 -1 -1 0.000 0 0 1 1 3134.312 6387.717 2340 6975 3150 5400 3960 6930
+ 0 0 1.00 60.00 120.00
+ 0 0 1.00 60.00 120.00
+5 1 1 1 0 7 50 -1 -1 3.000 0 0 0 0 3150.000 6403.393 2115 7065 3150 5175 4185 7065
+1 3 0 1 0 7 50 -1 -1 0.000 1 0.0000 3150 2835 1126 1126 3150 2835 3195 3960
+1 3 0 1 0 7 51 -1 -1 0.000 1 0.0000 3150 2835 1350 1350 3150 2835 4500 2835
+1 3 0 1 0 7 50 -1 -1 0.000 1 0.0000 3150 6390 1126 1126 3150 6390 3195 7515
+1 3 0 1 0 7 51 -1 -1 0.000 1 0.0000 3150 6390 1350 1350 3150 6390 4500 6390
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 3150 3960 3150 4185
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 4050 3510 4230 3690
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 2250 3510 2070 3690
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 4410 3285 4455 3150
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 4500 2655 4455 2475
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 4185 1980 4050 1845
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 3645 1575 3510 1530
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2295 1800 2160 1890
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 1980 2160 1935 2250
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 1800 2655 1800 2790
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 1800 3105 1845 3240
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2877 1519 2697 1564
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 3150 7515 3150 7740
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 4050 7065 4230 7245
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 2250 7065 2070 7245
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 4410 6840 4455 6705
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 4500 6210 4455 6030
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 4185 5535 4050 5400
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 3645 5130 3510 5085
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2295 5355 2160 5445
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 1980 5715 1935 5805
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 1800 6210 1800 6345
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 1800 6660 1845 6795
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2877 5074 2697 5119
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 4950 3510 4635 3690 4545 3375 4185 3600
+ 0.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 2115 3600 1800 3330 1305 3285 1260 3780
+ 0.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 4635 2205 4545 1890 4185 2115
+ 0.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 4950 7065 4635 7245 4545 6930 4185 7155
+ 0.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 2115 7155 1800 6885 1305 6840 1260 7335
+ 0.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 4635 5760 4545 5445 4185 5670
+ 0.000 1.000 0.000
+4 1 0 50 -1 12 8 0.0000 4 75 75 3150 3690 l\001
+4 1 0 50 -1 12 8 0.0000 4 75 450 3150 2025 size-l\001
+4 1 0 50 -1 12 8 5.4978 4 45 75 1935 3780 w\001
+4 1 0 50 -1 12 8 0.7854 4 45 75 4365 3825 r\001
+4 1 0 50 -1 12 8 0.0000 4 90 300 3150 4365 (lr)\001
+4 1 0 50 -1 14 10 5.7596 4 90 270 2520 3960 OUT\001
+4 1 0 50 -1 12 8 5.7596 4 75 525 2430 4140 sendmax\001
+4 1 0 50 -1 14 10 0.5236 4 90 180 3690 4005 IN\001
+4 1 0 50 -1 12 8 0.5236 4 75 675 3870 4185 l-sendmax\001
+4 0 0 50 -1 12 8 0.0000 4 90 750 4545 2340 free space\001
+4 0 0 50 -1 12 8 0.0000 4 90 975 4950 3555 [eg: recv()]\001
+4 1 0 50 -1 12 8 0.0000 4 90 900 1260 4095 [eg: send()]\001
+4 1 0 50 -1 16 12 0.0000 4 165 2370 3150 855 Principle of the circular buffer\001
+4 1 0 50 -1 12 8 0.0000 4 90 600 1260 3960 buffer_*\001
+4 0 0 50 -1 12 8 0.0000 4 90 600 4950 3420 buffer_*\001
+4 1 0 50 -1 16 12 0.0000 4 165 1605 3150 1125 Current (since v1.3)\001
+4 0 0 50 -1 12 8 0.0000 4 90 1050 4950 6975 buffer_input_*\001
+4 1 0 50 -1 14 10 5.7596 4 90 270 2520 7515 OUT\001
+4 1 0 50 -1 14 10 0.5236 4 90 180 3690 7560 IN\001
+4 1 0 50 -1 12 8 0.0000 4 90 1125 1260 7515 buffer_output_*\001
+4 0 0 50 -1 12 8 0.0000 4 90 750 4545 5895 free space\001
+4 0 0 50 -1 12 8 0.0000 4 90 975 4950 7110 [eg: recv()]\001
+4 1 0 50 -1 12 8 0.0000 4 90 900 1260 7650 [eg: send()]\001
+4 0 0 50 -1 0 10 0.0000 4 135 1860 6075 1755 Some http_msg fields point to\001
+4 0 0 50 -1 0 10 0.0000 4 120 2175 6075 1950 absolute locations within the buffer,\001
+4 0 0 50 -1 0 10 0.0000 4 135 2040 6075 2145 making realignments quite tricky.\001
+4 0 0 50 -1 0 10 0.0000 4 135 1890 6075 5400 http_msg owns a pointer to the\001
+4 0 0 50 -1 0 10 0.0000 4 135 2055 6075 5595 struct_buffer and only uses offsets\001
+4 0 0 50 -1 0 10 0.0000 4 135 1095 6075 5790 relative to buf->p.\001
+4 1 0 50 -1 12 8 0.0000 4 75 600 3150 5760 size-i-o\001
+4 1 0 50 -1 12 8 0.0000 4 75 225 3150 7200 o+i\001
+4 1 0 50 -1 12 8 5.7596 4 45 75 2430 7695 o\001
+4 1 0 50 -1 12 8 0.5236 4 75 75 3870 7740 i\001
+4 1 0 50 -1 16 12 0.0000 4 180 1965 3150 4905 New design (1.5-dev9+)\001
+4 1 0 50 -1 12 8 0.0000 4 60 75 3150 7920 p\001