blob: d4e26dbd998d8316b6532d4f1bb4f25dc7d20966 [file] [log] [blame]
Willy Tarreau9d752e92018-07-13 18:49:41 +020012018-07-13 - HAProxy Internal Buffer API
2
3
41. Background
5
6HAProxy uses a "struct buffer" internally to store data received from external
7agents, as well as data to be sent to external agents. These buffers are also
8used during data transformation such as compression, header insertion or
9defragmentation, and are used to carry intermediary representations between the
10various internal layers. They support wrapping at the end, and they carry their
11own size information so that in theory it would be possible to use different
Thayne McCombscdbcca92021-01-07 21:24:41 -070012buffer sizes in parallel even though this is not currently implemented.
Willy Tarreau9d752e92018-07-13 18:49:41 +020013
14The format of this structure has evolved over time, to reach a point where it
15is convenient and versatile enough to have permitted to make several internal
16types converge into a single one (specifically the struct chunk disappeared).
17
18
192. Representation as of 1.9-dev1
20
21The current buffer representation consists in a linear storage area of known
22size, with a head position indicating the oldest data, and a total data count
23expressed in bytes. The head position, data count and size are expressed as
24integers and are positive or null. By convention, the head position is strictly
25smaller than the buffer size and the data count is smaller than or equal to the
26size, so that wrapping can be resolved with a single subtract. A buffer not
27respecting these rules is said to be degenerate. Unless specified otherwise,
28the various API functions will adopt an undefined behaviour when passed such a
29degenerate buffer.
30
31 Buffer declaration :
32
33 struct buffer {
34 size_t size; // size of the storage area (wrapping point)
35 char *area; // start of the storage area
36 size_t data; // contents length after head
37 size_t head; // start offset of remaining data relative to area
38 };
39
40
41 Linear buffer representation :
42
43 area
44 |
45 V<--------------------------------------------------------->| size
46 +-----------+---------------------------------+-------------+
47 | |/////////////////////////////////| |
48 +-----------+---------------------------------+-------------+
49 |<--------->|<------------------------------->|
50 head data ^
51 |
52 tail
53
54
55 Wrapping buffer representation :
56
57 area
58 |
59 V<--------------------------------------------------------->| size
60 +---------------+------------------------+------------------+
61 |///////////////| |//////////////////|
62 +---------------+------------------------+------------------+
63 |<-------------------------------------->| head
64 |-------------->| ...data data...|<-----------------|
65 ^
66 |
67 tail
68
69
703. Terminology
71
72Manipulating a buffer just based on a head and a wrapping data count is not
73very convenient, so we define a certain number of terms for important elements
74characterizing a buffer :
75
76 - origin : pointer to relative position 0 in the storage area. Undefined
77 when the buffer is not allocated.
78
79 - size : the allocated size of the storage area starting at the origin,
80 expressed in bytes. A buffer whose size is zero is said not to
81 be allocated, and its origin in this case is undefined.
82
83 - data : the amount of data the buffer contains, in bytes. It is always
84 lower than or equal to the buffer's size, hence it is always 0
85 for an unallocated buffer.
86
87 - emptiness : a buffer is said to be empty when it contains no data, hence
88 data == 0. It is possible for such buffers not to be allocated
89 and to have size == 0 as well.
90
91 - room : the available space in the buffer. This is its size minus data.
92
93 - head : position relative to origin where the oldest data byte is found
94 (it typically is what send() uses to pick outgoing data). The
95 head is strictly smaller than the size.
96
97 - tail : position relative to origin where the first spare byte is found
98 (it typically is what recv() uses to store incoming data). It
99 is always equal to the buffer's data added to its head modulo
100 the buffer's size.
101
102 - wrapping : the byte following the last one of the storage area loops back
103 to position 0. This is called wrapping. The wrapping point is
104 the first position relative to origin which doesn't belong to
105 the storage area. There is no wrapping when a buffer is not
106 allocated. Wrapping requires special care and means that the
107 regular string manipulation functions are not usable on most
108 buffers, unless it is known that no wrapping happens. Free
109 space may wrap as well if the buffer only contains data in the
110 middle.
111
112 - alignment : a buffer is said to be aligned if its data do not wrap. That
113 is, its head is strictly before the tail, or the buffer is
114 empty and the head is null. Aligning a buffer may be required
115 to use regular string manipulation functions which have no
116 support for wrapping.
117
118
119A buffer may be in three different states :
120 - unallocated : size == 0, area == 0 (b_is_null() is true)
121 - waiting : size == 0, area != 0
122 - allocated : size > 0, area > 0
123
124It is not permitted to have area == 0 with a non-null size. In addition, the
125waiting state may also be used to indicate a read-only buffer which does not
126wrap and which must not be freed (e.g. for use with error messages).
127
128The basic API only covers allocated buffers. Switching to/from the other states
129is covered by the management API since it requires specific allocation and free
130calls.
131
132
1334. Using buffers
134
135Buffers are defined in a few files :
136 - include/common/buf.h : structure definition, and manipulation functions
137 - include/common/buffer.h : resource management (alloc/free/wait lists)
138 - include/common/istbuf.h : advanced string manipulation
139
140
1414.1. Basic API
142
143The basic API is made of the functions which abstract accesses to the buffers
144and which help calculating their state, free space or used space.
145
146====================+==================+=======================================
147Function | Arguments/Return | Description
148--------------------+------------------+---------------------------------------
149b_is_null() | const buffer *buf| returns true if (and only if) the
150 | ret: int | buffer is not yet allocated and thus
151 | | points to a NULL area
152--------------------+------------------+---------------------------------------
153b_orig() | const buffer *buf| returns the pointer to the origin of
154 | ret: char * | the storage, which is the location of
155 | | byte at offset zero. This is mostly
156 | | used by functions which handle the
157 | | wrapping by themselves
158--------------------+------------------+---------------------------------------
159b_size() | const buffer *buf| returns the size of the buffer
160 | ret: size_t |
161--------------------+------------------+---------------------------------------
162b_wrap() | const buffer *buf| returns the pointer to the wrapping
163 | ret: char * | position of the buffer area, which is
164 | | by definition the first byte not part
165 | | of the buffer
166--------------------+------------------+---------------------------------------
167b_data() | const buffer *buf| returns the number of bytes present in
168 | ret: size_t | the buffer
169--------------------+------------------+---------------------------------------
170b_room() | const buffer *buf| returns the amount of room left in the
171 | ret: size_t | buffer
172--------------------+------------------+---------------------------------------
173b_full() | const buffer *buf| returns true if the buffer is full
174 | ret: int |
175--------------------+------------------+---------------------------------------
176__b_stop() | const buffer *buf| returns a pointer to the byte
177 | ret: char * | following the end of the buffer, which
178 | | may be out of the buffer if the buffer
179 | | ends on the last byte of the area. It
180 | | is the caller's responsibility to
181 | | either know that the buffer does not
182 | | wrap or to check that the result does
183 | | not wrap
184--------------------+------------------+---------------------------------------
185__b_stop_ofs() | const buffer *buf| returns an origin-relative offset
186 | ret: size_t | pointing to the byte following the end
187 | | of the buffer, which may be out of the
188 | | buffer if the buffer ends on the last
189 | | byte of the area. It's the caller's
190 | | responsibility to either know that the
191 | | buffer does not wrap or to check that
192 | | the result does not wrap
193--------------------+------------------+---------------------------------------
194b_stop() | const buffer *buf| returns the pointer to the byte
195 | ret: char * | following the end of the buffer, which
196 | | may be out of the buffer if the buffer
197 | | ends on the last byte of the area
198--------------------+------------------+---------------------------------------
199b_stop_ofs() | const buffer *buf| returns an origin-relative offset
200 | ret: size_t | pointing to the byte following the end
201 | | of the buffer, which may be out of the
202 | | buffer if the buffer ends on the last
203 | | byte of the area
204--------------------+------------------+---------------------------------------
205__b_peek() | const buffer *buf| returns a pointer to the data at
206 | size_t ofs | position <ofs> relative to the head of
207 | ret: char * | the buffer. Will typically point to
208 | | input data if called with the amount
209 | | of output data. It's the caller's
210 | | responsibility to either know that the
211 | | buffer does not wrap or to check that
212 | | the result does not wrap
213--------------------+------------------+---------------------------------------
214__b_peek_ofs() | const buffer *buf| returns an origin-relative offset
215 | size_t ofs | pointing to the data at position <ofs>
216 | ret: size_t | relative to the head of the
217 | | buffer. Will typically point to input
218 | | data if called with the amount of
219 | | output data. It's the caller's
220 | | responsibility to either know that the
221 | | buffer does not wrap or to check that
222 | | the result does not wrap
223--------------------+------------------+---------------------------------------
224b_peek() | const buffer *buf| returns a pointer to the data at
225 | size_t ofs | position <ofs> relative to the head of
226 | ret: char * | the buffer. Will typically point to
227 | | input data if called with the amount
228 | | of output data. If applying <ofs> to
229 | | the buffers' head results in a
230 | | position between <size> and 2*>size>-1
231 | | included, a wrapping compensation is
232 | | applied to the result
233--------------------+------------------+---------------------------------------
234b_peek_ofs() | const buffer *buf| returns an origin-relative offset
235 | size_t ofs | pointing to the data at position <ofs>
236 | ret: size_t | relative to the head of the
237 | | buffer. Will typically point to input
238 | | data if called with the amount of
239 | | output data. If applying <ofs> to the
240 | | buffers' head results in a position
241 | | between <size> and 2*>size>-1
242 | | included, a wrapping compensation is
243 | | applied to the result
244--------------------+------------------+---------------------------------------
245__b_head() | const buffer *buf| returns the pointer to the buffer's
246 | ret: char * | head, which is the location of the
247 | | next byte to be dequeued. The result
248 | | is undefined for unallocated buffers
249--------------------+------------------+---------------------------------------
250__b_head_ofs() | const buffer *buf| returns an origin-relative offset
251 | ret: size_t | pointing to the buffer's head, which
252 | | is the location of the next byte to be
253 | | dequeued. The result is undefined for
254 | | unallocated buffers
255--------------------+------------------+---------------------------------------
256b_head() | const buffer *buf| returns the pointer to the buffer's
257 | ret: char * | head, which is the location of the
258 | | next byte to be dequeued. The result
259 | | is undefined for unallocated
260 | | buffers. If applying <ofs> to the
261 | | buffers' head results in a position
262 | | between <size> and 2*>size>-1
263 | | included, a wrapping compensation is
264 | | applied to the result
265--------------------+------------------+---------------------------------------
266b_head_ofs() | const buffer *buf| returns an origin-relative offset
267 | ret: size_t | pointing to the buffer's head, which
268 | | is the location of the next byte to be
269 | | dequeued. The result is undefined for
270 | | unallocated buffers. If applying
271 | | <ofs> to the buffers' head results in
272 | | a position between <size> and
273 | | 2*>size>-1 included, a wrapping
274 | | compensation is applied to the result
275--------------------+------------------+---------------------------------------
276__b_tail() | const buffer *buf| returns the pointer to the tail of the
277 | ret: char * | buffer, which is the location of the
278 | | first byte where it is possible to
279 | | enqueue new data. The result is
280 | | undefined for unallocated buffers
281--------------------+------------------+---------------------------------------
282__b_tail_ofs() | const buffer *buf| returns an origin-relative offset
283 | ret: size_t | pointing to the tail of the buffer,
284 | | which is the location of the first
285 | | byte where it is possible to enqueue
286 | | new data. The result is undefined for
287 | | unallocated buffers
288--------------------+------------------+---------------------------------------
289b_tail() | const buffer *buf| returns the pointer to the tail of the
290 | ret: char * | buffer, which is the location of the
291 | | first byte where it is possible to
292 | | enqueue new data. The result is
293 | | undefined for unallocated buffers
294--------------------+------------------+---------------------------------------
295b_tail_ofs() | const buffer *buf| returns an origin-relative offset
296 | ret: size_t | pointing to the tail of the buffer,
297 | | which is the location of the first
298 | | byte where it is possible to enqueue
299 | | new data. The result is undefined for
300 | | unallocated buffers
301--------------------+------------------+---------------------------------------
302b_next() | const buffer *buf| for an absolute pointer <p> pointing
303 | const char *p | to a valid location within buffer <b>,
304 | ret: char * | returns the absolute pointer to the
305 | | next byte, which usually is at (p + 1)
306 | | unless p reaches the wrapping point
307 | | and wrapping is needed
308--------------------+------------------+---------------------------------------
309b_next_ofs() | const buffer *buf| for an origin-relative offset <o>
310 | size_t o | pointing to a valid location within
311 | ret: size_t | buffer <b>, returns either the
312 | | relative offset pointing to the next
313 | | byte, which usually is at (o + 1)
314 | | unless o reaches the wrapping point
315 | | and wrapping is needed
316--------------------+------------------+---------------------------------------
317b_dist() | const buffer *buf| returns the distance between two
318 | const char *from | pointers, taking into account the
319 | const char *to | ability to wrap around the buffer's
320 | ret: size_t | end. The operation is not defined if
321 | | either of the pointers does not belong
322 | | to the buffer or if their distance is
323 | | greater than the buffer's size
324--------------------+------------------+---------------------------------------
325b_almost_full() | const buffer *buf| returns 1 if the buffer uses at least
326 | ret: int | 3/4 of its capacity, otherwise
327 | | zero. Buffers of size zero are
328 | | considered full
329--------------------+------------------+---------------------------------------
330b_space_wraps() | const buffer *buf| returns non-zero only if the buffer's
331 | ret: int | free space wraps, which means that the
332 | | buffer contains data that are not
333 | | touching at least one edge
334--------------------+------------------+---------------------------------------
335b_contig_data() | const buffer *buf| returns the amount of data that can
336 | size_t start | contiguously be read at once starting
337 | ret: size_t | from a relative offset <start> (which
338 | | allows to easily pre-compute blocks
339 | | for memcpy). The start point will
340 | | typically contain the amount of past
341 | | data already returned by a previous
342 | | call to this function
343--------------------+------------------+---------------------------------------
344b_contig_space() | const buffer *buf| returns the amount of bytes that can
345 | ret: size_t | be appended to the buffer at once
346--------------------+------------------+---------------------------------------
347b_getblk() | const buffer *buf| gets one full block of data at once
348 | char *blk | from a buffer, starting from offset
349 | size_t len | <offset> after the buffer's head, and
350 | size_t offset | limited to no more than <len> bytes.
351 | ret: size_t | The caller is responsible for ensuring
352 | | that neither <offset> nor <offset> +
353 | | <len> exceed the total number of bytes
354 | | available in the buffer. Return zero
355 | | if not enough data was available, in
356 | | which case blk is left undefined, or
357 | | the number of bytes read which is
358 | | equal to the requested size
359--------------------+------------------+---------------------------------------
360b_getblk_nc() | const buffer *buf| gets one or two blocks of data at once
361 | const char **blk1| from a buffer, starting from offset
362 | size_t *len1 | <ofs> after the beginning of its
363 | const char **blk2| output, and limited to no more than
364 | size_t *len2 | <max> bytes. The caller is responsible
365 | size_t ofs | for ensuring that neither <ofs> nor
366 | size_t max | <ofs>+<max> exceed the total number of
367 | ret: int | bytes available in the buffer. Returns
368 | | 0 if not enough data were available,
369 | | or the number of blocks filled (1 or
370 | | 2). <blk1> is always filled before
371 | | <blk2>. The unused blocks are left
372 | | undefined, and the buffer is left
373 | | unaffected. Unused buffers are left in
374 | | an undefined state
375--------------------+------------------+---------------------------------------
376b_reset() | buffer *buf | resets a buffer. The size is not
377 | ret: void | touched. In practice it resets the
378 | | head and the data length
379--------------------+------------------+---------------------------------------
380b_sub() | buffer *buf | decreases the buffer length by <count>
381 | size_t count | without touching the head position
382 | ret: void | (only the tail moves). this may mostly
383 | | be used to trim pending data before
384 | | reusing a buffer. The caller is
385 | | responsible for not removing more than
386 | | the available data
387--------------------+------------------+---------------------------------------
388b_add() | buffer *buf | increase the buffer length by <count>
389 | size_t count | without touching the head position
390 | ret: void | (only the tail moves). This is used
391 | | when adding data at the tail of a
392 | | buffer. The caller is responsible for
393 | | not adding more than the available
394 | | room
395--------------------+------------------+---------------------------------------
396b_set_data() | buffer *buf | sets the buffer's length, by adjusting
397 | size_t len | the buffer's tail only. The caller is
398 | ret: void | responsible for passing a valid length
399--------------------+------------------+---------------------------------------
400b_del() | buffer *buf | deletes <del> bytes at the head of
401 | size_t del | buffer <b> and updates the head. The
402 | ret: void | caller is responsible for not removing
403 | | more than the available data. This is
404 | | used after sending data from the
405 | | buffer
406--------------------+------------------+---------------------------------------
407b_realign_if_empty()| buffer *buf | realigns a buffer if it's empty, does
408 | ret: void | nothing otherwise. This is mostly used
409 | | after b_del() to make an empty
410 | | buffer's free space contiguous
411--------------------+------------------+---------------------------------------
412b_slow_realign() | buffer *buf | realigns a possibly wrapping buffer so
413 | size_t output | that the part remaining to be parsed
414 | ret: void | is contiguous and starts at the
415 | | beginning of the buffer and the
416 | | already parsed output part ends at the
417 | | end of the buffer. This provides the
418 | | best conditions since it allows the
419 | | largest inputs to be processed at once
420 | | and ensures that once the output data
421 | | leaves, the whole buffer is available
422 | | at once. The number of output bytes
423 | | supposedly present at the beginning of
424 | | the buffer and which need to be moved
425 | | to the end must be passed in <output>.
426 | | It will effectively make this offset
427 | | the new wrapping point. A temporary
428 | | swap area at least as large as b->size
429 | | must be provided in <swap>. It's up
430 | | to the caller to ensure <output> is no
431 | | larger than the difference between the
432 | | whole buffer's length and its input
433--------------------+------------------+---------------------------------------
434b_putchar() | buffer *buf | tries to append char <c> at the end of
435 | char c | buffer <b>. Supports wrapping. New
436 | ret: void | data are silently discarded if the
437 | | buffer is already full
438--------------------+------------------+---------------------------------------
439b_putblk() | buffer *buf | tries to append block <blk> at the end
440 | const char *blk | of buffer <b>. Supports wrapping. Data
441 | size_t len | are truncated if the buffer is too
442 | ret: size_t | short or if not enough space is
443 | | available. It returns the number of
444 | | bytes really copied
445--------------------+------------------+---------------------------------------
Willy Tarreauf48919a2018-12-22 19:19:50 +0100446b_move() | buffer *buf | moves block (src,len) left or right
447 | size_t src | by <shift> bytes, supporting wrapping
448 | size_t len | and overlapping.
449 | size_t shift |
450--------------------+------------------+---------------------------------------
Willy Tarreau9d752e92018-07-13 18:49:41 +0200451b_rep_blk() | buffer *buf | writes the block <blk> at position
452 | char *pos | <pos> which must be in buffer <b>, and
453 | char *end | moves the part between <end> and the
454 | const char *blk | buffer's tail just after the end of
455 | size_t len | the copy of <blk>. This effectively
456 | ret: int | replaces the part located between
457 | | <pos> and <end> with a copy of <blk>
458 | | of length <len>. The buffer's length
459 | | is automatically updated. This is used
460 | | to replace a block with another one
461 | | inside a buffer. The shift value
462 | | (positive or negative) is returned. If
463 | | there's no space left, the move is not
464 | | done. If <len> is null, the <blk>
465 | | pointer is allowed to be null, in
466 | | order to erase a block
Willy Tarreauf1488882018-07-20 16:24:39 +0200467--------------------+------------------+---------------------------------------
468b_xfer() | buffer *src | transfers at most <count> bytes from
469 | buffer *dst | buffer <src> to buffer <dst> and
470 | size_t cout | returns the number of bytes copied.
471 | ret: size_t | The bytes are removed from <src> and
472 | | added to <dst>. The caller guarantees
473 | | that <count> is <= b_room(dst)
Willy Tarreau9d752e92018-07-13 18:49:41 +0200474====================+==================+=======================================
475
476
4774.2. String API
478
479The string API aims at providing both convenient and efficient ways to read and
480write to/from buffers using indirect strings (ist). These strings and some
481associated functions are defined in ist.h.
482
483====================+==================+=======================================
484Function | Arguments/Return | Description
485--------------------+------------------+---------------------------------------
486b_isteq() | const buffer *b | b_isteq() : returns > 0 if the first
487 | size_t o | <n> characters of buffer <b> starting
488 | size_t n | at offset <o> relative to the buffer's
489 | const ist ist | head match <ist>. (empty strings do
490 | ret: int | match). It is designed to be used with
491 | | reasonably small strings (it matches a
492 | | single byte per loop iteration). It is
493 | | expected to be used with an offset to
494 | | skip old data. Return value number of
495 | | matching bytes if >0, not enough bytes
496 | | or empty string if 0, or non-matching
497 | | byte found if <0.
498--------------------+------------------+---------------------------------------
499b_isteat | struct buffer *b | b_isteat() : "eats" string <ist> from
500 | const ist ist | the head of buffer <b>. Wrapping data
501 | ret: ssize_t | is explicitly supported. It matches a
502 | | single byte per iteration so strings
503 | | should remain reasonably small.
504 | | Returns the number of bytes matched
505 | | and eaten if >0, not enough bytes or
506 | | matched empty string if 0, or non
507 | | matching byte found if <0.
508--------------------+------------------+---------------------------------------
509b_istput | struct buffer *b | b_istput() : injects string <ist> at
510 | const ist ist | the tail of output buffer <b> provided
511 | ret: ssize_t | that it fits. Wrapping is supported.
512 | | It's designed for small strings as it
513 | | only writes a single byte per
514 | | iteration. Returns the number of
515 | | characters copied (ist.len), 0 if it
516 | | temporarily does not fit, or -1 if it
517 | | will never fit. It will only modify
518 | | the buffer upon success. In all cases,
519 | | the contents are copied prior to
520 | | reporting an error, so that the
521 | | destination at least contains a valid
522 | | but truncated string.
523--------------------+------------------+---------------------------------------
524b_putist | struct buffer *b | b_putist() : tries to copy as much as
525 | const ist ist | possible of string <ist> into buffer
526 | ret: size_t | <b> and returns the number of bytes
527 | | copied (truncation is possible). It
528 | | uses b_putblk() and is suitable for
529 | | large blocks.
530====================+==================+=======================================
531
532
5334.3. Management API
534
535The management API makes a distinction between an empty buffer, which by
536definition is not allocated but is ready to be allocated at any time, and a
537buffer which failed an allocation and is waiting for an available area to be
538offered. The functions allow to register on a list to be notified about buffer
539availability, to notify others of a number of buffers just released, and to be
540and to be notified of buffer availability. All allocations are made through the
541standard buffer pools.
542
543====================+==================+=======================================
544Function | Arguments/Return | Description
545--------------------+------------------+---------------------------------------
546buffer_almost_full | const buffer *buf| returns true if the buffer is not null
547 | ret: int | and at least 3/4 of the buffer's space
548 | | are used. A waiting buffer will match.
549--------------------+------------------+---------------------------------------
550b_alloc | buffer *buf | allocates a buffer and assigns it to
551 | ret: buffer * | *buf. If no memory is available, (1)
552 | | is assigned instead with a zero size.
553 | | No control is made to check if *buf
554 | | already pointed to another buffer. The
555 | | allocated buffer is returned, or NULL
556 | | in case no memory is available
557--------------------+------------------+---------------------------------------
558b_alloc_fast | buffer *buf | allocates a buffer and assigns it to
559 | ret: buffer * | *buf. If no memory is available, (1)
560 | | is assigned instead with a zero size.
561 | | No control is made to check if *buf
562 | | already pointed to another buffer. The
563 | | allocated buffer is returned, or NULL
564 | | in case no memory is available. The
565 | | difference with b_alloc() is that this
566 | | function only picks from the pool and
567 | | never calls malloc(), so it can fail
568 | | even if some memory is available
569--------------------+------------------+---------------------------------------
Willy Tarreaue0d0b402019-08-08 08:06:27 +0200570__b_free | buffer *buf | releases <buf> which must be allocated
Willy Tarreau3b091f82019-08-08 07:53:20 +0200571 | ret: void | and marks it empty
Willy Tarreau9d752e92018-07-13 18:49:41 +0200572--------------------+------------------+---------------------------------------
573b_free | buffer *buf | releases <buf> only if it is allocated
574 | ret: void | and marks it empty
575--------------------+------------------+---------------------------------------
576b_alloc_margin | buffer *buf | ensures that <buf> is allocated. If an
577 | int margin | allocation is needed, it ensures that
578 | ret: buffer * | there are still at least <margin>
579 | | buffers available in the pool after
580 | | this allocation so that we don't leave
581 | | the pool in a condition where a
582 | | session or a response buffer could not
583 | | be allocated anymore, resulting in a
584 | | deadlock. This means that we sometimes
585 | | need to try to allocate extra entries
586 | | even if only one buffer is needed
587--------------------+------------------+---------------------------------------
588offer_buffers() | void *from | offer a buffer currently belonging to
589 | uint threshold | target <from> to whoever needs
590 | ret: void | one. Any pointer is valid for <from>,
591 | | including NULL. Its purpose is to
592 | | avoid passing a buffer to oneself in
593 | | case of failed allocations (e.g. need
594 | | two buffers, get one, fail, release it
595 | | and wake up self again). In case of
596 | | normal buffer release where it is
597 | | expected that the caller is not
598 | | waiting for a buffer, NULL is fine
599====================+==================+=======================================
600
601
6025. Porting code from older versions
603
604The previous buffer API introduced in 1.5-dev9 (May 2012) used to look like the
605following (with the struct renamed to old_buffer here to avoid confusion during
606quick lookups at the doc). It's worth noting that the "data" field used to be
607part of the struct but with a different type and meaning. It's important to be
608careful about potential code making use of &b->data as it will silently compile
609but fail.
610
611 Previous buffer declaration :
612
613 struct old_buffer {
614 char *p; /* buffer's start pointer, separates in and out data */
615 unsigned int size; /* buffer size in bytes */
616 unsigned int i; /* number of input bytes pending for analysis in the buffer */
617 unsigned int o; /* number of out bytes the sender can consume from this buffer */
618 char data[0]; /* <size> bytes */
619 };
620
621 Previous linear buffer representation :
622
623 data p
624 | |
625 V V
626 +-----------+--------------------+------------+-------------+
627 | |////////////////////|////////////| |
628 +-----------+--------------------+------------+-------------+
629 <---------------------------------------------------------> size
630 <------------------> <---------->
631 o i
632
Ilya Shipitsin2075ca82020-03-06 23:22:22 +0500633There is this correspondence between old and new fields (some will involve a
Willy Tarreau9d752e92018-07-13 18:49:41 +0200634knowledge of a channel when the output byte count is required) :
635
636 Old | New
637 --------+----------------------------------------------------
638 p | data + head + co_data(channel) // ci_head(channel)
639 size | size
640 i | data - co_data(channel) // ci_data(channel)
641 o | co_data(channel) // channel->output
642 data | area
643 --------+-----------------------------------------------------
644
645Then some common expressions can be mapped like this :
646
647 Old | New
648 -----------------------+---------------------------------------
649 b->data | b_orig(b)
650 &b->data | b_orig(b)
651 bi_ptr(b) | ci_head(channel)
652 bi_end(b) | b_tail(b)
653 bo_ptr(b) | b_head(b)
654 bo_end(b) | co_tail(channel)
655 bi_putblk(b,s,l) | b_putblk(b,s,l)
656 bo_getblk(b,s,l,o) | b_getblk(b,s,l,o)
657 bo_getblk_nc(b,s,l,o) | b_getblk_nc(b,s,l,o,0,co_data(channel))
658 b->i + b->o | b_data(b)
659 b->data + b->size | b_wrap(b)
660 b->i += len | b_add(b, len)
661 b->i -= len | b_sub(b, len)
662 b->i = len | b_set_data(b, co_data(channel) + len)
663 b->o += len | b_add(b, len); channel->output += len
664 b->o -= len | b_del(b, len); channel->output -= len
665 -----------------------+---------------------------------------
666
667The buffer modification functions are less straightforward and depend a lot on
668the context where they are used. It is strongly advised to figure in the list
669of functions above what is available based on what is attempted to be done in
670the existing code.
671
672Note that it is very likely that any out-of-tree code relying on buffers will
673not use both ->i and ->o but instead will use exclusively ->i on the side
674producing data and use exclusively ->o on the side consuming data (such as in a
675mux or in an applet). In both cases, it should be assumed that the other side
676is always zero and that either ->i or ->o is replaced with ->data, making the
677remaining code much simpler (no more code duplication based on the data
678direction).