tree 8f05f1ed1cfc4d4afd68b6a0452e0ac55f45e929
parent 0d6c75d7495091a615603152993378a2274a76b5
author Willy Tarreau <w@1wt.eu> 1558719738 +0200
committer Willy Tarreau <w@1wt.eu> 1558809113 +0200
encoding latin1

MEDIUM: mux-h2: avoid doing expensive buffer realigns when not absolutely needed

Transferring large objects over H2 sometimes shows unexplained performance
variations. A long analysis resulted in the following discovery. Often the
mux buffer looks like this :

    [ empty_head |    data     | empty_tail ]

Typical numbers are (very common) :
  - empty_head = 31
  - empty_tail = 16  (total free=47)
  - data = 16337
  - size = 16384
  - data to copy: 43

The reason for these holes are the blocking factors that are not always
the same in and out (due to keeping 9 bytes for the frame size, or the
56 bytes corresponding to the HTX header). This can easily happen 10000
times a second if the network bandwidth permits it!

In this case, while copying a DATA frame we find that the buffer has its
free space wrapped so we decide to realign it to optimize the copy. It's
possible that this practice stems from the code used to emit headers,
which do not support fragmentation and which had no other option left.
But it comes with two problems :
  - we don't check if the data fits, which results in a memcpy for nothing
  - we can move huge amounts of data to just copy a small block.

This patch addresses this two ways :
  - first, by not forcing a data realignment if what we have to copy does
    not fit, as this is totally pointless ;

  - second, by refusing to move too large data blocks. The threshold was
    set to 1 kB, because it may make sense to move 1 kB of data to copy
    a 15 kB one at once, which will leave as a single 16 kB block, but
    it doesn't make sense to mvoe 15 kB to copy just 1 kB. In all cases
    the data would fit and would just be split into two blocks, which is
    not very expensive, hence the low limit to 1 kB

With such changes, realignments are very rare, they show up around once
every 15 seconds at 60 Gbps, and look like this, resulting in a much more
stable bit rate :

  buf=0x7fe6ec0c3510,h=16333,d=35,s=16384 room=16349 in=16337

This patch should be safe for backporting to 1.9 if some performance
issues are reported there.