CLEANUP: doc: remove 21 totally obsolete docs
These were docs for very old design thoughts or internal subsystems
which are now totally irrelevant and even misleading. Those with some
outdated ideas mixed with useful stuff were kept though.
diff --git a/doc/close-options.txt b/doc/close-options.txt
deleted file mode 100644
index 0554bb8..0000000
--- a/doc/close-options.txt
+++ /dev/null
@@ -1,39 +0,0 @@
-2011/04/20 - List of keep-alive / close options with associated behaviours.
-
-PK="http-pretend-keepalive", HC="httpclose", SC="http-server-close",
-
-0 = option not set
-1 = option is set
-* = option doesn't matter
-
-Options can be split between frontend and backend, so some of them might have
-a meaning only when combined by associating a frontend to a backend. Some forms
-are not the normal ones and provide a behaviour compatible with another normal
-form. Those are considered alternate forms and are marked "(alt)".
-
-SC HC PK Behaviour
- 0 0 X tunnel mode
- 0 1 0 passive close, only set headers then tunnel
- 0 1 1 forced close with keep-alive announce (alt)
- 1 0 0 server close
- 1 0 1 server close with keep-alive announce
- 1 1 0 forced close (alt)
- 1 1 1 forced close with keep-alive announce (alt)
-
-At this point this results in 4 distinct effective modes for a request being
-processed :
- - tunnel mode : Connection header is left untouched and body is ignored
- - passive close : Connection header is changed and body is ignored
- - server close : Connection header set, body scanned, client-side keep-alive
- is made possible regardless of server-side capabilities
- - forced close : Connection header set, body scanned, connection closed.
-
-The "close" modes may be combined with a fake keep-alive announce to the server
-in order to workaround buggy servers that disable chunked encoding and content
-length announces when the client does not ask for keep-alive.
-
-Note: "http-pretend-keepalive" alone has no effect. However, if it is set in a
- backend while a frontend is in "http-close" mode, then the combination of
- both will result in a forced close with keep-alive announces for requests
- passing through both.
-
diff --git a/doc/design-thoughts/backends-v0.txt b/doc/design-thoughts/backends-v0.txt
deleted file mode 100644
index d350e22..0000000
--- a/doc/design-thoughts/backends-v0.txt
+++ /dev/null
@@ -1,27 +0,0 @@
-1 type générique "entité", avec les attributs suivants :
-
- - frontend *f
- - l7switch *s
- - backend *b
-
-des types spécifiques sont simplement des entités avec certains
-de ces champs remplis et pas forcément tous :
-
- listen = f [s] b
- frontend = f [s]
- l7switch = s
- backend = [s] b
-
-Ensuite, les traitements sont évalués dans l'ordre :
- - listen -> s'il a des règles de l7, on les évalue, et potentiellement on branche vers d'autres listen, l7 ou back, ou on travaille avec le back local.
- - frontend -> s'il a des règles de l7, on les évalue, et potentiellement on branche vers d'autres listen, l7 ou back
- - l7switch -> on évalue ses règles, potentiellement on branche vers d'autres listen, l7 ou backends
- - backend -> s'il a des règles l7, on les évalue (quitte à changer encore de backend) puis on traite.
-
-Les requêtes sont traitées dans l'ordre des chaînages f->s*->b, et les réponses doivent être
-traitées dans l'ordre inverse b->s*->f. Penser aux réécritures de champs Host à l'aller et
-Location en retour.
-
-D'autre part, prévoir des "profils" plutôt que des blocs de nouveaux paramètres par défaut.
-Ca permettra d'avoir plein de jeux de paramètres par défaut à utiliser dans chacun de ces
-types.
diff --git a/doc/design-thoughts/backends.txt b/doc/design-thoughts/backends.txt
deleted file mode 100644
index d2474ce..0000000
--- a/doc/design-thoughts/backends.txt
+++ /dev/null
@@ -1,125 +0,0 @@
-There has been a lot of confusion during the development because of the
-backends and frontends.
-
-What we want :
-
-- being able to still use a listener as it has always been working
-
-- being able to write a rule stating that we will *change* the backend when we
- match some pattern. Only one jump is allowed.
-
-- being able to write a "use_filters_from XXX" line stating that we will ignore
- any filter in the current listener, and that those from XXX will be borrowed
- instead. A warning would be welcome for options which will silently get
- masked. This is used to factor configuration.
-
-- being able to write a "use_backend_from XXX" line stating that we will ignore
- any server and timeout config in the current listener, and that those from
- XXX will be borrowed instead. A warning would be welcome for options which
- will silently get masked. This is used to factor configuration.
-
-
-
-Example :
----------
-
- | # frontend HTTP/80
- | listen fe_http 1.1.1.1:80
- | use_filters_from default_http
- | use_backend_from appli1
- |
- | # frontend HTTPS/443
- | listen fe_https 1.1.1.1:443
- | use_filters_from default_https
- | use_backend_from appli1
- |
- | # frontend HTTP/8080
- | listen fe_http-dev 1.1.1.1:8080
- | reqadd "X-proto: http"
- | reqisetbe "^Host: www1" appli1
- | reqisetbe "^Host: www2" appli2
- | reqisetbe "^Host: www3" appli-dev
- | use_backend_from appli1
- |
- |
- | # filters default_http
- | listen default_http
- | reqadd "X-proto: http"
- | reqisetbe "^Host: www1" appli1
- | reqisetbe "^Host: www2" appli2
- |
- | # filters default_https
- | listen default_https
- | reqadd "X-proto: https"
- | reqisetbe "^Host: www1" appli1
- | reqisetbe "^Host: www2" appli2
- |
- |
- | # backend appli1
- | listen appli1
- | reqidel "^X-appli1:.*"
- | reqadd "Via: appli1"
- | balance roundrobin
- | cookie app1
- | server srv1
- | server srv2
- |
- | # backend appli2
- | listen appli2
- | reqidel "^X-appli2:.*"
- | reqadd "Via: appli2"
- | balance roundrobin
- | cookie app2
- | server srv1
- | server srv2
- |
- | # backend appli-dev
- | listen appli-dev
- | reqadd "Via: appli-dev"
- | use_backend_from appli2
- |
- |
-
-
-Now we clearly see multiple things :
-------------------------------------
-
- - a frontend can EITHER have filters OR reference a use_filter
-
- - a backend can EITHER have servers OR reference a use_backend
-
- - we want the evaluation to cross TWO levels per request. When a request is
- being processed, it keeps track of its "frontend" side (where it came
- from), and of its "backend" side (where the server-side parameters have
- been found).
-
- - the use_{filters|backend} have nothing to do with how the request is
- decomposed.
-
-
-Conclusion :
-------------
-
- - a proxy is always its own frontend. It also has 2 parameters :
- - "fi_prm" : pointer to the proxy holding the filters (itself by default)
- - "be_prm" : pointer to the proxy holding the servers (itself by default)
-
- - a request has a frontend (fe) and a backend (be). By default, the backend
- is initialized to the frontend. Everything related to the client side is
- accessed through ->fe. Everything related to the server side is accessed
- through ->be.
-
- - request filters are first called from ->fe then ->be. Since only the
- filters can change ->be, it is possible to iterate the filters on ->be
- only and stop when ->be does not change anymore.
-
- - response filters are first called from ->be then ->fe IF (fe != be).
-
-
-When we parse the configuration, we immediately configure ->fi and ->be for
-all proxies.
-
-Upon session creation, s->fe and s->be are initialized to the proxy. Filters
-are executed via s->fe->fi_prm and s->be->fi_prm. Servers are found in
-s->be->be_prm.
-
diff --git a/doc/design-thoughts/be-fe-changes.txt b/doc/design-thoughts/be-fe-changes.txt
deleted file mode 100644
index f242f8a..0000000
--- a/doc/design-thoughts/be-fe-changes.txt
+++ /dev/null
@@ -1,74 +0,0 @@
-- PR_O_TRANSP => FE !!! devra peut-être changer vu que c'est un complément du mode dispatch.
-- PR_O_NULLNOLOG => FE
-- PR_O_HTTP_CLOSE => FE. !!! mettre BE aussi !!!
-- PR_O_TCP_CLI_KA => FE
-
-- PR_O_FWDFOR => BE. FE aussi ?
-- PR_O_FORCE_CLO => BE
-- PR_O_PERSIST => BE
-- PR_O_COOK_RW, PR_O_COOK_INS, PR_O_COOK_PFX, PR_O_COOK_POST => BE
-- PR_O_COOK_NOC, PR_O_COOK_IND => BE
-- PR_O_ABRT_CLOSE => BE
-- PR_O_REDISP => BE
-- PR_O_BALANCE, PR_O_BALANCE_RR, PR_O_BALANCE_SH => BE
-- PR_O_CHK_CACHE => BE
-- PR_O_TCP_SRV_KA => BE
-- PR_O_BIND_SRC => BE
-- PR_O_TPXY_MASK => BE
-
-
-- PR_MODE_TCP : BE côté serveur, FE côté client
-
-- nbconn -> fe->nbconn, be->nbconn.
- Pb: rendre impossible le fait que (fe == be) avant de faire ça,
- sinon on va compter les connexions en double. Ce ne sera possible
- que lorsque les FE et BE seront des entités distinctes. On va donc
- commencer par laisser uniquement fe->nbconn (vu que le fe ne change
- pas), et modifier ceci plus tard, ne serait-ce que pour prendre en
- compte correctement les minconn/maxconn.
- => solution : avoir beconn et feconn dans chaque proxy.
-
-- failed_conns, failed_secu (réponses bloquées), failed_resp... : be
- Attention: voir les cas de ERR_SRVCL, il semble que parfois on
- indique ça alors qu'il y a un write error côté client (ex: ligne
- 2044 dans proto_http).
-
- => be et pas be->beprm
-
-- logs du backup : ->be (idem)
-
-- queue : be
-
-- logs/debug : srv toujours associé à be (ex: proxy->id:srv->id). Rien
- pour le client pour le moment. D'une manière générale, les erreurs
- provoquées côté serveur vont sur BE et celles côté client vont sur
- FE.
-- logswait & LW_BYTES : FE (puisqu'on veut savoir si on logue tout de suite)
-
-- messages d'erreurs personnalisés (errmsg, ...) -> fe
-
-- monitor_uri -> fe
-- uri_auth -> (fe->firpm puis be->fiprm). Utilisation de ->be
-
-- req_add, req_exp => fe->fiprm, puis be->fiprm
-- req_cap, rsp_cap -> fe->fiprm
-- rsp_add, rsp_exp => be->fiprm, devrait être fait ensuite aussi sur fe->fiprm
-- capture_name, capture_namelen : fe->fiprm
-
- Ce n'est pas la solution idéale, mais au moins la capture et configurable
- par les filtres du FE et ne bouge pas lorsque le BE est réassigné. Cela
- résoud aussi un pb d'allocation mémoire.
-
-
-- persistance (appsessions, cookiename, ...) -> be
-- stats:scope "." = fe (celui par lequel on arrive)
- !!!ERREUR!!! => utiliser be pour avoir celui qui a été validé par
- l'uri_auth.
-
-
---------- corrections à effectuer ---------
-
-- remplacement de headers : parser le header et éventuellement le supprimer puis le(les) rajouter.
-- session->proto.{l4state,l7state,l7substate} pour CLI et SRV
-- errorloc : si définie dans backend, la prendre, sinon dans front.
-- logs : faire be sinon fe.
diff --git a/doc/design-thoughts/config-language.txt b/doc/design-thoughts/config-language.txt
deleted file mode 100644
index 20c4fbd..0000000
--- a/doc/design-thoughts/config-language.txt
+++ /dev/null
@@ -1,262 +0,0 @@
-Prévoir des commandes en plusieurs mots clés.
-Par exemple :
-
- timeout connection XXX
- connection scale XXX
-
-On doit aussi accepter les préfixes :
-
- tim co XXX
- co sca XXX
-
-Prévoir de ranger les combinaisons dans un tableau. On doit même
-pouvoir effectuer un mapping simplifiant le parseur.
-
-
-Pour les filtres :
-
-
- <direction> <where> <what> <operator> <pattern> <action> [ <args>* ]
-
- <direction> = [ req | rsp ]
- <where> = [ in | out ]
- <what> = [ line | LINE | METH | URI | h(hdr) | H(hdr) | c(cookie) | C(cookie) ]
- <operator> = [ == | =~ | =* | =^ | =/ | != | !~ | !* | !^ | !/ ]
- <pattern> = "<string>"
- <action> = [ allow | permit | deny | delete | replace | switch | add | set | redir ]
- <args> = optional action args
-
- examples:
-
- req in URI =^ "/images" switch images
- req in h(host) =* ".mydomain.com" switch mydomain
- req in h(host) =~ "localhost(.*)" replace "www\1"
-
- alternative :
-
- <direction> <where> <action> [not] <what> [<operator> <pattern> [ <args>* ]]
-
- req in switch URI =^ "/images" images
- req in switch h(host) =* ".mydomain.com" mydomain
- req in replace h(host) =~ "localhost(.*)" "www\1"
- req in delete h(Connection)
- req in deny not line =~ "((GET|HEAD|POST|OPTIONS) /)|(OPTIONS *)"
- req out set h(Connection) "close"
- req out add line "Server: truc"
-
-
- <direction> <action> <where> [not] <what> [<operator> <pattern> [ <args>* ]] ';' <action2> <what2>
-
- req in switch URI =^ "/images/" images ; replace "/"
- req in switch h(host) =* ".mydomain.com" mydomain
- req in replace h(host) =~ "localhost(.*)" "www\1"
- req in delete h(Connection)
- req in deny not line =~ "((GET|HEAD|POST|OPTIONS) /)|(OPTIONS *)"
- req out set h(Connection) "close"
- req out add line == "Server: truc"
-
-
-Extension avec des ACL :
-
- req in acl(meth_valid) METH =~ "(GET|POST|HEAD|OPTIONS)"
- req in acl(meth_options) METH == "OPTIONS"
- req in acl(uri_slash) URI =^ "/"
- req in acl(uri_star) URI == "*"
-
- req in deny acl !(meth_options && uri_star || meth_valid && uri_slash)
-
-Peut-être plus simplement :
-
- acl meth_valid METH =~ "(GET|POST|HEAD|OPTIONS)"
- acl meth_options METH == "OPTIONS"
- acl uri_slash URI =^ "/"
- acl uri_star URI == "*"
-
- req in deny not acl(meth_options uri_star, meth_valid uri_slash)
-
- req in switch URI =^ "/images/" images ; replace "/"
- req in switch h(host) =* ".mydomain.com" mydomain
- req in replace h(host) =~ "localhost(.*)" "www\1"
- req in delete h(Connection)
- req in deny not line =~ "((GET|HEAD|POST|OPTIONS) /)|(OPTIONS *)"
- req out set h(Connection) "close"
- req out add line == "Server: truc"
-
-Prévoir le cas du "if" pour exécuter plusieurs actions :
-
- req in if URI =^ "/images/" then replace "/" ; switch images
-
-Utiliser les noms en majuscules/minuscules pour indiquer si on veut prendre
-en compte la casse ou non :
-
- if uri =^ "/watch/" setbe watch rebase "/watch/" "/"
- if uri =* ".jpg" setbe images
- if uri =~ ".*dll.*" deny
- if HOST =* ".mydomain.com" setbe mydomain
- etc...
-
-Another solution would be to have a dedicated keyword to URI remapping. It
-would both rewrite the URI and optionally switch to another backend.
-
- uriremap "/watch/" "/" watch
- uriremap "/chat/" "/" chat
- uriremap "/event/" "/event/" event
-
-Or better :
-
- uriremap "/watch/" watch "/"
- uriremap "/chat/" chat "/"
- uriremap "/event/" event
-
-For the URI, using a regex is sometimes useful (eg: providing a set of possible prefixes.
-
-
-Sinon, peut-être que le "switch" peut prendre un paramètre de mapping pour la partie matchée :
-
- req in switch URI =^ "/images/" images:"/"
-
-
-2007/03/31 - Besoins plus précis.
-
-1) aucune extension de branchement ou autre dans les "listen", c'est trop complexe.
-
-Distinguer les données entrantes (in) et sortantes (out).
-
-Le frontend ne voit que les requetes entrantes et les réponses sortantes.
-Le backend voir les requêtes in/out et les réponses in/out.
-Le frontend permet les branchements d'ensembles de filtres de requêtes vers
-d'autres. Le frontend et les ensembles de filtres de requêtes peuvent brancher
-vers un backend.
-
------------+--------+----------+----------+---------+----------+
- \ Where | | | | | |
- \______ | Listen | Frontend | ReqRules | Backend | RspRules |
- \| | | | | |
-Capability | | | | | |
------------+--------+----------+----------+---------+----------+
-Frontend | X | X | | | |
------------+--------+----------+----------+---------+----------+
-FiltReqIn | X | X | X | X | |
------------+--------+----------+----------+---------+----------+
-JumpFiltReq| X | X | X | | | \
------------+--------+----------+----------+---------+----------+ > = ReqJump
-SetBackend | X | X | X | | | /
------------+--------+----------+----------+---------+----------+
-FiltReqOut | | | | X | |
------------+--------+----------+----------+---------+----------+
-FiltRspIn | X | | | X | X |
------------+--------+----------+----------+---------+----------+
-JumpFiltRsp| | | | X | X |
------------+--------+----------+----------+---------+----------+
-FiltRspOut | | X | | X | X |
------------+--------+----------+----------+---------+----------+
-Backend | X | | | X | |
------------+--------+----------+----------+---------+----------+
-
-En conclusion
--------------
-
-Il y a au moins besoin de distinguer 8 fonctionnalités de base :
- - capacité à recevoir des connexions (frontend)
- - capacité à filtrer les requêtes entrantes
- - capacité à brancher vers un backend ou un ensemble de règles de requêtes
- - capacité à filtrer les requêtes sortantes
- - capacité à filtrer les réponses entrantes
- - capacité à brancher vers un autre ensemble de règles de réponses
- - capacité à filtrer la réponse sortante
- - capacité à gérer des serveurs (backend)
-
-Remarque
---------
- - on a souvent besoin de pouvoir appliquer un petit traitement sur un ensemble
- host/uri/autre. Le petit traitement peut consister en quelques filtres ainsi
- qu'une réécriture du couple (host,uri).
-
-
-Proposition : ACL
-
-Syntaxe :
----------
-
- acl <name> <what> <operator> <value> ...
-
-Ceci créera une acl référencée sous le nom <name> qui sera validée si
-l'application d'au moins une des valeurs <value> avec l'opérateur <operator>
-sur le sujet <what> est validée.
-
-Opérateurs :
-------------
-
-Toujours 2 caractères :
-
- [=!][~=*^%/.]
-
-Premier caractère :
- '=' : OK si test valide
- '!' : OK si test échoué.
-
-Second caractère :
- '~' : compare avec une regex
- '=' : compare chaîne à chaîne
- '*' : compare la fin de la chaîne (ex: =* ".mydomain.com")
- '^' : compare le début de la chaîne (ex: =^ "/images/")
- '%' : recherche une sous-chaîne
- '/' : compare avec un mot entier en acceptant le '/' comme délimiteur
- '.' : compare avec un mot entier en acceptant el '.' comme délimiteur
-
-Ensuite on exécute une action de manière conditionnelle si l'ensemble des ACLs
-mentionnées sont validées (ou invalidées pour celles précédées d'un "!") :
-
- <what> <where> <action> on [!]<aclname> ...
-
-
-Exemple :
----------
-
- acl www_pub host =. www www01 dev preprod
- acl imghost host =. images
- acl imgdir uri =/ img
- acl imagedir uri =/ images
- acl msie h(user-agent) =% "MSIE"
-
- set_host "images" on www_pub imgdir
- remap_uri "/img" "/" on www_pub imgdir
- remap_uri "/images" "/" on www_pub imagedir
- setbe images on imghost
- reqdel "Cookie" on all
-
-
-
-Actions possibles :
-
- req {in|out} {append|delete|rem|add|set|rep|mapuri|rewrite|reqline|deny|allow|setbe|tarpit}
- resp {in|out} {append|delete|rem|add|set|rep|maploc|rewrite|stsline|deny|allow}
-
- req in append <line>
- req in delete <line_regex>
- req in rem <header>
- req in add <header> <new_value>
- req in set <header> <new_value>
- req in rep <header> <old_value> <new_value>
- req in mapuri <old_uri_prefix> <new_uri_prefix>
- req in rewrite <old_uri_regex> <new_uri>
- req in reqline <old_req_regex> <new_req>
- req in deny
- req in allow
- req in tarpit
- req in setbe <backend>
-
- resp out maploc <old_location_prefix> <new_loc_prefix>
- resp out stsline <old_sts_regex> <new_sts_regex>
-
-Les chaînes doivent être délimitées par un même caractère au début et à la fin,
-qui doit être échappé s'il est présent dans la chaîne. Tout ce qui se trouve
-entre le caractère de fin et les premiers espace est considéré comme des
-options passées au traitement. Par exemple :
-
- req in rep host /www/i /www/
- req in rep connection /keep-alive/i "close"
-
-Il serait pratique de pouvoir effectuer un remap en même temps qu'un setbe.
-
-Captures: les séparer en in/out. Les rendre conditionnelles ?
diff --git a/doc/design-thoughts/connection-sharing.txt b/doc/design-thoughts/connection-sharing.txt
deleted file mode 100644
index 99be1cd..0000000
--- a/doc/design-thoughts/connection-sharing.txt
+++ /dev/null
@@ -1,31 +0,0 @@
-2014/10/28 - Server connection sharing
-
-For HTTP/2 we'll have to use multiplexed connections to the servers and to
-share them between multiple streams. We'll also have to do this for H/1, but
-with some variations since H1 doesn't offer connection status verification.
-
-In order to validate that an idle connection is still usable, it is desirable
-to periodically send health checks over it. Normally, idle connections are
-meant to be heavily used, so there is no reason for having them idle for a long
-time. Thus we have two possibilities :
-
- - either we time them out after some inactivity, this saves server resources ;
- - or we check them after some inactivity. For this we can send the server-
- side HTTP health check (only when the server uses HTTP checks), and avoid
- using that to mark the server down, and instead consider the connection as
- dead.
-
-For HTTP/2 we'll have to send pings periodically over these connections, so
-it's worth considering a per-connection task to validate that the channel still
-works.
-
-In the current model, a connection necessarily belongs to a session, so it's
-not really possible to share them, at best they can be exchanged, but that
-doesn't make much sense as it means that it could disturb parallel traffic.
-
-Thus we need to have a per-server list of idle connections and a max-idle-conn
-setting to kill them when there are too many. In the case of H/1 it is also
-advisable to consider that if a connection was created to pass a first non-
-idempotent request while other idle connections were still existing, then a
-connection will have to be killed in order not to exceed the limit.
-
diff --git a/doc/design-thoughts/dynamic-buffers.txt b/doc/design-thoughts/dynamic-buffers.txt
deleted file mode 100644
index 564d868..0000000
--- a/doc/design-thoughts/dynamic-buffers.txt
+++ /dev/null
@@ -1,41 +0,0 @@
-2014/10/30 - dynamic buffer management
-
-Since HTTP/2 processing will significantly increase the need for buffering, it
-becomes mandatory to be able to support dynamic buffer allocation. This also
-means that at any moment some buffer allocation will fail and that a task or an
-I/O operation will have to be paused for the time needed to allocate a buffer.
-
-There are 3 places where buffers are needed :
-
- - receive side of a stream interface. A connection notifies about a pending
- recv() and the SI calls the receive function to put the data into a buffer.
- Here the buffer will have to be picked from a pool first, and if the
- allocation fails, the I/O will have to temporarily be disabled, the
- connection will have to subscribe for buffer release notification to be
- woken up once a buffer is available again. It's important to keep in mind
- that buffer availability doesn't necessarily mean a desire to enable recv
- again, just that recv is not paused anymore for resource reasons.
-
- - receive side of a stream interface when the other end point is an applet.
- The applet wants to write into the buffer and for this the buffer needs to
- be allocated as well. It is the same as above except that it is the applet
- which is put to a pause. Since the applet might be at the core of the task
- itself, it could become tricky to handle the situation correctly. Stats and
- peers are in this situation.
-
- - Tx of a task : some tasks perform spontaneous writes to a buffer. Checks
- are an example of this. The checks will have to be able to sleep while a
- buffer is being awaited.
-
-One important point is that such pauses must not prevent the task from timing
-out. There it becomes difficult because in the case of a time out, we could
-want to emit a timeout error message and for this, require a buffer. So it is
-important to keep the ability not to send messages upon error processing, and
-to be able to give up and stop waiting for buffers.
-
-The refill mechanism needs to be designed in a thread-safe way because this
-will become one of the rare cases of inter-task activity. Thus it is important
-to ensure that checking the state of the task and passing of the freshly
-released buffer are performed atomically, and that in case the task doesn't
-want it anymore, it is responsible for passing it to the next one.
-
diff --git a/doc/design-thoughts/entities-v2.txt b/doc/design-thoughts/entities-v2.txt
deleted file mode 100644
index 91c4fa9..0000000
--- a/doc/design-thoughts/entities-v2.txt
+++ /dev/null
@@ -1,276 +0,0 @@
-2012/07/05 - Connection layering and sequencing
-
-
-An FD has a state :
- - CLOSED
- - READY
- - ERROR (?)
- - LISTEN (?)
-
-A connection has a state :
- - CLOSED
- - ACCEPTED
- - CONNECTING
- - ESTABLISHED
- - ERROR
-
-A stream interface has a state :
- - INI, REQ, QUE, TAR, ASS, CON, CER, EST, DIS, CLO
-
-Note that CON and CER might be replaced by EST if the connection state is used
-instead. CON might even be more suited than EST to indicate that a connection
-is known.
-
-
-si_shutw() must do :
-
- data_shutw()
- if (shutr) {
- data_close()
- ctrl_shutw()
- ctrl_close()
- }
-
-si_shutr() must do :
- data_shutr()
- if (shutw) {
- data_close()
- ctrl_shutr()
- ctrl_close()
- }
-
-Each of these steps may fail, in which case the step must be retained and the
-operations postponed in an asynchronous task.
-
-The first asynchronous data_shut() might already fail so it is mandatory to
-save the other side's status with the connection in order to let the async task
-know whether the 3 next steps must be performed.
-
-The connection (or perhaps the FD) needs to know :
- - the desired close operations : DSHR, DSHW, CSHR, CSHW
- - the completed close operations : DSHR, DSHW, CSHR, CSHW
-
-
-On the accept() side, we probably need to know :
- - if a header is expected (eg: accept-proxy)
- - if this header is still being waited for
- => maybe both info might be combined into one bit
-
- - if a data-layer accept() is expected
- - if a data-layer accept() has been started
- - if a data-layer accept() has been performed
- => possibly 2 bits, to indicate the need to free()
-
-On the connect() side, we need to know :
- - the desire to send a header (eg: send-proxy)
- - if this header has been sent
- => maybe both info might be combined
-
- - if a data-layer connect() is expected
- - if a data-layer connect() has been started
- - if a data-layer connect() has been completed
- => possibly 2 bits, to indicate the need to free()
-
-On the response side, we also need to know :
- - the desire to send a header (eg: health check response for monitor-net)
- - if this header was sent
- => might be the same as sending a header over a new connection
-
-Note: monitor-net has precedence over proxy proto and data layers. Same for
- health mode.
-
-For multi-step operations, use 2 bits :
- 00 = operation not desired, not performed
- 10 = operation desired, not started
- 11 = operation desired, started but not completed
- 01 = operation desired, started and completed
-
- => X != 00 ==> operation desired
- X & 01 ==> operation at least started
- X & 10 ==> operation not completed
-
-Note: no way to store status information for error reporting.
-
-Note2: it would be nice if "tcp-request connection" rules could work at the
-connection level, just after headers ! This means support for tracking stick
-tables, possibly not too much complicated.
-
-
-Proposal for incoming connection sequence :
-
-- accept()
-- if monitor-net matches or if mode health => try to send response
-- if accept-proxy, wait for proxy request
-- if tcp-request connection, process tcp rules and possibly keep the
- pointer to stick-table
-- if SSL is enabled, switch to SSL handshake
-- then switch to DATA state and instantiate a session
-
-We just need a map of handshake handlers on the connection. They all manage the
-FD status themselves and set the callbacks themselves. If their work succeeds,
-they remove themselves from the list. If it fails, they remain subscribed and
-enable the required polling until they are woken up again or the timeout strikes.
-
-Identified handshake handlers for incoming connections :
- - HH_HEALTH (tries to send OK and dies)
- - HH_MONITOR_IN (matches src IP and adds/removes HH_SEND_OK/HH_SEND_HTTP_OK)
- - HH_SEND_OK (tries to send "OK" and dies)
- - HH_SEND_HTTP_OK (tries to send "HTTP/1.0 200 OK" and dies)
- - HH_ACCEPT_PROXY (waits for PROXY line and parses it)
- - HH_TCP_RULES (processes TCP rules)
- - HH_SSL_HS (starts SSL handshake)
- - HH_ACCEPT_SESSION (instantiates a session)
-
-Identified handshake handlers for outgoing connections :
- - HH_SEND_PROXY (tries to build and send the PROXY line)
- - HH_SSL_HS (starts SSL handshake)
-
-For the pollers, we could check that handshake handlers are not 0 and decide to
-call a generic connection handshake handler instead of usual callbacks. Problem
-is that pollers don't know connections, they know fds. So entities which manage
-handlers should update change the FD callbacks accordingly.
-
-With a bit of care, we could have :
- - HH_SEND_LAST_CHUNK (sends the chunk pointed to by a pointer and dies)
- => merges HEALTH, SEND_OK and SEND_HTTP_OK
-
-It sounds like the ctrl vs data state for the connection are per-direction
-(eg: support an async ctrl shutw while still reading data).
-
-Also support shutr/shutw status at L4/L7.
-
-In practice, what we really need is :
-
-shutdown(conn) =
- conn.data.shut()
- conn.ctrl.shut()
- conn.fd.shut()
-
-close(conn) =
- conn.data.close()
- conn.ctrl.close()
- conn.fd.close()
-
-With SSL over Remote TCP (RTCP + RSSL) to reach the server, we would have :
-
- HTTP -> RTCP+RSSL connection <-> RTCP+RRAW connection -> TCP+SSL connection
-
-The connection has to be closed at 3 places after a successful response :
- - DATA (RSSL over RTCP)
- - CTRL (RTCP to close connection to server)
- - SOCK (FD to close connection to second process)
-
-Externally, the connection is seen with very few flags :
- - SHR
- - SHW
- - ERR
-
-We don't need a CLOSED flag as a connection must always be detached when it's closed.
-
-The internal status doesn't need to be exposed :
- - FD allocated (Y/N)
- - CTRL initialized (Y/N)
- - CTRL connected (Y/N)
- - CTRL handlers done (Y/N)
- - CTRL failed (Y/N)
- - CTRL shutr (Y/N)
- - CTRL shutw (Y/N)
- - DATA initialized (Y/N)
- - DATA connected (Y/N)
- - DATA handlers done (Y/N)
- - DATA failed (Y/N)
- - DATA shutr (Y/N)
- - DATA shutw (Y/N)
-
-(note that having flags for operations needing to be completed might be easier)
---------------
-
-Maybe we need to be able to call conn->fdset() and conn->fdclr() but it sounds
-very unlikely since the only functions manipulating this are in the code of
-the data/ctrl handlers.
-
-FDSET/FDCLR cannot be directly controlled by the stream interface since it also
-depends on the DATA layer (WANT_READ/WANT_WRITE).
-
-But FDSET/FDCLR is probably controlled by who owns the connection (eg: DATA).
-
-Example: an SSL conn relies on an FD. The buffer is full, and wants the conn to
-stop reading. It must not stop the FD itself. It is the read function which
-should notice that it has nothing to do with a read wake-up, which needs to
-disable reading.
-
-Conversely, when calling conn->chk_rcv(), the reader might get a WANT_READ or
-even WANT_WRITE and adjust the FDs accordingly.
-
-------------------------
-
-OK, the problem is simple : we don't manipulate the FD at the right level.
-We should have :
- ->connect(), ->chk_snd(), ->chk_rcv(), ->shutw(), ->shutr() which are
- called from the upper layer (buffer)
- ->recv(), ->send(), called from the lower layer
-
-Note that the SHR is *reported* by lower layer but can be forced by upper
-layer. In this case it's like a delayed abort. The difficulty consists in
-knowing the output data were correctly read. Probably we'd need to drain
-incoming data past the active shutr().
-
-The only four purposes of the top-down shutr() call are :
- - acknowledge a shut read report : could probably be done better
- - read timeout => disable reading : it's a delayed abort. We want to
- report that the buffer is SHR, maybe even the connection, but the
- FD clearly isn't.
- - read abort due to error on the other side or desire to close (eg:
- http-server-close) : delayed abort
- - complete abort
-
-The active shutr() is problematic as we can't disable reading if we expect some
-exchanges for data acknowledgement. We probably need to drain data only until
-the shutw() has been performed and ACKed.
-
-A connection shut down for read would behave like this :
-
- 1) bidir exchanges
-
- 2) shutr() => read_abort_pending=1
-
- 3) drain input, still send output
-
- 4) shutw()
-
- 5) drain input, wait for read0 or ack(shutw)
-
- 6) close()
-
---------------------- 2012/07/05 -------------------
-
-Communications must be performed this way :
-
- connection <-> channel <-> connection
-
-A channel is composed of flags and stats, and may store data in either a buffer
-or a pipe. We need low-layer operations between sockets and buffers or pipes.
-Right now we only support sockets, but later we might support remote sockets
-and maybe pipes or shared memory segments.
-
-So we need :
-
- - raw_sock_to_buf() => receive raw data from socket into buffer
- - raw_sock_to_pipe => receive raw data from socket into pipe (splice in)
- - raw_sock_from_buf() => send raw data from buffer to socket
- - raw_sock_from_pipe => send raw data from pipe to socket (splice out)
-
- - ssl_sock_to_buf() => receive ssl data from socket into buffer
- - ssl_sock_to_pipe => receive ssl data from socket into a pipe (NULL)
- - ssl_sock_from_buf() => send ssl data from buffer to socket
- - ssl_sock_from_pipe => send ssl data from pipe to socket (NULL)
-
-These functions should set such status flags :
-
-#define ERR_IN 0x01
-#define ERR_OUT 0x02
-#define SHUT_IN 0x04
-#define SHUT_OUT 0x08
-#define EMPTY_IN 0x10
-#define FULL_OUT 0x20
-
diff --git a/doc/design-thoughts/how-it-works.txt b/doc/design-thoughts/how-it-works.txt
deleted file mode 100644
index 2d1cb89..0000000
--- a/doc/design-thoughts/how-it-works.txt
+++ /dev/null
@@ -1,60 +0,0 @@
-How it works ? (unfinished and inexact)
-
-For TCP and HTTP :
-
-- listeners create listening sockets with a READ callback pointing to the
- protocol-specific accept() function.
-
-- the protocol-specific accept() function then accept()'s the connection and
- instantiates a "server TCP socket" (which is dedicated to the client side),
- and configures it (non_block, get_original_dst, ...).
-
-For TCP :
-- in case of pure TCP, a request buffer is created, as well as a "client TCP
- socket", which tries to connect to the server.
-
-- once the connection is established, the response buffer is allocated and
- connected to both ends.
-
-- both sockets are set to "autonomous mode" so that they only wake up their
- supervising session when they encounter a special condition (error or close).
-
-
-For HTTP :
-- in case of HTTP, a request buffer is created with the "HOLD" flag set and
- a read limit to support header rewriting (may be this one will be removed
- eventually because it's better to limit only to the buffer size and report
- an error when rewritten data overflows)
-
-- a "flow analyzer" is attached to the buffer (or possibly multiple flow
- analyzers). For the request, the flow analyzer is "http_lb_req". The flow
- analyzer is a function which gets called when new data is present and
- blocked. It has a timeout (request timeout). It can also be bypassed on
- demand.
-
-- when the "http_lb_req" has received the whole request, it creates a client
- socket with all the parameters needed to try to connect to the server. When
- the connection establishes, the response buffer is allocated on the fly,
- put to HOLD mode, and a an "http_lb_resp" flow analyzer is attached to the
- buffer.
-
-
-For client-side HTTPS :
-
-- the accept() function must completely instantiate a TCP socket + an SSL
- reader. It is when the SSL session is complete that we call the
- protocol-specific accept(), and create its buffer.
-
-
-
-
-Conclusions
------------
-
-- we need a generic TCP accept() function with a lot of flags set by the
- listener, to tell it what info we need to get at the accept() time, and
- what flags will have to be set on the socket.
-
-- once the TCP accept() function ends, it wakes up the protocol supervisor
- which is in charge of creating the buffers, etc, switch states, etc...
-
diff --git a/doc/design-thoughts/http2.txt b/doc/design-thoughts/http2.txt
deleted file mode 100644
index c21ac10..0000000
--- a/doc/design-thoughts/http2.txt
+++ /dev/null
@@ -1,277 +0,0 @@
-2014/10/23 - design thoughts for HTTP/2
-
-- connections : HTTP/2 depends a lot more on a connection than HTTP/1 because a
- connection holds a compression context (headers table, etc...). We probably
- need to have an h2_conn struct.
-
-- multiple transactions will be handled in parallel for a given h2_conn. They
- are called streams in HTTP/2 terminology.
-
-- multiplexing : for a given client-side h2 connection, we can have multiple
- server-side h2 connections. And for a server-side h2 connection, we can have
- multiple client-side h2 connections. Streams circulate in N-to-N fashion.
-
-- flow control : flow control will be applied between multiple streams. Special
- care must be taken so that an H2 client cannot block some H2 servers by
- sending requests spread over multiple servers to the point where one server
- response is blocked and prevents other responses from the same server from
- reaching their clients. H2 connection buffers must always be empty or nearly
- empty. The per-stream flow control needs to be respected as well as the
- connection's buffers. It is important to implement some fairness between all
- the streams so that it's not always the same which gets the bandwidth when
- the connection is congested.
-
-- some clients can be H1 with an H2 server (is this really needed ?). Most of
- the initial use case will be H2 clients to H1 servers. It is important to keep
- in mind that H1 servers do not do flow control and that we don't want them to
- block transfers (eg: post upload).
-
-- internal tasks : some H2 clients will be internal tasks (eg: health checks).
- Some H2 servers will be internal tasks (eg: stats, cache). The model must be
- compatible with this use case.
-
-- header indexing : headers are transported compressed, with a reference to a
- static or a dynamic header, or a literal, possibly huffman-encoded. Indexing
- is specific to the H2 connection. This means there is no way any binary data
- can flow between both sides, headers will have to be decoded according to the
- incoming connection's context and re-encoded according to the outgoing
- connection's context, which can significantly differ. In order to avoid the
- parsing trouble we currently face, headers will have to be clearly split
- between name and value. It is worth noting that neither the incoming nor the
- outgoing connections' contexts will be of any use while processing the
- headers. At best we can have some shortcuts for well-known names that map
- well to the static ones (eg: use the first static entry with same name), and
- maybe have a few special cases for static name+value as well. Probably we can
- classify headers in such categories :
-
- - static name + value
- - static name + other value
- - dynamic name + other value
-
- This will allow for better processing in some specific cases. Headers
- supporting a single value (:method, :status, :path, ...) should probably
- be stored in a single location with a direct access. That would allow us
- to retrieve a method using hdr[METHOD]. All such indexing must be performed
- while parsing. That also means that HTTP/1 will have to be converted to this
- representation very early in the parser and possibly converted back to H/1
- after processing.
-
- Header names/values will have to be placed in a small memory area that will
- inevitably get fragmented as headers are rewritten. An automatic packing
- mechanism must be implemented so that when there's no more room, headers are
- simply defragmented/packet to a new table and the old one is released. Just
- like for the static chunks, we need to have a few such tables pre-allocated
- and ready to be swapped at any moment. Repacking must not change any index
- nor affect the way headers are compressed so that it can happen late after a
- retry (send-name-header for example).
-
-- header processing : can still happen on a (header, value) basis. Reqrep/
- rsprep completely disappear and will have to be replaced with something else
- to support renaming headers and rewriting url/path/...
-
-- push_promise : servers can push dummy requests+responses. They advertise
- the stream ID in the push_promise frame indicating the associated stream ID.
- This means that it is possible to initiate a client-server stream from the
- information coming from the server and make the data flow as if the client
- had made it. It's likely that we'll have to support two types of server
- connections: those which support push and those which do not. That way client
- streams will be distributed to existing server connections based on their
- capabilities. It's important to keep in mind that PUSH will not be rewritten
- in responses.
-
-- stream ID mapping : since the stream ID is per H2 connection, stream IDs will
- have to be mapped. Thus a given stream is an entity with two IDs (one per
- side). Or more precisely a stream has two end points, each one carrying an ID
- when it ends on an HTTP2 connection. Also, for each stream ID we need to
- quickly find the associated transaction in progress. Using a small quick
- unique tree seems indicated considering the wide range of valid values.
-
-- frame sizes : frame have to be remapped between both sides as multiplexed
- connections won't always have the same characteristics. Thus some frames
- might be spliced and others will be sliced.
-
-- error processing : care must be taken to never break a connection unless it
- is dead or corrupt at the protocol level. Stats counter must exist to observe
- the causes. Timeouts are a great problem because silent connections might
- die out of inactivity. Ping frames should probably be scheduled a few seconds
- before the connection timeout so that an unused connection is verified before
- being killed. Abnormal requests must be dealt with using RST_STREAM.
-
-- ALPN : ALPN must be observed on the client side, and transmitted to the server
- side.
-
-- proxy protocol : proxy protocol makes little to no sense in a multiplexed
- protocol. A per-stream equivalent will surely be needed if implementations
- do not quickly generalize the use of Forward.
-
-- simplified protocol for local devices (eg: haproxy->varnish in clear and
- without handshake, and possibly even with splicing if the connection's
- settings are shared)
-
-- logging : logging must report a number of extra information such as the
- stream ID, and whether the transaction was initiated by the client or by the
- server (which can be deduced from the stream ID's parity). In case of push,
- the number of the associated stream must also be reported.
-
-- memory usage : H2 increases memory usage by mandating use of 16384 bytes
- frame size minimum. That means slightly more than 16kB of buffer in each
- direction to process any frame. It will definitely have an impact on the
- deployed maxconn setting in places using less than this (4..8kB are common).
- Also, the header list is persistent per connection, so if we reach the same
- size as the request, that's another 16kB in each direction, resulting in
- about 48kB of memory where 8 were previously used. A more careful encoder
- can work with a much smaller set even if that implies evicting entries
- between multiple headers of the same message.
-
-- HTTP/1.0 should very carefully be transported over H2. Since there's no way
- to pass version information in the protocol, the server could use some
- features of HTTP/1.1 that are unsafe in HTTP/1.0 (compression, trailers,
- ...).
-
-- host / :authority : ":authority" is the norm, and "host" will be absent when
- H2 clients generate :authority. This probably means that a dummy Host header
- will have to be produced internally from :authority and removed when passing
- to H2 behind. This can cause some trouble when passing H2 requests to H1
- proxies, because there's no way to know if the request should contain scheme
- and authority in H1 or not based on the H2 request. Thus a "proxy" option
- will have to be explicitly mentioned on HTTP/1 server lines. One of the
- problem that it creates is that it's not longer possible to pass H/1 requests
- to H/1 proxies without an explicit configuration. Maybe a table of the
- various combinations is needed.
-
- :scheme :authority host
- HTTP/2 request present present absent
- HTTP/1 server req absent absent present
- HTTP/1 proxy req present present present
-
- So in the end the issue is only with H/2 requests passed to H/1 proxies.
-
-- ping frames : they don't indicate any stream ID so by definition they cannot
- be forwarded to any server. The H2 connection should deal with them only.
-
-There's a layering problem with H2. The framing layer has to be aware of the
-upper layer semantics. We can't simply re-encode HTTP/1 to HTTP/2 then pass
-it over a framing layer to mux the streams, the frame type must be passed below
-so that frames are properly arranged. Header encoding is connection-based and
-all streams using the same connection will interact in the way their headers
-are encoded. Thus the encoder *has* to be placed in the h2_conn entity, and
-this entity has to know for each stream what its headers are.
-
-Probably that we should remove *all* headers from transported data and move
-them on the fly to a parallel structure that can be shared between H1 and H2
-and consumed at the appropriate level. That means buffers only transport data.
-Trailers have to be dealt with differently.
-
-So if we consider an H1 request being forwarded between a client and a server,
-it would look approximately like this :
-
- - request header + body land into a stream's receive buffer
- - headers are indexed and stripped out so that only the body and whatever
- follows remain in the buffer
- - both the header index and the buffer with the body stay attached to the
- stream
- - the sender can rebuild the whole headers. Since they're found in a table
- supposed to be stable, it can rebuild them as many times as desired and
- will always get the same result, so it's safe to build them into the trash
- buffer for immediate sending, just as we do for the PROXY protocol.
- - the upper protocol should probably provide a build_hdr() callback which
- when called by the socket layer, builds this header block based on the
- current stream's header list, ready to be sent.
- - the socket layer has to know how many bytes from the headers are left to be
- forwarded prior to processing the body.
- - the socket layer needs to consume only the acceptable part of the body and
- must not release the buffer if any data remains in it (eg: pipelining over
- H1). This is already handled by channel->o and channel->to_forward.
- - we could possibly have another optional callback to send a preamble before
- data, that could be used to send chunk sizes in H1. The danger is that it
- absolutely needs to be stable if it has to be retried. But it could
- considerably simplify de-chunking.
-
-When the request is sent to an H2 server, an H2 stream request must be made
-to the server, we find an existing connection whose settings are compatible
-with our needs (eg: tls/clear, push/no-push), and with a spare stream ID. If
-none is found, a new connection must be established, unless maxconn is reached.
-
-Servers must have a maxstream setting just like they have a maxconn. The same
-queue may be used for that.
-
-The "tcp-request content" ruleset must apply to the TCP layer. But with HTTP/2
-that becomes impossible (and useless). We still need something like the
-"tcp-request session" hook to apply just after the SSL handshake is done.
-
-It is impossible to defragment the body on the fly in HTTP/2. Since multiple
-messages are interleaved, we cannot wait for all of them and block the head of
-line. Thus if body analysis is required, it will have to use the stream's
-buffer, which necessarily implies a copy. That means that with each H2 end we
-necessarily have at least one copy. Sometimes we might be able to "splice" some
-bytes from one side to the other without copying into the stream buffer (same
-rules as for TCP splicing).
-
-In theory, only data should flow through the channel buffer, so each side's
-connector is responsible for encoding data (H1: linear/chunks, H2: frames).
-Maybe the same mechanism could be extrapolated to tunnels / TCP.
-
-Since we'd use buffers only for data (and for receipt of headers), we need to
-have dynamic buffer allocation.
-
-Thus :
-- Tx buffers do not exist. We allocate a buffer on the fly when we're ready to
- send something that we need to build and that needs to be persistent in case
- of partial send. H1 headers are built on the fly from the header table to a
- temporary buffer that is immediately sent and whose amount of sent bytes is
- the only information kept (like for PROXY protocol). H2 headers are more
- complex since the encoding depends on what was successfully sent. Thus we
- need to build them and put them into a temporary buffer that remains
- persistent in case send() fails. It is possible to have a limited pool of
- Tx buffers and refrain from sending if there is no more buffer available in
- the pool. In that case we need a wake-up mechanism once a buffer is
- available. Once the data are sent, the Tx buffer is then immediately recycled
- in its pool. Note that no tx buffer being used (eg: for hdr or control) means
- that we have to be able to serialize access to the connection and retry with
- the same stream. It also means that a stream that times out while waiting for
- the connector to read the second half of its request has to stay there, or at
- least needs to be handled gracefully. However if the connector cannot read
- the data to be sent, it means that the buffer is congested and the connection
- is dead, so that probably means it can be killed.
-
-- Rx buffers have to be pre-allocated just before calling recv(). A connection
- will first try to pick a buffer and disable reception if it fails, then
- subscribe to the list of tasks waiting for an Rx buffer.
-
-- full Rx buffers might sometimes be moved around to the next buffer instead of
- experiencing a copy. That means that channels and connectors must use the
- same format of buffer, and that only the channel will have to see its
- pointers adjusted.
-
-- Tx of data should be made as much as possible without copying. That possibly
- means by directly looking into the connection buffer on the other side if
- the local Tx buffer does not exist and the stream buffer is not allocated, or
- even performing a splice() call between the two sides. One of the problem in
- doing this is that it requires proper ordering of the operations (eg: when
- multiple readers are attached to a same buffer). If the splitting occurs upon
- receipt, there's no problem. If we expect to retrieve data directly from the
- original buffer, it's harder since it contains various things in an order
- which does not even indicate what belongs to whom. Thus possibly the only
- mechanism to implement is the buffer permutation which guarantees zero-copy
- and only in the 100% safe case. Also it's atomic and does not cause HOL
- blocking.
-
-It makes sense to chose the frontend_accept() function right after the
-handshake ended. It is then possible to check the ALPN, the SNI, the ciphers
-and to accept to switch to the h2_conn_accept handler only if everything is OK.
-The h2_conn_accept handler will have to deal with the connection setup,
-initialization of the header table, exchange of the settings frames and
-preparing whatever is needed to fire new streams upon receipt of unknown
-stream IDs. Note: most of the time it will not be possible to splice() because
-we need to know in advance the amount of bytes to write the header, and here it
-will not be possible.
-
-H2 health checks must be seen as regular transactions/streams. The check runs a
-normal client which seeks an available stream from a server. The server then
-finds one on an existing connection or initiates a new H2 connection. The H2
-checks will have to be configurable for sharing streams or not. Another option
-could be to specify how many requests can be made over existing connections
-before insisting on getting a separate connection. Note that such separate
-connections might end up stacking up once released. So probably that they need
-to be recycled very quickly (eg: fix how many unused ones can exist max).
-
diff --git a/doc/design-thoughts/rate-shaping.txt b/doc/design-thoughts/rate-shaping.txt
deleted file mode 100644
index ca09408..0000000
--- a/doc/design-thoughts/rate-shaping.txt
+++ /dev/null
@@ -1,90 +0,0 @@
-2010/01/24 - Design of multi-criteria request rate shaping.
-
-We want to be able to rate-shape traffic on multiple cirteria. For instance, we
-may want to support shaping of per-host header requests, as well as per source.
-
-In order to achieve this, we will use checkpoints, one per criterion. Each of
-these checkpoints will consist in a test, a rate counter and a queue.
-
-A request reaches the checkpoint and checks the counter. If the counter is
-below the limit, it is updated and the request continues. If the limit is
-reached, the request attaches itself into the queue and sleeps. The sleep time
-is computed from the queue status, and updates the queue status.
-
-A task is dedicated to each queue. Its sole purpose is to be woken up when the
-next task may wake up, to check the frequency counter, wake as many requests as
-possible and update the counter. All the woken up requests are detached from
-the queue. Maybe the task dedicated to the queue can be avoided and replaced
-with all queued tasks's sleep counters, though this looks tricky. Or maybe it's
-just the first request in the queue that should be responsible for waking up
-other tasks, and not to forget to pass on this responsibility to next tasks if
-it leaves the queue.
-
-The woken up request then goes on evaluating other criteria and possibly sleeps
-again on another one. In the end, the task will have waited the amount of time
-required to pass all checkpoints, and all checkpoints will be able to maintain
-a permanent load of exactly their limit if enough streams flow through them.
-
-Since a request can only sleep in one queue at a time, it makes sense to use a
-linked list element in each session to attach it to any queue. It could very
-well be shared with the pendconn hooks which could then be part of the session.
-
-This mechanism could be used to rate-shape sessions and requests per backend
-and per server.
-
-When rate-shaping on dynamic criteria, such as the source IP address, we have
-to first extract the data pattern, then look it up in a table very similar to
-the stickiness tables, but with a frequency counter. At the checkpoint, the
-pattern is looked up, the entry created or refreshed, and the frequency counter
-updated and checked. Then the request either goes on or sleeps as described
-above, but if it sleeps, it's still in the checkpoint's queue, but with a date
-computed from the criterion's status.
-
-This means that we need 3 distinct features :
-
- - optional pattern extraction
- - per-pattern or per-queue frequency counter
- - time-ordered queue with a task
-
-Based on past experiences with frequency counters, it does not appear very easy
-to exactly compute sleep delays in advance for multiple requests. So most
-likely we'll have to run per-criterion queues too, with only the head of the
-queue holding a wake-up timeout.
-
-This finally leads us to the following :
-
- - optional pattern extraction
- - per-pattern or per-queue frequency counter
- - per-frequency counter queue
- - head of the queue serves as a global queue timer.
-
-This brings us to a very flexible architecture :
- - 1 list of rule-based checkpoints per frontend
- - 1 list of rule-based checkpoints per backend
- - 1 list of rule-based checkpoints per server
-
-Each of these lists have a lot of rules conditioned by ACLs, just like the
-use-backend rules, except that all rules are evaluated in turn.
-
-Since we might sometimes just want to enable that without setting any limit and
-just for enabling control in ACLs (or logging ?), we should probably try to
-find a flexible way of declaring just a counter without a queue.
-
-These checkpoints could be of two types :
- - rate-limit (described here)
- - concurrency-limit (very similar with the counter and no timer). This
- feature would require to keep track of all accounted criteria in a
- request so that they can be released upon request completion.
-
-It should be possible to define a max of requests in the queue, above which a
-503 is returned. The same applies for the max delay in the queue. We could have
-it per-task (currently it's the connection timeout) and abort tasks with a 503
-when the delay is exceeded.
-
-Per-server connection concurrency could be converted to use this mechanism
-which is very similar.
-
-The construct should be flexible enough so that the counters may be checked
-from ACLs. That would allow to reject connections or switch to an alternate
-backend when some limits are reached.
-
diff --git a/doc/design-thoughts/sess_par_sec.txt b/doc/design-thoughts/sess_par_sec.txt
deleted file mode 100644
index e936374..0000000
--- a/doc/design-thoughts/sess_par_sec.txt
+++ /dev/null
@@ -1,13 +0,0 @@
-Graphe des nombres de traitements par seconde unité de temps avec
- - un algo linéaire et très peu coûteux unitairement (0.01 ut)
- - un algo en log(2) et 5 fois plus coûteux (0.05 ut)
-
-set yrange [0:1]
-plot [0:1000] 1/(1+0.01*x), 1/(1+0.05*log(x+1)/log(2))
-
-Graphe de la latence induite par ces traitements en unités de temps :
-
-set yrange [0:1000]
-plot [0:1000] x/(1+0.01*x), x/(1+0.05*log(x+1)/log(2))
-
-
diff --git a/doc/internals/entities-v2.txt b/doc/internals/entities-v2.txt
deleted file mode 100644
index 86782c3..0000000
--- a/doc/internals/entities-v2.txt
+++ /dev/null
@@ -1,193 +0,0 @@
-An FD has a state :
- - CLOSED
- - READY
- - ERROR (?)
- - LISTEN (?)
-
-A connection has a state :
- - CLOSED
- - ACCEPTED
- - CONNECTING
- - ESTABLISHED
- - ERROR
-
-A stream interface has a state :
- - INI, REQ, QUE, TAR, ASS, CON, CER, EST, DIS, CLO
-
-Note that CON and CER might be replaced by EST if the connection state is used
-instead. CON might even be more suited than EST to indicate that a connection
-is known.
-
-
-si_shutw() must do :
-
- data_shutw()
- if (shutr) {
- data_close()
- ctrl_shutw()
- ctrl_close()
- }
-
-si_shutr() must do :
- data_shutr()
- if (shutw) {
- data_close()
- ctrl_shutr()
- ctrl_close()
- }
-
-Each of these steps may fail, in which case the step must be retained and the
-operations postponed in an asynchronous task.
-
-The first asynchronous data_shut() might already fail so it is mandatory to
-save the other side's status with the connection in order to let the async task
-know whether the 3 next steps must be performed.
-
-The connection (or perhaps the FD) needs to know :
- - the desired close operations : DSHR, DSHW, CSHR, CSHW
- - the completed close operations : DSHR, DSHW, CSHR, CSHW
-
-
-On the accept() side, we probably need to know :
- - if a header is expected (eg: accept-proxy)
- - if this header is still being waited for
- => maybe both info might be combined into one bit
-
- - if a data-layer accept() is expected
- - if a data-layer accept() has been started
- - if a data-layer accept() has been performed
- => possibly 2 bits, to indicate the need to free()
-
-On the connect() side, we need to know :
- - the desire to send a header (eg: send-proxy)
- - if this header has been sent
- => maybe both info might be combined
-
- - if a data-layer connect() is expected
- - if a data-layer connect() has been started
- - if a data-layer connect() has been completed
- => possibly 2 bits, to indicate the need to free()
-
-On the response side, we also need to know :
- - the desire to send a header (eg: health check response for monitor-net)
- - if this header was sent
- => might be the same as sending a header over a new connection
-
-Note: monitor-net has precedence over proxy proto and data layers. Same for
- health mode.
-
-For multi-step operations, use 2 bits :
- 00 = operation not desired, not performed
- 10 = operation desired, not started
- 11 = operation desired, started but not completed
- 01 = operation desired, started and completed
-
- => X != 00 ==> operation desired
- X & 01 ==> operation at least started
- X & 10 ==> operation not completed
-
-Note: no way to store status information for error reporting.
-
-Note2: it would be nice if "tcp-request connection" rules could work at the
-connection level, just after headers ! This means support for tracking stick
-tables, possibly not too much complicated.
-
-
-Proposal for incoming connection sequence :
-
-- accept()
-- if monitor-net matches or if mode health => try to send response
-- if accept-proxy, wait for proxy request
-- if tcp-request connection, process tcp rules and possibly keep the
- pointer to stick-table
-- if SSL is enabled, switch to SSL handshake
-- then switch to DATA state and instantiate a session
-
-We just need a map of handshake handlers on the connection. They all manage the
-FD status themselves and set the callbacks themselves. If their work succeeds,
-they remove themselves from the list. If it fails, they remain subscribed and
-enable the required polling until they are woken up again or the timeout strikes.
-
-Identified handshake handlers for incoming connections :
- - HH_HEALTH (tries to send OK and dies)
- - HH_MONITOR_IN (matches src IP and adds/removes HH_SEND_OK/HH_SEND_HTTP_OK)
- - HH_SEND_OK (tries to send "OK" and dies)
- - HH_SEND_HTTP_OK (tries to send "HTTP/1.0 200 OK" and dies)
- - HH_ACCEPT_PROXY (waits for PROXY line and parses it)
- - HH_TCP_RULES (processes TCP rules)
- - HH_SSL_HS (starts SSL handshake)
- - HH_ACCEPT_SESSION (instantiates a session)
-
-Identified handshake handlers for outgoing connections :
- - HH_SEND_PROXY (tries to build and send the PROXY line)
- - HH_SSL_HS (starts SSL handshake)
-
-For the pollers, we could check that handshake handlers are not 0 and decide to
-call a generic connection handshake handler instead of usual callbacks. Problem
-is that pollers don't know connections, they know fds. So entities which manage
-handlers should update change the FD callbacks accordingly.
-
-With a bit of care, we could have :
- - HH_SEND_LAST_CHUNK (sends the chunk pointed to by a pointer and dies)
- => merges HEALTH, SEND_OK and SEND_HTTP_OK
-
-It sounds like the ctrl vs data state for the connection are per-direction
-(eg: support an async ctrl shutw while still reading data).
-
-Also support shutr/shutw status at L4/L7.
-
-In practice, what we really need is :
-
-shutdown(conn) =
- conn.data.shut()
- conn.ctrl.shut()
- conn.fd.shut()
-
-close(conn) =
- conn.data.close()
- conn.ctrl.close()
- conn.fd.close()
-
-With SSL over Remote TCP (RTCP + RSSL) to reach the server, we would have :
-
- HTTP -> RTCP+RSSL connection <-> RTCP+RRAW connection -> TCP+SSL connection
-
-The connection has to be closed at 3 places after a successful response :
- - DATA (RSSL over RTCP)
- - CTRL (RTCP to close connection to server)
- - SOCK (FD to close connection to second process)
-
-Externally, the connection is seen with very few flags :
- - SHR
- - SHW
- - ERR
-
-We don't need a CLOSED flag as a connection must always be detached when it's closed.
-
-The internal status doesn't need to be exposed :
- - FD allocated (Y/N)
- - CTRL initialized (Y/N)
- - CTRL connected (Y/N)
- - CTRL handlers done (Y/N)
- - CTRL failed (Y/N)
- - CTRL shutr (Y/N)
- - CTRL shutw (Y/N)
- - DATA initialized (Y/N)
- - DATA connected (Y/N)
- - DATA handlers done (Y/N)
- - DATA failed (Y/N)
- - DATA shutr (Y/N)
- - DATA shutw (Y/N)
-
-(note that having flags for operations needing to be completed might be easier)
---------------
-
-Maybe we need to be able to call conn->fdset() and conn->fdclr() but it sounds
-very unlikely since the only functions manipulating this are in the code of
-the data/ctrl handlers.
-
-FDSET/FDCLR cannot be directly controlled by the stream interface since it also
-depends on the DATA layer (WANT_READ/wANT_WRITE).
-
-But FDSET/FDCLR is probably controlled by who owns the connection (eg: DATA).
-
diff --git a/doc/internals/entities.txt b/doc/internals/entities.txt
deleted file mode 100644
index cdde82e..0000000
--- a/doc/internals/entities.txt
+++ /dev/null
@@ -1,96 +0,0 @@
-2011/02/25 - Description of the different entities in haproxy - w@1wt.eu
-
-
-1) Definitions
---------------
-
-Listener
---------
-
-A listener is the entity which is part of a frontend and which accepts
-connections. There are as many listeners as there are ip:port couples.
-There is at least one listener instantiated for each "bind" entry, and
-port ranges will lead to as many listeners as there are ports in the
-range. A listener just has a listening file descriptor ready to accept
-incoming connections and to dispatch them to upper layers.
-
-
-Initiator
----------
-
-An initiator is instantiated for each incoming connection on a listener. It may
-also be instantiated by a task pretending to be a client. An initiator calls
-the next stage's accept() callback to present it with the parameters of the
-incoming connection.
-
-
-Session
--------
-
-A session is the only entity located between an initiator and a connector.
-This is the last stage which offers an accept() callback, and all of its
-processing will continue with the next stage's connect() callback. It holds
-the buffers needed to forward the protocol data between each side. This entity
-sees the native protocol, and is able to call analysers on these buffers. As it
-is used in both directions, it always has two buffers.
-
-When transformations are required, some of them may be done on the initiator
-side and other ones on the connector side. If additional buffers are needed for
-such transforms, those buffers cannot replace the session's buffers, but they
-may complete them.
-
-A session only needs to be instantiated when forwarding of data is required
-between two sides. Accepting and filtering on layer 4 information only does not
-require a session.
-
-For instance, let's consider the case of a proxy which receives and decodes
-HTTPS traffic, processes it as HTTP and recodes it as HTTPS before forwarding
-it. We'd have 3 layers of buffers, where the middle ones are used for
-forwarding of the protocol data (HTTP here) :
-
- <-- ssl dec --> <-forwarding-> <-- ssl enc -->
-
- ,->[||||]--. ,->[||||]--. ,->[||||]--.
- client (|) (|) (|) (|) server
- ^--[||||]<-' ^--[||||]<-' ^--[||||]<-'
-
- HTTPS HTTP HTTPS
-
-The session handling code is only responsible for monitoring the forwarding
-buffers here. It may declare the end of the session once those buffers are
-closed and no analyser wants to re-open them. The session is also the entity
-which applies the load balancing algorithm and decides the server to use.
-
-The other sides are responsible for propagating the state up to the session
-which takes decisions.
-
-
-Connector
----------
-
-A connector is the entity which permits to instantiate a connection to a known
-destination. It presents a connect() callback, and as such appears on the right
-side of diagrams.
-
-
-Connection
-----------
-
-A connection is the entity instantiated by a connector. It may be composed of
-multiple stages linked together. Generally it is the part of the stream
-interface holding a file descriptor, but it can also be a processing block or a
-transformation block terminated by a connection. A connection presents a
-server-side interface.
-
-
-2) Sequencing
--------------
-
-Upon startup, listeners are instantiated by the configuration. When an incoming
-connection reaches a listening file descriptor, its read() callback calls the
-corresponding listener's accept() function which instantiates an initiator and
-in turn recursively calls upper layers' accept() callbacks until
-accept_session() is called. accept_session() instantiates a new session which
-starts protocol analysis via process_session(). When all protocol analysis is
-done, process_session() calls the connect() callback of the connector in order
-to get a connection.
diff --git a/doc/internals/header-parser-speed.txt b/doc/internals/header-parser-speed.txt
deleted file mode 100644
index 285e2fa..0000000
--- a/doc/internals/header-parser-speed.txt
+++ /dev/null
@@ -1,92 +0,0 @@
-TEST 3:
-
- printf "GET /\r\nbla: truc\r\n\r\n"
-
-
-NO SPEEDUP :
-
-WHL: hdr_st=0x00, hdr_used=1 hdr_tail=0 hdr_last=1, h=0x8071080, lr=0x8071080, r=0x8071094
-WHL: hdr_st=0x01, hdr_used=1 hdr_tail=0 hdr_last=1, h=0x8071080, lr=0x8071080, r=0x8071094
-WHL: hdr_st=0x32, hdr_used=1 hdr_tail=0 hdr_last=1, h=0x8071080, lr=0x8071086, r=0x8071094
-WHL: hdr_st=0x03, hdr_used=2 hdr_tail=1 hdr_last=2, h=0x8071087, lr=0x8071087, r=0x8071094
-WHL: hdr_st=0x34, hdr_used=2 hdr_tail=1 hdr_last=2, h=0x8071087, lr=0x8071091, r=0x8071094
-WHL: hdr_st=0x03, hdr_used=3 hdr_tail=2 hdr_last=3, h=0x8071092, lr=0x8071092, r=0x8071094
-WHL: hdr_st=0x34, hdr_used=3 hdr_tail=2 hdr_last=3, h=0x8071092, lr=0x8071093, r=0x8071094
-WHL: hdr_st=0x06, hdr_used=3 hdr_tail=2 hdr_last=3, h=0x8071092, lr=0x8071093, r=0x8071094
-END: hdr_st=0x06, hdr_used=3 hdr_tail=2 hdr_last=3, h=0x8071092, lr=0x8071094, r=0x8071094
-=> 9 trans
-
-
-FULL SPEEDUP :
-
-WHL: hdr_st=0x00, hdr_used=1 hdr_tail=0 hdr_last=1, h=0x806a770, lr=0x806a770, r=0x806a784
-WHL: hdr_st=0x32, hdr_used=1 hdr_tail=0 hdr_last=1, h=0x806a770, lr=0x806a776, r=0x806a784
-WHL: hdr_st=0x03, hdr_used=2 hdr_tail=1 hdr_last=2, h=0x806a777, lr=0x806a777, r=0x806a784
-WHL: hdr_st=0x34, hdr_used=2 hdr_tail=1 hdr_last=2, h=0x806a777, lr=0x806a781, r=0x806a784
-WHL: hdr_st=0x26, hdr_used=3 hdr_tail=2 hdr_last=3, h=0x806a782, lr=0x806a783, r=0x806a784
-END: hdr_st=0x06, hdr_used=3 hdr_tail=2 hdr_last=3, h=0x806a782, lr=0x806a784, r=0x806a784
-=> 6 trans
-
-
-
-TEST 4:
-
-
- printf "GET /\nbla: truc\n\n"
-
-
-NO SPEEDUP :
-
-WHL: hdr_st=0x00, hdr_used=1 hdr_tail=0 hdr_last=1, h=0x80750d0, lr=0x80750d0, r=0x80750e1
-WHL: hdr_st=0x01, hdr_used=1 hdr_tail=0 hdr_last=1, h=0x80750d0, lr=0x80750d0, r=0x80750e1
-WHL: hdr_st=0x02, hdr_used=1 hdr_tail=0 hdr_last=1, h=0x80750d0, lr=0x80750d5, r=0x80750e1
-WHL: hdr_st=0x03, hdr_used=2 hdr_tail=1 hdr_last=2, h=0x80750d6, lr=0x80750d6, r=0x80750e1
-WHL: hdr_st=0x04, hdr_used=2 hdr_tail=1 hdr_last=2, h=0x80750d6, lr=0x80750df, r=0x80750e1
-WHL: hdr_st=0x03, hdr_used=3 hdr_tail=2 hdr_last=3, h=0x80750e0, lr=0x80750e0, r=0x80750e1
-WHL: hdr_st=0x04, hdr_used=3 hdr_tail=2 hdr_last=3, h=0x80750e0, lr=0x80750e0, r=0x80750e1
-WHL: hdr_st=0x06, hdr_used=3 hdr_tail=2 hdr_last=3, h=0x80750e0, lr=0x80750e0, r=0x80750e1
-END: hdr_st=0x06, hdr_used=3 hdr_tail=2 hdr_last=3, h=0x80750e0, lr=0x80750e1, r=0x80750e1
-=> 9 trans
-
-
-FULL SPEEDUP :
-
-WHL: hdr_st=0x00, hdr_used=1 hdr_tail=0 hdr_last=1, h=0x8072010, lr=0x8072010, r=0x8072021
-WHL: hdr_st=0x03, hdr_used=2 hdr_tail=1 hdr_last=2, h=0x8072016, lr=0x8072016, r=0x8072021
-END: hdr_st=0x06, hdr_used=3 hdr_tail=2 hdr_last=3, h=0x8072020, lr=0x8072021, r=0x8072021
-=> 3 trans
-
-
-TEST 5:
-
-
- printf "GET /\r\nbla: truc\r\n truc2\r\n\r\n"
-
-
-NO SPEEDUP :
-
-WHL: hdr_st=0x00, hdr_used=1 hdr_tail=0 hdr_last=1, h=0x8071080, lr=0x8071080, r=0x807109d
-WHL: hdr_st=0x01, hdr_used=1 hdr_tail=0 hdr_last=1, h=0x8071080, lr=0x8071080, r=0x807109d
-WHL: hdr_st=0x32, hdr_used=1 hdr_tail=0 hdr_last=1, h=0x8071080, lr=0x8071086, r=0x807109d
-WHL: hdr_st=0x03, hdr_used=2 hdr_tail=1 hdr_last=2, h=0x8071087, lr=0x8071087, r=0x807109d
-WHL: hdr_st=0x34, hdr_used=2 hdr_tail=1 hdr_last=2, h=0x8071087, lr=0x8071091, r=0x807109d
-WHL: hdr_st=0x05, hdr_used=2 hdr_tail=1 hdr_last=2, h=0x8071087, lr=0x8071092, r=0x807109d
-WHL: hdr_st=0x03, hdr_used=2 hdr_tail=1 hdr_last=2, h=0x8071087, lr=0x8071094, r=0x807109d
-WHL: hdr_st=0x34, hdr_used=2 hdr_tail=1 hdr_last=2, h=0x8071087, lr=0x807109a, r=0x807109d
-WHL: hdr_st=0x03, hdr_used=3 hdr_tail=2 hdr_last=3, h=0x807109b, lr=0x807109b, r=0x807109d
-WHL: hdr_st=0x34, hdr_used=3 hdr_tail=2 hdr_last=3, h=0x807109b, lr=0x807109c, r=0x807109d
-WHL: hdr_st=0x06, hdr_used=3 hdr_tail=2 hdr_last=3, h=0x807109b, lr=0x807109c, r=0x807109d
-END: hdr_st=0x06, hdr_used=3 hdr_tail=2 hdr_last=3, h=0x807109b, lr=0x807109d, r=0x807109d
-=> 12 trans
-
-
-FULL SPEEDUP :
-
-WHL: hdr_st=0x00, hdr_used=1 hdr_tail=0 hdr_last=1, h=0x806dfc0, lr=0x806dfc0, r=0x806dfdd
-WHL: hdr_st=0x32, hdr_used=1 hdr_tail=0 hdr_last=1, h=0x806dfc0, lr=0x806dfc6, r=0x806dfdd
-WHL: hdr_st=0x03, hdr_used=2 hdr_tail=1 hdr_last=2, h=0x806dfc7, lr=0x806dfc7, r=0x806dfdd
-WHL: hdr_st=0x34, hdr_used=2 hdr_tail=1 hdr_last=2, h=0x806dfc7, lr=0x806dfd1, r=0x806dfdd
-WHL: hdr_st=0x34, hdr_used=2 hdr_tail=1 hdr_last=2, h=0x806dfc7, lr=0x806dfda, r=0x806dfdd
-WHL: hdr_st=0x26, hdr_used=3 hdr_tail=2 hdr_last=3, h=0x806dfdb, lr=0x806dfdc, r=0x806dfdd
-END: hdr_st=0x06, hdr_used=3 hdr_tail=2 hdr_last=3, h=0x806dfdb, lr=0x806dfdd, r=0x806dfdd
-=> 7 trans
diff --git a/doc/internals/header-tree.txt b/doc/internals/header-tree.txt
deleted file mode 100644
index 9a97361..0000000
--- a/doc/internals/header-tree.txt
+++ /dev/null
@@ -1,124 +0,0 @@
-2007/03/30 - Header storage in trees
-
-This documentation describes how to store headers in radix trees, providing
-fast access to any known position, while retaining the ability to grow/reduce
-any arbitrary header without having to recompute all positions.
-
-Principle :
- We have a radix tree represented in an integer array, which represents the
- total number of bytes used by all headers whose position is below it. This
- ensures that we can compute any header's position in O(log(N)) where N is
- the number of headers.
-
-Example with N=16 :
-
- +-----------------------+
- | |
- +-----------+ +-----------+
- | | | |
- +-----+ +-----+ +-----+ +-----+
- | | | | | | | |
- +--+ +--+ +--+ +--+ +--+ +--+ +--+ +--+
- | | | | | | | | | | | | | | | |
-
- 0 1 2 3 4 5 6 7 8 9 A B C D E F
-
- To reach header 6, we have to compute hdr[0]+hdr[4]+hdr[6]
-
- With this method, it becomes easy to grow any header and update the array.
- To achieve this, we have to replace one after the other all bits on the
- right with one 1 followed by zeroes, and update the position if it's higher
- than current position, and stop when it's above number of stored headers.
-
- For instance, if we want to grow hdr[6], we proceed like this :
-
- 6 = 0110 (BIN)
-
- Let's consider the values to update :
-
- (bit 0) : (0110 & ~0001) | 0001 = 0111 = 7 > 6 => update
- (bit 1) : (0110 & ~0011) | 0010 = 0110 = 6 <= 6 => leave it
- (bit 2) : (0110 & ~0111) | 0100 = 0100 = 4 <= 6 => leave it
- (bit 4) : (0110 & ~1111) | 1000 = 1000 = 8 > 6 => update
- (bit 5) : larger than array size, stop.
-
-
-It's easy to walk through the tree too. We only have one iteration per bit
-changing from X to the ancestor, and one per bit from the ancestor to Y.
-The ancestor is found while walking. To go from X to Y :
-
- pos = pos(X)
-
- while (Y != X) {
- if (Y > X) {
- // walk from Y to ancestor
- pos += hdr[Y]
- Y &= (Y - 1)
- } else {
- // walk from X to ancestor
- pos -= hdr[X]
- X &= (X - 1)
- }
- }
-
-However, it is not trivial anymore to linearly walk the tree. We have to move
-from a known place to another known place, but a jump to next entry costs the
-same as a jump to a random place.
-
-Other caveats :
- - it is not possible to remove a header, it is only possible to empty it.
- - it is not possible to insert a header, as that would imply a renumbering.
- => this means that a "defrag" function is required. Headers should preferably
- be added, then should be stuffed on top of destroyed ones, then only
- inserted if absolutely required.
-
-
-When we have this, we can then focus on a 32-bit header descriptor which would
-look like this :
-
-{
- unsigned line_len :13; /* total line length, including CRLF */
- unsigned name_len :6; /* header name length, max 63 chars */
- unsigned sp1 :5; /* max spaces before value : 31 */
- unsigned sp2 :8; /* max spaces after value : 255 */
-}
-
-Example :
-
- Connection: close \r\n
- <---------+-----+-----+-------------> line_len
- <-------->| | | name_len
- <-----> | sp1
- <-------------> sp2
-Rem:
- - if there are more than 31 spaces before the value, the buffer will have to
- be moved before being registered
-
- - if there are more than 255 spaces after the value, the buffer will have to
- be moved before being registered
-
- - we can use the empty header name as an indicator for a deleted header
-
- - it would be wise to format a new request before sending lots of random
- spaces to the servers.
-
- - normal clients do not send such crap, so those operations *may* reasonably
- be more expensive than the rest provided that other ones are very fast.
-
-It would be handy to have the following macros :
-
- hdr_eon(hdr) => end of name
- hdr_sov(hdr) => start of value
- hdr_eof(hdr) => end of value
- hdr_vlen(hdr) => length of value
- hdr_hlen(hdr) => total header length
-
-
-A 48-bit encoding would look like this :
-
- Connection: close \r\n
- <---------+------+---+--------------> eoh = 16 bits
- <-------->| | | eon = 8 bits
- <--------------->| | sov = 8 bits
- <---> vlen = 16 bits
-
diff --git a/doc/internals/http-cookies.txt b/doc/internals/http-cookies.txt
deleted file mode 100644
index 6d65c54..0000000
--- a/doc/internals/http-cookies.txt
+++ /dev/null
@@ -1,45 +0,0 @@
-2010/08/31 - HTTP Cookies - Theory and reality
-
-HTTP cookies are not uniformly supported across browsers, which makes it very
-hard to build a widely compatible implementation. At least four conflicting
-documents exist to describe how cookies should be handled, and browsers
-generally don't respect any but a sensibly selected mix of them :
-
- - Netscape's original spec (also mirrored at Curl's site among others) :
- http://web.archive.org/web/20070805052634/http://wp.netscape.com/newsref/std/cookie_spec.html
- http://curl.haxx.se/rfc/cookie_spec.html
-
- Issues: uses an unquoted "Expires" field that includes a comma.
-
- - RFC 2109 :
- http://www.ietf.org/rfc/rfc2109.txt
-
- Issues: specifies use of "Max-Age" (not universally implemented) and does
- not talk about "Expires" (generally supported). References quoted
- strings, not generally supported (eg: MSIE). Stricter than browsers
- about domains. Ambiguous about allowed spaces in values and attrs.
-
- - RFC 2965 :
- http://www.ietf.org/rfc/rfc2965.txt
-
- Issues: same as RFC2109 + describes Set-Cookie2 which only Opera supports.
-
- - Current internet draft :
- https://datatracker.ietf.org/wg/httpstate/charter/
-
- Issues: as of -p10, does not explain how the Set-Cookie2 header must be
- emitted/handled, while suggesting a stricter approach for Cookie.
- Documents reality and as such reintroduces the widely used unquoted
- "Expires" attribute with its error-prone syntax. States that a
- server should not emit more than one cookie per Set-Cookie header,
- which is incompatible with HTTP which says that multiple headers
- are allowed only if they can be folded.
-
-See also the following URL for a browser * feature matrix :
- http://code.google.com/p/browsersec/wiki/Part2#Same-origin_policy_for_cookies
-
-In short, MSIE and Safari neither support quoted strings nor max-age, which
-make it mandatory to continue to send an unquoted Expires value (maybe the
-day of week could be omitted though). Only Safari supports comma-separated
-lists of Set-Cookie headers. Support for cross-domains is not uniform either.
-
diff --git a/doc/internals/http-docs.txt b/doc/internals/http-docs.txt
deleted file mode 100644
index 4ed2480..0000000
--- a/doc/internals/http-docs.txt
+++ /dev/null
@@ -1,5 +0,0 @@
-Many interesting RFC and drafts linked to from this site :
-
- http://www.web-cache.com/Writings/protocols-standards.html
-
-
diff --git a/doc/internals/http-parsing.txt b/doc/internals/http-parsing.txt
deleted file mode 100644
index 8b3f239..0000000
--- a/doc/internals/http-parsing.txt
+++ /dev/null
@@ -1,335 +0,0 @@
---- Relevant portions of RFC2616 ---
-
-OCTET = <any 8-bit sequence of data>
-CHAR = <any US-ASCII character (octets 0 - 127)>
-UPALPHA = <any US-ASCII uppercase letter "A".."Z">
-LOALPHA = <any US-ASCII lowercase letter "a".."z">
-ALPHA = UPALPHA | LOALPHA
-DIGIT = <any US-ASCII digit "0".."9">
-CTL = <any US-ASCII control character (octets 0 - 31) and DEL (127)>
-CR = <US-ASCII CR, carriage return (13)>
-LF = <US-ASCII LF, linefeed (10)>
-SP = <US-ASCII SP, space (32)>
-HT = <US-ASCII HT, horizontal-tab (9)>
-<"> = <US-ASCII double-quote mark (34)>
-CRLF = CR LF
-LWS = [CRLF] 1*( SP | HT )
-TEXT = <any OCTET except CTLs, but including LWS>
-HEX = "A" | "B" | "C" | "D" | "E" | "F"
- | "a" | "b" | "c" | "d" | "e" | "f" | DIGIT
-separators = "(" | ")" | "<" | ">" | "@"
- | "," | ";" | ":" | "\" | <">
- | "/" | "[" | "]" | "?" | "="
- | "{" | "}" | SP | HT
-token = 1*<any CHAR except CTLs or separators>
-
-quoted-pair = "\" CHAR
-ctext = <any TEXT excluding "(" and ")">
-qdtext = <any TEXT except <">>
-quoted-string = ( <"> *(qdtext | quoted-pair ) <"> )
-comment = "(" *( ctext | quoted-pair | comment ) ")"
-
-
-
-
-
-4 HTTP Message
-4.1 Message Types
-
-HTTP messages consist of requests from client to server and responses from
-server to client. Request (section 5) and Response (section 6) messages use the
-generic message format of RFC 822 [9] for transferring entities (the payload of
-the message). Both types of message consist of :
-
- - a start-line
- - zero or more header fields (also known as "headers")
- - an empty line (i.e., a line with nothing preceding the CRLF) indicating the
- end of the header fields
- - and possibly a message-body.
-
-
-HTTP-message = Request | Response
-
-start-line = Request-Line | Status-Line
-generic-message = start-line
- *(message-header CRLF)
- CRLF
- [ message-body ]
-
-In the interest of robustness, servers SHOULD ignore any empty line(s) received
-where a Request-Line is expected. In other words, if the server is reading the
-protocol stream at the beginning of a message and receives a CRLF first, it
-should ignore the CRLF.
-
-
-4.2 Message headers
-
-- Each header field consists of a name followed by a colon (":") and the field
- value.
-- Field names are case-insensitive.
-- The field value MAY be preceded by any amount of LWS, though a single SP is
- preferred.
-- Header fields can be extended over multiple lines by preceding each extra
- line with at least one SP or HT.
-
-
-message-header = field-name ":" [ field-value ]
-field-name = token
-field-value = *( field-content | LWS )
-field-content = <the OCTETs making up the field-value and consisting of
- either *TEXT or combinations of token, separators, and
- quoted-string>
-
-
-The field-content does not include any leading or trailing LWS occurring before
-the first non-whitespace character of the field-value or after the last
-non-whitespace character of the field-value. Such leading or trailing LWS MAY
-be removed without changing the semantics of the field value. Any LWS that
-occurs between field-content MAY be replaced with a single SP before
-interpreting the field value or forwarding the message downstream.
-
-
-=> format des headers = 1*(CHAR & !ctl & !sep) ":" *(OCTET & (!ctl | LWS))
-=> les regex de matching de headers s'appliquent sur field-content, et peuvent
- utiliser field-value comme espace de travail (mais de préférence après le
- premier SP).
-
-(19.3) The line terminator for message-header fields is the sequence CRLF.
-However, we recommend that applications, when parsing such headers, recognize
-a single LF as a line terminator and ignore the leading CR.
-
-
-
-
-
-message-body = entity-body
- | <entity-body encoded as per Transfer-Encoding>
-
-
-
-5 Request
-
-Request = Request-Line
- *(( general-header
- | request-header
- | entity-header ) CRLF)
- CRLF
- [ message-body ]
-
-
-
-5.1 Request line
-
-The elements are separated by SP characters. No CR or LF is allowed except in
-the final CRLF sequence.
-
-Request-Line = Method SP Request-URI SP HTTP-Version CRLF
-
-(19.3) Clients SHOULD be tolerant in parsing the Status-Line and servers
-tolerant when parsing the Request-Line. In particular, they SHOULD accept any
-amount of SP or HT characters between fields, even though only a single SP is
-required.
-
-4.5 General headers
-Apply to MESSAGE.
-
-general-header = Cache-Control
- | Connection
- | Date
- | Pragma
- | Trailer
- | Transfer-Encoding
- | Upgrade
- | Via
- | Warning
-
-General-header field names can be extended reliably only in combination with a
-change in the protocol version. However, new or experimental header fields may
-be given the semantics of general header fields if all parties in the
-communication recognize them to be general-header fields. Unrecognized header
-fields are treated as entity-header fields.
-
-
-
-
-5.3 Request Header Fields
-
-The request-header fields allow the client to pass additional information about
-the request, and about the client itself, to the server. These fields act as
-request modifiers, with semantics equivalent to the parameters on a programming
-language method invocation.
-
-request-header = Accept
- | Accept-Charset
- | Accept-Encoding
- | Accept-Language
- | Authorization
- | Expect
- | From
- | Host
- | If-Match
- | If-Modified-Since
- | If-None-Match
- | If-Range
- | If-Unmodified-Since
- | Max-Forwards
- | Proxy-Authorization
- | Range
- | Referer
- | TE
- | User-Agent
-
-Request-header field names can be extended reliably only in combination with a
-change in the protocol version. However, new or experimental header fields MAY
-be given the semantics of request-header fields if all parties in the
-communication recognize them to be request-header fields. Unrecognized header
-fields are treated as entity-header fields.
-
-
-
-7.1 Entity header fields
-
-Entity-header fields define metainformation about the entity-body or, if no
-body is present, about the resource identified by the request. Some of this
-metainformation is OPTIONAL; some might be REQUIRED by portions of this
-specification.
-
-entity-header = Allow
- | Content-Encoding
- | Content-Language
- | Content-Length
- | Content-Location
- | Content-MD5
- | Content-Range
- | Content-Type
- | Expires
- | Last-Modified
- | extension-header
-extension-header = message-header
-
-The extension-header mechanism allows additional entity-header fields to be
-defined without changing the protocol, but these fields cannot be assumed to be
-recognizable by the recipient. Unrecognized header fields SHOULD be ignored by
-the recipient and MUST be forwarded by transparent proxies.
-
-----------------------------------
-
-The format of Request-URI is defined by RFC3986 :
-
- URI = scheme ":" hier-part [ "?" query ] [ "#" fragment ]
-
- hier-part = "//" authority path-abempty
- / path-absolute
- / path-rootless
- / path-empty
-
- URI-reference = URI / relative-ref
-
- absolute-URI = scheme ":" hier-part [ "?" query ]
-
- relative-ref = relative-part [ "?" query ] [ "#" fragment ]
-
- relative-part = "//" authority path-abempty
- / path-absolute
- / path-noscheme
- / path-empty
-
- scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )
-
- authority = [ userinfo "@" ] host [ ":" port ]
- userinfo = *( unreserved / pct-encoded / sub-delims / ":" )
- host = IP-literal / IPv4address / reg-name
- port = *DIGIT
-
- IP-literal = "[" ( IPv6address / IPvFuture ) "]"
-
- IPvFuture = "v" 1*HEXDIG "." 1*( unreserved / sub-delims / ":" )
-
- IPv6address = 6( h16 ":" ) ls32
- / "::" 5( h16 ":" ) ls32
- / [ h16 ] "::" 4( h16 ":" ) ls32
- / [ *1( h16 ":" ) h16 ] "::" 3( h16 ":" ) ls32
- / [ *2( h16 ":" ) h16 ] "::" 2( h16 ":" ) ls32
- / [ *3( h16 ":" ) h16 ] "::" h16 ":" ls32
- / [ *4( h16 ":" ) h16 ] "::" ls32
- / [ *5( h16 ":" ) h16 ] "::" h16
- / [ *6( h16 ":" ) h16 ] "::"
-
- h16 = 1*4HEXDIG
- ls32 = ( h16 ":" h16 ) / IPv4address
- IPv4address = dec-octet "." dec-octet "." dec-octet "." dec-octet
- dec-octet = DIGIT ; 0-9
- / %x31-39 DIGIT ; 10-99
- / "1" 2DIGIT ; 100-199
- / "2" %x30-34 DIGIT ; 200-249
- / "25" %x30-35 ; 250-255
-
- reg-name = *( unreserved / pct-encoded / sub-delims )
-
- path = path-abempty ; begins with "/" or is empty
- / path-absolute ; begins with "/" but not "//"
- / path-noscheme ; begins with a non-colon segment
- / path-rootless ; begins with a segment
- / path-empty ; zero characters
-
- path-abempty = *( "/" segment )
- path-absolute = "/" [ segment-nz *( "/" segment ) ]
- path-noscheme = segment-nz-nc *( "/" segment )
- path-rootless = segment-nz *( "/" segment )
- path-empty = 0<pchar>
-
- segment = *pchar
- segment-nz = 1*pchar
- segment-nz-nc = 1*( unreserved / pct-encoded / sub-delims / "@" )
- ; non-zero-length segment without any colon ":"
-
- pchar = unreserved / pct-encoded / sub-delims / ":" / "@"
-
- query = *( pchar / "/" / "?" )
-
- fragment = *( pchar / "/" / "?" )
-
- pct-encoded = "%" HEXDIG HEXDIG
-
- unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~"
- reserved = gen-delims / sub-delims
- gen-delims = ":" / "/" / "?" / "#" / "[" / "]" / "@"
- sub-delims = "!" / "$" / "&" / "'" / "(" / ")"
- / "*" / "+" / "," / ";" / "="
-
-=> so the list of allowed characters in a URI is :
-
- uri-char = unreserved / gen-delims / sub-delims / "%"
- = ALPHA / DIGIT / "-" / "." / "_" / "~"
- / ":" / "/" / "?" / "#" / "[" / "]" / "@"
- / "!" / "$" / "&" / "'" / "(" / ")" /
- / "*" / "+" / "," / ";" / "=" / "%"
-
-Note that non-ascii characters are forbidden ! Spaces and CTL are forbidden.
-Unfortunately, some products such as Apache allow such characters :-/
-
----- The correct way to do it ----
-
-- one http_session
- It is basically any transport session on which we talk HTTP. It may be TCP,
- SSL over TCP, etc... It knows a way to talk to the client, either the socket
- file descriptor or a direct access to the client-side buffer. It should hold
- information about the last accessed server so that we can guarantee that the
- same server can be used during a whole session if needed. A first version
- without optimal support for HTTP pipelining will have the client buffers tied
- to the http_session. It may be possible that it is not sufficient for full
- pipelining, but this will need further study. The link from the buffers to
- the backend should be managed by the http transaction (http_txn), provided
- that they are serialized. Each http_session, has 0 to N http_txn. Each
- http_txn belongs to one and only one http_session.
-
-- each http_txn has 1 request message (http_req), and 0 or 1 response message
- (http_rtr). Each of them has 1 and only one http_txn. An http_txn holds
- information such as the HTTP method, the URI, the HTTP version, the
- transfer-encoding, the HTTP status, the authorization, the req and rtr
- content-length, the timers, logs, etc... The backend and server which process
- the request are also known from the http_txn.
-
-- both request and response messages hold header and parsing information, such
- as the parsing state, start of headers, start of message, captures, etc...
-
diff --git a/doc/internals/naming.txt b/doc/internals/naming.txt
deleted file mode 100644
index 0c3d5b5..0000000
--- a/doc/internals/naming.txt
+++ /dev/null
@@ -1,54 +0,0 @@
-Naming rules for manipulated objects and structures.
-
-Previously, there were ambiguities between sessions, transactions and requests,
-as well as in the way responses are noted ("resp", "rep", "rsp").
-
-Here is a proposal for a better naming scheme.
-
-The "session" is above the transport level, which means at ISO layer 5.
-We can talk about "http sessions" when we consider the entity which lives
-between the accept() and the close(), or the connect() and the close().
-
-=> This demonstrates that it is not possible to have the same http session from
- the client to the server.
-
-A session can carry one or multiple "transactions", which are each composed of
-one "request" and zero or one "response". Both "request" and "response" are
-described in RFC2616 as "HTTP messages". RFC2616 also seldom references the
-word "transaction" without explicitly defining it.
-
-An "HTTP message" is composed of a "start line" which can be either a
-"request line" or a "status line", followed by a number of "message headers"
-which can be either "request headers" or "response headers", and an "entity",
-itself composed of "entity headers" and an "entity body".Most probably,
-"message headers" and "entity headers" will always be processed together as
-"headers", while the "entity body" will design the payload.
-
-We must try to always use the same abbreviations when naming objects. Here are
-a few common ones :
-
- - txn : transaction
- - req : request
- - rtr : response to request
- - msg : message
- - hdr : header
- - ent : entity
- - bdy : body
- - sts : status
- - stt : state
- - idx : index
- - cli : client
- - srv : server
- - svc : service
- - ses : session
- - tsk : task
-
-Short names for unions or cascaded structs :
- - sl : start line
- - sl.rq : request line
- - sl.st : status line
- - cl : client
- - px : proxy
- - sv : server
- - st : state / status
-
diff --git a/doc/internals/repartition-be-fe-fi.txt b/doc/internals/repartition-be-fe-fi.txt
deleted file mode 100644
index 261d073..0000000
--- a/doc/internals/repartition-be-fe-fi.txt
+++ /dev/null
@@ -1,20 +0,0 @@
-- session : ajouter ->fiprm et ->beprm comme raccourcis
-- px->maxconn: ne s'applique qu'au FE. Pour le BE, on utilise fullconn,
- initialisé par défaut à la même chose que maxconn.
-
-
- \ from: proxy session server actuellement
-field \
-rules px->fiprm sess->fiprm -
-srv,cookies px->beprm sess->beprm srv->px
-options(log) px-> sess->fe -
-options(fe) px-> sess->fe -
-options(be) px->beprm sess->beprm srv->px
-captures px-> sess->fe - ->fiprm
-
-
-logs px-> sess->fe srv->px
-errorloc px-> sess->beprm|fe -
-maxconn px-> sess->fe - ->be
-fullconn px-> sess->beprm srv->px -
-