Gitiles
Code Review Sign In
git01.mediatek.com / haproxy / 4a5c44a3981bdb51912a0ef2f04035d0f677f59f / . / doc / internals / filters.txt
blob: d1e0c4da18aa1cfb7da9bf7d53dd551d64d9b99c [file] [log] [blame]
-----------------------------------------
Filters Guide - version 2.0
( Last update: 2017-07-27 )
------------------------------------------
Author : Christopher Faulet
Contact : christopher dot faulet at capflam dot org
ABSTRACT
--------
The filters support is a new feature of HAProxy 1.7. It is a way to extend
HAProxy without touching its core code and, in certain extent, without knowing
its internals. This feature will ease contributions, reducing impact of
changes. Another advantage will be to simplify HAProxy by replacing some parts
by filters. As we will see, and as an example, the HTTP compression is the first
feature moved in a filter.
This document describes how to write a filter and what you have to keep in mind
to do so. It also talks about the known limits and the pitfalls to avoid.
As said, filters are quite new for now. The API is not freezed and will be
updated/modified/improved/extended as needed.
SUMMARY
-------
1. Filters introduction
2. How to use filters
3. How to write a new filter
3.1. API Overview
3.2. Defining the filter name and its configuration
3.3. Managing the filter lifecycle
3.3.1. Dealing with threads
3.4. Handling the streams activity
3.5. Analyzing the channels activity
3.6. Filtering the data exchanged
4. FAQ
1. FILTERS INTRODUCTION
-----------------------
First of all, to fully understand how filters work and how to create one, it is
best to know, at least from a distance, what is a proxy (frontend/backend), a
stream and a channel in HAProxy and how these entities are linked to each other.
doc/internals/entities.pdf is a good overview.
Then, to support filters, many callbacks has been added to HAProxy at different
places, mainly around channel analyzers. Their purpose is to allow filters to
be involved in the data processing, from the stream creation/destruction to
the data forwarding. Depending of what it should do, a filter can implement all
or part of these callbacks. For now, existing callbacks are focused on
streams. But future improvements could enlarge filters scope. For example, it
could be useful to handle events at the connection level.
In HAProxy configuration file, a filter is declared in a proxy section, except
default. So the configuration corresponding to a filter declaration is attached
to a specific proxy, and will be shared by all its instances. it is opaque from
the HAProxy point of view, this is the filter responsibility to manage it. For
each filter declaration matches a uniq configuration. Several declarations of
the same filter in the same proxy will be handle as different filters by
HAProxy.
A filter instance is represented by a partially opaque context (or a state)
attached to a stream and passed as arguments to callbacks. Through this context,
filter instances are stateful. Depending the filter is declared in a frontend or
a backend section, its instances will be created, respectively, when a stream is
created or when a backend is selected. Their behaviors will also be
different. Only instances of filters declared in a frontend section will be
aware of the creation and the destruction of the stream, and will take part in
the channels analyzing before the backend is defined.
It is important to remember the configuration of a filter is shared by all its
instances, while the context of an instance is owned by a uniq stream.
Filters are designed to be chained. It is possible to declare several filters in
the same proxy section. The declaration order is important because filters will
be called one after the other respecting this order. Frontend and backend
filters are also chained, frontend ones called first. Even if the filters
processing is serialized, each filter will bahave as it was alone (unless it was
developed to be aware of other filters). For all that, some constraints are
imposed to filters, especially when data exchanged between the client and the
server are processed. We will discuss again these constraints when we will tackle
the subject of writing a filter.
2. HOW TO USE FILTERS
---------------------
To use a filter, you must use the parameter 'filter' followed by the filter name
and, optionnaly, its configuration in the desired listen, frontend or backend
section. For example:
listen test
...
filter trace name TST
...
See doc/configuration.txt for a formal definition of the parameter 'filter'.
Note that additional parameters on the filter line must be parsed by the filter
itself.
The list of available filters is reported by 'haproxy -vv':
$> haproxy -vv
HA-Proxy version 1.7-dev2-3a1d4a-33 2016/03/21
Copyright 2000-2016 Willy Tarreau <willy@haproxy.org>
[...]
Available filters :
[COMP] compression
[TRACE] trace
Multiple filter lines can be used in a proxy section to chain filters. Filters
will be called in the declaration order.
Some filters can support implicit declarations in certain circumstances
(without the filter line). This is not recommended for new features but are
useful for existing ones moved in a filter, for backward compatibility
reasons. Implicit declarations are supported when there is only one filter used
on a proxy. When several filters are used, explicit declarations are mandatory.
The HTTP compression filter is one of these filters. Alone, using 'compression'
keywords is enough to use it. But when at least a second filter is used, a
filter line must be added.
# filter line is optionnal
listen t1
bind *:80
compression algo gzip
compression offload
server srv x.x.x.x:80
# filter line is mandatory for the compression filter
listen t2
bind *:81
filter trace name T2
filter compression
compression algo gzip
compression offload
server srv x.x.x.x:80
3. HOW TO WRITE A NEW FILTER
----------------------------
If you want to write a filter, there are 2 header files that you must know:
* include/types/filters.h: This is the main header file, containing all
important structures you will use. It represents
the filter API.
* include/proto/filters.h: This header file contains helper functions that
you may need to use. It also contains the internal
API used by HAProxy to handle filters.
To ease the filters integration, it is better to follow some conventions:
* Use 'flt_' prefix to name your filter (e.g: flt_http_comp or flt_trace).
* Keep everything related to your filter in a same file.
The filter 'trace' can be used as a template to write your own filter. It is a
good start to see how filters really work.
3.1 API OVERVIEW
----------------
Writing a filter can be summarized to write functions and attach them to the
existing callbacks. Available callbacks are listed in the following structure:
struct flt_ops {
/*
* Callbacks to manage the filter lifecycle
*/
int (*init) (struct proxy *p, struct flt_conf *fconf);
void (*deinit) (struct proxy *p, struct flt_conf *fconf);
int (*check) (struct proxy *p, struct flt_conf *fconf);
int (*init_per_thread) (struct proxy *p, struct flt_conf *fconf);
void (*deinit_per_thread)(struct proxy *p, struct flt_conf *fconf);
/*
* Stream callbacks
*/
int (*attach) (struct stream *s, struct filter *f);
int (*stream_start) (struct stream *s, struct filter *f);
int (*stream_set_backend)(struct stream *s, struct filter *f, struct proxy *be);
void (*stream_stop) (struct stream *s, struct filter *f);
void (*detach) (struct stream *s, struct filter *f);
void (*check_timeouts) (struct stream *s, struct filter *f);
/*
* Channel callbacks
*/
int (*channel_start_analyze)(struct stream *s, struct filter *f,
struct channel *chn);
int (*channel_pre_analyze) (struct stream *s, struct filter *f,
struct channel *chn,
unsigned int an_bit);
int (*channel_post_analyze) (struct stream *s, struct filter *f,
struct channel *chn,
unsigned int an_bit);
int (*channel_end_analyze) (struct stream *s, struct filter *f,
struct channel *chn);
/*
* HTTP callbacks
*/
int (*http_headers) (struct stream *s, struct filter *f,
struct http_msg *msg);
int (*http_data) (struct stream *s, struct filter *f,
struct http_msg *msg);
int (*http_chunk_trailers)(struct stream *s, struct filter *f,
struct http_msg *msg);
int (*http_end) (struct stream *s, struct filter *f,
struct http_msg *msg);
int (*http_forward_data) (struct stream *s, struct filter *f,
struct http_msg *msg,
unsigned int len);
void (*http_reset) (struct stream *s, struct filter *f,
struct http_msg *msg);
void (*http_reply) (struct stream *s, struct filter *f,
short status,
const struct buffer *msg);
/*
* TCP callbacks
*/
int (*tcp_data) (struct stream *s, struct filter *f,
struct channel *chn);
int (*tcp_forward_data)(struct stream *s, struct filter *f,
struct channel *chn,
unsigned int len);
};
We will explain in following parts when these callbacks are called and what they
should do.
Filters are declared in proxy sections. So each proxy have an ordered list of
filters, possibly empty if no filter is used. When the configuration of a proxy
is parsed, each filter line represents an entry in this list. In the structure
'proxy', the filters configurations are stored in the field 'filter_configs',
each one of type 'struct flt_conf *':
/*
* Structure representing the filter configuration, attached to a proxy and
* accessible from a filter when instantiated in a stream
*/
struct flt_conf {
const char *id; /* The filter id */
struct flt_ops *ops; /* The filter callbacks */
void *conf; /* The filter configuration */
struct list list; /* Next filter for the same proxy */
};
* 'flt_conf.id' is an identifier, defined by the filter. It can be
NULL. HAProxy does not use this field. Filters can use it in log messages or
as a uniq identifier to check multiple declarations. It is the filter
responsibility to free it, if necessary.
* 'flt_conf.conf' is opaque. It is the internal configuration of a filter,
generally allocated and filled by its parsing function (See § 3.2). It is
the filter responsibility to free it.
* 'flt_conf.ops' references the callbacks implemented by the filter. This
field must be set during the parsing phase (See § 3.2) and can be refine
during the initialization phase (See § 3.3). If it is dynamically allocated,
it is the filter responsibility to free it.
The filter configuration is global and shared by all its instances. A filter
instance is created in the context of a stream and attached to this stream. in
the structure 'stream', the field 'strm_flt' is the state of all filter
instances attached to a stream:
/*
* Structure representing the "global" state of filters attached to a
* stream.
*/
struct strm_flt {
struct list filters; /* List of filters attached to a stream */
struct filter *current[2]; /* From which filter resume processing, for a specific channel.
* This is used for resumable callbacks only,
* If NULL, we start from the first filter.
* 0: request channel, 1: response channel */
unsigned short flags; /* STRM_FL_* */
unsigned char nb_req_data_filters; /* Number of data filters registered on the request channel */
unsigned char nb_rsp_data_filters; /* Number of data filters registered on the response channel */
};
Filter instances attached to a stream are stored in the field
'strm_flt.filters', each instance is of type 'struct filter *':
/*
* Structure representing a filter instance attached to a stream
*
* 2D-Array fields are used to store info per channel. The first index
* stands for the request channel, and the second one for the response
* channel. Especially, <next> and <fwd> are offets representing amount of
* data that the filter are, respectively, parsed and forwarded on a
* channel. Filters can access these values using FLT_NXT and FLT_FWD
* macros.
*/
struct filter {
struct flt_conf *config; /* the filter's configuration */
void *ctx; /* The filter context (opaque) */
unsigned short flags; /* FLT_FL_* */
unsigned int next[2]; /* Offset, relative to buf->p, to the next
* byte to parse for a specific channel
* 0: request channel, 1: response channel */
unsigned int fwd[2]; /* Offset, relative to buf->p, to the next
* byte to forward for a specific channel
* 0: request channel, 1: response channel */
unsigned int pre_analyzers; /* bit field indicating analyzers to
* pre-process */
unsigned int post_analyzers; /* bit field indicating analyzers to
* post-process */
struct list list; /* Next filter for the same proxy/stream */
};
* 'filter.config' is the filter configuration previously described. All
instances of a filter share it.
* 'filter.ctx' is an opaque context. It is managed by the filter, so it is its
responsibility to free it.
* 'filter.pre_analyzers and 'filter.post_analyzers will be described later
(See § 3.5).
* 'filter.next' and 'filter.fwd' will be described later (See § 3.6).
3.2. DEFINING THE FILTER NAME AND ITS CONFIGURATION
---------------------------------------------------
When you write a filter, the first thing to do is to add it in the supported
filters. To do so, you must register its name as a valid keyword on the filter
line:
/* Declare the filter parser for "my_filter" keyword */
static struct flt_kw_list flt_kws = { "MY_FILTER_SCOPE", { }, {
{ "my_filter", parse_my_filter_cfg, NULL /* private data */ },
{ NULL, NULL, NULL },
}
};
INITCALL1(STG_REGISTER, flt_register_keywords, &flt_kws);
Then you must define the internal configuration your filter will use. For
example:
struct my_filter_config {
struct proxy *proxy;
char *name;
/* ... */
};
You also must list all callbacks implemented by your filter. Here, we use a
global variable:
struct flt_ops my_filter_ops {
.init = my_filter_init,
.deinit = my_filter_deinit,
.check = my_filter_config_check,
/* ... */
};
Finally, you must define the function to parse your filter configuration, here
'parse_my_filter_cfg'. This function must parse all remaining keywords on the
filter line:
/* Return -1 on error, else 0 */
static int
parse_my_filter_cfg(char **args, int *cur_arg, struct proxy *px,
struct flt_conf *flt_conf, char **err, void *private)
{
struct my_filter_config *my_conf;
int pos = *cur_arg;
/* Allocate the internal configuration used by the filter */
my_conf = calloc(1, sizeof(*my_conf));
if (!my_conf) {
memprintf(err, "%s: out of memory", args[*cur_arg]);
return -1;
}
my_conf->proxy = px;
/* ... */
/* Parse all keywords supported by the filter and fill the internal
* configuration */
pos++; /* Skip the filter name */
while (*args[pos]) {
if (!strcmp(args[pos], "name")) {
if (!*args[pos + 1]) {
memprintf(err, "'%s' : '%s' option without value",
args[*cur_arg], args[pos]);
goto error;
}
my_conf->name = strdup(args[pos + 1]);
if (!my_conf->name) {
memprintf(err, "%s: out of memory", args[*cur_arg]);
goto error;
}
pos += 2;
}
/* ... parse other keywords ... */
}
*cur_arg = pos;
/* Set callbacks supported by the filter */
flt_conf->ops = &my_filter_ops;
/* Last, save the internal configuration */
flt_conf->conf = my_conf;
return 0;
error:
if (my_conf->name)
free(my_conf->name);
free(my_conf);
return -1;
}
WARNING: In your parsing function, you must define 'flt_conf->ops'. You must
also parse all arguments on the filter line. This is mandatory.
In the previous example, we expect to read a filter line as follows:
filter my_filter name MY_NAME ...
Optionnaly, by implementing the 'flt_ops.check' callback, you add a step to
check the internal configuration of your filter after the parsing phase, when
the HAProxy configuration is fully defined. For example:
/* Check configuration of a trace filter for a specified proxy.
* Return 1 on error, else 0. */
static int
my_filter_config_check(struct proxy *px, struct flt_conf *my_conf)
{
if (px->mode != PR_MODE_HTTP) {
Alert("The filter 'my_filter' cannot be used in non-HTTP mode.\n");
return 1;
}
/* ... */
return 0;
}
3.3. MANAGING THE FILTER LIFECYCLE
----------------------------------
Once the configuration parsed and checked, filters are ready to by used. There
are two main callbacks to manage the filter lifecycle:
* 'flt_ops.init': It initializes the filter for a proxy. You may define this
callback if you need to complete your filter configuration.
* 'flt_ops.deinit': It cleans up what the parsing function and the init
callback have done. This callback is useful to release
memory allocated for the filter configuration.
Here is an example:
/* Initialize the filter. Returns -1 on error, else 0. */
static int
my_filter_init(struct proxy *px, struct flt_conf *fconf)
{
struct my_filter_config *my_conf = fconf->conf;
/* ... */
return 0;
}
/* Free ressources allocated by the trace filter. */
static void
my_filter_deinit(struct proxy *px, struct flt_conf *fconf)
{
struct my_filter_config *my_conf = fconf->conf;
if (my_conf) {
free(my_conf->name);
/* ... */
free(my_conf);
}
fconf->conf = NULL;
}
TODO: Add callbacks to handle creation/destruction of filter instances. And
document it.
3.3.1 DEALING WITH THREADS
--------------------------
When HAProxy is compiled with the threads support and started with more that one
thread (global.nbthread > 1), then it is possible to manage the filter per
thread with following callbacks:
* 'flt_ops.init_per_thread': It initializes the filter for each thread. It
works the same way than 'flt_ops.init' but in the
context of a thread. This callback is called
after the thread creation.
* 'flt_ops.deinit_per_thread': It cleans up what the init_per_thread callback
have done. It is called in the context of a
thread, before exiting it.
This is the filter's responsibility to deal with concurrency. check, init and
deinit callbacks are called on the main thread. All others are called on a
"worker" thread (not always the same). This is also the filter's responsibility
to know if HAProxy is started with more than one thread. If it is started with
one thread (or compiled without the threads support), these callbacks will be
silently ignored (in this case, global.nbthread will be always equal to one).
3.4. HANDLING THE STREAMS ACTIVITY
-----------------------------------
You may be interested to handle streams activity. For now, there is three
callbacks that you should define to do so:
* 'flt_ops.stream_start': It is called when a stream is started. This callback
can fail by returning a negative value. It will be
considered as a critical error by HAProxy which
disabled the listener for a short time.
* 'flt_ops.stream_set_backend': It is called when a backend is set for a
stream. This callbacks will be called for all
filters attached to a stream (frontend and
backend). Note this callback is not called if
the frontend and the backend are the same.
* 'flt_ops.stream_stop': It is called when a stream is stopped. This callback
always succeed. Anyway, it is too late to return an
error.
For example:
/* Called when a stream is created. Returns -1 on error, else 0. */
static int
my_filter_stream_start(struct stream *s, struct filter *filter)
{
struct my_filter_config *my_conf = FLT_CONF(filter);
/* ... */
return 0;
}
/* Called when a backend is set for a stream */
static int
my_filter_stream_set_backend(struct stream *s, struct filter *filter,
struct proxy *be)
{
struct my_filter_config *my_conf = FLT_CONF(filter);
/* ... */
return 0;
}
/* Called when a stream is destroyed */
static void
my_filter_stream_stop(struct stream *s, struct filter *filter)
{
struct my_filter_config *my_conf = FLT_CONF(filter);
/* ... */
}
WARNING: Handling the streams creation and destuction is only possible for
filters defined on proxies with the frontend capability.
In addition, it is possible to handle creation and destruction of filter
instances using following callbacks:
* 'flt_ops.attach': It is called after a filter instance creation, when it is
attached to a stream. This happens when the stream is
started for filters defined on the stream's frontend and
when the backend is set for filters declared on the
stream's backend. It is possible to ignore the filter, if
needed, by returning 0. This could be useful to have
conditional filtering.
* 'flt_ops.detach': It is called when a filter instance is detached from a
stream, before its destruction. This happens when the
stream is stopped for filters defined on the stream's
frontend and when the analyze ends for filters defined on
the stream's backend.
For example:
/* Called when a filter instance is created and attach to a stream */
static int
my_filter_attach(struct stream *s, struct filter *filter)
{
struct my_filter_config *my_conf = FLT_CONF(filter);
if (/* ... */)
return 0; /* Ignore the filter here */
return 1;
}
/* Called when a filter instance is detach from a stream, just before its
* destruction */
static void
my_filter_detach(struct stream *s, struct filter *filter)
{
struct my_filter_config *my_conf = FLT_CONF(filter);
/* ... */
}
Finally, you may be interested to be notified when the stream is woken up
because of an expired timer. This could let you a chance to check your own
timeouts, if any. To do so you can use the following callback:
* 'flt_opt.check_timeouts': It is called when a stream is woken up because
of an expired timer.
For example:
/* Called when a stream is woken up because of an expired timer */
static void
my_filter_check_timeouts(struct stream *s, struct filter *filter)
{
struct my_filter_config *my_conf = FLT_CONF(filter);
/* ... */
}
3.5. ANALYZING THE CHANNELS ACTIVITY
------------------------------------
The main purpose of filters is to take part in the channels analyzing. To do so,
there is 2 callbacks, 'flt_ops.channel_pre_analyze' and
'flt_ops.channel_post_analyze', called respectively before and after each
analyzer attached to a channel, except analyzers responsible for the data
parsing/forwarding (TCP or HTTP data). Concretely, on the request channel, these
callbacks could be called before following analyzers:
* tcp_inspect_request (AN_REQ_INSPECT_FE and AN_REQ_INSPECT_BE)
* http_wait_for_request (AN_REQ_WAIT_HTTP)
* http_wait_for_request_body (AN_REQ_HTTP_BODY)
* http_process_req_common (AN_REQ_HTTP_PROCESS_FE)
* process_switching_rules (AN_REQ_SWITCHING_RULES)
* http_process_req_ common (AN_REQ_HTTP_PROCESS_BE)
* http_process_tarpit (AN_REQ_HTTP_TARPIT)
* process_server_rules (AN_REQ_SRV_RULES)
* http_process_request (AN_REQ_HTTP_INNER)
* tcp_persist_rdp_cookie (AN_REQ_PRST_RDP_COOKIE)
* process_sticking_rules (AN_REQ_STICKING_RULES)
And on the response channel:
* tcp_inspect_response (AN_RES_INSPECT)
* http_wait_for_response (AN_RES_WAIT_HTTP)
* process_store_rules (AN_RES_STORE_RULES)
* http_process_res_common (AN_RES_HTTP_PROCESS_BE)
Unlike the other callbacks previously seen before, 'flt_ops.channel_pre_analyze'
can interrupt the stream processing. So a filter can decide to not execute the
analyzer that follows and wait the next iteration. If there are more than one
filter, following ones are skipped. On the next iteration, the filtering resumes
where it was stopped, i.e. on the filter that has previously stopped the
processing. So it is possible for a filter to stop the stream processing on a
specific analyzer for a while before continuing. Moreover, this callback can be
called many times for the same analyzer, until it finishes its processing. For
example:
/* Called before a processing happens on a given channel.
* Returns a negative value if an error occurs, 0 if it needs to wait,
* any other value otherwise. */
static int
my_filter_chn_pre_analyze(struct stream *s, struct filter *filter,
struct channel *chn, unsigned an_bit)
{
struct my_filter_config *my_conf = FLT_CONF(filter);
switch (an_bit) {
case AN_REQ_WAIT_HTTP:
if (/* wait that a condition is verified before continuing */)
return 0;
break;
/* ... * /
}
return 1;
}
* 'an_bit' is the analyzer id. All analyzers are listed in
'include/types/channels.h'.
* 'chn' is the channel on which the analyzing is done. You can know if it is
the request or the response channel by testing if CF_ISRESP flag is set:
│ ((chn->flags & CF_ISRESP) == CF_ISRESP)
In previous example, the stream processing is blocked before receipt of the HTTP
request until a condition is verified.
'flt_ops.channel_post_analyze', for its part, is not resumable. It returns a
negative value if an error occurs, any other value otherwise. It is called when
a filterable analyzer finishes its processing. So it called once for the same
analyzer. For example:
/* Called after a processing happens on a given channel.
* Returns a negative value if an error occurs, any other
* value otherwise. */
static int
my_filter_chn_post_analyze(struct stream *s, struct filter *filter,
struct channel *chn, unsigned an_bit)
{
struct my_filter_config *my_conf = FLT_CONF(filter);
struct http_msg *msg;
switch (an_bit) {
case AN_REQ_WAIT_HTTP:
if (/* A test on received headers before any other treatment */) {
msg = ((chn->flags & CF_ISRESP) ? &s->txn->rsp : &s->txn->req);
txn->status = 400;
msg->msg_state = HTTP_MSG_ERROR;
http_reply_and_close(s, s->txn->status,
http_error_message(s, HTTP_ERR_400));
return -1; /* This is an error ! */
}
break;
/* ... * /
}
return 1;
}
Pre and post analyzer callbacks of a filter are not automatically called. You
must register it explicitly on analyzers, updating the value of
'filter.pre_analyzers' and 'filter.post_analyzers' bit fields. All analyzer bits
are listed in 'include/types/channels.h'. Here is an example:
static int
my_filter_stream_start(struct stream *s, struct filter *filter)
{
/* ... * /
/* Register the pre analyzer callback on all request and response
* analyzers */
filter->pre_analyzers |= (AN_REQ_ALL | AN_RES_ALL)
/* Register the post analyzer callback of only on AN_REQ_WAIT_HTTP and
* AN_RES_WAIT_HTTP analyzers */
filter->post_analyzers |= (AN_REQ_WAIT_HTTP | AN_RES_WAIT_HTTP)
/* ... * /
return 0;
}
To surround activity of a filter during the channel analyzing, two new analyzers
has been added:
* 'flt_start_analyze' (AN_REQ/RES_FLT_START_FE/AN_REQ_RES_FLT_START_BE): For
a specific filter, this analyzer is called before any call to the
'channel_analyze' callback. From the filter point of view, it calls the
'flt_ops.channel_start_analyze' callback.
* 'flt_end_analyze' (AN_REQ/RES_FLT_END): For a specific filter, this analyzer
is called when all other analyzers have finished their processing. From the
filter point of view, it calls the 'flt_ops.channel_end_analyze' callback.
For TCP streams, these analyzers are called only once. For HTTP streams, if the
client connection is kept alive, this happens at each request/response roundtip.
'flt_ops.channel_start_analyze' and 'flt_ops.channel_end_analyze' callbacks can
interrupt the stream processing, as 'flt_ops.channel_analyze'. Here is an
example:
/* Called when analyze starts for a given channel
* Returns a negative value if an error occurs, 0 if it needs to wait,
* any other value otherwise. */
static int
my_filter_chn_start_analyze(struct stream *s, struct filter *filter,
struct channel *chn)
{
struct my_filter_config *my_conf = FLT_CONF(filter);
/* ... TODO ... */
return 1;
}
/* Called when analyze ends for a given channel
* Returns a negative value if an error occurs, 0 if it needs to wait,
* any other value otherwise. */
static int
my_filter_chn_end_analyze(struct stream *s, struct filter *filter,
struct channel *chn)
{
struct my_filter_config *my_conf = FLT_CONF(filter);
/* ... TODO ... */
return 1;
}
Workflow on channels can be summarized as following:
FE: Called for filters defined on the stream's frontend
BE: Called for filters defined on the stream's backend
+------->---------+
| | |
+----------------------+ | +----------------------+
| flt_ops.attach (FE) | | | flt_ops.attach (BE) |
+----------------------+ | +----------------------+
| | |
V | V
+--------------------------+ | +------------------------------------+
| flt_ops.stream_start (FE)| | | flt_ops.stream_set_backend (FE+BE) |
+--------------------------+ | +------------------------------------+
| | |
... | ...
| | |
+-<-- [1] ^ |
| --+ | | --+
+------<----------+ | | +--------<--------+ |
| | | | | | |
V | | | V | |
+-------------------------------+ | | | +-------------------------------+ | |
| flt_start_analyze (FE) +-+ | | | flt_start_analyze (BE) +-+ |
|(flt_ops.channel_start_analyze)| | F | |(flt_ops.channel_start_analyze)| |
+---------------+---------------+ | R | +-------------------------------+ |
| | O | | |
+------<---------+ | N ^ +--------<-------+ | B
| | | T | | | | A
+---------------|------------+ | | E | +---------------|------------+ | | C
|+--------------V-------------+ | | N | |+--------------V-------------+ | | K
||+----------------------------+ | | D | ||+----------------------------+ | | E
|||flt_ops.channel_pre_analyze | | | | |||flt_ops.channel_pre_analyze | | | N
||| V | | | | ||| V | | | D
||| analyzer (FE) +-+ | | ||| analyzer (FE+BE) +-+ |
+|| V | | | +|| V | |
+|flt_ops.channel_post_analyze| | | +|flt_ops.channel_post_analyze| |
+----------------------------+ | | +----------------------------+ |
| --+ | | |
+------------>------------+ ... |
| |
[ data filtering (see below) ] |
| |
... |
| |
+--------<--------+ |
| | |
V | |
+-------------------------------+ | |
| flt_end_analyze (FE+BE) +-+ |
| (flt_ops.channel_end_analyze) | |
+---------------+---------------+ |
| --+
V
+----------------------+
| flt_ops.detach (BE) |
+----------------------+
|
If HTTP stream, go back to [1] --<--+
|
...
|
V
+--------------------------+
| flt_ops.stream_stop (FE) |
+--------------------------+
|
V
+----------------------+
| flt_ops.detach (FE) |
+----------------------+
|
V
By zooming on an analyzer box we have:
...
|
V
|
+-----------<-----------+
| |
+-----------------+--------------------+ |
| | | |
| +--------<---------+ | |
| | | | |
| V | | |
| flt_ops.channel_pre_analyze ->-+ | ^
| | | |
| | | |
| V | |
| analyzer --------->-----+--+
| | |
| | |
| V |
| flt_ops.channel_post_analyze |
| | |
| | |
+-----------------+--------------------+
|
V
...
3.6. FILTERING THE DATA EXCHANGED
-----------------------------------
WARNING: To fully understand this part, you must be aware on how the buffers
work in HAProxy. In particular, you must be comfortable with the idea
of circular buffers. See doc/internals/buffer-operations.txt and
doc/internals/buffer-ops.fig for details.
doc/internals/body-parsing.txt could also be useful.
An extended feature of the filters is the data filtering. By default a filter
does not look into data exchanged between the client and the server because it
is expensive. Indeed, instead of forwarding data without any processing, each
byte need to be buffered.
So, to enable the data filtering on a channel, at any time, in one of previous
callbacks, you should call 'register_data_filter' function. And conversely, to
disable it, you should call 'unregister_data_filter' function. For example:
my_filter_http_headers(struct stream *s, struct filter *filter,
struct http_msg *msg)
{
struct my_filter_config *my_conf = FLT_CONF(filter);
/* 'chn' must be the request channel */
if (!(msg->chn->flags & CF_ISRESP)) {
struct http_txn *txn = s->txn;
struct buffer *req = msg->chn->buf;
struct hdr_ctx ctx;
/* Enable the data filtering for the request if 'X-Filter' header
* is set to 'true'. */
if (http_find_header2("X-Filter", 8, req->p, &txn->hdr_idx, &ctx) &&
ctx.vlen >= 3 && memcmp(ctx.line + ctx.val, "true", 4) == 0)
register_data_filter(s, chn, filter);
}
return 1;
}
Here, the data filtering is enabled if the HTTP header 'X-Filter' is found and
set to 'true'.
If several filters are declared, the evaluation order remains the same,
regardless the order of the registrations to the data filtering.
Depending on the stream type, TCP or HTTP, the way to handle data filtering will
be slightly different. Among other things, for HTTP streams, there are more
callbacks to help you to fully handle all steps of an HTTP transaction. But the
basis is the same. The data filtering is done in 2 stages:
* The data parsing: At this stage, filters will analyze input data on a
channel. Once a filter has parsed some data, it cannot parse it again. At
any time, a filter can choose to not parse all available data. So, it is
possible for a filter to retain data for a while. Because filters are
chained, a filter cannot parse more data than its predecessors. Thus only
data considered as parsed by the last filter will be available to the next
stage, the data forwarding.
* The data forwarding: At this stage, filters will decide how much data
HAProxy can forward among those considered as parsed at the previous
stage. Once a filter has marked data as forwardable, it cannot analyze it
anymore. At any time, a filter can choose to not forward all parsed
data. So, it is possible for a filter to retain data for a while. Because
filters are chained, a filter cannot forward more data than its
predecessors. Thus only data marked as forwardable by the last filter will
be actually forwarded by HAProxy.
Internally, filters own 2 offsets, relatively to 'buf->p', representing the
number of bytes already parsed in the available input data and the number of
bytes considered as forwarded. We will call these offsets, respectively, 'nxt'
and 'fwd'. Following macros reference these offsets:
* FLT_NXT(flt, chn), flt_req_nxt(flt) and flt_rsp_nxt(flt)
* FLT_FWD(flt, chn), flt_req_fwd(flt) and flt_rsp_fwd(flt)
where 'flt' is the 'struct filter' passed as argument in all callbacks and 'chn'
is the considered channel.
Using these offsets, following operations on buffers are possible:
chn->buf->p + FLT_NXT(flt, chn) // the pointer on parsable data for
// the filter 'flt' on the channel 'chn'.
// Everything between chn->buf->p and 'nxt' offset was already parsed
// by the filter.
chn->buf->i - FLT_NXT(flt, chn) // the number of bytes of parsable data for
// the filter 'flt' on the channel 'chn'.
chn->buf->p + FLT_FWD(flt, chn) // the pointer on forwardable data for
// the filter 'flt' on the channel 'chn'.
// Everything between chn->buf->p and 'fwd' offset was already forwarded
// by the filter.
Note that at any time, for a filter, 'nxt' offset is always greater or equal to
'fwd' offset.
TODO: Add schema with buffer states when there is 2 filters that analyze data.
3.6.1 FILTERING DATA ON TCP STREAMS
-----------------------------------
The TCP data filtering is the easy case, because HAProxy do not parse these
data. So you have only two callbacks that you need to consider:
* 'flt_ops.tcp_data': This callback is called when unparsed data are
available. If not defined, all available data will be considered as parsed
for the filter.
* 'flt_ops.tcp_forward_data': This callback is called when parsed data are
available. If not defined, all parsed data will be considered as forwarded
for the filter.
Here is an example:
/* Returns a negative value if an error occurs, else the number of
* consumed bytes. */
static int
my_filter_tcp_data(struct stream *s, struct filter *filter,
struct channel *chn)
{
struct my_filter_config *my_conf = FLT_CONF(filter);
int avail = chn->buf->i - FLT_NXT(filter, chn);
int ret = avail;
/* Do not parse more than 'my_conf->max_parse' bytes at a time */
if (my_conf->max_parse != 0 && ret > my_conf->max_parse)
ret = my_conf->max_parse;
/* if available data are not completely parsed, wake up the stream to
* be sure to not freeze it. */
if (ret != avail)
task_wakeup(s->task, TASK_WOKEN_MSG);
return ret;
}
/* Returns a negative value if an error occurs, else * or the number of
* forwarded bytes. */
static int
my_filter_tcp_forward_data(struct stream *s, struct filter *filter,
struct channel *chn, unsigned int len)
{
struct my_filter_config *my_conf = FLT_CONF(filter);
int ret = len;
/* Do not forward more than 'my_conf->max_forward' bytes at a time */
if (my_conf->max_forward != 0 && ret > my_conf->max_forward)
ret = my_conf->max_forward;
/* if parsed data are not completely forwarded, wake up the stream to
* be sure to not freeze it. */
if (ret != len)
task_wakeup(s->task, TASK_WOKEN_MSG);
return ret;
}
3.6.2 FILTERING DATA ON HTTP STREAMS
------------------------------------
The HTTP data filtering is a bit tricky because HAProxy will parse the body
structure, especially chunked body. So basically there is the HTTP counterpart
to the previous callbacks:
* 'flt_ops.http_data': This callback is called when unparsed data are
available. If not defined, all available data will be considered as parsed
for the filter.
* 'flt_ops.http_forward_data': This callback is called when parsed data are
available. If not defined, all parsed data will be considered as forwarded
for the filter.
But the prototype for these callbacks is slightly different. Instead of having
the channel as parameter, we have the HTTP message (struct http_msg). You need
to be careful when you use 'http_msg.chunk_len' size. This value is the number
of bytes remaining to parse in the HTTP body (or the chunk for chunked
messages). The HTTP parser of HAProxy uses it to have the number of bytes that
it could consume:
/* Available input data in the current chunk from the HAProxy point of view.
* msg->next bytes were already parsed. Without data filtering, HAProxy
* will consume all of it. */
Bytes = MIN(msg->chunk_len, chn->buf->i - msg->next);
But in your filter, you need to recompute it:
/* Available input data in the current chunk from the filter point of view.
* 'nxt' bytes were already parsed. */
Bytes = MIN(msg->chunk_len + msg->next, chn->buf->i) - FLT_NXT(flt, chn);
In addition to these callbacks, there are three others:
* 'flt_ops.http_headers': This callback is called just before the HTTP body
parsing and after any processing on the request/response HTTP headers. When
defined, this callback is always called for HTTP streams (i.e. without needs
of a registration on data filtering).
* 'flt_ops.http_end': This callback is called when the whole HTTP
request/response is processed. It can interrupt the stream processing. So,
it could be used to synchronize the HTTP request with the HTTP response, for
example:
/* Returns a negative value if an error occurs, 0 if it needs to wait,
* any other value otherwise. */
static int
my_filter_http_end(struct stream *s, struct filter *filter,
struct http_msg *msg)
{
struct my_filter_ctx *my_ctx = filter->ctx;
if (!(msg->chn->flags & CF_ISRESP)) /* The request */
my_ctx->end_of_req = 1;
else /* The response */
my_ctx->end_of_rsp = 1;
/* Both the request and the response are finished */
if (my_ctx->end_of_req == 1 && my_ctx->end_of_rsp == 1)
return 1;
/* Wait */
return 0;
}
* 'flt_ops.http_chunk_trailers': This callback is called for chunked HTTP
messages only when all chunks were parsed. HTTP trailers can be parsed into
several passes. This callback will be called each time. The number of bytes
parsed by HAProxy at each iteration is stored in 'msg->sol'.
Then, to finish, there are 2 informational callbacks:
* 'flt_ops.http_reset': This callback is called when a HTTP message is
reset. This happens either when a '100-continue' response is received, or
if we're retrying to send the request to the server after it failed. It
could be useful to reset the filter context before receiving the true
response.
You can know why the callback is called by checking s->txn->status. If it's
10X, we're called because of a '100-continue', if not, it's a L7 retry.
* 'flt_ops.http_reply': This callback is called when, at any time, HAProxy
decides to stop the processing on a HTTP message and to send an internal
response to the client. This mainly happens when an error or a redirect
occurs.
3.6.3 REWRITING DATA
--------------------
The last part, and the trickiest one about the data filtering, is about the data
rewriting. For now, the filter API does not offer a lot of functions to handle
it. There are only functions to notify HAProxy that the data size has changed to
let it update internal state of filters. This is your responsibility to update
data itself, i.e. the buffer offsets. For a HTTP message, you also must update
'msg->next' and 'msg->chunk_len' values accordingly:
* 'flt_change_next_size': This function must be called when a filter alter
incoming data. It updates 'nxt' offset value of all its predecessors. Do not
call this function when a filter change the size of incoming data leads to
an undefined behavior.
unsigned int avail = MIN(msg->chunk_len + msg->next, chn->buf->i) -
flt_rsp_next(filter);
if (avail > 10 and /* ...Some condition... */) {
/* Move the buffer forward to have buf->p pointing on unparsed
* data */
c_adv(msg->chn, flt_rsp_nxt(filter));
/* Skip first 10 bytes. To simplify this example, we consider a
* non-wrapping buffer */
memmove(buf->p + 10, buf->p, avail - 10);
/* Restore buf->p value */
c_rew(msg->chn, flt_rsp_nxt(filter));
/* Now update other filters */
flt_change_next_size(filter, msg->chn, -10);
/* Update the buffer state */
buf->i -= 10;
/* And update the HTTP message state */
msg->chunk_len -= 10;
return (avail - 10);
}
else
return 0; /* Wait for more data */
* 'flt_change_forward_size': This function must be called when a filter alter
parsed data. It updates offset values ('nxt' and 'fwd') of all filters. Do
not call this function when a filter change the size of parsed data leads to
an undefined behavior.
/* len is the number of bytes of forwardable data */
if (len > 10 and /* ...Some condition... */) {
/* Move the buffer forward to have buf->p pointing on non-forwarded
* data */
c_adv(msg->chn, flt_rsp_fwd(filter));
/* Skip first 10 bytes. To simplify this example, we consider a
* non-wrapping buffer */
memmove(buf->p + 10, buf->p, len - 10);
/* Restore buf->p value */
c_rew(msg->chn, flt_rsp_fwd(filter));
/* Now update other filters */
flt_change_forward_size(filter, msg->chn, -10);
/* Update the buffer state */
buf->i -= 10;
/* And update the HTTP message state */
msg->next -= 10;
return (len - 10);
}
else
return 0; /* Wait for more data */
TODO: implement all the stuff to easily rewrite data. For HTTP messages, this
requires to have a chunked message. Else the size of data cannot be
changed.
4. FAQ
------
4.1. Detect multiple declarations of the same filter
----------------------------------------------------
TODO