blob: d716c6b6b6419083bbb530ee1632099a1e0d36a6 [file] [log] [blame]
Christopher Fauletf7e4e7e2016-10-27 22:29:49 +02001 -----------------------------------------------
2 Stream Processing Offload Engine (SPOE)
3 Version 1.0
4 ( Last update: 2016-11-07 )
5 -----------------------------------------------
6 Author : Christopher Faulet
7 Contact : cfaulet at haproxy dot com
8
9
10SUMMARY
11--------
12
13 0. Terms
14 1. Introduction
15 2. SPOE configuration
16 2.1. SPOE scope
17 2.2. "spoe-agent" section
18 2.3. "spoe-message" section
19 2.4. Example
20 3. SPOP specification
21 3.1. Data types
22 3.2. Frames
23 3.2.1. Frame capabilities
24 3.2.2. Frame types overview
25 3.2.3. Workflow
26 3.2.4. Frame: HAPROXY-HELLO
27 3.2.5. Frame: AGENT-HELLO
28 3.2.6. Frame: NOTIFY
29 3.2.7. Frame: ACK
30 3.2.8. Frame: HAPROXY-DISCONNECT
31 3.2.9. Frame: AGENT-DISCONNECT
32 3.3. Events & messages
33 3.4. Actions
34 3.5. Error & timeouts
35
36
370. Terms
38---------
39
40* SPOE : Stream Processing Offload Engine.
41
42 A SPOE is a filter talking to servers managed ba a SPOA to offload the
43 stream processing. An engine is attached to a proxy. A proxy can have
44 several engine. Each engine is linked to an agent and only one.
45
46* SPOA : Stream Processing Offload Agent.
47
48 A SPOA is a service that will receive info from a SPOE to offload the
49 stream processing. An agent manages several servers. It uses a backend to
50 reference all of them. By extension, these servers can also be called
51 agents.
52
53* SPOP : Stream Processing Offload Protocol, used by SPOEs to talk to SPOA
54 servers.
55
56 This protocol is used by engines to talk to agents. It is an in-house
57 binary protocol described in this documentation.
58
59
601. Introduction
61----------------
62
63SPOE is a feature introduced in HAProxy 1.7. It makes possible the
64communication with external components to retrieve some info. The idea started
65with the problems caused by most ldap libs not working fine in event-driven
66systems (often at least the connect() is blocking). So, it is hard to properly
67implement Single Sign On solution (SSO) in HAProxy. The SPOE will ease this
68kind of processing, or we hope so.
69
70Now, the aim of SPOE is to allow any kind of offloading on the streams. First
71releases, besides being experimental, won't do lot of things. As we will see,
72there are few handled events and even less actions supported. Actually, for
73now, the SPOE can offload the processing before "tcp-request content",
74"tcp-response content", "http-request" and "http-response" rules. And it only
75supports variables definition. But, in spite of these limited features, we can
76easily imagine to implement SSO solution, ip reputation or ip geolocation
77services.
78
79
802. SPOE configuration
81----------------------
82
83Because SPOE is implemented as a filter, To use it, you must declare a "filter
84spoe" line in a proxy section (frontend/backend/listen) :
85
86 frontend my-front
87 ...
88 filter spoe [engine <name>] config <file>
89 ...
90
91The "config" parameter is mandatory. It specififies the SPOE configuration
92file. The engine name is optional. It can be set to declare the scope to use in
93the SPOE configuration. So it is possible to use the same SPOE configuration
94for several engines. If no name is provided, the SPOE configuration must not
95contain any scope directive.
96
97We use a separate configuration file on purpose. By commenting SPOE filter
98line, you completly disable the feature, including the parsing of sections
99reserved to SPOE. This is also a way to keep the HAProxy configuration clean.
100
101A SPOE configuration file must contains, at least, the SPOA configuration
102("spoe-agent" section) and SPOE messages ("spoe-message" section) attached to
103this agent. Unused messages (not reference in "spoe-agent" section) will be
104ignored.
105
106IMPORTANT : The configuration of a SPOE filter must be located in a dedicated
107file. But the backend used by a SPOA must be declared in HAProxy configuration
108file.
109
1102.1. SPOE scope
111-------------------------
112
113If you specify an engine name on the SPOE filter line, then you need to define
114scope in the SPOE configuration with the same name. You can have several SPOE
115scope in the same file. In each scope, you must define one and only one
116"spoe-agent" section to configure the SPOA linked to your SPOE and several
117"spoe-message" sections to describe messages sent to servers mananger by your
118SPOA.
119
120A SPOE scope starts with this kind of line :
121
122 [<name>]
123
124where <name> is the same engine name specified on the SPOE filter line. The
125scope ends when the file ends or when another scope is found.
126
127 Example :
128 [my-first-engine]
129 spoe-agent my-agent
130 ...
131 spoe-message msg1
132 ...
133 spoe-message msg2
134 ...
135
136 [my-second-engine]
137 ...
138
139If no engine name is provided on the SPOE filter line, no SPOE scope must be
140found in the SPOE configuration file. All the file is considered to be in the
141same anonymous and implicit scope.
142
1432.2. "spoe-agent" section
144--------------------------
145
146For each engine, you must define one and only one "spoe-agent" section. In this
147section, you will declare SPOE messages and the backend you will use. You will
148also set timeouts and options to customize your agent's behaviour.
149
150
151spoe-agent <name>
152 Create a new SPOA with the name <name>. It must have one and only one
153 "spoe-agent" definition by SPOE scope.
154
155 Arguments :
156 <name> is the name of the agent section.
157
158 following keywords are supported :
Christopher Faulet48026722016-11-16 15:01:12 +0100159 - maxconnrate
160 - maxerrrate
Christopher Fauletf7e4e7e2016-10-27 22:29:49 +0200161 - messages
Christopher Fauletea62c2a2016-11-14 10:54:21 +0100162 - option continue-on-error
Christopher Faulet985532d2016-11-16 15:36:19 +0100163 - option set-on-error
Christopher Fauletf7e4e7e2016-10-27 22:29:49 +0200164 - option var-prefix
Christopher Faulet03a34492016-11-19 16:47:56 +0100165 - timeout hello|idle|processing
Christopher Fauletf7e4e7e2016-10-27 22:29:49 +0200166 - use-backend
167
168
Christopher Faulet48026722016-11-16 15:01:12 +0100169maxconnrate <number>
170 Set the maximum number of connections per second to <number>. The SPOE will
171 stop to open new connections if the maximum is reached and will wait to
172 acquire an existing one. So it is important to set "timeout hello" to a
173 relatively small value.
174
175
176maxerrrate <number>
177 Set the maximum number of errors per second to <number>. The SPOE will stop
178 its processing if the maximum is reached.
179
180
Christopher Fauletf7e4e7e2016-10-27 22:29:49 +0200181messages <msg-name> ...
182 Declare the list of SPOE messages that an agent will handle.
183
184 Arguments :
185 <msg-name> is the name of a SPOE message.
186
187 Messages declared here must be found in the same engine scope, else an error
188 is triggered during the configuration parsing. You can have many "messages"
189 lines.
190
191 See also: "spoe-message" section.
192
193
Christopher Fauletea62c2a2016-11-14 10:54:21 +0100194option continue-on-error
195 Do not stop the events processing when an error occurred on a stream.
196
197 By default, for a specific stream, when an abnormal/unexpected error occurs,
198 the SPOE is disabled for all the transaction. So if you have several events
199 configured, such error on an event will disabled all followings. For TCP
200 streams, this will disable the SPOE for the whole session. For HTTP streams,
201 this will disable it for the transaction (request and response).
202
203 When set, this option bypass this behaviour and only the current event will
204 be ignored.
205
206
Christopher Faulet985532d2016-11-16 15:36:19 +0100207option set-on-error <var name>
208 Define the variable to set when an error occurred during an event processing.
209
210 Arguments :
211
212 <var name> is the variable name, without the scope. The name may only
213 contain characters 'a-z', 'A-Z', '0-9', '.' and '_'.
214
215 This variable will only be set when an error occurred in the scope of the
216 transaction. As for all other variables define by the SPOE, it will be
217 prefixed. So, if your variable name is "error" and your prefix is
218 "my_spoe_pfx", the variable will be "txn.my_spoe_pfx.error".
219
220 When set, the variable is the boolean "true". Note that if "option
221 continue-on-error" is set, the variable is not automatically removed between
222 events processing.
223
224 See also: "option continue-on-error", "option var-prefix".
225
Christopher Fauletf7e4e7e2016-10-27 22:29:49 +0200226option var-prefix <prefix>
227 Define the prefix used when variables are set by an agent.
228
229 Arguments :
230
231 <prefix> is the prefix used to limit the scope of variables set by an
232 agent.
233
234 To avoid conflict with other variables defined by HAProxy, all variables
235 names will be prefixed. By default, the "spoe-agent" name is used. This
236 option can be used to customize it.
237
238 The prefix will be added between the variable scope and its name, separated
239 by a '.'. It may only contain characters 'a-z', 'A-Z', '0-9', '.' and '_', as
240 for variables name. In HAProxy configuration, you need to use this prefix as
241 a part of the variables name. For example, if an agent define the variable
242 "myvar" in the "txn" scope, with the prefix "my_spoe_pfx", then you should
243 use "txn.my_spoe_pfx.myvar" name in your HAProxy configuration.
244
245 An agent will never set new variables at runtime. It can only set new value
246 for existing ones.
247
248
Christopher Fauletf7e4e7e2016-10-27 22:29:49 +0200249timeout hello <timeout>
250 Set the maximum time to wait for an agent to receive the AGENT-HELLO frame.
Christopher Fauletf7a30922016-11-10 15:04:51 +0100251 It is applied on the stream that handle the connection with the agent.
Christopher Fauletf7e4e7e2016-10-27 22:29:49 +0200252
253 Arguments :
254 <timeout> is the timeout value specified in milliseconds by default, but
255 can be in any other unit if the number is suffixed by the unit,
256 as explained at the top of this document.
257
258 This timeout is an applicative timeout. It differ from "timeout connect"
259 defined on backends.
260
261
262timeout idle <timeout>
Christopher Fauletf7a30922016-11-10 15:04:51 +0100263 Set the maximum time to wait for an agent to close an idle connection. It is
264 applied on the stream that handle the connection with the agent.
265
266 Arguments :
267 <timeout> is the timeout value specified in milliseconds by default, but
268 can be in any other unit if the number is suffixed by the unit,
269 as explained at the top of this document.
270
271
272timeout processing <timeout>
273 Set the maximum time to wait for a stream to process an event, i.e to acquire
274 a stream to talk with an agent, to encode all messages, to send the NOTIFY
275 frame, to receive the corrsponding acknowledgement and to process all
276 actions. It is applied on the stream that handle the client and the server
277 sessions.
Christopher Fauletf7e4e7e2016-10-27 22:29:49 +0200278
279 Arguments :
280 <timeout> is the timeout value specified in milliseconds by default, but
281 can be in any other unit if the number is suffixed by the unit,
282 as explained at the top of this document.
283
284
285use-backend <backend>
286 Specify the backend to use. It must be defined.
287
288 Arguments :
289 <backend> is the name of a valid "backend" section.
290
291
2922.3. "spoe-message" section
293----------------------------
294
295To offload the stream processing, SPOE will send messages with specific
296information at a specific moment in the stream life and will wait for
297corresponding replies to know what to do.
298
299
300spoe-message <name>
301 Create a new SPOE message with the name <name>.
302
303 Arguments :
304 <name> is the name of the SPOE message.
305
306 Here you define a message that can be referenced in a "spoe-agent"
307 section. Following keywords are supported :
308 - args
309 - event
310
311 See also: "spoe-agent" section.
312
313
314args [name=]<sample> ...
315 Define arguments passed into the SPOE message.
316
317 Arguments :
318 <sample> is a sample expression.
319
320 When the message is processed, if a sample expression is not available, it is
321 set to NULL. Arguments are processed in their declaration order and added in
322 the message in that order. It is possible to declare named arguements.
323
324 For example:
325 args frontend=fe_id src dst
326
327
328event <name>
329 Set the event that triggers sending of the message.
330
331 Argument :
332 <name> is the event name.
333
334 Supported events are:
335 - on-client-session
Christopher Faulet1002aac2016-12-09 17:41:54 +0100336 - on-server-session
Christopher Fauletf7e4e7e2016-10-27 22:29:49 +0200337 - on-frontend-tcp-request
338 - on-backend-tcp-request
339 - on-tcp-response
340 - on-frontend-http-request
341 - on-backend-http-request
342 - on-http-response
343
344 See section 3.5 about Events.
345
3462.4. Example
347-------------
348
349Here is a simple but complete example that sends client-ip address to a ip
350reputation service. This service can set the variable "ip_score" which is an
351integer between 0 and 100, indicating its reputation (100 means totally safe
352and 0 a blacklisted IP with no doubt).
353
354 ###
355 ### HAProxy configuration
356 frontend www
357 mode http
358 bind *:80
359
360 filter spoe engine ip-reputation config spoe-ip-reputation.conf
361
362 # Reject connection if the IP reputation is under 20
363 tcp-request content reject if { var(sess.iprep.ip_score) -m int lt 20 }
364
365 default_backend http-servers
366
367 backend http-servers
368 mode http
369 server http A.B.C.D:80
370
371 backend iprep-servers
372 mode tcp
373 balance roundrobin
374
375 timeout connect 5s # greater than hello timeout
376 timeout server 3m # greater than idle timeout
377
378 server iprep1 A1.B1.C1.D1:12345
379 server iprep2 A2.B2.C2.D2:12345
380
381 ####
382 ### spoe-ip-reputation.conf
383 [ip-reputation]
384
385 spoe-agent iprep-agent
386 messages get-ip-reputation
387
388 option var-prefix iprep
389
Christopher Faulet03a34492016-11-19 16:47:56 +0100390 timeout hello 2s
391 timeout idle 2m
392 timeout processing 10ms
Christopher Fauletf7e4e7e2016-10-27 22:29:49 +0200393
394 use-backend iprep-servers
395
396 spoe-message get-ip-reputation
397 args ip=src
398 event on-client-session
399
400
4013. SPOP specification
402----------------------
403
4043.1. Data types
405----------------
406
407Here is the bytewise representation of typed data:
408
409 TYPED-DATA : <TYPE:4 bits><FLAGS:4 bits><DATA>
410
411Supported types and their representation are:
412
413 TYPE | ID | DESCRIPTION
414 -----------------------------+-----+----------------------------------
415 NULL | 0 | NULL : <0>
416 Boolean | 1 | BOOL : <1+FLAG>
417 32bits signed integer | 2 | INT32 : <2><VALUE:varint>
418 32bits unsigned integer | 3 | UINT32 : <3><VALUE:varint>
419 64bits signed integer | 4 | INT64 : <4><VALUE:varint>
420 32bits unsigned integer | 5 | UNIT64 : <5><VALUE:varint>
421 IPV4 | 6 | IPV4 : <6><STRUCT IN_ADDR:4 bytes>
422 IPV6 | 7 | IPV6 : <7><STRUCT IN_ADDR6:16 bytes>
423 String | 8 | STRING : <8><LENGTH:varint><BYTES>
424 Binary | 9 | BINARY : <9><LENGTH:varint><BYTES>
425 10 -> 15 unused/reserved | - | -
426 -----------------------------+-----+----------------------------------
427
428Variable-length integer (varint) are encoded using Peers encoding:
429
430
431 0 <= X < 240 : 1 byte (7.875 bits) [ XXXX XXXX ]
432 240 <= X < 2288 : 2 bytes (11 bits) [ 1111 XXXX ] [ 0XXX XXXX ]
433 2288 <= X < 264432 : 3 bytes (18 bits) [ 1111 XXXX ] [ 1XXX XXXX ] [ 0XXX XXXX ]
434 264432 <= X < 33818864 : 4 bytes (25 bits) [ 1111 XXXX ] [ 1XXX XXXX ]*2 [ 0XXX XXXX ]
435 33818864 <= X < 4328786160 : 5 bytes (32 bits) [ 1111 XXXX ] [ 1XXX XXXX ]*3 [ 0XXX XXXX ]
436 ...
437
438For booleans, the value (true or false) is the first bit in the FLAGS
439bitfield. if this bit is set to 0, then the boolean is evaluated as false,
440otherwise, the boolean is evaluated as true.
441
4423.2. Frames
443------------
444
445Exchange between HAProxy and agents are made using FRAME packets. All frames
446must be prefixed with their size encoded on 4 bytes in network byte order:
447
448 <FRAME-LENGTH:4 bytes> <FRAME>
449
450A frame always starts with its type, on one byte, followed by metadata
451containing flags, on 4 bytes and a two variable-length integer representing the
452stream identifier and the frame identifier inside the stream:
453
454 FRAME : <FRAME-TYPE:1 byte> <METADATA> <FRAME-PAYLOAD>
455 METADATA : <FLAGS:4 bytes> <STREAM-ID:varint> <FRAME-ID:varint>
456
457Then comes the frame payload. Depending on the frame type, the payload can be
458of three types: a simple key/value list, a list of messages or a list of
459actions.
460
461 FRAME-PAYLOAD : <LIST-OF-MESSAGES> | <LIST-OF-ACTIONS> | <KV-LIST>
462
463 LIST-OF-MESSAGES : [ <MESSAGE-NAME> <NB-ARGS:1 byte> <KV-LIST> ... ]
464 MESSAGE-NAME : <STRING>
465
466 LIST-OF-ACTIONS : [ <ACTION-TYPE:1 byte> <NB-ARGS:1 byte> <ACTION-ARGS> ... ]
467 ACTION-ARGS : [ <TYPED-DATA>... ]
468
469 KV-LIST : [ <KV-NAME> <KV-VALUE> ... ]
470 KV-NAME : <STRING>
471 KV-VALUE : <TYPED-DATA>
472
473 FLAGS : 0 1-31
474 +---+-----------+
475 | F| |
476 | I| RESERVED |
477 | N| |
478 +--+------------+
479
480 FIN: Indicates that this is the final payload fragment. The first fragment
481 may also be the final fragment.
482
483Frames cannot exceed a maximum size negociated between HAProxy and agents
484during the HELLO handshake. Most of time, payload will be small enough to send
485it in one frame. But when supported by the peer, it will be possible to
486fragment huge payload on many frames. This ability is announced during the
487HELLO handshake and it can be asynmetric (supported by agents but not by
488HAProxy or the opposite). The following rules apply to fragmentation:
489
490 * An unfragemnted payload consists of a single frame with the FIN bit set.
491
492 * A fragemented payload consists of several frames with the FIN bit clear and
493 terminated by a single frame with the FIN bit set. All these frames must
494 share the same STREAM-ID and FRAME-ID. And, of course, the FRAME-TYPE must
495 be the same.
496
497Beside the support of fragmented payload by a peer, some payload must not be
498fragmented. See below for details.
499
500IMPORTANT : The maximum size supported by peers for a frame must be greater or
501equal to 256 bytes.
502
5033.2.1. Frame capabilities
504--------------------------
505
506Here are the list of official capabilities that HAProxy and agents can support:
507
Christopher Fauleta1cda022016-12-21 08:58:06 +0100508 * fragmentation: This is the ability for a peer to support fragmented
509 payload in received frames. This is an asymmectical
510 capability, it only concerns the peer that announces
511 it. This is the responsibility to the other peer to use it
512 or not.
513
514 * pipelining: This is the ability for a peer to decouple NOTIFY and ACK
515 frames. This is a symmectical capability. To be used, it must
516 be supported by HAproxy and agents. Unlike HTTP pipelining, the
517 ACK frames can be send in any order, but always on the same TCP
518 connection used for the corresponding NOTIFY frame.
519
520 * async: This ability is similar to the pipelining, but here any TCP
521 connection established between HAProxy and the agent can be used to
522 send ACK frames. if an agent accepts connections from multiple
523 HAProxy, it can use the "engine-id" value to group TCP
524 connections. See details about HAPROXY-HELLO frame.
Christopher Fauletf7e4e7e2016-10-27 22:29:49 +0200525
526Unsupported or unknown capabilities are silently ignored, when possible.
527
5283.2.2. Frame types overview
529----------------------------
530
531Here are types of frame supported by SPOE. Frames sent by HAProxy come first,
532then frames sent by agents :
533
534 TYPE | ID | DESCRIPTION
535 -----------------------------+-----+-------------------------------------
536 HAPROXY-HELLO | 1 | Sent by HAProxy when it opens a
537 | | connection on an agent.
538 | |
539 HAPROXY-DISCONNECT | 2 | Sent by HAProxy when it want to close
540 | | the connection or in reply to an
541 | | AGENT-DISCONNECT frame
542 | |
543 NOTIFY | 3 | Sent by HAProxy to pass information
544 | | to an agent
545 -----------------------------+-----+-------------------------------------
546 AGENT-HELLO | 101 | Reply to a HAPROXY-HELLO frame, when
547 | | the connection is established
548 | |
549 AGENT-DISCONNECT | 102 | Sent by an agent just before closing
550 | | the connection
551 | |
552 ACK | 103 | Sent to acknowledge a NOTIFY frame
553 -----------------------------+-----+-------------------------------------
554
555Unknown frames may be silently skipped.
556
5573.2.3. Workflow
558----------------
559
560 * Successful HELLO handshake:
561
562 HAPROXY AGENT SRV
563 | HAPROXY-HELLO |
Christopher Fauletba7bc162016-11-07 21:07:38 +0100564 | (healthcheck: false) |
Christopher Fauletf7e4e7e2016-10-27 22:29:49 +0200565 | --------------------------> |
566 | |
567 | AGENT-HELLO |
568 | <-------------------------- |
569 | |
570
Christopher Fauletba7bc162016-11-07 21:07:38 +0100571 * Successful HELLO healthcheck:
572
573 HAPROXY AGENT SRV
574 | HAPROXY-HELLO |
575 | (healthcheck: true) |
576 | --------------------------> |
577 | |
578 | AGENT-HELLO + close() |
579 | <-------------------------- |
580 | |
581
Christopher Fauletf7e4e7e2016-10-27 22:29:49 +0200582
583 * Error encountered by agent during the HELLO handshake:
584
585 HAPROXY AGENT SRV
586 | HAPROXY-HELLO |
587 | --------------------------> |
588 | |
589 | DISCONNECT + close() |
590 | <-------------------------- |
591 | |
592
593 * Error encountered by HAProxy during the HELLO handshake:
594
595 HAPROXY AGENT SRV
596 | HAPROXY-HELLO |
597 | --------------------------> |
598 | |
599 | AGENT-HELLO |
600 | <-------------------------- |
601 | |
602 | DISCONNECT |
603 | --------------------------> |
604 | |
605 | DISCONNECT + close() |
606 | <-------------------------- |
607 | |
608
609 * Notify / Ack exchange:
610
611 HAPROXY AGENT SRV
612 | NOTIFY |
613 | --------------------------> |
614 | |
615 | ACK |
616 | <-------------------------- |
617 | |
618
619 * Connection closed by haproxy:
620
621 HAPROXY AGENT SRV
622 | DISCONNECT |
623 | --------------------------> |
624 | |
625 | DISCONNECT + close() |
626 | <-------------------------- |
627 | |
628
629 * Connection closed by agent:
630
631 HAPROXY AGENT SRV
632 | DISCONNECT + close() |
633 | <-------------------------- |
634 | |
635
6363.2.4. Frame: HAPROXY-HELLO
637----------------------------
638
639This frame is the first one exchanged between HAProxy and an agent, when the
640connection is established. The payload of this frame is a KV-LIST. It cannot be
641fragmented. STREAM-ID and FRAME-ID are must be set 0.
642
643Following items are mandatory in the KV-LIST:
644
645 * "supported-versions" <STRING>
646
647 Last SPOP major versions supported by HAProxy. It is a comma-separated list
648 of versions, following the format "Major.Minor". Spaces must be ignored, if
649 any. When a major version is announced by HAProxy, it means it also support
650 all previous minor versions.
651
652 Example: "2.0, 1.5" means HAProxy supports SPOP 2.0 and 1.0 to 1.5
653
654 * "max-frame-size" <UINT32>
655
656 This is the maximum size allowed for a frame. The HAPROXY-HELLO frame must
657 be lower or equal to this value.
658
659 * "capabilities" <STRING>
660
661 This a comma-separated list of capabilities supported by HAProxy. Spaces
662 must be ignored, if any.
663
Christopher Fauletba7bc162016-11-07 21:07:38 +0100664Following optional items can be added in the KV-LIST:
665
666 * "healthcheck" <BOOLEAN>
667
668 If this item is set to TRUE, then the HAPROXY-HELLO frame is sent during a
669 SPOE health check. When set to FALSE, this item can be ignored.
670
Christopher Fauleta1cda022016-12-21 08:58:06 +0100671 * "engine-id" <STRING>
672
673 This is a uniq string that identify a SPOE engine.
674
Christopher Fauletf7e4e7e2016-10-27 22:29:49 +0200675To finish the HELLO handshake, the agent must return an AGENT-HELLO frame with
676its supported SPOP version, the lower value between its maximum size allowed
677for a frame and the HAProxy one and capabilities it supports. If an error
678occurs or if an incompatibility is detected with the agent configuration, an
679AGENT-DISCONNECT frame must be returned.
680
6813.2.5. Frame: AGENT-HELLO
682--------------------------
683
684This frame is sent in reply to a HAPROXY-HELLO frame to finish a HELLO
685handshake. As for HAPROXY-HELLO frame, STREAM-ID and FRAME-ID are also set
6860. The payload of this frame is a KV-LIST and it cannot be fragmented.
687
688Following items are mandatory in the KV-LIST:
689
690 * "version" <STRING>
691
692 This is the SPOP version the agent supports. It must follow the format
693 "Major.Minor" and it must be lower or equal than one of major versions
694 announced by HAProxy.
695
696 * "max-frame-size" <UINT32>
697
698 This is the maximum size allowed for a frame. It must be lower or equal to
699 the value in the HAPROXY-HELLO frame. This value will be used for all
700 subsequent frames.
701
702 * "capabilities" <STRING>
703
704 This a comma-separated list of capabilities supported by agent. Spaces must
705 be ignored, if any.
706
707At this time, if everything is ok for HAProxy (supported version and valid
708max-frame-size value), the HELLO handshake is successfully completed. Else,
709HAProxy sends a HAPROXY-DISCONNECT frame with the corresponding error.
710
Christopher Fauletba7bc162016-11-07 21:07:38 +0100711If "healthcheck" item was set to TRUE in the HAPROXY-HELLO frame, the agent can
712safely close the connection without DISCONNECT frame. In all cases, HAProxy
713will close the connexion at the end of the health check.
714
Christopher Fauletf7e4e7e2016-10-27 22:29:49 +02007153.2.6. Frame: NOTIFY
716---------------------
717
718Information are sent to the agents inside NOTIFY frames. These frames are
719attached to a stream, so STREAM-ID and FRAME-ID must be set. The payload of
720NOTIFY frames is a LIST-OF-MESSAGES and, if supported by agents, it can be
721fragmented.
722
723NOTIFY frames must be acknowledge by agents sending an ACK frame, repeating
724right STREAM-ID and FRAME-ID.
725
7263.2.7. Frame: ACK
727------------------
728
729ACK frames must be sent by agents to reply to NOTIFY frames. STREAM-ID and
730FRAME-ID found in a NOTIFY frame must be reuse in the corresponding ACK
731frame. The payload of ACK frames is a LIST-OF-ACTIONS and, if supported by
732HAProxy, it can be fragmented.
733
7343.2.8. Frame: HAPROXY-DISCONNECT
735---------------------------------
736
737If an error occurs, at anytime, from the HAProxy side, a HAPROXY-DISCONNECT
738frame is sent with information describing the error. HAProxy will wait an
739AGENT-DISCONNECT frame in reply. All other frames will be ignored. The agent
740must then close the socket.
741
742The payload of this frame is a KV-LIST. It cannot be fragmented. STREAM-ID and
743FRAME-ID are must be set 0.
744
745Following items are mandatory in the KV-LIST:
746
747 * "status-code" <UINT32>
748
749 This is the code corresponding to the error.
750
751 * "message" <STRING>
752
753 This is a textual message describing the error.
754
755For more information about known errors, see section "Errors & timeouts"
756
7573.2.9. Frame: AGENT-DISCONNECT
758-------------------------------
759
760If an error occurs, at anytime, from the agent size, a AGENT-DISCONNECT frame
761is sent, with information desribing the error. such frame is also sent in reply
762to a HAPROXY-DISCONNECT. The agent must close the socket just after sending
763this frame.
764
765The payload of this frame is a KV-LIST. It cannot be fragmented. STREAM-ID and
766FRAME-ID are must be set 0.
767
768Following items are mandatory in the KV-LIST:
769
770 * "status-code" <UINT32>
771
772 This is the code corresponding to the error.
773
774 * "message" <STRING>
775
776 This is a textual message describing the error.
777
778For more information about known errors, see section "Errors & timeouts"
779
7803.3. Events & Messages
781-----------------------
782
783Information about streams are sent in NOTIFY frames. You can specify which kind
784of information to send by defining "spoe-message" sections in your SPOE
785configuration file. for each "spoe-message" there will be a message in a NOTIFY
786frame when the right event is triggered.
787
788A NOTIFY frame is sent for an specific event when there is at least one
789"spoe-message" attached to this event. All messages for an event will be added
790in the same NOTIFY frame.
791
792Here is the list of supported events:
793
794 * on-client-session is triggered when a new client session is created.
795 This event is only available for SPOE filters
796 declared in a frontend or a listen section.
797
798 * on-frontend-tcp-request is triggered just before the evaluation of
799 "tcp-request content" rules on the frontend side.
800 This event is only available for SPOE filters
801 declared in a frontend or a listen section.
802
803 * on-backend-tcp-request is triggered just before the evaluation of
804 "tcp-request content" rules on the backend side.
805 This event is skipped for SPOE filters declared
806 in a listen section.
807
808 * on-frontend-http-request is triggered just before the evaluation of
809 "http-request" rules on the frontend side. This
810 event is only available for SPOE filters declared
811 in a frontend or a listen section.
812
813 * on-backend-http-request is triggered just before the evaluation of
814 "http-request" rules on the backend side. This
815 event is skipped for SPOE filters declared in a
816 listen section.
817
818 * on-server-session is triggered when the session with the server is
819 established.
820
821 * on-tcp-response is triggered just before the evaluation of
822 "tcp-response content" rules.
823
824 * on-http-response is triggered just before the evaluation of
825 "http-response" rules.
826
827
828The stream processing will loop on these events, when triggered, waiting the
829agent reply.
830
8313.4. Actions
832-------------
833
834An agent must acknowledge each NOTIFY frame by sending the corresponding ACK
835frame. Actions can be added in these frames to dynamically take action on the
836processing of a stream.
837
838Here is the list of supported actions:
839
840 * set-var set the value for an existing variable. 3 arguments must be
841 attached to this action: the variable scope (proc, sess, txn,
842 req or req), the variable name (a string) and its value.
843
844 ACTION-SET-VAR : <SET-VAR:1 byte><NB-ARGS:1 byte><VAR-SCOPE:1 byte><VAR-NAME><VAR-VALUE>
845
846 SET-VAR : <1>
847 NB-ARGS : <3>
848 VAR-SCOPE : <PROCESS> | <SESSION> | <TRANSACTION> | <REQUEST> | <RESPONSE>
849 VAR-NAME : <STRING>
850 VAR-VALUE : <TYPED-DATA>
851
852 PROCESS : <0>
853 SESSION : <1>
854 TRANSACTION : <2>
855 REQUEST : <3>
Christopher Fauleta1cda022016-12-21 08:58:06 +0100856 RESPONSE : <4>
Christopher Fauletf7e4e7e2016-10-27 22:29:49 +0200857
858 * unset-var unset the value for an existing variable. 2 arguments must be
859 attached to this action: the variable scope (proc, sess, txn,
860 req or req) and the variable name (a string).
861
Christopher Faulet1002aac2016-12-09 17:41:54 +0100862 ACTION-UNSET-VAR : <UNSET-VAR:1 byte><NB-ARGS:1 byte><VAR-SCOPE:1 byte><VAR-NAME>
Christopher Fauletf7e4e7e2016-10-27 22:29:49 +0200863
Christopher Faulet1002aac2016-12-09 17:41:54 +0100864 UNSET-VAR : <2>
865 NB-ARGS : <2>
Christopher Fauletf7e4e7e2016-10-27 22:29:49 +0200866 VAR-SCOPE : <PROCESS> | <SESSION> | <TRANSACTION> | <REQUEST> | <RESPONSE>
867 VAR-NAME : <STRING>
868
869 PROCESS : <0>
870 SESSION : <1>
871 TRANSACTION : <2>
872 REQUEST : <3>
Christopher Fauleta1cda022016-12-21 08:58:06 +0100873 RESPONSE : <4>
Christopher Fauletf7e4e7e2016-10-27 22:29:49 +0200874
875
876NOTE: Name of the variables will be automatically prefixed by HAProxy to avoid
877 name clashes with other variables used in HAProxy. Moreover, unknown
878 variable will be silently ignored.
879
8803.5. Error & timeouts
881----------------------
882
883Here is the list of all known errors:
884
885 STATUS CODE | DESCRIPTION
886 ----------------+--------------------------------------------------------
887 0 | normal (no error occurred)
888 1 | I/O error
889 2 | A timeout occurred
890 3 | frame is too big
891 4 | invalid frame received
892 5 | version value not found
893 6 | max-frame-size value not found
894 7 | capabilities value not found
895 8 | unsupported version
896 9 | max-frame-size too big or too small
897 99 | an unknown error occurrde
898 ----------------+--------------------------------------------------------
899
900An agent can define its own errors using a not yet assigned status code.
901
Christopher Fauletea62c2a2016-11-14 10:54:21 +0100902IMPORTANT NOTE: By default, for a specific stream, when an abnormal/unexpected
903 error occurs, the SPOE is disabled for all the transaction. So
904 if you have several events configured, such error on an event
905 will disabled all followings. For TCP streams, this will
906 disable the SPOE for the whole session. For HTTP streams, this
907 will disable it for the transaction (request and response).
908 See 'option continue-on-error' to bypass this limitation.
Christopher Fauletf7e4e7e2016-10-27 22:29:49 +0200909
910To avoid a stream to wait infinitly, you must carefully choose the
911acknowledgement timeout. In most of cases, it will be quiet low. But it depends
912on the responsivness of your service.
913
914You must also choose idle timeout carefully. Because connection with your
915service depends on the backend configuration used by the SPOA, it is important
916to use a lower value for idle timeout than the server timeout. Else the
917connection will be closed by HAProxy. The same is true for hello timeout. You
918should choose a lower value than the connect timeout.
919
920
921/*
922 * Local variables:
923 * fill-column: 79
924 * End:
925 */