blob: 743276d50b7cb657b49627b383428a54c0105b60 [file] [log] [blame]
Thierry FOURNIER79c10512015-11-08 22:02:21 +01001 Lua: Architecture and first steps
2 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Willy Tarreau5a753bd2020-03-09 14:57:20 +01003 version 2.2
Thierry FOURNIER79c10512015-11-08 22:02:21 +01004
5 author: Thierry FOURNIER
6 contact: tfournier at arpalert dot org
7
8
9
10HAProxy is a powerful load balancer. It embeds many options and many
11configuration styles in order to give a solution to many load balancing
12problems. However, HAProxy is not universal and some special or specific
Godbach06c80992016-02-16 11:59:17 +080013problems do not have solution with the native software.
Thierry FOURNIER79c10512015-11-08 22:02:21 +010014
15This text is not a full explanation of the Lua syntax.
16
17This text is not a replacement of the HAProxy Lua API documentation. The API
18documentation can be found at the project root, in the documentation directory.
19The goal of this text is to discover how Lua is implemented in HAProxy and using
20it efficiently.
21
22However, this can be read by Lua beginners. Some examples are detailed.
23
24Why a scripting language in HAProxy
25===================================
26
27HAProxy 1.5 makes at possible to do many things using samples, but some people
Godbach06c80992016-02-16 11:59:17 +080028want to more combining results of samples fetches, programming conditions and
Ilya Shipitsin2075ca82020-03-06 23:22:22 +050029loops which is not possible. Sometimes people implement these functionalities
Thierry FOURNIER79c10512015-11-08 22:02:21 +010030in patches which have no meaning outside their network. These people must
31maintain these patches, or worse we must integrate them in the HAProxy
32mainstream.
33
34Their need is to have an embedded programming language in order to no longer
35modify the HAProxy source code, but to write their own control code. Lua is
36encountered very often in the software industry, and in some open source
37projects. It is easy to understand, efficient, light without external
Joseph Herlant71b4b152018-11-13 16:55:16 -080038dependencies, and leaves the resource control to the implementation. Its design
Thierry FOURNIER79c10512015-11-08 22:02:21 +010039is close to the HAProxy philosophy which uses components for what they do
40perfectly.
41
42The HAProxy control block allows one to take a decision based on the comparison
43between samples and patterns. The samples are extracted using fetch functions
44easily extensible, and are used by actions which are also extensible. It seems
45natural to allow Lua to give samples, modify them, and to be an action target.
46So, Lua uses the same entities as the configuration language. This is the most
Godbach06c80992016-02-16 11:59:17 +080047natural and reliable way for the Lua integration. So, the Lua engine allows one
Thierry FOURNIER79c10512015-11-08 22:02:21 +010048to add new sample fetch functions, new converter functions and new actions.
49These new entities can access the existing samples fetches and converters
50allowing to extend them without rewriting them.
51
52The writing of the first Lua functions shows that implementing complex concepts
53like protocol analysers is easy and can be extended to full services. It appears
54that these services are not easy to implement with the HAProxy configuration
Godbach06c80992016-02-16 11:59:17 +080055model which is based on four steps: fetch, convert, compare and action. HAProxy
Thierry FOURNIER79c10512015-11-08 22:02:21 +010056is extended with a notion of services which are a formalisation of the existing
57services like stats, cli and peers. The service is an autonomous entity with a
58behaviour pattern close to that of an external client or server. The Lua engine
59inherits from this new service and offers new possibilities for writing
60services.
61
62This scripting language is useful for testing new features as proof of concept.
63Later, if there is general interest, the proof of concept could be integrated
64with C language in the HAProxy core.
65
Godbach06c80992016-02-16 11:59:17 +080066The HAProxy Lua integration also provides a simple way for distributing Lua
Thierry FOURNIER79c10512015-11-08 22:02:21 +010067packages. The final user needs only to install the Lua file, load it in HAProxy
68and follow the attached documentation.
69
70Design and technical things
71===========================
72
73Lua is integrated into the HAProxy event driven core. We want to preserve the
74fast processing of HAProxy. To ensure this, we implement some technical concepts
75between HAProxy and the Lua library.
76
77The following paragraph also describes the interactions between Lua and HAProxy
78from a technical point of view.
79
80Prerequisite
81-----------
82
83Reading the following documentation links is required to understand the
84current paragraph:
85
Willy Tarreaudaac1e42018-04-19 15:12:26 +020086 HAProxy doc: http://cbonte.github.io/haproxy-dconv/
Thierry FOURNIER79c10512015-11-08 22:02:21 +010087 Lua API: http://www.lua.org/manual/5.3/
Willy Tarreaudaac1e42018-04-19 15:12:26 +020088 HAProxy API: http://www.arpalert.org/src/haproxy-lua-api/1.9dev/index.html
Thierry FOURNIER79c10512015-11-08 22:02:21 +010089 Lua guide: http://www.lua.org/pil/
90
91more about Lua choice
92---------------------
93
94Lua language is very simple to extend. It is easy to add new functions written
Godbach06c80992016-02-16 11:59:17 +080095in C in the core language. It is not required to embed very intrusive libraries,
96and we do not change compilation processes.
Thierry FOURNIER79c10512015-11-08 22:02:21 +010097
98The amount of memory consumed can be controlled, and the issues due to lack of
99memory are perfectly caught. The maximum amount of memory allowed for the Lua
100processes is configurable. If some memory is missing, the current Lua action
101fails, and the HAProxy processing flow continues.
102
103Lua provides a way for implementing event driven design. When the Lua code
104wants to do a blocking action, the action is started, it executes non blocking
105operations, and returns control to the HAProxy scheduler when it needs to wait
106for some external event.
107
108The Lua process can be interrupted after a number of instructions executed. The
109Lua execution will resume later. This is a useful way for controlling the
110execution time. This system also keeps HAProxy responsive. When the Lua
111execution is interrupted, HAProxy accepts some connections or transfers pending
112data. The Lua execution does not block the main HAProxy processing, except in
113some cases which we will see later.
114
115Lua function integration
116------------------------
117
118The Lua actions, sample fetches, converters and services are integrated in
119HAProxy with "register_*" functions. The register system is a choice for
120providing HAProxy Lua packages easily. The register system adds new sample
121fetches, converters, actions or services usable in the HAProxy configuration
122file.
123
124The register system is defined in the "core" functions collection. This
125collection is provided by HAProxy and is always available. Below, the list of
126these functions:
127
128 - core.register_action()
129 - core.register_converters()
130 - core.register_fetches()
131 - core.register_init()
132 - core.register_service()
133 - core.register_task()
134
135These functions are the execution entry points.
136
137HTTP action must be used for manipulating HTTP request headers. This action
138can not manipulates HTTP content. It is dangerous to use the channel
139manipulation object with an HTTP request in an HTTP action. The channel
140manipulation can transform a valid request in an invalid request. In this case,
141the action will never resume and the processing will be frozen. HAProxy
142discards the request after the reception timeout.
143
144Non blocking design
145-------------------
146
147HAProxy is an event driven software, so blocking system calls are absolutely
148forbidden. However, the Lua allows to do blocking actions. When an action
149blocks, HAProxy is waiting and do nothing, so the basic functionalities like
150accepting connections or forwarding data are blocked while the end of the system
151call. In this case HAProxy will be less responsive.
152
153This is very insidious because when the developer tries to execute its Lua code
154with only one stream, HAProxy seems to run fine. When the code is used with
155production stream, HAProxy encounters some slow processing, and it cannot
156hold the load.
157
158However, during the initialisation state, you can obviously using blocking
159functions. There are typically used for loading files.
160
161The list of prohibited standard Lua functions during the runtime contains all
162that do filesystem access:
163
164 - os.remove()
165 - os.rename()
166 - os.tmpname()
167 - package.*()
168 - io.*()
169 - file.*()
170
171Some other functions are prohibited:
172
173 - os.execute(), waits for the end of the required execution blocking HAProxy.
174
Godbach06c80992016-02-16 11:59:17 +0800175 - os.exit(), is not really dangerous for the process, but it's not the good way
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100176 for exiting the HAProxy process.
177
178 - print(), writes data on stdout. In some cases these writes are blocking, the
179 best practice is reserving this call for debugging. We must prefer
180 to use core.log() or TXN.log() for sending messages.
181
182Some HAProxy functions have a blocking behaviour pattern in the Lua code, but
183there are compatible with the non blocking design. These functions are:
184
185 - All the socket class
186 - core.sleep()
187
188Responsive design
189-----------------
190
Godbach06c80992016-02-16 11:59:17 +0800191HAProxy must process connections accept, forwarding data and processing timeouts
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100192as soon as possible. The first thing is to believe that a Lua script with a long
193execution time should impact the expected responsive behaviour.
194
195It is not the case, the Lua script execution are regularly interrupted, and
196HAProxy can process other things. These interruptions are exprimed in number of
197Lua instructions. The number of interruptions between two interrupts is
198configured with the following "tune" option:
199
200 tune.lua.forced-yield <nb>
201
202The default value is 10 000. For determining it, I ran benchmark on my laptop.
Godbach06c80992016-02-16 11:59:17 +0800203I executed a Lua loop between 10 seconds with different values for the
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100204"tune.lua.forced-yield" option, and I noted the results:
205
206 configured | Number of
207 instructions | loops executed
Godbach06c80992016-02-16 11:59:17 +0800208 between two | in millions
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100209 forced yields |
210 ---------------+---------------
211 10 | 160
212 500 | 670
213 1000 | 680
214 5000 | 700
215 7000 | 700
216 8000 | 700
217 9000 | 710 <- ceil
218 10000 | 710
219 100000 | 710
220 1000000 | 710
221
222The result showed that from 9000 instructions between two interrupt, we reached
223a ceil, so the default parameter is 10 000.
224
225When HAProxy interrupts the Lua processing, we have two states possible:
226
227 - Lua is resumable, and it returns control to the HAProxy scheduler,
228 - Lua is not resumable, and we just check the execution timeout.
229
230The second case occurs if it is required by the HAProxy core. This state is
231forced if the Lua is processed in a non resumable HAProxy part, like sample
232fetches or converters.
233
234It occurs also if the Lua is non resumable. For example, if some code is
235executed through the Lua pcall() function, the execution is not resumable. This
236is explained later.
237
238So, the Lua code must be fast and simple when is executed as sample fetches and
239converters, it could be slow and complex when is executed as actions and
240services.
241
242Execution time
243--------------
244
Godbach06c80992016-02-16 11:59:17 +0800245The Lua execution time is measured and limited. Each group of functions has its
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100246own timeout configured. The time measured is the real Lua execution time, and
247not the difference between the end time and the start time. The groups are:
248
249 - main code and init are not submitted to the timeout,
250 - fetches, converters and action have a default timeout of 4s,
251 - task, by default does not have timeout,
252 - service have a default timeout of 4s.
253
Godbach06c80992016-02-16 11:59:17 +0800254The corresponding tune options are:
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100255
256 - tune.lua.session-timeout (fetches, converters and action)
257 - tune.lua.task-timeout (task)
258 - tune.lua.service-timeout (services)
259
Godbach06c80992016-02-16 11:59:17 +0800260The task does not have a timeout because it runs in background along the
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100261HAProxy process life.
262
Godbach06c80992016-02-16 11:59:17 +0800263For example, if an Lua script is executed during 1.1s and the script executes a
264sleep of 1 second, the effective measured running time is 0.1s.
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100265
266This timeout is useful for preventing infinite loops. During the runtime, it
Godbach06c80992016-02-16 11:59:17 +0800267should be never triggered.
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100268
269The stack and the coprocess
270---------------------------
271
272The Lua execution is organized around a stack. Each Lua action, even out of the
273effective execution, affects the stack. HAProxy integration uses one main stack,
274which is common for all the process, and a secondary one used as coprocess.
275After the initialization, the main stack is no longer used by HAProxy, except
276for global storage. The second type of stack is used by all the Lua functions
277called from different Lua actions declared in HAProxy. The main stack permits
278to store coroutines pointers, and some global variables.
279
280Do you want to see an example of how seems Lua C development around a stack ?
281Some examples follows. This first one, is a simple addition:
282
283 lua_pushnumber(L, 1)
284 lua_pushnumber(L, 2)
285 lua_arith(L, LUA_OPADD)
286
Godbach06c80992016-02-16 11:59:17 +0800287It's easy, we push 1 on the stack, after, we push 2, and finally, we perform an
Ilya Shipitsin2075ca82020-03-06 23:22:22 +0500288addition. The two top entries of the stack are added, popped, and the result is
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100289pushed. It is a classic way with a stack.
290
Godbach06c80992016-02-16 11:59:17 +0800291Now an example for constructing array and objects. It's a little bit more
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100292complicated. The difficult consist to keep in mind the state of the stack while
293we write the code. The goal is to create the entity described below. Note that
294the notation "*1" is a metatable reference. The metatable will be explained
295later.
296
297 name*1 = {
298 [0] = <userdata>,
299 }
300
301 *1 = {
302 "__index" = {
303 "method1" = <function>,
304 "method2" = <function>
305 }
306 "__gc" = <function>
307 }
308
309Let's go:
310
311 lua_newtable() // The "name" table
312 lua_newtable() // The metatable *1
313 lua_pushstring("__index")
314 lua_newtable() // The "__index" table
315 lua_pushstring("method1")
316 lua_pushfunction(function)
317 lua_settable(-3) // -3 is an index in the stack. insert method1
318 lua_pushstring("method2")
319 lua_pushfunction(function)
320 lua_settable(-3) // insert method2
321 lua_settable(-3) // insert "__index"
322 lua_pushstring("__gc")
323 lua_pushfunction(function)
324 lua_settable() // insert "__gc"
325 lua_setmetatable(-1) // attach metatable to "name"
326 lua_pushnumber(0)
327 lua_pushuserdata(userdata)
328 lua_settable(-3)
329 lua_setglobal("name")
330
331So, coding for Lua in C, is not complex, but it needs some mental gymnastic.
332
333The object concept and the HAProxy format
334-----------------------------------------
335
Godbach06c80992016-02-16 11:59:17 +0800336The object seems to be not a native concept. An Lua object is a table. We can
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100337note that the table notation accept three forms:
338
339 1. mytable["entry"](mytable, "param")
340 2. mytable.entry(mytable, "param")
341 3. mytable:entry("param")
342
343These three notation have the same behaviour pattern: a function is executed
Godbach06c80992016-02-16 11:59:17 +0800344with the table itself as first parameter and string "param" as second parameter
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100345The notation with [] is commonly used for storing data in a hash table, and the
346dotted notation is used for objects. The notation with ":" indicates that the
347first parameter is the element at the left of the symbol ":".
348
349So, an object is a table and each entry of the table is a variable. A variable
350can be a function. These are the first concepts of the object notation in the
351Lua, but it is not the end.
352
353With the objects, we usually expect classes and inheritance. This is the role of
354the metable. A metable is a table with predefined entries. These entries modify
355the default behaviour of the table. The simplest example is the "__index" entry.
356If this entry exists, it is called when a value is requested in the table. The
357behaviour is the following:
358
359 1 - looks in the table if the entry exists, and if it the case, return it
360
361 2 - looks if a metatable exists, and if the "__index" entry exists
362
363 3 - if "__index" is a function, execute it with the key as parameter, and
364 returns the result of the function.
365
366 4 - if "__index" is a table, looks if the requested entry exists, and if
367 exists, return it.
368
369 5 - if not exists, return to step 2
370
371The behaviour of the point 5 represents the inheritance.
372
373In HAProxy all the provided objects are tables, the entry "[0]" contains private
Ilya Shipitsin11057a32020-06-21 21:18:27 +0500374data, there are often userdata or lightuserdata. The metatable is registered in
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100375the global part of the main Lua stack, and it is called with the case sensitive
376class name. A great part of these class must not be used directly because it
377requires an initialisation using the HAProxy internal structs.
378
Godbach06c80992016-02-16 11:59:17 +0800379The HAProxy objects use unified conventions. An Lua object is always a table.
380In most cases, an HAProxy Lua object needs some private data. These are always
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100381set in the index [0] of the array. The metatable entry "__tostring" returns the
382object name.
383
384The Lua developer can add entries to the HAProxy object. He just works carefully
385and prevent to modify the index [0].
386
387Common HAproxy objects are:
388
389 - TXN : manipulates the transaction between the client and the server
390 - Channel : manipulates proxified data between the client and the server
391 - HTTP : manipulates HTTP between the client and the server
392 - Map : manipulates HAProxy maps.
393 - Fetches : access to all HAProxy sample fetches
394 - Converters : access to all HAProxy sample converters
395 - AppletTCP : process client request like a TCP server
396 - AppletHTTP : process client request like an HTTP server
397 - Socket : establish tcp connection to a server (ipv4/ipv6/socket/ssl/...)
398
399The garbage collector and the memory allocation
400-----------------------------------------------
401
402Lua doesn't really have a global memory limit, but HAProxy implements it. This
403permits to control the amount of memory dedicated to the Lua processes. It is
404specially useful with embedded environments.
405
406When the memory limit is reached, HAProxy refuses to give more memory to the Lua
407scripts. The current Lua execution is terminated with an error and HAProxy
Godbach06c80992016-02-16 11:59:17 +0800408continues its processing.
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100409
410The max amount of memory is configured with the option:
411
412 tune.lua.maxmem
413
414As many other script languages, Lua uses a garbage collector for reusing its
Michael Prokop4438c602019-05-24 10:25:45 +0200415memory. The Lua developer can work without memory preoccupation. Usually, the
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100416garbage collector is controlled by the Lua core, but sometimes it will be useful
417to run when the user/developer requires. So the garbage collector can be called
418from C part or Lua part.
419
420Sometimes, objects using lightuserdata or userdata requires to free some memory
421block or close filedescriptor not controlled by the Lua. A dedicated garbage
Godbach06c80992016-02-16 11:59:17 +0800422collection function is provided through the metatable. It is referenced with the
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100423special entry "__gc".
424
425Generally, in HAProxy, the garbage collector does this job without any
Godbach06c80992016-02-16 11:59:17 +0800426intervention. However some objects use a great amount of memory, and we want to
427release as quickly as possible. The problem is that only the GC knows if the
428object is in use or not. The reason is simple variable containing objects can be
429shared between coroutines and the main thread, so an object can be used
430everywhere in HAProxy.
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100431
432The only one example is the HAProxy sockets. These are explained later, just for
433understanding the GC issues, a quick overview of the socket follows. The HAProxy
Godbach06c80992016-02-16 11:59:17 +0800434socket uses an internal session and stream, the session uses resources like
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100435memory and file descriptor and in some cases keeps a socket open while it is no
Godbach06c80992016-02-16 11:59:17 +0800436longer used by Lua.
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100437
438If the HAProxy socket is used, we forcing a garbage collector cycle after the
439end of each function using HAProxy socket. The reason is simple: if the socket
440is no longer used, we want to close the connection quickly.
441
442A special flag is used in HAProxy indicating that a HAProxy socket is created.
443If this flag is set, a full GC cycle is started after each Lua action. This is
444not free, we loose about 10% of performances, but it is the only way for closing
445sockets quickly.
446
447The yield concept / longjmp issues
448----------------------------------
449
450The "yield" is an action which does some Lua processing in pause and give back
451the hand to the HAProxy core. This action is do when the Lua needs to wait about
Godbach06c80992016-02-16 11:59:17 +0800452data or other things. The most basically example is the sleep() function. In an
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100453event driven software the code must not process blocking systems call, so the
454sleep blocks the software between a lot of time. In HAProxy, an Lua sleep does a
Godbach06c80992016-02-16 11:59:17 +0800455yield, and ask to the scheduler to be woken up in a required sleep time.
456Meanwhile, the HAProxy scheduler does other things, like accepting new
457connection or forwarding data.
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100458
Godbach06c80992016-02-16 11:59:17 +0800459A yield is also executed regularly, after a lot of Lua instructions processed.
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100460This yield permits to control the effective execution time, and also give back
Godbach06c80992016-02-16 11:59:17 +0800461the hand to the HAProxy core. When HAProxy finishes to process the pending jobs,
462the Lua execution continues.
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100463
464This special "yield" uses the Lua "debug" functions. Lua provides a debug method
465called "lua_sethook()" which permits to interrupt the execution after some
466configured condition and call a function. This condition used in HAProxy is
Godbach06c80992016-02-16 11:59:17 +0800467a number of instructions processed and when a function returns. The function
468called controls the effective execution time, and if it is possible to send a
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100469"yield".
470
471The yield system is based on a couple setjmp/longjmp. In brief, the setjmp()
472stores a stack state, and the longjmp restores the stack in its state which had
473before the last Lua execution.
474
Godbach06c80992016-02-16 11:59:17 +0800475Lua can immediately stop its execution if an error occurs. This system uses also
Joseph Herlant71b4b152018-11-13 16:55:16 -0800476the longjmp system. In HAProxy, we try to use this system only for unrecoverable
Godbach06c80992016-02-16 11:59:17 +0800477errors. Maybe some trivial errors target an exception, but we try to remove it.
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100478
479It seems that Lua uses the longjmp system for having a behaviour like the java
Godbach06c80992016-02-16 11:59:17 +0800480try / catch. We can use the function pcall() to execute some code. The function
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100481pcall() run a setjmp(). So, if any error occurs while the Lua code execution,
Godbach06c80992016-02-16 11:59:17 +0800482the flow immediately returns from the pcall() with an error.
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100483
484The big issue of this behaviour is that we cannot do a yield. So if some Lua code
485executes a library using pcall for catching errors, HAProxy must be wait for the
486end of execution without processing any accept or any stream. The cause is the
Godbach06c80992016-02-16 11:59:17 +0800487yield must be jump to the root of execution. The intermediate setjmp() avoids
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100488this behaviour.
489
490
491 HAproxy start Lua execution
492 + Lua puts a setjmp()
493 + Lua executes code
494 + Some code is executed in a pcall()
495 + pcall() puts a setjmp()
496 + Lua executes code
497 + A yield is require for a sleep function
498 it cannot be jumps to the Lua root execution.
499
500
501Another issue with the processing of strong errors is the manipulation of the
502Lua stack outside of an Lua processing. If one of the functions called occurs a
503strong error, the default behaviour is an abort(). It is not acceptable when
504HAProxy is in runtime mode. The Lua documentation propose to use another
Godbach06c80992016-02-16 11:59:17 +0800505setjmp/longjmp to avoid the abort(). The goal is to put a setjmp between
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100506manipulating the Lua stack and using an alternative "panic" function which jumps
507to the setjmp() in error case.
508
509All of these behaviours are very dangerous for the stability, and the internal
510HAProxy code must be modified with many precautions.
511
512For preserving a good behaviour of HAProxy, the yield is mandatory.
Godbach06c80992016-02-16 11:59:17 +0800513Unfortunately, some HAProxy parts are not adapted for resuming an execution
514after a yield. These parts are the sample fetches and the sample converters. So,
515the Lua code written in these parts of HAProxy must be quickly executed, and can
516not do actions which require yield like TCP connection or simple sleep.
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100517
518HAproxy socket object
519---------------------
520
521The HAProxy design is optimized for the data transfers between a client and a
522server, and processing the many errors which can occurs during these exchanges.
523HAProxy is not designed for having a third connection established to a third
524party server.
525
Godbach06c80992016-02-16 11:59:17 +0800526The solution consist to put the main stream in pause waiting for the end of the
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100527exchanges with the third connection. This is completed by a signal between
528internal tasks. The following graph shows the HAProxy Lua socket:
529
530
531 +--------------------+
532 | Lua processing |
533 ------------------\ | creates socket | ------------------\
534 incoming request > | and puts the | Outgoing request >
535 ------------------/ | current processing | ------------------/
536    | in pause waiting |
537 | for TCP applet |
538 +-----------------+--+
539 ^ |
540 | |
541 | signal | read / write
542 | | data
543 | |
544 +-------------+---------+ v
545 | HAProxy internal +----------------+
546 | applet send signals | |
547 | when data is received | | -------------------\
548 | or some room is | Attached I/O | Client TCP stream >
549 | available | Buffers | -------------------/
550 +--------------------+--+ |
551 | |
552 +-------------------+
553
554
555A more detailed graph is available in the "doc/internals" directory.
556
557The HAProxy Lua socket uses a full HAProxy session / stream for establishing the
558connection. This mechanism provides all the facilities and HAProxy features,
559like the SSL stack, many socket type, and support for namespaces.
Godbach06c80992016-02-16 11:59:17 +0800560Technically it supports the proxy protocol, but there are no way to enable it.
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100561
562How compiling HAProxy with Lua
563==============================
564
Godbach06c80992016-02-16 11:59:17 +0800565HAProxy 1.6 requires Lua 5.3. Lua 5.3 offers some features which make easy the
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100566integration. Lua 5.3 is young, and some distros do not distribute it. Luckily,
567Lua is a great product because it does not require exotic dependencies, and its
568build process is really easy.
569
570The compilation process for linux is easy:
571
572 - download the source tarball
573 wget http://www.lua.org/ftp/lua-5.3.1.tar.gz
574
575 - untar it
576 tar xf lua-5.3.1.tar.gz
577
578 - enter the directory
579 cd lua-5.3.1
580
581 - build the library for linux
582 make linux
583
584 - install it:
Godbach06c80992016-02-16 11:59:17 +0800585 sudo make INSTALL_TOP=/opt/lua-5.3.1 install
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100586
587HAProxy builds with your favourite options, plus the following options for
588embedding the Lua script language:
589
590 - download the source tarball
591 wget http://www.haproxy.org/download/1.6/src/haproxy-1.6.2.tar.gz
592
593 - untar it
594 tar xf haproxy-1.6.2.tar.gz
595
596 - enter the directory
597 cd haproxy-1.6.2
598
599 - build HAProxy:
Willy Tarreaud254aa82019-06-14 18:40:48 +0200600 make TARGET=linux-glibc \
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100601 USE_LUA=1 \
602 LUA_LIB=/opt/lua-5.3.1/lib \
603 LUA_INC=/opt/lua-5.3.1/include
604
605 - install it:
606 sudo make PREFIX=/opt/haproxy-1.6.2 install
607
608First steps with Lua
609====================
610
Godbach06c80992016-02-16 11:59:17 +0800611Now, it's time to use Lua in HAProxy.
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100612
613Start point
614-----------
615
Godbach06c80992016-02-16 11:59:17 +0800616The HAProxy global directive "lua-load <file>" allows to load an Lua file. This
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100617is the entry point. This load become during the configuration parsing, and the
618Lua file is immediately executed.
619
Godbach06c80992016-02-16 11:59:17 +0800620All the register_*() functions must be called at this time because they are used
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100621just after the processing of the global section, in the frontend/backend/listen
622sections.
623
624The most simple "Hello world !" is the following line a loaded Lua file:
625
626 core.Alert("Hello World !");
627
Godbach06c80992016-02-16 11:59:17 +0800628It displays a log during the HAProxy startup:
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100629
630 [alert] 285/083533 (14465) : Hello World !
631
632Default path and libraries
633--------------------------
634
635Lua can embed some libraries. These libraries can be included from different
Godbach06c80992016-02-16 11:59:17 +0800636paths. It seems that Lua doesn't like subdirectories. In the following example,
637I try to load a compiled library, so the first line is Lua code, the second line
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100638is an 'strace' extract proving that the library was opened. The next lines are
639the associated error.
640
641 require("luac/concat")
642
643 open("./luac/concat.so", O_RDONLY|O_CLOEXEC) = 4
644
645 [ALERT] 293/175822 (22806) : parsing [commonstats.conf:15] : lua runtime
646 error: error loading module 'luac/concat' from file './luac/concat.so':
647 ./luac/concat.so: undefined symbol: luaopen_luac/concat
648
649Lua tries to load the C symbol 'luaopen_luac/concat'. When Lua tries to open a
650library, it tries to execute the function associated to the symbol
651"luaopen_<libname>".
652
653The variable "<libname>" is defined using the content of the variable
654"package.cpath" and/or "package.path". The default definition of the
655"package.cpath" (on my computer is ) variable is:
656
657 /usr/local/lib/lua/5.3/?.so;/usr/local/lib/lua/5.3/loadall.so;./?.so
658
Godbach06c80992016-02-16 11:59:17 +0800659The "<libname>" is the content which replaces the symbol "<?>". In the previous
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100660example, its "luac/concat", and obviously the Lua core try to load the function
661associated with the symbol "luaopen_luac/concat".
662
663My conclusion is that Lua doesn't support subdirectories. So, for loading
664libraries in subdirectory, it must fill the variable with the name of this
665subdirectory. The extension .so must disappear, otherwise Lua try to execute the
666function associated with the symbol "luaopen_concat.so". The following syntax is
667correct:
668
669 package.cpath = package.cpath .. ";./luac/?.so"
670 require("concat")
671
672First useful example
673--------------------
674
675 core.register_fetches("my-hash", function(txn, salt)
676 return txn.sc:sdbm(salt .. txn.sf:req_fhdr("host") .. txn.sf:path() .. txn.sf:src(), 1)
677 end)
678
Godbach06c80992016-02-16 11:59:17 +0800679You will see that these 3 lines can generate a lot of explanations :)
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100680
681Core.register_fetches() is executed during the processing of the global section
682by the HAProxy configuration parser. A new sample fetch is declared with name
683"my-hash", this name is always prefixed by "lua.". So this new declared
684sample fetch will be used calling "lua.my-hash" in the HAProxy configuration
685file.
686
687The second parameter is an inline declared anonymous function. Note the closed
Godbach06c80992016-02-16 11:59:17 +0800688parenthesis after the keyword "end" which ends the function. The first parameter
689of this anonymous function is "txn". It is an object of class TXN. It provides
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100690access functions. The second parameter is an arbitrary value provided by the
691HAProxy configuration file. This parameter is optional, the developer must
Godbach06c80992016-02-16 11:59:17 +0800692check if it is present.
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100693
694The anonymous function registration is executed when the HAProxy backend or
695frontend configuration references the sample fetch "lua.my-hash".
696
Godbach06c80992016-02-16 11:59:17 +0800697This example can be written with another style, like below:
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100698
699 function my_hash(txn, salt)
700 return txn.sc:sdbm(salt .. txn.sf:req_fhdr("host") .. txn.sf:path() .. txn.sf:src(), 1)
701 end
702
703 core.register_fetches("my-hash", my_hash)
704
705This second form is clearer, but the first one is compact.
706
Godbach06c80992016-02-16 11:59:17 +0800707The operator ".." is a string concatenation. If one of the two operands is not a
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100708string, an error occurs and the execution is immediately stopped. This is
709important to keep in mind for the following things.
710
711Now I write the example on more than one line. Its an easiest way for commenting
712the code:
713
714 1. function my_hash(txn, salt)
715 2. local str = ""
716 3. str = str .. salt
717 4. str = str .. txn.sf:req_fhdr("host")
718 5. str = str .. txn.sf:path()
719 6. str = str .. txn.sf:src()
720 7. local result = txn.sc:sdbm(str, 1)
721 8. return result
722 9. end
723 10.
724 11. core.register_fetches("my-hash", my_hash)
725
726local
727~~~~~
728
729The first keyword is "local". This is a really important keyword. You must
730understand that the function "my_hash" will be called for each HAProxy request
731using the declared sample fetch. So, this function can be executed many times in
732parallel.
733
Godbach06c80992016-02-16 11:59:17 +0800734By default, Lua uses global variables. So in this example, if the variable "str"
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100735is declared without the keyword "local", it will be shared by all the parallel
736executions of the function and obviously, the content of the requests will be
737shared.
738
739This warning is very important. I tried to write useful Lua code like a rewrite
Godbach06c80992016-02-16 11:59:17 +0800740of the statistics page, and it is very hard thing to declare each variable as
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100741"local".
742
Godbach06c80992016-02-16 11:59:17 +0800743I guess that this behaviour will be the cause of many troubles on the mailing
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100744list.
745
746str = str ..
747~~~~~~~~~~~~
748
Godbach06c80992016-02-16 11:59:17 +0800749Now a parenthesis about the form "str = str ..". This form allows to do string
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100750concatenations. Remember that Lua uses a garbage collector, so what happens when
751we do "str = str .. 'another string'" ?
752
753 str = str .. "another string"
754 ^ ^ ^ ^
755 1 2 3 4
756
Godbach06c80992016-02-16 11:59:17 +0800757Lua executes first the concatenation operator (3), it allocates memory for the
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100758resulting string and fill this memory with the concatenation of the operands 2
Godbach06c80992016-02-16 11:59:17 +0800759and 4. Next, it frees the variable 1, now the old content of 1 can be garbage
760collected. And finally, the new content of 1 is the concatenation.
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100761
762what the matter ? when we do this operation many times, we consume a lot of
763memory, and the string data is duplicated and move many times. So, this practice
764is expensive in execution time and memory consumption.
765
766There are easy ways to prevent this behaviour. I guess that a C binding for
767concatenation with chunks will be available ASAP (it is already written). I do
768some benchmarks. I compare the execution time of 1 000 times, 1 000
769concatenation of 10 bytes written in pure Lua and with a C library. The result is
77010 times faster in C (1s in Lua, and 0.1s in C).
771
772txn
773~~~
774
775txn is an HAProxy object of class TXN. The documentation is available in the
776HAProxy Lua API reference. This class allow the access to the native HAProxy
777sample fetches and converters. The object txn contains 2 members dedicated to
778the sample fetches and 2 members dedicated to the converters.
779
780The sample fetches members are "f" (as sample-Fetch) and "sf" (as String
Godbach06c80992016-02-16 11:59:17 +0800781sample-Fetch). These two members contain exactly the same functions. All the
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100782HAProxy native sample fetches are available, obviously, the Lua registered sample
783fetches are not available. Unfortunately, HAProxy sample fetches names are not
Godbach06c80992016-02-16 11:59:17 +0800784compatible with the Lua function names, and they are renamed. The rename
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100785convention is simple, we replace all the '.', '+' and '-' by '_'. The '.' is the
786object member separator, and the "-" and "+" is math operator.
787
Godbach06c80992016-02-16 11:59:17 +0800788Now, that I'm writing this article, I know the Lua better than I wrote the
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100789sample-fetches wrapper. The original HAProxy sample-fetches name should be used
790using alternative manner to call an object member, so the sample-fetch
Godbach06c80992016-02-16 11:59:17 +0800791"req.fhdr" (actually renamed req_fhdr") should be used like this:
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100792
793 txn.f["req.fhdr"](txn.f, ...)
794
795However, I think that this form is not elegant.
796
797The "s" collection return a data with a type near to the original returned type.
Godbach06c80992016-02-16 11:59:17 +0800798A string returns an Lua string, an integer returns an Lua integer and an IP
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100799address returns an Lua string. Sometime the data is not or not yet available, in
800this case it returns the Lua nil value.
801
Godbach06c80992016-02-16 11:59:17 +0800802The "sf" collection guarantees that a string will be always returned. If the data
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100803is not available, an empty string is returned. The main usage of these collection
804is to concatenate the returned sample-fetches without testing each function.
805
Godbach06c80992016-02-16 11:59:17 +0800806The parameters of the sample-fetches are according with the HAProxy
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100807documentation.
808
Godbach06c80992016-02-16 11:59:17 +0800809The converters run exactly with the same manner as the sample fetches. The
810only one difference is that the first parameter is the converter entry element.
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100811The "c" collection returns a precise result, and the "sc" collection returns
812always a string.
813
814The sample-fetches used in the example function are "txn.sf:req_fhdr()",
Godbach06c80992016-02-16 11:59:17 +0800815"txn.sf:path()" and "txn.sf:src()". The converter is "txn.sc:sdbm()". The same
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100816function with the "s" collection of sample-fetches and the "c" collection of
817converter should be written like this:
818
819 1. function my_hash(txn, salt)
820 2. local str = ""
821 3. str = str .. salt
822 4. str = str .. tostring(txn.f:req_fhdr("host"))
823 5. str = str .. tostring(txn.f:path())
824 6. str = str .. tostring(txn.f:src())
825 7. local result = tostring(txn.c:sdbm(str, 1))
826 8. return result
827 9. end
828 10.
829 11. core.register_fetches("my-hash", my_hash)
830
831tostring
832~~~~~~~~
833
Godbach06c80992016-02-16 11:59:17 +0800834The function tostring ensures that its parameter is returned as a string. If the
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100835parameter is a table or a thread or anything that will not have any sense as a
836string, a form like the typename followed by a pointer is returned. For example:
837
838 t = {}
839 print(tostring(t))
840
841returns:
842
843 table: 0x15facc0
844
845For objects, if the special function __tostring() is registered in the attached
846metatable, it will be called with the table itself as first argument. The
Godbach06c80992016-02-16 11:59:17 +0800847HAProxy object returns its own type.
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100848
849About the converters entry point
850--------------------------------
851
852In HAProxy, a converter is a stateless function that takes a data as entry and
853returns a transformation of this data as output. In Lua it is exactly the same
854behaviour.
855
Godbach06c80992016-02-16 11:59:17 +0800856So, the registered Lua function doesn't have any special parameters, just a
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100857variable as input which contains the value to convert, and it must return data.
858
859The data required as input by the Lua converter is a string. So HAProxy will
860always provide a string as input. If the native sample fetch is not a string it
Godbach06c80992016-02-16 11:59:17 +0800861will be converted in best effort.
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100862
863The returned value will have anything type, it will be converted as sample of
864the near HAProxy type. The conversion rules from Lua variables to HAProxy
865samples are:
866
867 Lua | HAProxy sample types
868 -----------+---------------------
869 "number" | "sint"
870 "boolean" | "bool"
871 "string" | "str"
872 "userdata" | "bool" (false)
873 "nil" | "bool" (false)
874 "table" | "bool" (false)
875 "function" | "bool" (false)
876 "thread" | "bool" (false)
877
878The function used for registering a converter is:
879
880 core.register_converters()
881
882The task entry point
883--------------------
884
885The function "core.register_task(fcn)" executes once the function "fcn" when the
886scheduler starts. This way is used for executing background task. For example,
Michael Prokop4438c602019-05-24 10:25:45 +0200887you can use this functionality for periodically checking the health of another
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100888service, and giving the result to each proxy needing it.
889
890The task is started once, if you want periodic actions, you can use the
891"core.sleep()" or "core.msleep()" for waiting the next runtime.
892
893Storing Lua variable between function in the same session
894---------------------------------------------------------
895
896All the functions registered as action or sample fetch can share an Lua context.
Godbach06c80992016-02-16 11:59:17 +0800897This context is a memory zone in the stack. sample fetch and action use the
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100898same stack, so both can access to the context.
899
900The context is accessible via the function get_priv and set_priv provided by an
901object of class TXN. The value given to set_priv replaces the current stored
902value. This value can be a table, it is useful if a lot of data can be shared.
903
904If the value stored is a table, you can add or remove entries from the table
905without storing again the new table. Maybe an example will be clearer:
906
907 local t = {}
908 txn:set_priv(t)
909
910 t["entry1"] = "foo"
911 t["entry2"] = "bar"
912
913 -- this will display "foo"
914 print(txn:get_priv()["entry1"])
915
916HTTP actions
917============
918
Joseph Herlant71b4b152018-11-13 16:55:16 -0800919 ... coming soon ...
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100920
921Lua is fast, but my service require more execution speed
922========================================================
923
Godbach06c80992016-02-16 11:59:17 +0800924We can write C modules for Lua. These modules must run with HAProxy while they
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100925are compliant with the HAProxy Lua version. A simple example is the "concat"
926module.
927
928It is very easy to write and compile a C Lua library, however, I don't see
929documentation about this process. So the current chapter is a quick howto.
930
931The entry point
932---------------
933
934The entry point is called "luaopen_<name>", where <name> is the name of the ".so"
935file. An hello world is like this:
936
937 #include <stdio.h>
938 #include <lua.h>
939 #include <lauxlib.h>
940
941 int luaopen_mymod(lua_State *L)
942 {
943 printf("Hello world\n");
944 return 0;
945 }
946
947The build
948---------
949
950The compilation of the source file requires the Lua "include" directory. The
Godbach06c80992016-02-16 11:59:17 +0800951compilation and the link of the object file requires the -fPIC option. That's
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100952all.
953
954 cc -I/opt/lua/include -fPIC -shared -o mymod.so mymod.c
955
956Usage
957-----
958
959You can load this module with the following Lua syntax:
960
961 require("mymod")
962
Godbach06c80992016-02-16 11:59:17 +0800963When you start HAProxy, this module just print "Hello world" when it is loaded.
Thierry FOURNIER79c10512015-11-08 22:02:21 +0100964Please, remember that HAProxy doesn't allow blocking method, so if you write a
965function doing filesystem access or synchronous network access, all the HAProxy
966process will fail.