BUG/MEDIUM: hlua: prevent deadlocks with main lua lock

Main lua lock is used at various places in the code.

Most of the time it is used from unprotected lua environments,
in which case the locking is mandatory.
But there are some cases where the lock is attempted from protected
lua environments, meaning that lock is already owned by the current
thread. Thus new locking attempt should be skipped to prevent any
deadlocks from occuring.

To address this, "already_safe" lock hint was implemented in
hlua_ctx_init() function with commit bf90ce1
("BUG/MEDIUM: lua: dead lock when Lua tasks are trigerred")

But this approach is not very safe, for 2 reasons:

First reason is that there are still some code paths that could lead
to deadlocks.
For instance, in register_task(), hlua_ctx_init() is called with
already_safe set to 1 to prevent deadlock from occuring.

But in case of task init failure, hlua_ctx_destroy() will be called
from the same environment (protected environment), and hlua_ctx_destroy()
does not offer the already_safe lock hint.. resulting in a deadlock.

Second reason is that already_safe hint is used to completely skip
SET_LJMP macros (which manipulates the lock internally), resulting
in some logics in the function being unprotected from lua aborts in
case of unexpected errors when manipulating the lua stack (the lock
does not protect against longjmps)

Instead of leaving the locking responsibility to the caller, which is
quite error prone since we must find out ourselves if we are or not in
a protected environment (and is not robust against code re-use),
we move the deadlock protection logic directly in hlua_lock() function.

Thanks to a thread-local lock hint, we can easily guess if the current
thread already owns the main lua lock, in which case the locking attempt
is skipped. The thread-local lock hint is implemented as a counter so
that the lock is properly dropped when the counter reaches 0.
(to match actual lock() and unlock() calls)

This commit depends on "MINOR: hlua: simplify lua locking"
It may be backported to every stable versions.
[prior to 2.5 lua filter API did not exist, filter-related parts
should be skipped]

(cherry picked from commit 6b0b9bd39f5a2ac63ecd0093cf385bfab9864efe)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit 63ec5a3ac46e11f4b655510c8be93e1265570435)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit 0a36e2604ec775d32b8a425523ed06776c316189)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit 67f1fd71feb035eec74e46fccdd9bbba4b9c46b3)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
1 file changed