Thierry FOURNIER | 4d7bfa1 | 2018-02-25 21:28:05 +0100 | [diff] [blame] | 1 | Multi script langyage Stream Processing Offload Agent |
| 2 | ----------------------------------------------------- |
Thierry FOURNIER | d8b5c77 | 2018-02-23 11:40:03 +0100 | [diff] [blame] | 3 | |
Thierry FOURNIER | 4d7bfa1 | 2018-02-25 21:28:05 +0100 | [diff] [blame] | 4 | This agent receive SPOP message and process it with script languages. The |
| 5 | language register callback with a message. Each callback receive the list |
| 6 | of arguments with types according with the language capabilities. The |
| 7 | callback write variables which are sent as response when the processing |
| 8 | is done. |
Thierry FOURNIER | d8b5c77 | 2018-02-23 11:40:03 +0100 | [diff] [blame] | 9 | |
| 10 | |
Thierry FOURNIER | 4d7bfa1 | 2018-02-25 21:28:05 +0100 | [diff] [blame] | 11 | Compilation |
| 12 | --------------- |
| 13 | |
William Dauchy | fcae4b6 | 2020-08-01 16:28:52 +0200 | [diff] [blame] | 14 | The server currently supports Lua and Python. Type "make" with the options: |
Thierry FOURNIER | 4d7bfa1 | 2018-02-25 21:28:05 +0100 | [diff] [blame] | 15 | USE_LUA=1 and/or USE_PYTHON=1. |
| 16 | |
| 17 | |
Thierry FOURNIER | d8b5c77 | 2018-02-23 11:40:03 +0100 | [diff] [blame] | 18 | Start the service |
| 19 | --------------------- |
| 20 | |
| 21 | After you have compiled it, to start the service, you just need to use "spoa" |
| 22 | binary: |
| 23 | |
| 24 | $> ./spoa -h |
| 25 | Usage: ./spoa [-h] [-d] [-p <port>] [-n <num-workers>] |
| 26 | -h Print this message |
| 27 | -d Enable the debug mode |
| 28 | -p <port> Specify the port to listen on (default: 12345) |
| 29 | -n <num-workers> Specify the number of workers (default: 5) |
Thierry FOURNIER | 4d7bfa1 | 2018-02-25 21:28:05 +0100 | [diff] [blame] | 30 | -f <file> Load script according with the supported languages |
Thierry FOURNIER | d8b5c77 | 2018-02-23 11:40:03 +0100 | [diff] [blame] | 31 | |
Thierry FOURNIER | 4d7bfa1 | 2018-02-25 21:28:05 +0100 | [diff] [blame] | 32 | The file processor is recognized using the extension. .lua or .luac for lua and |
| 33 | .py for python. Start example: |
Thierry FOURNIER | d8b5c77 | 2018-02-23 11:40:03 +0100 | [diff] [blame] | 34 | |
Thierry FOURNIER | 4d7bfa1 | 2018-02-25 21:28:05 +0100 | [diff] [blame] | 35 | $> ./spoa -d -f ps_lua.lua |
Thierry FOURNIER | d8b5c77 | 2018-02-23 11:40:03 +0100 | [diff] [blame] | 36 | |
Daniel Corbett | c802921 | 2019-06-11 10:08:53 -0400 | [diff] [blame] | 37 | $> ./spoa -d -f ps_python.py |
Thierry FOURNIER | d8b5c77 | 2018-02-23 11:40:03 +0100 | [diff] [blame] | 38 | |
Thierry FOURNIER | d8b5c77 | 2018-02-23 11:40:03 +0100 | [diff] [blame] | 39 | |
Thierry FOURNIER | 4d7bfa1 | 2018-02-25 21:28:05 +0100 | [diff] [blame] | 40 | Configure |
| 41 | ------------- |
Thierry FOURNIER | d8b5c77 | 2018-02-23 11:40:03 +0100 | [diff] [blame] | 42 | |
Thierry FOURNIER | 4d7bfa1 | 2018-02-25 21:28:05 +0100 | [diff] [blame] | 43 | Sample configuration are join to this server: |
Thierry FOURNIER | d8b5c77 | 2018-02-23 11:40:03 +0100 | [diff] [blame] | 44 | |
Thierry FOURNIER | 4d7bfa1 | 2018-02-25 21:28:05 +0100 | [diff] [blame] | 45 | spoa-server.conf : The HAProxy configuration file using SPOE server |
| 46 | spoa-server.spoe.conf : The SPOP description file used by HAProxy |
| 47 | ps_lua.lua : Processing Lua example |
| 48 | ps_python.py : Processing Python example |
Thierry FOURNIER | d8b5c77 | 2018-02-23 11:40:03 +0100 | [diff] [blame] | 49 | |
| 50 | |
Thierry FOURNIER | 4d7bfa1 | 2018-02-25 21:28:05 +0100 | [diff] [blame] | 51 | Considerations |
| 52 | ------------------ |
Thierry FOURNIER | d8b5c77 | 2018-02-23 11:40:03 +0100 | [diff] [blame] | 53 | |
Thierry FOURNIER | 4d7bfa1 | 2018-02-25 21:28:05 +0100 | [diff] [blame] | 54 | This server is a beta version. It works fine, but some improvement will be |
| 55 | welcome: |
Thierry FOURNIER | d8b5c77 | 2018-02-23 11:40:03 +0100 | [diff] [blame] | 56 | |
Thierry FOURNIER | 4d7bfa1 | 2018-02-25 21:28:05 +0100 | [diff] [blame] | 57 | Main process: |
Thierry FOURNIER | d8b5c77 | 2018-02-23 11:40:03 +0100 | [diff] [blame] | 58 | |
Thierry FOURNIER | 4d7bfa1 | 2018-02-25 21:28:05 +0100 | [diff] [blame] | 59 | * Improve log management: Today the log are sent on stdout. |
| 60 | * Improve process management: The dead process are ignored. |
| 61 | * Implement systemd integration. |
| 62 | * Implement threads: It would be fine to implement thread working. Shared |
| 63 | memory is welcome for managing database connection pool and something like |
| 64 | that. |
| 65 | * Add PHP support and some other languages. |
Thierry FOURNIER | d8b5c77 | 2018-02-23 11:40:03 +0100 | [diff] [blame] | 66 | |
Thierry FOURNIER | 4d7bfa1 | 2018-02-25 21:28:05 +0100 | [diff] [blame] | 67 | Python: |
Thierry FOURNIER | d8b5c77 | 2018-02-23 11:40:03 +0100 | [diff] [blame] | 68 | |
Ilya Shipitsin | e011bc3 | 2020-03-14 17:47:28 +0500 | [diff] [blame] | 69 | * Improve reporting: Catch python error message and report it in the right |
Thierry FOURNIER | 4d7bfa1 | 2018-02-25 21:28:05 +0100 | [diff] [blame] | 70 | place. Today the error are dumped on stdout. How using syslog for logging |
| 71 | stack traces ? |
Thierry FOURNIER | d8b5c77 | 2018-02-23 11:40:03 +0100 | [diff] [blame] | 72 | |
Thierry FOURNIER | 4d7bfa1 | 2018-02-25 21:28:05 +0100 | [diff] [blame] | 73 | Maybe some other things... |