Blame - contrib/prometheus-exporter/README - haproxy

2019-02-07 15:38:42 +0100

[diff] [blame]

1

PROMEX: A Prometheus exporter for HAProxy

2

-------------------------------------------

3

4

Prometheus is a monitoring and alerting system. More and more people use it to

5

monitor their environment (this is written February 2019). It collects metrics

6

from monitored targets by scraping metrics HTTP endpoints on these targets. For

7

HAProxy, The Prometheus team offically supports an exporter written in Go

8

(https://github.com/prometheus/haproxy_exporter). But it requires an extra

9

software to deploy and monitor. PROMEX, on its side, is a built-in Prometheus

10

exporter for HAProxy. It was developed as a service and is directly available in

11

HAProxy, like the stats applet.

12

13

However, PROMEX is not built by default with HAProxy. It is provided as an extra

14

component for everyone want to use it. So you need to explicity build HAProxy

15

with the PROMEX service, using the Makefile variable "EXTRA_OBJS". For instance:

16

Willy Tarreau

d254aa8

2019-06-14 18:40:48 +0200

[diff] [blame]

17

> make TARGET=linux-glibc EXTRA_OBJS="contrib/prometheus-exporter/service-prometheus.o"

Christopher Faulet

2019-02-07 15:38:42 +0100

[diff] [blame]

18

19

if HAProxy provides the PROMEX service, the following build option will be

20

reported by the command "haproxy -vv":

21

22

Built with the Prometheus exporter as a service

23

24

To be used, it must be enabled in the configuration with an "http-request" rule

25

and the corresponding HTTP proxy must enable the HTX support. For instance:

frontend test

mode http

...

option http-use-htx

http-request use-service prometheus-exporter if { path /metrics }

...

This service has been developed as a third-party component because it could

36

become obsolete, depending on how much time Prometheus will remain heavily

37

used. This is said with no ulterior motive of course. Prometheus is a great

38

software and I hope all the well for it. But we involve in a environment moving

39

quickly and a solution may be obvious today could be deprecated the next

40

year. And because PROMEX is not integrated by default into the HAProxy codebase,

41

it will need some interest to be actively supported. All contribution of any

42

kind are welcome.

43

44

You must also be careful if you use with huge configurations. Unlike the stats

45

applet, all metrics are not grouped by service (proxy, listener or server). With

46

PROMEX, all lines for a given metric are provided as one single group. So

47

instead of collecting all metrics for a proxy before moving to the next one, we

48

must loop on all proxies for each metric. Same for the servers. Thus, it will

49

spend much more ressources to produce the Prometheus metrics than the CSV export

50

through the stats page. To give a comparison order, quick benchmarks shown that

51

a PROMEX dump is 5x slower and 20x more verbose than a CSV export.

Exported metrics

------------------

* Globals metrics

+------------------------------------------------+-------------------------------------------------------------------------------+

59

| Metric name | Description |

60

+------------------------------------------------+-------------------------------------------------------------------------------+

61

| haproxy_process_nbthread | Configured number of threads. |

62

| haproxy_process_nbproc | Configured number of processes. |

63

| haproxy_process_relative_process_id | Relative process id, starting at 1. |

Christopher Faulet

8c8e4b1

2019-04-18 10:15:15 +0200

[diff] [blame]

64

| haproxy_process_start_time_seconds | Start time in seconds. |

65

| haproxy_process_max_memory_bytes | Per-process memory limit (in bytes); 0=unset. |

66

| haproxy_process_pool_allocated_bytes | Total amount of memory allocated in pools (in bytes). |

67

| haproxy_process_pool_used_bytes | Total amount of memory used in pools (in bytes). |

Christopher Faulet

2019-02-07 15:38:42 +0100

[diff] [blame]

68

| haproxy_process_pool_failures_total | Total number of failed pool allocations. |

69

| haproxy_process_max_fds | Maximum number of open file descriptors; 0=unset. |

70

| haproxy_process_max_sockets | Maximum numer of open sockets. |

71

| haproxy_process_max_connections | Maximum number of concurrent connections. |

72

| haproxy_process_hard_max_connections | Initial Maximum number of concurrent connections. |

73

| haproxy_process_current_connections | Number of active sessions. |

Christopher Faulet

8c8e4b1

2019-04-18 10:15:15 +0200

[diff] [blame]

74

| haproxy_process_connections_total | Total number of created sessions. |

Christopher Faulet

2019-02-07 15:38:42 +0100

[diff] [blame]

75

| haproxy_process_requests_total | Total number of requests (TCP or HTTP). |

76

| haproxy_process_max_ssl_connections | Configured maximum number of concurrent SSL connections. |

77

| haproxy_process_current_ssl_connections | Number of opened SSL connections. |

78

| haproxy_process_ssl_connections_total | Total number of opened SSL connections. |

79

| haproxy_process_max_pipes | Configured maximum number of pipes. |

80

| haproxy_process_pipes_used_total | Number of pipes in used. |

81

| haproxy_process_pipes_free_total | Number of pipes unused. |

82

| haproxy_process_current_connection_rate | Current number of connections per second over last elapsed second. |

83

| haproxy_process_limit_connection_rate | Configured maximum number of connections per second. |

84

| haproxy_process_max_connection_rate | Maximum observed number of connections per second. |

85

| haproxy_process_current_session_rate | Current number of sessions per second over last elapsed second. |

86

| haproxy_process_limit_session_rate | Configured maximum number of sessions per second. |

87

| haproxy_process_max_session_rate | Maximum observed number of sessions per second. |

88

| haproxy_process_current_ssl_rate | Current number of SSL sessions per second over last elapsed second. |

89

| haproxy_process_limit_ssl_rate | Configured maximum number of SSL sessions per second. |

90

| haproxy_process_max_ssl_rate | Maximum observed number of SSL sessions per second. |

91

| haproxy_process_current_frontend_ssl_key_rate | Current frontend SSL Key computation per second over last elapsed second. |

92

| haproxy_process_max_frontend_ssl_key_rate | Maximum observed frontend SSL Key computation per second. |

93

| haproxy_process_frontent_ssl_reuse | SSL session reuse ratio (percent). |

94

| haproxy_process_current_backend_ssl_key_rate | Current backend SSL Key computation per second over last elapsed second. |

95

| haproxy_process_max_backend_ssl_key_rate | Maximum observed backend SSL Key computation per second. |

Christopher Faulet

8c8e4b1

2019-04-18 10:15:15 +0200

[diff] [blame]

96

| haproxy_process_ssl_cache_lookups_total | Total number of SSL session cache lookups. |

97

| haproxy_process_ssl_cache_misses_total | Total number of SSL session cache misses. |

Christopher Faulet

2019-02-07 15:38:42 +0100

[diff] [blame]

98

| haproxy_process_http_comp_bytes_in_total | Number of bytes per second over last elapsed second, before http compression. |

99

| haproxy_process_http_comp_bytes_out_total | Number of bytes per second over last elapsed second, after http compression. |

100

| haproxy_process_limit_http_comp | Configured maximum input compression rate in bytes. |

101

| haproxy_process_current_zlib_memory | Current memory used for zlib in bytes. |

102

| haproxy_process_max_zlib_memory | Configured maximum amount of memory for zlib in bytes. |

103

| haproxy_process_current_tasks | Current number of tasks. |

104

| haproxy_process_current_run_queue | Current number of tasks in the run-queue. |

105

| haproxy_process_idle_time_percent | Idle to total ratio over last sample (percent). |

106

| haproxy_process_stopping | Non zero means stopping in progress. |

107

| haproxy_process_jobs | Current number of active jobs (listeners, sessions, open devices). |

108

| haproxy_process_unstoppable_jobs | Current number of active jobs that can't be stopped during a soft stop. |

109

| haproxy_process_listeners | Current number of active listeners. |

110

| haproxy_process_active_peers | Current number of active peers. |

111

| haproxy_process_connected_peers | Current number of connected peers. |

112

| haproxy_process_dropped_logs_total | Total number of dropped logs. |

113

| haproxy_process_busy_polling_enabled | Non zero if the busy polling is enabled. |

114

+------------------------------------------------+-------------------------------------------------------------------------------+

* Frontend metrics

+-------------------------------------------------+------------------------------------------------------------------------------+

119

| Metric name | Description |

120

+-------------------------------------------------+------------------------------------------------------------------------------+

121

| haproxy_frontend_status | Current status of the service. |

122

| haproxy_frontend_current_sessions | Current number of active sessions. |

123

| haproxy_frontend_max_sessions | Maximum observed number of active sessions. |

124

| haproxy_frontend_limit_sessions | Configured session limit. |

125

| haproxy_frontend_sessions_total | Total number of sessions. |

Christopher Faulet

2019-02-07 15:38:42 +0100

[diff] [blame]

126

| haproxy_frontend_limit_session_rate | Configured limit on new sessions per second. |

127

| haproxy_frontend_max_session_rate | Maximum observed number of sessions per second. |

128

| haproxy_frontend_connections_rate_current | Current number of connections per second over the last elapsed second. |

129

| haproxy_frontend_connections_rate_max | Maximum observed number of connections per second. |

130

| haproxy_frontend_connections_total | Total number of connections. |

131

| haproxy_frontend_bytes_in_total | Current total of incoming bytes. |

132

| haproxy_frontend_bytes_out_total | Current total of outgoing bytes. |

133

| haproxy_frontend_requests_denied_total | Total number of denied requests. |

134

| haproxy_frontend_responses_denied_total | Total number of denied responses. |

135

| haproxy_frontend_request_errors_total | Total number of request errors. |

136

| haproxy_frontend_denied_connections_total | Total number of requests denied by "tcp-request connection" rules. |

137

| haproxy_frontend_denied_sessions_total | Total number of requests denied by "tcp-request session" rules. |

138

| haproxy_frontend_failed_header_rewriting_total | Total number of failed header rewriting warnings. |

Christopher Faulet

2019-02-07 15:38:42 +0100

[diff] [blame]

139

| haproxy_frontend_http_requests_rate_max | Maximum observed number of HTTP requests per second. |

140

| haproxy_frontend_http_requests_total | Total number of HTTP requests received. |

141

| haproxy_frontend_http_responses_total | Total number of HTTP responses. |

142

| haproxy_frontend_intercepted_requests_total | Total number of intercepted HTTP requests. |

143

| haproxy_frontend_http_cache_lookups_total | Total number of HTTP cache lookups. |

144

| haproxy_frontend_http_cache_hits_total | Total number of HTTP cache hits. |

145

| haproxy_frontend_http_comp_bytes_in_total | Total number of HTTP response bytes fed to the compressor. |

146

| haproxy_frontend_http_comp_bytes_out_total | Total number of HTTP response bytes emitted by the compressor. |

147

| haproxy_frontend_http_comp_bytes_bypassed_total | Total number of bytes that bypassed the HTTP compressor (CPU/BW limit). |

148

| haproxy_frontend_http_comp_responses_total | Total number of HTTP responses that were compressed. |

149

+-------------------------------------------------+------------------------------------------------------------------------------+

150

Rick Rackow

35efbe2

2019-10-08 11:28:06 +0200

[diff] [blame]

151

* Backend metrics

Christopher Faulet

2019-02-07 15:38:42 +0100

[diff] [blame]

152

153

+-----------------------------------------------------+--------------------------------------------------------------------------+

154

| Metric name | Description |

155

+-----------------------------------------------------+--------------------------------------------------------------------------+

156

| haproxy_backend_status | Current status of the service. |

157

| haproxy_backend_current_sessions | Current number of active sessions. |

158

| haproxy_backend_max_sessions | Maximum observed number of active sessions. |

159

| haproxy_backend_limit_sessions | Configured session limit. |

160

| haproxy_backend_sessions_total | Total number of sessions. |

Christopher Faulet

2019-02-07 15:38:42 +0100

[diff] [blame]

161

| haproxy_backend_max_session_rate | Maximum observed number of sessions per second. |

162

| haproxy_backend_last_session_seconds | Number of seconds since last session assigned to server/backend. |

163

| haproxy_backend_current_queue | Current number of queued requests. |

164

| haproxy_backend_max_queue | Maximum observed number of queued requests. |

165

| haproxy_backend_connection_attempts_total | Total number of connection establishment attempts. |

166

| haproxy_backend_connection_reuses_total | Total number of connection reuses. |

167

| haproxy_backend_bytes_in_total | Current total of incoming bytes. |

168

| haproxy_backend_bytes_out_total | Current total of outgoing bytes. |

Christopher Faulet

2019-11-08 15:12:29 +0100

[diff] [blame^]

169

| haproxy_backend_queue_time_average_seconds | Avg. queue time for last 1024 successful connections. |

170

| haproxy_backend_connect_time_average_seconds | Avg. connect time for last 1024 successful connections. |

171

| haproxy_backend_response_time_average_seconds | Avg. response time for last 1024 successful connections. (0 for TCP) |

172

| haproxy_backend_total_time_average_seconds | Avg. total time for last 1024 successful connections. |

Christopher Faulet

2019-11-08 15:05:31 +0100

[diff] [blame]

173

| haproxy_backend_max_queue_time_seconds | Maximum observed queue time. |

174

| haproxy_backend_max_connect_time_seconds | Maximum observed connect time. |

Christopher Faulet

2019-11-08 15:12:29 +0100

[diff] [blame^]

175

| haproxy_backend_max_response_time_seconds | Maximum observed response time. (0 for TCP) |

Christopher Faulet

2019-11-08 15:05:31 +0100

[diff] [blame]

176

| haproxy_backend_max_total_time_seconds | Maximum observed total time. |

Christopher Faulet

2019-02-07 15:38:42 +0100

[diff] [blame]

177

| haproxy_backend_requests_denied_total | Total number of denied requests. |

178

| haproxy_backend_responses_denied_total | Total number of denied responses. |

179

| haproxy_backend_connection_errors_total | Total number of connection errors. |

180

| haproxy_backend_response_errors_total | Total number of response errors. |

181

| haproxy_backend_retry_warnings_total | Total number of retry warnings. |

182

| haproxy_backend_redispatch_warnings_total | Total number of redispatch warnings. |

183

| haproxy_backend_failed_header_rewriting_total | Total number of failed header rewriting warnings. |

184

| haproxy_backend_client_aborts_total | Total number of data transfers aborted by the client. |

185

| haproxy_backend_server_aborts_total | Total number of data transfers aborted by the server. |

186

| haproxy_backend_weight | Service weight. |

187

| haproxy_backend_active_servers | Current number of active servers. |

188

| haproxy_backend_backup_servers | Current number of backup servers. |

189

| haproxy_backend_check_up_down_total | Total number of UP->DOWN transitions. |

190

| haproxy_backend_check_last_change_seconds | Number of seconds since the last UP<->DOWN transition. |

191

| haproxy_backend_downtime_seconds_total | Total downtime (in seconds) for the service. |

192

| haproxy_backend_loadbalanced_total | Total number of times a service was selected. |

193

| haproxy_backend_http_requests_total | Total number of HTTP requests received. |

194

| haproxy_backend_http_responses_total | Total number of HTTP responses. |

195

| haproxy_backend_http_cache_lookups_total | Total number of HTTP cache lookups. |

196

| haproxy_backend_http_cache_hits_total | Total number of HTTP cache hits. |

197

| haproxy_backend_http_comp_bytes_in_total | Total number of HTTP response bytes fed to the compressor. |

198

| haproxy_backend_http_comp_bytes_out_total | Total number of HTTP response bytes emitted by the compressor. |

199

| haproxy_backend_http_comp_bytes_bypassed_total | Total number of bytes that bypassed the HTTP compressor (CPU/BW limit). |

200

| haproxy_backend_http_comp_responses_total | Total number of HTTP responses that were compressed. |

201

+-----------------------------------------------------+--------------------------------------------------------------------------+

* Server metrics

+----------------------------------------------------+---------------------------------------------------------------------------+

206

| Metric name | Description |

207

+----------------------------------------------------+---------------------------------------------------------------------------+

208

| haproxy_server_status | Current status of the service. |

209

| haproxy_server_current_sessions | Current number of active sessions. |

210

| haproxy_server_max_sessions | Maximum observed number of active sessions. |

211

| haproxy_server_limit_sessions | Configured session limit. |

212

| haproxy_server_sessions_total | Total number of sessions. |

Christopher Faulet

2019-02-07 15:38:42 +0100

[diff] [blame]

213

| haproxy_server_max_session_rate | Maximum observed number of sessions per second. |

214

| haproxy_server_last_session_seconds | Number of seconds since last session assigned to server/backend. |

215

| haproxy_server_current_queue | Current number of queued requests. |

216

| haproxy_server_max_queue | Maximum observed number of queued requests. |

217

| haproxy_server_queue_limit | Configured maxqueue for the server (0 meaning no limit). |

218

| haproxy_server_bytes_in_total | Current total of incoming bytes. |

219

| haproxy_server_bytes_out_total | Current total of outgoing bytes. |

Christopher Faulet

2019-11-08 15:12:29 +0100

[diff] [blame^]

220

| haproxy_server_queue_time_average_seconds | Avg. queue time for last 1024 successful connections. |

221

| haproxy_server_connect_time_average_seconds | Avg. connect time for last 1024 successful connections. |

222

| haproxy_server_response_time_average_seconds | Avg. response time for last 1024 successful connections. (0 for TCP) |

223

| haproxy_server_total_time_average_seconds | Avg. total time for last 1024 successful connections. |

Christopher Faulet

2019-11-08 15:05:31 +0100

[diff] [blame]

224

| haproxy_server_max_queue_time_seconds | Maximum observed queue time. |

225

| haproxy_server_max_connect_time_seconds | Maximum observed connect time. |

Christopher Faulet

2019-11-08 15:12:29 +0100

[diff] [blame^]

226

| haproxy_server_max_response_time_seconds | Maximum observed response time. (0 for TCP) |

Christopher Faulet

2019-11-08 15:05:31 +0100

[diff] [blame]

227

| haproxy_server_max_total_time_seconds | Maximum observed total time. |

Christopher Faulet