检测后端relaserver 真实状态,使用前端负载均衡器nginx做到后端服务出错时,自动将出错的节点路踢掉,使得正常请求不发往出错的后端节点,当出错的后端节点恢复后,又能将节点自动加入集群中。nginx自身虽然带有简单的健康检测,但并不有效。
些处使用第三方插件:
nginx_upstream_check_module Health check HTTP servers inside an upstream
https://github.com/yaoweibin/nginx_upstream_check_modue
安装插件
下载
$ wget https://github.com/yaoweibin/nginx_upstream_check_module/archive/v0.3.0.tar.gz
$ tar xf v0.3.0.tar.gz
$ cd nginx_upstream_check_module-0.3.0
$ ls -l
total 244
-rw-rw-r-- 1 root root
0 Oct 2 2014 CHANGES
-rw-rw-r-- 1 root root 5483 Oct 2 2014 check_1.2.1.patch
#相应版本的补丁
-rw-rw-r-- 1 root root 7130 Oct 2 2014 check_1.2.2+.patch
-rw-rw-r-- 1 root root 7094 Oct 2 2014 check_1.2.6+.patch
-rw-rw-r-- 1 root root 6791 Oct 2 2014 check_1.5.12+.patch
-rw-rw-r-- 1 root root 6701 Oct 2 2014 check_1.7.2+.patch
...
安装
$ nginx -V #此处省略了很多模块,只是为了看清而已
...
configure arguments: --prefix=http://www.likecs.com/usr/local/nginx --add-module=../lua-nginx-module-0.10.8
$ cd /usr/local/src/nginx-1.8.1
#给nginx打补丁(根据nginx版本号选择补丁包),打补丁这步操作不可少,否则会出出"[error] 29841#0: *23 http upstream check module can not find any check server, make sure you\'ve added the check servers," 错误
$ patch -p1 < /usr/local/src/nginx_upstream_check_module-0.3.0/check_1.7.2+.patch
patching file src/http/modules/ngx_http_upstream_ip_hash_module.c
patching file src/http/modules/ngx_http_upstream_least_conn_module.c
patching file src/http/ngx_http_upstream_round_robin.c
Hunk #1 succeeded at 9 with fuzz 2.
Hunk #2 succeeded at 95 (offset 4 lines).
Hunk #3 succeeded at 159 (offset 4 lines).
Hunk #4 succeeded at 227 (offset 4 lines).
Hunk #5 succeeded at 339 (offset 4 lines).
Hunk #6 succeeded at 381 (offset 4 lines).
Hunk #7 succeeded at 443 (offset 4 lines).
Hunk #8 succeeded at 541 (offset -1 lines).
patching file src/http/ngx_http_upstream_round_robin.h
$ ./configure --prefix=http://www.likecs.com/usr/local/nginx --add-module=../lua-nginx-module-0.10.8 --add-module=../nginx_upstream_check_module-0.3.0
....
$ make
#make (注意:此处只make,编译参数需要和之前的一样,不要执行make install,否则就会覆盖正在使用的nginx)
$ mv /usr/loca/nginx/sbin/nginx{,_bak}
$ mv objs/nginx /usr/local/nginx/sbin/
$ nginx -t
nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful
$ kill -USER2 `cat /usr/local/nginx/logs/nginx.pid` #热升级nginx,之前的文章有讲过,如果当前nginx不是用绝对路径下的nginx命令启动的话,热升级无效。只能`nginx -s stop`&& /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf`
配置说明
$ less nginx.conf #配置段
http {
upstream cluster {
# simple round-robin
server 192.168.0.1:80;
server 192.168.0.2:80;
check interval=5000 rise=1 fall=3 timeout=4000;
#check interval=3000 rise=2 fall=5 timeout=1000 type=ssl_hello;
#check interval=3000 rise=2 fall=5 timeout=1000 type=http;
#check_http_send "HEAD / HTTP/1.0\r\n\r\n";
#check_http_expect_alive http_2xx http_3xx;
}
...
check
syntax: *check interval=milliseconds [fall=count] [rise=count]
[timeout=milliseconds] [default_down=true|false]
[type=tcp|http|ssl_hello|mysql|ajp|fastcgi]*
默认配置:interval=3000 fall=5 rise=2 timeout=1000 default_down=true type=tcp*
...
interval: 检测间隔3秒
fall: 连续检测失败次数5次时,认定relaserver is down
rise: 连续检测成功2次时,认定relaserver is up
timeout: 超时1秒
default_down: 初始状态为down,只有检测通过后才为up
type: 检测类型方式 tcp
1. tcp :tcp 套接字,不建议使用,后端业务未100%启动完成,前端已经放开访问的情况
2. ssl_hello: 发送hello报文并接收relaserver 返回的hello报文
3. http: 自定义发送一个请求,判断上游relaserver 接收并处理
4. mysql: 连接到mysql服务器,判断上游relaserver是否还存在
5. ajp: 发送AJP Cping数据包,接收并解析AJP Cpong响应以诊断上游relaserver是否还存活(AJP tomcat内置的一种协议)
6. fastcgi: php程序是否存活
check_http_send "HEAD / HTTP/1.0\r\n\r\n";
check_http_expect_alive http_2xx http_3xx ;
如果将检查类型设置为http,则可以通过请求指定资源来判断后端relaserver是否error。同时判断后端返回的状态码是否为2xx和3xx,是表示检查通过,否则表示失败。
示例
upstream cluster {
server 192.168.20.12:80;
server 192.168.20.3:80;
#未启动web服务,默认为down
check interval=3000 rise=2 fall=3 timeout=1000 type=http;
check_http_send "GET /index.html HTTP/1.0\r\n\r\n"; #获取后端资源,资源变动后,经历三次检查周期后立即将状态改为down
check_http_expect_alive http_2xx http_3xx ; #check_http_send 返回状态码,2xx和3xx为默认值。
}
server {
listen 80;
location / {
proxy_pass ;
}
location = /status {
check_status;
#allow xxx;
#deny all;
}
}