Kafka使用jmxtrans+influxdb+grafana监控JMX指标

最近在搞Kafka集群监控,之前也是看了网上的很多资料。之所以使用jmxtrans+influxdb+grafana是因为界面酷炫,可以定制化,缺点是不能操作Kafka集群,可能需要配合Kafka Manager一起使用。

环境信息

CentOS Linux release 7.6.1810 (Core)

jdk1.8.0_201

zookeeper-3.4.14

kafka_2.11-2.2.0

开启Kafka JMX端口

JMX(Java Management Extensions,即Java管理扩展)是一个为应用程序、设备、系统等植入管理功能的框架。JMX可以跨越一系列异构操作系统平台、系统体系结构和网络传输协议,灵活的开发无缝集成的系统、网络和服务管理应用。Kafka做为一款Java应用,已经定义了丰富的性能指标,(可以参考Kafka监控指标),通过JMX可以轻松对其进行监控。

在${KAFKA_HOME}/bin/路径下修改kafka-run-class.sh脚本,第一行增加JMX_PORT=9999即可。

JMX_PORT=9999

重启Kafka

./bin/kafka-server-stop.sh
./bin/kafka-server-start.sh -daemon ./config/server.properties

重启后查看Kafka以及JMX端口状态

ps -ef | grep kafka
root      8273      1 99 02:32 pts/0    00:00:09 /opt/jdk1.8.0_201/bin/java -Xmx1G -Xms1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 ......  kafka.Kafka ./config/server.properties

netstat -anop | grep 9999
tcp6      0      0 :::9999                :::*                    LISTEN      8273/java            off (0.00/0/0)

安装InfluxDB

InfluxDB是一个时间序列数据库,用于处理海量写入与负载查询。InfluxDB旨在用作涉及大量时间戳数据的任何用例(包括DevOps监控,应用程序指标,物联网传感器数据和实时分析)的后端存储。

下载InfluxDB rpm安装包

wget https://dl.influxdata.com/influxdb/releases/influxdb-1.7.5.x86_64.rpm
--2019-04-10 02:52:30--  https://dl.influxdata.com/influxdb/releases/influxdb-1.7.5.x86_64.rpm
Resolving dl.influxdata.com (dl.influxdata.com)... 54.192.151.21, 54.192.151.81, 54.192.151.87, ...
Connecting to dl.influxdata.com (dl.influxdata.com)|54.192.151.21|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 46536692 (44M) [application/octet-stream]
Saving to: ‘influxdb-1.7.5.x86_64.rpm’

100%[================================================================================================================================================================================>] 46,536,692  440KB/s  in 60s   

2019-04-10 02:53:37 (756 KB/s) - ‘influxdb-1.7.5.x86_64.rpm’ saved [46536692/46536692]

安装rpm包

rpm -ivh influxdb-1.7.5.x86_64.rpm
Preparing...                          ################################# [100%]
Updating / installing...
  1:influxdb-1.7.5-1                ################################# [100%]
Created symlink from /etc/systemd/system/influxd.service to /usr/lib/systemd/system/influxdb.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/influxdb.service to /usr/lib/systemd/system/influxdb.service.

启动InfluxDB

service influxdb start
Redirecting to /bin/systemctl start influxdb.service

查看InfluxDB状态

ps -ef | grep influxdb
influxdb  8475      1  2 03:01 ?        00:00:00 /usr/bin/influxd -config /etc/influxdb/influxdb.conf
root      8486  7007  0 03:02 pts/0    00:00:00 grep --color=auto influxdb

service influxdb status
Redirecting to /bin/systemctl status influxdb.service
● influxdb.service - InfluxDB is an open-source, distributed, time series database
  Loaded: loaded (/usr/lib/systemd/system/influxdb.service; enabled; vendor preset: disabled)
  Active: active (running) since Wed 2019-04-10 03:01:48 EDT; 22s ago
    Docs: https://docs.influxdata.com/influxdb/
 Main PID: 8475 (influxd)
  CGroup: /system.slice/influxdb.service
          └─8475 /usr/bin/influxd -config /etc/influxdb/influxdb.conf

Apr 10 03:01:48 node1 influxd[8475]: ts=2019-04-10T07:01:48.375804Z lvl=info msg="Starting precreation service" log_id=0EiWgWRl000 service=shard-precreation check_interval=10m advance_period=30m
Apr 10 03:01:48 node1 influxd[8475]: ts=2019-04-10T07:01:48.375810Z lvl=info msg="Starting snapshot service" log_id=0EiWgWRl000 service=snapshot
Apr 10 03:01:48 node1 influxd[8475]: ts=2019-04-10T07:01:48.375816Z lvl=info msg="Starting continuous query service" log_id=0EiWgWRl000 service=continuous_querier
Apr 10 03:01:48 node1 influxd[8475]: ts=2019-04-10T07:01:48.375826Z lvl=info msg="Starting HTTP service" log_id=0EiWgWRl000 service=httpd authentication=false
Apr 10 03:01:48 node1 influxd[8475]: ts=2019-04-10T07:01:48.375830Z lvl=info msg="opened HTTP access log" log_id=0EiWgWRl000 service=httpd path=stderr
Apr 10 03:01:48 node1 influxd[8475]: ts=2019-04-10T07:01:48.375936Z lvl=info msg="Listening on HTTP" log_id=0EiWgWRl000 service=httpd addr=[::]:8086 https=false
Apr 10 03:01:48 node1 influxd[8475]: ts=2019-04-10T07:01:48.375949Z lvl=info msg="Starting retention policy enforcement service" log_id=0EiWgWRl000 service=retention check_interval=30m
Apr 10 03:01:48 node1 influxd[8475]: ts=2019-04-10T07:01:48.376138Z lvl=info msg="Listening for signals" log_id=0EiWgWRl000
Apr 10 03:01:48 node1 influxd[8475]: ts=2019-04-10T07:01:48.376389Z lvl=info msg="Storing statistics" log_id=0EiWgWRl000 service=monitor db_instance=_internal db_rp=monitor interval=10s
Apr 10 03:01:48 node1 influxd[8475]: ts=2019-04-10T07:01:48.376534Z lvl=info msg="Sending usage statistics to usage.influxdata.com" log_id=0EiWgWRl000

使用InfluxDB客户端

influx
Connected to :8086 version 1.7.5
InfluxDB shell version: 1.7.5
Enter an InfluxQL query
>

创建用户和数据库

> CREATE USER "admin" WITH PASSWORD 'admin' WITH ALL PRIVILEGES
> create database "jmxDB"

创建完成InfluxDB的用户和数据库暂时就够用了,其它简单操作如下,后面会用到

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:https://www.heiqu.com/6c9d2b24e66e4d4e601f49a516ffa762.html