CentOS 7.2部署Elasticsearch-5.0.1+Kibana-5.0.1+Zookeeper3.4.6+Kafka_2.9.2-0.8.2.1详解。
环境准备：
操作系统：centos7.2 x86_64
# uname -a
Linux u04elk03.yaya.corp 3.10.0-327.22.2.el7.x86_64 #1 SMP Thu Jun 23 17:05:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
软件包：
jdk-8u102-linux-x64.rpm
kibana-5.0.1-x86_64.rpm
elasticsearch-5.0.1.rpm
kafka版本：2.9.2-0.8.2.1.tgz
zookeeper3.4.6.tar.gz
zook和kafka集群：
u04elk03.yaya.corp
u04elk02.yaya.corp
u04elk.yaya.corp
elasticsearch集群：
u04elk03.yaya.corp
u04elk02.yaya.corp
1.安装jdk1.8，记得删除之前的openjdk或者低版本jdk
rpm -e --nodeps Java-1.8.0-openjdk-headless-1.8.0.101-3.b13.el7_2.x86_64
rpm -e --nodeps Python-javapackages-3.4.1-11.el7.noarch
rpm -e --nodeps tzdata-java-2016f-1.el7.noarch
rpm -e --nodeps java-1.8.0-openjdk-1.8.0.101-3.b13.el7_2.x86_64
rpm -e --nodeps javapackages-tools-3.4.1-11.el7.noarch
2.安装elasticsearch-5.0.1
下载地址：https://www.elastic.co/downloads
rpm -ivh elasticsearch-5.0.1.rpm
u04elk02.yaya.corp的elasticsearch.yml配置
cluster.name: yunva_elk_cluster
node.name: u04elk02.yaya.corp
node.master: true
node.data: true
path.data: /data/es/data
path.logs: /data/es/logs
bootstrap.memory_lock: false
network.host: 0.0.0.0
http.port: 9200
u04elk03.yaya.corp的配置
# cat /etc/elasticsearch/elasticsearch.yml
cluster.name: yunva_elk_cluster
node.name: u04elk03.yaya.corp
node.master: true
path.data: /data/es/data
path.logs: /data/es/logs
bootstrap.memory_lock: false
network.host: 0.0.0.0
http.port: 9200
创建日志和数据存放目录
mkdir -p /data/es/data
mkdir /data/es/logs
chown -R elasticsearch.elasticsearch /data/es
启动elasticsearch
systemctl start elasticsearch
systemctl enable elasticsearch
访问以下web地址验证是否ok：
:9200/
出现如下信息，说明没有问题
{
"name" : "u04elk02.yaya.corp",
"cluster_name" : "yunva_elk_cluster",
"cluster_uuid" : "_6G7IvUeT6-2Qdh_AXo2mA",
"version" : {
"number" : "5.0.1",
"build_hash" : "080bb47",
"build_date" : "2016-11-11T22:08:49.812Z",
"build_snapshot" : false,
"lucene_version" : "6.2.1"
},
"tagline" : "You Know, for Search"
}
可以查看相关集群的状态信息
# curl -XGET 'http://localhost:9200/_cluster/state/{metrics}/{indices}'
{"cluster_name":"yunva_elk_cluster"}
# curl -XGET 'http://localhost:9200/_cluster/state?pretty'
{
"cluster_name" : "yunva_elk_cluster",
"version" : 2,
"state_uuid" : "hHaLqrlyT4KszbCCZ5Yy-Q",
"master_node" : "oe-62CJxS_G5sKLYWeepqA",
"blocks" : { },
"nodes" : {
"oe-62CJxS_G5sKLYWeepqA" : {
"name" : "u04elk02.yaya.corp",
"ephemeral_id" : "qhW1YiSURwKZUenjKDRWZw",
"transport_address" : "10.19.124.119:9300",
"attributes" : { }
}
},
"metadata" : {
"cluster_uuid" : "4b6LLl0GRdWNm9r7acGE8Q",
"templates" : { },
"indices" : { },
"index-graveyard" : {
"tombstones" : [ ]
}
},
"routing_table" : {
"indices" : { }
},
"routing_nodes" : {
"unassigned" : [ ],
"nodes" : {
"oe-62CJxS_G5sKLYWeepqA" : [ ]
}
}
}
elasticsearch-5.0.1 只支持内核3.5以上版本的linux操作系统
centos6.5报错：
[2016-11-23T16:16:24,329][WARN ][o.e.b.JNANatives ] unable to install syscall filter:
java.lang.UnsupportedOperationException: seccomp unavailable: requires kernel 3.5+ with CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER compiled in
①优化elasticsearch内存配置：
修改配置文件：
vim /etc/elasticsearch/jvm.options
-Xms10g
-Xmx10g
建议配置为物理最大内存的一半以上，在/usr/share/elasticsearch/bin/elasticsearch.in.sh配置没有用
查看日志出现类似heap size [9.9gb] 说明配置成功：
[2016-11-23T20:02:52,900][INFO ][o.e.e.NodeEnvironment ] [u04elk02.yaya.corp] heap size [9.9gb], compressed ordinary object pointers [true]
②配置文件句柄
vim /etc/security/limits.d/20-nproc.conf
删除这行：
* soft nproc 4096
3.安装kibana
rpm -ivh kibana-5.0.1-x86_64.rpm
编辑配置文件
vim /etc/kibana/kibana.yml
修改这两项，其他默认不用动
server.host: "0.0.0.0"
elasticsearch.url: "http://localhost:9200"
启动命令：
systemctl start kibana
systemctl enable kibana
4.安装zookeeper集群，单独配置，不用kafka自带的(集群节点建议是单数个，本次我们有启用3个节点)
下载地址：
tar -zxf zookeeper-3.4.6.tar.gz
cd /data/zookeeper-3.4.6/
cp zoo_sample.cfg zoo.cfg