在Docker容器中运行Elasticsearch Kibana和Cerebro
机器信息
10.160.13.139 10.160.9.162 10.160.11.171 1. 安装docker和docker compose安装docker
# 安装 docker 社区版 yum -y install docker-ce docker-ce-cli containerd.io # 启动 Docker systemctl start docker # 查看 docker 版本 docker version # 开机启动 Docker systemctl enable docker安装docker-compose
sudo curl -L https://github.com/docker/compose/releases/download/1.27.4/docker-compose-`uname -s`-`uname -m` \ > /usr/local/bin/docker-compose # 添加执行权限 sudo chmod +x /usr/local/bin/docker-compose # 验证 docker-compose --version docker-compose version 1.27.4, build 40524192 2. 下载镜像 docker pull elasticsearch:7.10.1 docker pull kibana:7.10.1 3. 将 kibana 上传到私有harbor [@master software]# docker images |grep kibana kibana 7.10.1 3e014820ee3f 3 weeks ago 992MB [@master software]# docker tag 3e014820ee3f :7.10.1 [@master software]# docker login Username: song Password: Login Succeeded [@master software]# docker push :7.10.1 The push refers to repository [] 00cfe116c0b3: Pushed 20889a7593bc: Pushed d1409dea493d: Pushed 3448a6e61306: Pushed f3a4630cb1b0: Pushed 0e3f51adbdfe: Pushed 7bb6eb357f15: Pushed 0263a48f9a3a: Pushed 5acb536ef82f: Pushed 58d51bd5abeb: Pushed e116279745ca: Pushed cc33c1bea858: Pushed 5ec5e9c8cf6f: Pushed 291f6e44771a: Pushed 7.10.1: digest: sha256:1731793b7f3e453c65ebaf92ec0b55f4029310ba8abae9e04753a4680dd8210b size: 3247 [@master software]# 4. 将 elasticsearch 上传到私有harbor [@master software]# docker images |grep elastic elasticsearch 7.10.1 558380375f1a 3 weeks ago 774MB [@master software]# docker tag 558380375f1a :7.10.1 [@master software]# docker push :7.10.1 The push refers to repository [] d6208dd58075: Pushed 9117213e7470: Pushed b662ef6f4fc1: Pushed 3f481e912123: Pushed b456348ecd84: Pushed e2834bada95f: Pushed 8f84abff69b8: Pushed 911490c8ab00: Pushed 291f6e44771a: Mounted from elk/kibana 7.10.1: digest: sha256:e9a1fe65f68b2d2b9583287d1190f67f23af08582eac4d2a8dc342e4219c7306 size: 2202 [@master software]# 5 .操作系统优化操作系统以及JVM调优主要是针对安装elasticsearch的机器。对于操作系统，需要调整几个内核参数
vim /etc/sysctl.conf # fs.file-max主要是配置系统最大打开文件描述符数，建议修改为655360或者更高 fs.file-max=655360 # vm.max_map_count影响Java线程数量，用于限制一个进程可以拥有的VMA(虚拟内存区域)的大小 # 系统默认是65530，建议修改成262144或者更高。 vm.max_map_count = 262144 sysctl -p sysctl -a另外，还需要调整进程最大打开文件描述符(nofile)、最大用户进程数(nproc)和最大锁定内存地址空间(memlock)，
vim /etc/security/limits.conf * soft nproc 20480 * hard nproc 20480 * soft nofile 65536 * hard nofile 65536 * soft memlock unlimited * hard memlock unlimited # 备注： * 代表Linux所有用户名称（比如es），保存，退出，重新登录才可生效 # 查看 ulimit -a最后，还需要修改
# /etc/security/limits.d/90-nproc.conf（centos6.x系统） # /etc/security/limits.d/20-nproc.conf（centos7.x系统） vim /etc/security/limits.d/20-nproc.conf * soft nproc 4096 # 修改为： * soft nproc 20480或者直接删除/etc/security/limits.d/20-nproc.conf文件也行。
6. JVM调优JVM调优主要是针对elasticsearch的JVM内存资源进行优化，elasticsearch的内存资源配置文件为jvm.options，7.1下载的默认设置是1GB
从ES 6开始，只支持64位的JVM（配置config/jvm.options）
配置的建议
将内存Xms和Xmx设置成一样，避免heap resize时引发停顿
Xmx设置不要超过物理内存的50%，单个节点上，最大内存建议不要超过32G内存
生产环境，JVM必须使用Server模式
关闭JVM Swapping
vim /usr/local/elasticsearch/config/jvm.options -Xms30g -Xmx30g 7. 安装 elasticsearch你可以先新建一个 elasticsearch 文件夹。然后在这个文件夹中，新建一个 docker-compose.yml 文件。
运行docker-compose，本地构建更高效的开发环境，更直观地了解 Elasticsearch 分布式特性
集成Cerebro，方便查看集群状态
mkdir -p /search/odin/elasticsearch/{data01,data02,data03} mkdir -p /Users/song/study/elasticsearch/plugins/ik # ES_JAVA_OPTS="-Xms64m -Xmx512m" 内存限制，最小64m，最大512m，可根据需要自行调整，除此之外其他的配置不用做修改 cat /search/odin/elasticsearch/docker-compose.yml version: '2.2' services: cerebro: image: lmenezes/cerebro:0.8.3 container_name: cerebro ports: - "9000:9000" command: - -Dhosts.0.host=http://elasticsearch:9200 networks: - elastic kibana: image: kibana:7.10.1 container_name: kibana7 environment: - I18N_LOCALE=zh-CN - XPACK_GRAPH_ENABLED=true - TIMELION_ENABLED=true - XPACK_MONITORING_COLLECTION_ENABLED="true" ports: - "5601:5601" networks: - elastic elasticsearch: image: elasticsearch:7.10.1 container_name: es01 environment: - node.name=es01 - cluster.name=es-docker-cluster - discovery.seed_hosts=es02,es03 - cluster.initial_master_nodes=es01,es02,es03 - bootstrap.memory_lock=true - "ES_JAVA_OPTS=-Xms1024m -Xmx1024m" ulimits: nproc: 65535 memlock: soft: -1 hard: -1 volumes: - data01:/usr/share/elasticsearch/data - /Users/song/study/elasticsearch/plugins/ik:/usr/share/elasticsearch/plugins/ik ports: - 9200:9200 - 9300:9300 networks: - elastic elasticsearch2: image: elasticsearch:7.10.1 container_name: es02 environment: - node.name=es02 - cluster.name=es-docker-cluster - discovery.seed_hosts=es01,es03 - cluster.initial_master_nodes=es01,es02,es03 - bootstrap.memory_lock=true - "ES_JAVA_OPTS=-Xms1024m -Xmx1024m" ulimits: nproc: 65535 memlock: soft: -1 hard: -1 volumes: - data02:/usr/share/elasticsearch/data - /Users/song/study/elasticsearch/plugins/ik:/usr/share/elasticsearch/plugins/ik ports: - 9201:9201 - 9301:9301 networks: - elastic elasticsearch3: image: elasticsearch:7.10.1 container_name: es03 environment: - node.name=es03 - cluster.name=es-docker-cluster - discovery.seed_hosts=es01,es02 - cluster.initial_master_nodes=es01,es02,es03 - bootstrap.memory_lock=true - "ES_JAVA_OPTS=-Xms1024m -Xmx1024m" ulimits: nproc: 65535 memlock: soft: -1 hard: -1 volumes: - data03:/usr/share/elasticsearch/data - /Users/song/study/elasticsearch/plugins/ik:/usr/share/elasticsearch/plugins/ik ports: - 9202:9202 - 9302:9302 networks: - elastic elastichd: image: containerize/elastichd:latest container_name: elasticsearch-hd networks: - elastic ports: - 9800:9800 depends_on: - "elasticsearch" links: - "elasticsearch:demo" volumes: data01: driver: local data02: driver: local data03: driver: local networks: elastic: driver: bridge运行docker-compose，安装Elasticsearch
启动
screen docker-compose up访问测试
curl -X GET "localhost:9200/_cat/nodes?v&pretty"停止容器
docker-compose down停止容器并且移除数据
docker-compose down -v