/var/log/httpd/error_log
hosts: 【存放路径】
③添加数据
for i in {1..20};do echo "test $i" > /var/www/html/test${i}.html;done
启动httpd服务和filebeat服务
systemctl start httpd
随意找个主机进行访问，为了得到日志
for i in {1..20};do j=$[$RANDOM%20+1]；curl ${j}.html
④输出给elasticsearch
vim /etc/filebeat.yml
output.elasticsearch :
hosts: ["server1:9200“，“server2:9200”，“server3:9200”]
直接输出给els不用输出插件了
⑤输出给logstash
vim /etc/filebeat.yml
output.logstash :
hosts: ["172.16.0.4:5044]
vim /etc/logstash/conf.d/Apachelog.conf
input {
beats {
port => 5044
}
filter {
grok {
match => {
"message" => "%{HTTPD_COMBINEDLOG}"
}
}
date {
match => ["timestamp","dd/MMM/YYYY:H:m:s Z"]
}
mutate {
rename => {
"agent" => "user_agent"
}
}
geoip {
source => "clientip"
target => "geoip"
database => "/etc/logstash/maxmind/GeoLite2-City.mmdb"
}
output {
elasticsearsh {
hosts => ["http://server1:9200","http://server2:9200","http://master:9200"]
index => "logstash-%{+YYYY.MM.DD}
document_type => "http_access_logs"
}
}
启动：logstash -f apachelog.conf
⑥输出给redis
编辑filebeat配置文件
vim /etc/filebeat.yml
添加：
output.redis:
hosts: ["redis服务器"]
password: "iliunx.io"
key: "httplog"
db: 0
timeout: 5
重启filebeat
systemctl restart filebeat
进入redis查看数据
redis-cli -a ilinux.io
查看有多少数据
LLEN httplogs
在els server端配置输入机制
vim /etc/elasticsearch/conf.d/redis2.conf
input {
redis {
batch_count => 1
data_type => "list"
key => "httpdlogs"
host => "192.168.0.2"
port => 6379
threads => 5
password => "ilinux.io"
}
}
filter {
grok {
match => {
"message" => "%{HTTPD_COMBINEDLOG}"
}
}
date {
match => ["timestamp","dd/MMM/YYYY:H:m:s Z"]
}
mutate {
rename => {
"agent" => "user_agent"
}
}
geoip {
source => "clientip"
target => "geoip"
database => "/etc/logstash/maxmind/GeoLite2-City.mmdb"
}
output {
elasticsearsh {
hosts => ["http://server1:9200","http://server2:9200","http://server3:9200"]
index => "logstash-%{+YYYY.MM.DD}
document_type => "http_access_logs"
}
}