CenttOS 6.6下部署ELK日志告警系统(3)

[Confluent]
name=Confluent repository
baseurl=https://packages.confluent.io/rpm/4.0
gpgcheck=1
gpgkey=https://packages.confluent.io/rpm/4.0/archive.key
enabled=1
EOF

yum install confluent-platform-oss-2.11

Elastalert

Elastalert可以部署到任何一台能够读取到ES的服务器上；配置文件中modules.eagle_post.EagleAlerter blacklist_v2经过修改，后面会介绍到

rules/system_log.yaml

es_host: <亚马逊ES地址>
es_port: 80
name: system log rule
type: blacklist_v2
index: system_log*
timeframe:
    minutes: 1

# 监控key
compare_key: system.syslog.message

# 出现下面任意关键字将告警，按需添加
blacklist_v2:
  - "ERROR"
  - "error"

alert: "modules.eagle_post.EagleAlerter"
eagle_post_url: "<eagle>"
eagle_post_all_values: False
eagle_post_payload:
  server: "fields.server_name"
  info: "system.syslog.message"
  source: "source"

rules/system_log.yaml

es_host: <亚马逊ES地址>
es_port: 80
name: system secure rule
type: frequency
index: system_secure*
num_events: 1
timeframe:
    minutes: 1
filter:
- query:
    wildcard:
      system.auth.user : "*"
alert: "modules.eagle_post.EagleAlerter"
eagle_post_url: "<eagle>"
eagle_post_all_values: False

# 非工作时间
eagle_time_start: "09:00"
eagle_time_end: "18:00"
eagle_post_payload:
  user: "system.auth.user"
  server: "fields.server_name"
  ip: "system.auth.ssh.ip"
  event: "system.auth.ssh.event"

Elastalert

自定义type与alert

为了能够将告警接入到Eagle（自研统一接口平台）在尝试使用http_post做告警类型过程中，发现无法传入ES结果作为POST参数，所以对其进行简单修改，新增类型，实现能够无缝接入Eagle

Alert

moudules/eagle_post.py

将文件夹保存到site-packages/elastalert

import json
import requests
import dateutil.parser
import datetime
from elastalert.alerts import Alerter
from elastalert.util import EAException
from elastalert.util import elastalert_logger
from elastalert.util import lookup_es_key

class EagleAlerter(Alerter):
    def __init__(self, rule):
        super(EagleAlerter, self).__init__(rule)

# 设定时间有效范围
        self.post_time_start = self.rule.get('eagle_time_start','00:00')
        self.post_time_end = self.rule.get('eagle_time_end','00:00')

# post链接
        self.post_url = self.rule.get('eagle_post_url','')
        self.post_payload = self.rule.get('eagle_post_payload', {})
        self.post_static_payload = self.rule.get('eagle_post_static_payload', {})
        self.post_all_values = self.rule.get('eagle_post_all_values', False)
        self.post_lock = False

def alert(self, matches):
        if not self.post_url:
            elastalert_logger.info('Please input eagle url!')
            return False

for match in matches:

# 获取所有payload
            payload = match if self.post_all_values else {}

# 构建字典
            for post_key, es_key in self.post_payload.items():
                payload[post_key] = lookup_es_key(match, es_key)

# 获取当前时间
            login_time = datetime.datetime.now().time()

# 获取时间限制
            time_start = dateutil.parser.parse(self.post_time_start).time()
            time_end =  dateutil.parser.parse(self.post_time_end).time()

# 如果在时间范围内，将不做告警
            self.post_lock = False if login_time > time_start and \
                                login_time < time_end else True

# 合并两种类型payload
            data = self.post_static_payload
            data.update(payload)

# 发送告警
            if self.post_lock:
                myRequests = requests.Session()
                myRequests.post(url=self.post_url,data=data,verify=False)
                elastalert_logger.info("[-] eagle alert sent.")
            else:
                elastalert_logger.info("[*] nothing to do.")
    def get_info(self):
        return {'type': 'http_post'}

type

在使用blaklist过程发现改类型是全匹配，为了方便编写配置文件，所以对其做了简单修改

elastalert/ruletypes.py

# 新增

class BlacklistV2Rule(CompareRule):
    required_options = frozenset(['compare_key', 'blacklist_v2'])