Fluent将日志转储到多个文件和MongoDB

1、转储到单个文件的配置

<source>
 type tail 
 path /home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1001.log
format /^*(?<message>.*)$/
 tag mongo.apache
</source>
<source>
 type tail
 path /home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1002.log
format /^*(?<message>.*)$/

 tag mongo.apache
</source>

匹配的时候第一个mongo.apache结果生效，所以如下转储到mongodb会生效，后面的转储到文件不会生效，因此如下的配置已经注释掉了，为的是让后面的转储文件生效。

#<match mongo.**>
# type mongo
# database apache
# collection fmongo
# host Server202
# port 10001
# flush_interval 1s
#</match>

<match mongo.**>

  type file
  path /var/log/fluent/access

</match>

到目录下验证转储效果

[root@Server202 fluent]# pwd
/var/log/fluent
[root@Server202 fluent]# ll
total 64
-rw-rw-rw- 1 root root 57824 Sep 27 14:06 access.20120927.b4caa87ded0f11576

有一个文件access.20120927.b4caa87ded0f11576 

tail这个文件，可以看到日志在不断变化着

tail -f access.20120927.b4caa87ded0f11576 
2012-09-27T14:07:14+08:00       mongo.apache    {"message":"Thu Sep 27 14:07:14 [conn10450] end connection 192.168.19.202:42236"}
2012-09-27T14:07:14+08:00       mongo.apache    {"message":"Thu Sep 27 14:07:14 [initandlisten] connection accepted from 192.168.19.

2、转储到多个文件的配置

<match mongo.**>
  type copy
 <store>
  type file
  path /var/log/fluent/access
</store>
<store>
 type file
path /var/log/fluent/bccess
</store>
</match>

copy指定多个输出

store规则与match相同

验证结果：

[root@Server202 fluent]# ll
total 184
-rw-rw-rw- 1 root root 165074 Sep 27 15:20 access.20120927.b4caa87ded0f11576
-rw-rw-rw- 1 root root  16034 Sep 27 15:20 bccess.20120927.b4caa9b5665aeb9b4

有2个转储后的文件分别为access.20120927.b4caa87ded0f11576 和  bccess.20120927.b4caa9b5665aeb9b4

分别tail 会看到相同的日志输出

tail -f access.20120927.b4caa87ded0f11576 

2012-09-27T15:22:20+08:00       mongo.apache    {"message":"Thu Sep 27 15:22:20 [conn10831] end connection 192.168.19.201:54006"}
2012-09-27T15:22:20+08:00       mongo.apache    {"message":"Thu Sep 27 15:22:20 [initandlisten] connection accepted from 192.168.19.201:54012 #10834"}

tail -f bccess.20120927.b4caa9b5665aeb9b4 

2012-09-27T15:22:20+08:00       mongo.apache    {"message":"Thu Sep 27 15:22:20 [conn10831] end connection 192.168.19.201:54006"}
2012-09-27T15:22:20+08:00       mongo.apache    {"message":"Thu Sep 27 15:22:20 [initandlisten] connection accepted from 192.168.19.201:54012 #10834"}

3、转储到2个文件，一个mongodb的配置。

<match mongo.**>
  type copy
 <store>
  type file
  path /var/log/fluent/access
</store>
<store>
 type file
path /var/log/fluent/bccess
</store>
<store>
 type mongo
 database apache
 collection fmongo
 host Server202
 port 10001
flush_interval 1s
</store>
</match>