在CentOS 6上使用AWStats分析httpd和Tomcat日志(3)

三、用awstats分析tomcat的访问日志
1、要分析tomcat的日志，就要首先了解其日志格式。
并比较与httpd的访问日志格式有什么不同之处，然后就可以参照awstats分析httpd日志的格式来定义awstats分析tomcat的日志。
我的tomcat服务器上定义的访问日志格式如下：
<Valve className="org.apache.catalina.valves.
AccessLogValve" directory="logs"
prefix="localhost_access_log." suffix=".txt"
      pattern="%h %l %u %t &quot;%r&quot; %s %b" />
%...a: 远程IP地址 
%...A: 本地IP地址 
%...B: 已发送的字节数，不包含HTTP头 
%...b: CLF格式的已发送字节数量，不包含HTTP头。 
例如当没有发送数据时，写入‘-’而不是0。 
%e: 环境变量FOOBAR的内容 
%...f: 文件名字 
%...h: 远程主机 
%...H 请求的协议 
%i: Foobar的内容，发送给服务器的请求的标头行。 
%...l: 远程登录名字（来自identd，如提供的话） 
%...m 请求的方法 
%n: 来自另外一个模块的注解“Foobar”的内容 
%o: Foobar的内容，应答的标头行 
%...p: 服务器响应请求时使用的端口 
%...P: 响应请求的子进程ID。 
%...q 查询字符串（如果存在查询字符串，则包含“?”后面的 
部分；否则，它是一个空字符串。） 
%...r: 请求的第一行 
%...s: 状态。对于进行内部重定向的请求，这是指*原来*请求 
的状态。如果用%...>s，则是指后来的请求。 
%...t: 以公共日志时间格式表示的时间（或称为标准英文格式） 
%t: 以指定格式format表示的时间 
%...T: 为响应请求而耗费的时间，以秒计 
%...u: 远程用户（来自auth；如果返回状态（%s）是401则可能是伪造的） 
%...U: 用户所请求的URL路径 
%...v: 响应请求的服务器的ServerName 
%...V: 依照UseCanonicalName设置得到的服务器名字 
最后的tomcat的访问日志内容如下：
203.156.200.162 - - [29/Aug/2012:11:16:58 +0800] "GET /front/magazine/getContent.htm?contentId=124504 HTTP/1.1" 200 20001
2、由于我的tomcat服务器是在其他机器上，所以我将tomcat的服务日志copy到本机的/var/log/httpd/下即可。
如copy的文件是：localhost_access_log.2012-08-29.txt
3、配置awstats分析此日志(tomcat 的域名并不是httpd的虚拟主机，所以没有写进httpd.conf文件里面)
# cd /usr/local/awstats/tools
# perl awstats_configure.pl
----- AWStats awstats_configure 1.0 (build 1.9) (c) Laurent Destailleur -----
This tool will help you to configure AWStats to analyze statistics for
one web server. You can try to use it to let it do all that is possible
in AWStats setup, however following the step by step manual setup
documentation (docs/index.html) is often a better idea. Above all if:
- You are not an administrator user,
- You want to analyze downloaded log files without web server,
- You want to analyze mail or ftp log files instead of web log files,
- You need to analyze load balanced servers log files,
- You want to 'understand' all possible ways to use AWStats...
Read the AWStats documentation (docs/index.html).
 
-----> Running OS detected: Linux, BSD or Unix
 
-----> Check for web server install
 
Enter full config file path of your Web server.
Example: /etc/httpd/httpd.conf
Example: /usr/local/apache2/conf/httpd.conf
Example: c:\Program files\apache group\apache\conf\httpd.conf
Config file path ('none' to skip web server setup):
>none
Your web server config file(s) could not be found.
You will need to setup your web server manually to declare AWStats
script as a CGI, if you want to build reports dynamically.
See AWStats setup documentation (file docs/index.html)
 
-----> Update model config file '/usr/local/awstats/wwwroot/cgi-bin/awstats.model.conf'
 File awstats.model.conf updated.
 
-----> Need to create a new config file ?
Do you want me to build a new AWStats config/profile
file (required if first install) [y/N] ? y
 
-----> Define config file name to create
What is the name of your web site or profile analysis ?
Example:
Example: demo
Your web site, virtual server or profile name:
>linuxidc.com
-----> Define config file path
In which directory do you plan to store your config file(s) ?
Default: /etc/awstats
Directory path to store config file(s) (Enter for default):
>
 
-----> Create config file '/etc/awstats/awstats.linuxidc.com.conf'
 Config file /etc/awstats/awstats.linuxidc.com.conf created.
 
-----> Add update process inside a scheduler
Sorry, configure.pl does not support automatic add to cron yet.
You can do it manually by adding the following command to your cron:
/usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=linuxidc.com
Or if you have several config files and prefer having only one command:
/usr/local/awstats/tools/awstats_updateall.pl now
Press ENTER to continue...
 
 
A SIMPLE config file has been created: /etc/awstats/awstats.linuxidc.com.conf
You should have a look inside to check and change manually main parameters.
You can then manually update your statistics for 'linuxidc.com' with command:
> perl awstats.pl -update -config=linuxidc.com
You can also build static report pages for 'linuxidc.com' with command:
> perl awstats.pl -output=pagetype -config=linuxidc.com
 
Press ENTER to finish...
4、修改要分析日志文件
# vim /etc/awstats/awstats.linuxidc.com.conf
将LogFile="/var/log/httpd/mylog.log"
改为LogFile="/var/log/httpd/localhost_access_log.2012-08-29.txt"
LogFile="/var/log/httpd/localhost_access_log. YYY-0%MM-0%DD-0.txt"都可以。
5、重启httpd服务，并分析日志
# service httpd restart
# cd /usr/local/awstats/wwwroot/cgi-bin
# perl awstats.pl -update -config=linuxidc.com
Create/Update database for config "/etc/awstats/awstats.linuxidc.com.conf" by AWStats version 7.0 (build 1.976)
From data in log file "/var/log/httpd/localhost_access_log.2012-08-29.txt"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
AWStats did not find any valid log lines that match your LogFormat parameter, in the 50th first non commented lines read of your log.
Your log file /var/log/httpd/localhost_access_log.2012-08-29.txt must have a bad format or LogFormat parameter setup does not match this format.
Your AWStats LogFormat parameter is:
1
This means each line in your web server log file need to have "combined log format" like this:
111.22.33.44 - - [10/Jan/2001:02:14:14 +0200] "GET / HTTP/1.1" 200 1234 "http://www.fromserver.com/from.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)"
And this is an example of records AWStats found in your log file (the record number 50 in your log):
203.208.60.231 - - [29/Aug/2012:00:02:47 +0800] "GET /front/magazine/getContent.htm?contentId=52253 HTTP/1.1" 200 18419
Setup ('/etc/awstats/awstats.linuxidc.com.conf' file, web server or permissions) may be wrong.
Check config file, permissions and AWStats documentation (in 'docs' directory).