Awstats with virtual hosts on Apache

Installing awstats

For this I used Fedora Core 2 with Apache 1.3 installed in /opt/apache-1.3/. I install awstats from an RPM downloaded from rpmfind.net. At the time of my writing this the latest rpm available is awstats-6.2-1.1.fc2.rf.noarch.rpm. The default installation directory is /var/www/awstats and the config files are placed in /etc/awstats. The installation directory is not the optimal as didn’t install apache in that location but it doesn’t really matter. After installation these lines should be present at the bottom of your httpd.conf files. If commented out, uncomment them.

Alias /awstatsclasses "/var/www/awstats/classes/"
Alias /awstatscss "/var/www/awstats/css/"
Alias /awstatsicons "/var/www/awstats/icon/"
ScriptAlias /awstats "/var/www/awstats/"

<Directory "/var/www/awstats">
    Options None
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>

The way I set it up requires basic authentication. Adding the required lines to the Directory part leaves it like below. In this case I specified the password file to reside in the same place as the rest of the awstats configuration. The awstats.pwd file will be created later.

<Directory "/var/www/awstats">
    Options None
    AllowOverride None
    Order allow,deny
    Allow from all
    AuthType Basic
    AuthName stats
    AuthUserFile /etc/awstats/awstats.pwd
    require valid-user
</Directory>

Setup apache logging

Awstats recommend you to split your logs between your virtual hosts so that each virtual host has its own apache access_log file. Even so I like to keep logging for all the virtual hosts in the same log file. I think it is a bit easier to handle the logfile and it also simplifies logrotating.

As I’m keeping all virtual hosts in the same logfile I need to make sure the virtual host is written to each line. This does not come out of the box and a small change in httpd.conf is needed. In httpd.conf you will find a few lines that specify different log formats. Per default the combined log format should be used. This is also what awstats recommend. It is defined in httpd.conf.

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

As I want to be able to differantiate log entries between different virtual hosts I want the combined format plus the virtual host. So I create my own log format that I call cplus. This format has %v prepended to it which will put the virtual host first on each log entry. I add the following line to httpd.conf

LogFormat "%v %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" cplus

I also need to tell apache to use the cplus log format instead of the combined format by changing the CustomLog directive to look like below.

CustomLog /opt/apache-1.3/logs/access_log cplus

I restart apache and have a look in access_log to verify that the virtual host is written on each log entry.

Configure awstats

With apache done it is time to configure awstats. I’m only interested in the file awstats.model.conf in /etc/awstats. So the first thing I do is to remove the everything else from /etc/awstats that was created during installation. This file, awstats.model.conf, is going to be the template for all other awstats.*.conf files as awstats needs one config file per virtual host. Setting up a new virtual host manually is boring as well as error prone so I create scripts to do most of the job. This is also the case with awstats.

There are several configuration changes needed in awstats.model.conf. As I don’t use the combined log format I have to specify a custom log format. In several places I change the values to something that is easier to parse with a script. Applicable entries and their values are:

LogFile="/opt/apache-1.3/logs/access_log"
LogFormat = "%virtualname %host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot"
SiteDomain="$DOMAIN"
HostAliases="$ALIASES"
DirData="/home/$USERNAME/awstats"
AllowAccessFromWebToAuthenticatedUsersOnly=1
AllowAccessFromWebToFollowingAuthenticatedUsers="$USERNAME"

There are loads of other stuff you can tweak but these are the ones important for this setup.

Set up statistics for a host

<VirtualHost *:80>
    ServerName www.mysite300.org
    ServerAlias mysite300.org
    DocumentRoot /home/mysite300_org/www
    ServerAdmin info@mysite300.org
</VirtualHost>

Everything is now in place to setup awstats for an individual virtual host. The above is a minimal example of a virtual host section in httpd.conf. If I want to manually setup awstats for www.mysite300.org I have to create a directory to host the statistics files. I have to copy awstats.model.conf to awstats.www.mysite300.org and change all the configuration values by hand. Then I need to run htpasswd to add a username and password to /etc/awstats/awstats.pwd to allow someone to see the statistics.

As I am lazy I want to cut this three minute job to a few seconds. I have created the following small script, addstat.sh, to ask the right questions and then perform the task. This script is run as root. It could be prettier but it gets the job done.

#!/bin/bash
echo "Enter the username:"
read WUSERNAME

ACCESS_FILE="/etc/awstats/awstats.pwd"
STAT_DIR="/home/$WUSERNAME/awstats"

echo "Enter the password:"
read PASSWORD

echo "Enter the main domain:"
read DOMAIN

echo "Enter aliases separated by space:"
read ALIASES

# Create the statistics directory
if [ -d $STAT_DIR ]; then
    echo "Statistics dir already exist"
else
    mkdir $STAT_DIR
fi

# Create the virtual host awstats.conf
cat /etc/awstats/awstats.model.conf | \
sed -e "s/\\\$DOMAIN/$DOMAIN/g" | \
sed -e "s/\\\$USERNAME/$WUSERNAME/g" | \
sed -e "s/\\\$ALIASES/$ALIASES/g" > \
"/etc/awstats/awstats.$DOMAIN.conf"

# Add user/password to password file
if [ -e $ACCESS_FILE ]; then
    /opt/apache-1.3/bin/htpasswd -bm $ACCESS_FILE $WUSERNAME $PASSWORD
else
    /opt/apache-1.3/bin/htpasswd -bm -c $ACCESS_FILE $WUSERNAME $PASSWORD
fi

After running this script to setup awstats for the example virtual host the user mysite300_org can go to http://www.mysite300.org/awstats/awstats.pl to view his statistics. The user will be presented with a login form and only allowed through if he presents the correct credentials.

Generating statistics

Now that I have run the script and setup statistics for this virtual host I want to generate statistics as well. Because it is still empty. I can do this manually through

/var/www/awstats/awstats.pl -update -config=www.mysite300.org

I have many virtual hosts and I want the statistics to be updated every day. So this is not an option. Creating a script and run it from a cron job could be a good idea. But an even better idea is to use logrotate to rotate the apache logs and have logrotate run our script.

A simple script, genstat.sh, to generate statistics for all awstats.*.conf files in /etc/awstats except awstats.model.conf is needed. I put the script in /root/scripts.

#!/bin/bash
CONF_DIR="/etc/awstats"
CONF_EXT="conf"

if [ ! -d "$CONF_DIR" ]; then
  echo "$CONF_DIR does not exist"
  exit 1
else
  cd $CONF_DIR
fi

for cfg in *.$CONF_EXT; do
if [ "$cfg" != "awstats.model.conf" ]; then
   site=`echo $cfg|sed -e "s/awstats.//g" | sed -e "s/.conf//g"`
   /var/www/awstats/awstats.pl -update -config="$site"
fi
done

Then I need to configure logrotate.conf in /etc to run this script prior to rotating the logs. If rpm was used to install apache it could have its own file in /etc/logrotate.d/. I normally install apache manually, so I add this entry directly in logrotate.conf.

/opt/apache-1.3/logs/access_log {
   rotate 31
   daily
   compress
   sharedscripts
   prerotate
     /root/scripts/dailystat.sh
   endscript
   postrotate
     /opt/apache-1.3/bin/apachectl restart
   endscript
}

There is one problem one can encounter with logrotate. There are many vulnerable applications on the web. When hosting applications I don’t have any control over I (ofcourse) try to secure the system as good as possible. As a security measure I have /tmp mounted with the options nosuid and noexec. The specific line in /etc/fstab looks like below.

LABEL=/tmp /tmp ext3 defaults,nosuid,noexec 1 2

Crackers and worms often exploit vulnerable web applications to upload scripts and run them from /tmp. Mounting /tmp with the noexec option is a good way to make sure this cannot happen even if there are vulnerable applications on the system. If you haven’t done so already – now is a good time to fix this. However, this also stops logrotate as it runs scripts from /tmp (or whatever you have TMPDIR defined as).

Logrotate is run daily per default. The acutal script can be found in /etc/cron.daily/logrotate. To fix this I create a tmp directory, /tmp_safe, that is going to be used by logrotate only. Then I add the following to the top of /etc/cron.daily/logrotate.

export TMPDIR=/tmp_safe

No logrotate will use /tmp_safe as a tmp directory instead of /tmp. All other applications running on the system will still use /tmp as normal.

Wrapping it up

Now I have a working awstats installation and logrotate rotates my apache access_log. The logs are compressed and saved for 31 days. When I want to add statistics possibilites for a virtual host I just run the script addstat.sh. Life is simple again.

Files created

• /root/scripts/addstat.sh
• /root/scripts/genstat.sh

Files changed

• /opt/apache-1.3/conf/httpd.conf
• /etc/awstats/awstats.model.conf
• /etc/logrotate.conf
• /etc/cron.daily/logrotate
• /etc/fstab