The reason is that many web developers don’t have a clue about the difference between the http methods GET and POST. Some know approximately the technical difference but not why one should use one method over the other. According to the RFC the GET method should never “alter the world”. In other words – POST should be used whenever there is a need to alter data and GET should only ever be used to fetch data.

So what does this have to do the Google Web Accelerator? It is very common in web applications to present, for example, lists of data like users or articles with links to either display or delete the data.

Consider what happens if the application user has GWA installed when all the links are prefetched. All the links will be invisibly followed and all data deleted. Many applications protect links like this with onclick handlers that ask the user if they want to delete the data. As the GWA has no concept of javascript it just ignores this fact and the links will be prefetched anyway with potentially disastrous results.

It is not Google that has done anything wrong. They follow the guidelines in the RFC on the correct usage of GET and POST.

Personally I think my applications are safe. I usually follow the guidelines on GET and POST but not always. But as I use only javascript calls to fetch or do things in my applications I am safe.

Safe

<a href="#" onclick="deleteUser(372);">Delete user</a>

Very dangerous (and wrong)

<a href="deleteUser.php?id=372">Delete user</a>

A safeguard for Apache

RewriteCond %{HTTP:x-moz} ^prefetch
RewriteRule ^/*admin/.* - [F,L]

I found the Apache solution on the blog of Christian Stocker who got it from the Syntax CMS blog through Planet OSCOM. (The blogging world of today…) Anyway, it is most likely a good idea to review your applications before users of your applications starts using GWA and this becomes a serious issue.

Installing awstats

For this I used Fedora Core 2 with Apache 1.3 installed in /opt/apache-1.3/. I install awstats from an RPM downloaded from rpmfind.net. At the time of my writing this the latest rpm available is awstats-6.2-1.1.fc2.rf.noarch.rpm. The default installation directory is /var/www/awstats and the config files are placed in /etc/awstats. The installation directory is not the optimal as didn’t install apache in that location but it doesn’t really matter. After installation these lines should be present at the bottom of your httpd.conf files. If commented out, uncomment them.

Alias /awstatsclasses "/var/www/awstats/classes/"
Alias /awstatscss "/var/www/awstats/css/"
Alias /awstatsicons "/var/www/awstats/icon/"
ScriptAlias /awstats "/var/www/awstats/"

<Directory "/var/www/awstats">
    Options None
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>

The way I set it up requires basic authentication. Adding the required lines to the Directory part leaves it like below. In this case I specified the password file to reside in the same place as the rest of the awstats configuration. The awstats.pwd file will be created later.

<Directory "/var/www/awstats">
    Options None
    AllowOverride None
    Order allow,deny
    Allow from all
    AuthType Basic
    AuthName stats
    AuthUserFile /etc/awstats/awstats.pwd
    require valid-user
</Directory>

Setup apache logging

Awstats recommend you to split your logs between your virtual hosts so that each virtual host has its own apache access_log file. Even so I like to keep logging for all the virtual hosts in the same log file. I think it is a bit easier to handle the logfile and it also simplifies logrotating.

As I’m keeping all virtual hosts in the same logfile I need to make sure the virtual host is written to each line. This does not come out of the box and a small change in httpd.conf is needed. In httpd.conf you will find a few lines that specify different log formats. Per default the combined log format should be used. This is also what awstats recommend. It is defined in httpd.conf.

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

As I want to be able to differantiate log entries between different virtual hosts I want the combined format plus the virtual host. So I create my own log format that I call cplus. This format has %v prepended to it which will put the virtual host first on each log entry. I add the following line to httpd.conf

LogFormat "%v %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" cplus

I also need to tell apache to use the cplus log format instead of the combined format by changing the CustomLog directive to look like below.

CustomLog /opt/apache-1.3/logs/access_log cplus

I restart apache and have a look in access_log to verify that the virtual host is written on each log entry.

Configure awstats

With apache done it is time to configure awstats. I’m only interested in the file awstats.model.conf in /etc/awstats. So the first thing I do is to remove the everything else from /etc/awstats that was created during installation. This file, awstats.model.conf, is going to be the template for all other awstats.*.conf files as awstats needs one config file per virtual host. Setting up a new virtual host manually is boring as well as error prone so I create scripts to do most of the job. This is also the case with awstats.

There are several configuration changes needed in awstats.model.conf. As I don’t use the combined log format I have to specify a custom log format. In several places I change the values to something that is easier to parse with a script. Applicable entries and their values are:

LogFile="/opt/apache-1.3/logs/access_log"
LogFormat = "%virtualname %host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot"
SiteDomain="$DOMAIN"
HostAliases="$ALIASES"
DirData="/home/$USERNAME/awstats"
AllowAccessFromWebToAuthenticatedUsersOnly=1
AllowAccessFromWebToFollowingAuthenticatedUsers="$USERNAME"

There are loads of other stuff you can tweak but these are the ones important for this setup.

Set up statistics for a host

<VirtualHost *:80>
    ServerName www.mysite300.org
    ServerAlias mysite300.org
    DocumentRoot /home/mysite300_org/www
    ServerAdmin info@mysite300.org
</VirtualHost>

Everything is now in place to setup awstats for an individual virtual host. The above is a minimal example of a virtual host section in httpd.conf. If I want to manually setup awstats for www.mysite300.org I have to create a directory to host the statistics files. I have to copy awstats.model.conf to awstats.www.mysite300.org and change all the configuration values by hand. Then I need to run htpasswd to add a username and password to /etc/awstats/awstats.pwd to allow someone to see the statistics.

As I am lazy I want to cut this three minute job to a few seconds. I have created the following small script, addstat.sh, to ask the right questions and then perform the task. This script is run as root. It could be prettier but it gets the job done.

#!/bin/bash
echo "Enter the username:"
read WUSERNAME

ACCESS_FILE="/etc/awstats/awstats.pwd"
STAT_DIR="/home/$WUSERNAME/awstats"

echo "Enter the password:"
read PASSWORD

echo "Enter the main domain:"
read DOMAIN

echo "Enter aliases separated by space:"
read ALIASES

# Create the statistics directory
if [ -d $STAT_DIR ]; then
    echo "Statistics dir already exist"
else
    mkdir $STAT_DIR
fi

# Create the virtual host awstats.conf
cat /etc/awstats/awstats.model.conf | \
sed -e "s/\\\$DOMAIN/$DOMAIN/g" | \
sed -e "s/\\\$USERNAME/$WUSERNAME/g" | \
sed -e "s/\\\$ALIASES/$ALIASES/g" > \
"/etc/awstats/awstats.$DOMAIN.conf"

# Add user/password to password file
if [ -e $ACCESS_FILE ]; then
    /opt/apache-1.3/bin/htpasswd -bm $ACCESS_FILE $WUSERNAME $PASSWORD
else
    /opt/apache-1.3/bin/htpasswd -bm -c $ACCESS_FILE $WUSERNAME $PASSWORD
fi

After running this script to setup awstats for the example virtual host the user mysite300_org can go to http://www.mysite300.org/awstats/awstats.pl to view his statistics. The user will be presented with a login form and only allowed through if he presents the correct credentials.

Generating statistics

Now that I have run the script and setup statistics for this virtual host I want to generate statistics as well. Because it is still empty. I can do this manually through

/var/www/awstats/awstats.pl -update -config=www.mysite300.org

I have many virtual hosts and I want the statistics to be updated every day. So this is not an option. Creating a script and run it from a cron job could be a good idea. But an even better idea is to use logrotate to rotate the apache logs and have logrotate run our script.

A simple script, genstat.sh, to generate statistics for all awstats.*.conf files in /etc/awstats except awstats.model.conf is needed. I put the script in /root/scripts.

#!/bin/bash
CONF_DIR="/etc/awstats"
CONF_EXT="conf"

if [ ! -d "$CONF_DIR" ]; then
  echo "$CONF_DIR does not exist"
  exit 1
else
  cd $CONF_DIR
fi

for cfg in *.$CONF_EXT; do
if [ "$cfg" != "awstats.model.conf" ]; then
   site=`echo $cfg|sed -e "s/awstats.//g" | sed -e "s/.conf//g"`
   /var/www/awstats/awstats.pl -update -config="$site"
fi
done

Then I need to configure logrotate.conf in /etc to run this script prior to rotating the logs. If rpm was used to install apache it could have its own file in /etc/logrotate.d/. I normally install apache manually, so I add this entry directly in logrotate.conf.

/opt/apache-1.3/logs/access_log {
   rotate 31
   daily
   compress
   sharedscripts
   prerotate
     /root/scripts/dailystat.sh
   endscript
   postrotate
     /opt/apache-1.3/bin/apachectl restart
   endscript
}

There is one problem one can encounter with logrotate. There are many vulnerable applications on the web. When hosting applications I don’t have any control over I (ofcourse) try to secure the system as good as possible. As a security measure I have /tmp mounted with the options nosuid and noexec. The specific line in /etc/fstab looks like below.

LABEL=/tmp /tmp ext3 defaults,nosuid,noexec 1 2

Crackers and worms often exploit vulnerable web applications to upload scripts and run them from /tmp. Mounting /tmp with the noexec option is a good way to make sure this cannot happen even if there are vulnerable applications on the system. If you haven’t done so already – now is a good time to fix this. However, this also stops logrotate as it runs scripts from /tmp (or whatever you have TMPDIR defined as).

Logrotate is run daily per default. The acutal script can be found in /etc/cron.daily/logrotate. To fix this I create a tmp directory, /tmp_safe, that is going to be used by logrotate only. Then I add the following to the top of /etc/cron.daily/logrotate.

export TMPDIR=/tmp_safe

No logrotate will use /tmp_safe as a tmp directory instead of /tmp. All other applications running on the system will still use /tmp as normal.

Wrapping it up

Now I have a working awstats installation and logrotate rotates my apache access_log. The logs are compressed and saved for 31 days. When I want to add statistics possibilites for a virtual host I just run the script addstat.sh. Life is simple again.

Files created

• /root/scripts/addstat.sh
• /root/scripts/genstat.sh

Files changed

• /opt/apache-1.3/conf/httpd.conf
• /etc/awstats/awstats.model.conf
• /etc/logrotate.conf
• /etc/cron.daily/logrotate
• /etc/fstab

Preparations

The first thing I had to do was to download the different GNU packages needed to compile MySQL, PHP and many other open source projects on Solaris. These packages can all be found on Sunfreeware.com. The packages I installed, by default under /usr/local,  were:

• gcc
• gmake
• flex
• bison
• m4
• autoconf
• automake
• gzip
• tar
• GNU sed
• libiconv

All packages are easy to install using pkgadd. Care needs to be taken to set the system paths correctly after the installation. I used the command crle -u -l /usr/local/lib to add the lib directory to the system defaults. Then I added /usr/local/bin first in my PATH to make sure GNU versions take precedence if there are any conflicts. For example the package m4 could be found on the system already. I also appended /usr/ccs/bin last to the PATH as there are programs there needed for compilation and linking.

MySQL

Having the environment setup as I wanted it I set my eyes on installing MySQL. I couldn’t find an official binary package for Solaris 9 x86 so I had to build it myself. I downloaded the source from mysql.com, quickly unpacked it and failed miserably at compiling it.

A quick research using Google gave me the answer. The command mkheaders refused due to paths not being found. So additionally I had to create a softlink as the paths for some reason differed. After creating the symlink I could generate the gcc header files. The exact commands issued are as below.

# ln -s /usr/local/lib/gcc /usr/local/lib/gcc-lib
# cd /usr/local/libexec/gcc/i386-pc-solaris2.9/\
  3.4.1/install-tools
# ./mkheaders

Then I tried compiling MySQL again. This time, everything could be done by the book. For sake of convenience – here’s a near copy from the MySQL manual.

# ./configure --prefix=/usr/local/mysql
# make && make install
# cp support-files/my-medium.cnf /etc/my.cnf

# cd /usr/local/mysql
# groupadd mysql
# useradd -g mysql mysql
# chown -R root .
# chown -R mysql var
# chgrp -R mysql .

The only thing left then is to start the MySQL daemon and set the root password.

# bin/mysqld_safe --user=mysql
# bin/mysqladmin -u root password XXXXXX
# bin/mysqladmin -u root -h myhost password XXXXXX

Installing gd

I need gd for, among other things, thumbnail generation. As I wanted to download and compile as little as possible a quick check revealed that freetype, libjpeg, libpng and libiconv already existed in my default Solaris 9 installation. The libraries can be found under the /usr/sfw directory.

I wanted the newest version of gd as GIF support is now added again after being removed for several years due to Unisys suddenly starting do demand licensing fees for their LZW patent a few years ago. So I could not use the slightly older gd available as a package at sunfreeware.

I downloaded version 2.0.28 and tried to compile it. However, I could not get it to compile using the libiconv already installed. Instead I had to download libiconv from sunfreeware and install it under /usr/local. The following configure command was what I ended up with to get gd to compile.

# ./configure \
  --prefix=/usr/local \
  --with-freetype=/usr/sfw \
  --with-png=/usr/sfw \
  --with-jpeg=/usr/sfw \
  --with-libiconv-prefix=/usr/local

After that compiling the package was a breeze.

# make && make install

On to apache

Apache must be the easiest application around to build on different platforms. I have never experienced any problems building a stable apache release on any unix platform. I did once have problems getting it to run on NT 4 when the windows version still was marked unstable. I guess that could be expected though. So creating a standard build of apache is very straight forward.

# ./configure --prefix=/usr/local/apache-1.3.31 \
  --enable-module=so
# make && make install

PHP – finally

Compiling PHP became, as it turned out, a bit tricky becaused of my decision to enable gd support. The reason was that the PHP configure script didn’t list libiconv when checking that all the gd functions were correct. The configure script just said “no” to all the gd functions and refused to create the makefiles. After awhile I looked through the generated config.log. This revealed that what really failed was not actually gd. When loading gd to check what functions were available it failed to find symbols from libiconv. This must either be a bug in the configure script or more probable a problem with my Solaris 9 environment and the library paths.

So prior to running the configure script I had to manually amend the configure script and add -liconv in all places where -lgd could be found. That did the trick and the script happily trotted along as it should.

# ./configure \
  --with-apxs=/usr/local/apache-1.3.31/bin/apxs \
  --with-mysql=/usr/local/mysql/ \
  --with-gd=/usr/local \
  --with-freetype-dir=/usr/sfw \
  --enable-gd-native-ttf \
  --with-png-dir=/usr/sfw \
  --with-zlib-dir=/usr/sfw \
  --with-jpeg-dir=/usr/sfw

# make && make install

After that I added the directives needed in httpd.conf to enable PHP4. Actually I only had to add the second line below as the LoadModule directive was already present.

LoadModule php4_module libexec/libphp4.so
AddType application/x-httpd-php .php

And finally, at 03:43 AM, I could verify the installation and configuration using the simple php script with a call to phpinfo(). Of course it worked like a charm.

All that remains now is to get Apache 2 and PHP 5 up and running alongside. I still haven’t figured out if the good PHP folks recommend Apache 2 for production yet. Anyways,
that’ll be a project for another night.