Advanced 404 Pages
| Category: PC KNOW HOW | Date: 2002-08-30 |
In a previous article, I discussed the advantages of using a customized 404 error page on your site and gave instructions on how to create one. As I explained back then, these pages are highly useful because they enable you to benefit from traffic that would otherwise be lost. They however do have one dark side that can make maintaining your site significantly more difficult than what it has been before.
What is this problem I'm talking about? Well, it is quite simple. Usually when a server encounters a 404 error, it records the details about the event into its log file. Should you suddenly notice that the log shows multiple 404 errors due to a page named "abutme.html" not being found, you can deduct that you've probably accidentally linked to "abutme.html" somewhere on your pages instead of using the correct filename "aboutme.html". As you can see from the above, the functionality offered by log files makes them a great help in tracking down such simple mistakes and enables you to keep your site relatively free of in-site broken links.
However, if you happen to have replaced the standard 404 error with your own HTML 404 error page, you can't utilize this useful feature. Yes, you will still see that a 404 error has occurred, but you won't get any further details that would help you figure out what exactly happened. Finding and fixing these errors becomes nearly impossible, causing them to pile up over time and deteriorate the professional image of your site. Now you're facing the tough choice of either showing your visitors a very unfriendly error message that drives them away or accepting the fact that your stylish custom 404 error page will become an all too familiar sight to those who click around your site.
All hope is not lost
After reading the above, you must be feeling pretty down. I sure know I did after having installed my own 404 page only to notice that I had corrected one problem, but caused another one while doing it. Still, there is a solution to every difficult situation and this one is no exception. If you want to keep your 404 page and still get informed when you mess things up and create a broken link, you'll be pleased to hear that I happen to have just the thing for the job. Before we begin, please take into account that in order to use this fix, your host has to be running Apache with support for .htaccess files, Server Side Includes (SSI) and CGI. Contact your technical support for information on whether you have access to these valuable features or not.
Without further ado, let's roll up our sleeves and get to work. The first thing you will need is a CGI script that will log the errors and let you know about them. There are several ones out there that you can use, but I personally prefer Matrix Vault's free 404 Helper that can be found at http://www.apromotionguide.com/404code.txt . Download the source code to your hard drive and open it in a text editor. You can get free editors from the Net, but the old MS-DOS Edit supplied with just about every version of Windows will do just fine. To run Edit, go to Start, Run, type "edit" without the quotes into the box and click OK.
Before you start editing the file, you'll need to know where your host has installed the Perl interpreter and Sendmail. Once you have figured it out, check if the paths used in the CGI script match those your host uses. The Perl interpreter's location is set to /usr/bin/perl in the first row and the location of Sendmail is set to /usr/lib/sendmail in the 21st row. Make changes, if necessary.
After you have made sure that the paths are correct, modify the rest of the script to suit your needs. Be sure to replace the E-mail address in the $email field with the one you want the error report to be sent to. You might also wish to use a smaller value in the $mailon field than the default of 10, as it can take quite a while for a small site to generate enough 404 errors to fill up a 10K log. I suggest using a value of 1 or 2 at the beginning and raising it later if you feel that you are receiving the error reports more often than you'd like to.
You're now done with the CGI script. Save it as "404helper.cgi", without the quotes of course. However, there's still work to be done, so take a deep breath and prepare yourself for the next challenge.
Editing your custom 404 page and .htaccess file
Just having the script will not be enough. In order for it to work, it has to be executed when an error is encountered. This is the part where the SSI's step into the picture. Open up your 404 error page in a text editor and add the following line into it:
Because the script prints out a few rows of HTML after it has been run, the best place for that line is at the bottom of your 404 error page, but before the
What is this problem I'm talking about? Well, it is quite simple. Usually when a server encounters a 404 error, it records the details about the event into its log file. Should you suddenly notice that the log shows multiple 404 errors due to a page named "abutme.html" not being found, you can deduct that you've probably accidentally linked to "abutme.html" somewhere on your pages instead of using the correct filename "aboutme.html". As you can see from the above, the functionality offered by log files makes them a great help in tracking down such simple mistakes and enables you to keep your site relatively free of in-site broken links.
However, if you happen to have replaced the standard 404 error with your own HTML 404 error page, you can't utilize this useful feature. Yes, you will still see that a 404 error has occurred, but you won't get any further details that would help you figure out what exactly happened. Finding and fixing these errors becomes nearly impossible, causing them to pile up over time and deteriorate the professional image of your site. Now you're facing the tough choice of either showing your visitors a very unfriendly error message that drives them away or accepting the fact that your stylish custom 404 error page will become an all too familiar sight to those who click around your site.
All hope is not lost
After reading the above, you must be feeling pretty down. I sure know I did after having installed my own 404 page only to notice that I had corrected one problem, but caused another one while doing it. Still, there is a solution to every difficult situation and this one is no exception. If you want to keep your 404 page and still get informed when you mess things up and create a broken link, you'll be pleased to hear that I happen to have just the thing for the job. Before we begin, please take into account that in order to use this fix, your host has to be running Apache with support for .htaccess files, Server Side Includes (SSI) and CGI. Contact your technical support for information on whether you have access to these valuable features or not.
Without further ado, let's roll up our sleeves and get to work. The first thing you will need is a CGI script that will log the errors and let you know about them. There are several ones out there that you can use, but I personally prefer Matrix Vault's free 404 Helper that can be found at http://www.apromotionguide.com/404code.txt . Download the source code to your hard drive and open it in a text editor. You can get free editors from the Net, but the old MS-DOS Edit supplied with just about every version of Windows will do just fine. To run Edit, go to Start, Run, type "edit" without the quotes into the box and click OK.
Before you start editing the file, you'll need to know where your host has installed the Perl interpreter and Sendmail. Once you have figured it out, check if the paths used in the CGI script match those your host uses. The Perl interpreter's location is set to /usr/bin/perl in the first row and the location of Sendmail is set to /usr/lib/sendmail in the 21st row. Make changes, if necessary.
After you have made sure that the paths are correct, modify the rest of the script to suit your needs. Be sure to replace the E-mail address in the $email field with the one you want the error report to be sent to. You might also wish to use a smaller value in the $mailon field than the default of 10, as it can take quite a while for a small site to generate enough 404 errors to fill up a 10K log. I suggest using a value of 1 or 2 at the beginning and raising it later if you feel that you are receiving the error reports more often than you'd like to.
You're now done with the CGI script. Save it as "404helper.cgi", without the quotes of course. However, there's still work to be done, so take a deep breath and prepare yourself for the next challenge.
Editing your custom 404 page and .htaccess file
Just having the script will not be enough. In order for it to work, it has to be executed when an error is encountered. This is the part where the SSI's step into the picture. Open up your 404 error page in a text editor and add the following line into it:
Because the script prints out a few rows of HTML after it has been run, the best place for that line is at the bottom of your 404 error page, but before the