Posted by Alan J. Flavell on 07/01/06 15:17

On Sat, 1 Jul 2006, Benjamin Niemann wrote:

> > Hits by Response Code
> > Code 200 - OK 12033
[...]
> > Code 304 - Not Modified 2362

> Looks pretty normal to me. But without knowing which documents
> generated which codes, there's not much more to say.
>
> A few hints:
>
> 'More 304 is good'. It means that unmodified documents are not
> repeatedly served to the same client or are cached by a proxy, thus
> reducing the load on your server. If you are using static HTML
> documents, the server should take care of this. When using
> dynamically generated documents, you may have a look at the code, if
> it generated 'Last-Modified' and/or 'ETag' response headers and
> recognizes and handles 'If-Modified-Since' and/or 'If-Match' request
> headers.

Indeed (it's "If-None-Match", to be accurate).

I'd recommend reading Mark Nottingham's cacheing tutorial
http://www.mnot.net/cache_docs/
and use the cited "cacheability engine" to report on the site,
to help identify any areas of concern.

> You should have a closer look at which documents generated 4xx
> codes. Many 4xx hits are generated by script kiddies testing for
> vulnerable software on your server that is not present or
> vulnerabilities of the server itself.

Yes, we see lots of those. But also garbled URLs that appear to be
defective web searches. This makes it quite hard to spot genuine
problems.

> But it could also indicate broken links on your site.

Yes, but IMHO those can be more accurately identified by running a
link checker on your site.[1]

4xx errors in the log can also indicate broken links to your pages
from *somebody else's* site. In which case you could alert the other
webmaster, and/or set up a redirection to the correct URL on your own
site.

h t h

[1] If you run a windows system, then Xenu link checker is amazingly
fast, and generally to be recommended. However, it uses Windows
routines "under the covers" to access URLs, and there's a couple of
kinds of URL error which Windows silently fixes-up "under the covers",
with the consequence that Xenu checker can't see the error. For
example, using "\" instead of "/" in a URL path. So it's recommended,
but not ideal.

[Back to original message]