Posted by Jukka K. Korpela on 04/10/06 15:42

Michael Winter wrote:

> Though
> there's content, the status line returns the 404 response code.

This, in turn, means that the content shall be interpreted as an _error
message_, which gives explanations related to the basic message that
says that the URL does not correspond to any resource on the WWW.

Search engines generally behave according to the protocol, i.e. treat
the resource as nonexistent; they have no use for the error message, so
they discard it. Thus, even for testing purposes, it is unadvisable to
make a server send 404 for any URL that is supposed to refer to an
existing resource.

Server administration sometimes makes the mistake of setting the server
to send 404 responses during a maintenance break. This can be very
harmful, since the pages will be removed from search engine data bases,
if a robot happens to check them. It typically takes months, if not an
eternity, to make pages appear in the data bases again.

[Back to original message]