Posted by ctclibby on 10/07/02 11:51

StevePBurgess@gmail.com wrote:
[snip]
> I have noticed that the book links are getting lots of hit. At first, I
> was pleased about the potential income this might mean - but then it
> occurred to me that many of these hits are web crawlers (this was
> confirmed by webaliser).
>
> Any suggestions of ways of checking if the link is being "clicked" by a
> webcrawler so that I can not increment the field in the sql database?

Steve

You should be able to take care of this in your robots.txt file ( root
of your web directory ). I had a problem with a login link and the
bots were getting to it and I would see one reference to the login
file, but no POST. scratched me head for a bit... Anyway, Google it
and you will find a wealth of info.

todh

[Back to original message]