Posted by Adrienne Boswell on 06/10/07 16:08

Gazing into my crystal ball I observed Desmond <otuatail@aol.com> writing
in news:1181469765.276407.219910@k79g2000hse.googlegroups.com:

> I know I can prevent robots by creating a robot text file. but how do
> you know what to put in. for example the folowing user_agent
> Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)
>
> problem what is the exact name to put in the text file as this in not
> always the same.
>
> Is their a another server variable to acompany
> $_SERVER['HTTP_USER_AGENT'];
>
> Desmond.
>

I don't restrict by user agent, I just do it on a page or folder basis.
You could go kuku trying to account for every bot.

--
Adrienne Boswell at Home
Arbpen Web Site Design Services
http://www.cavalcade-of-coding.info
Please respond to the group so others can share

[Back to original message]