Posted by Desmond on 06/10/07 19:53

On 10 Jun, 17:08, Adrienne Boswell <arb...@yahoo.com> wrote:
> Gazing into my crystal ball I observed Desmond <otuat...@aol.com> writing
> innews:1181469765.276407.219910@k79g2000hse.googlegroups.com:
>
> > I know I can prevent robots by creating a robot text file. but how do
> > you know what to put in. for example the folowing user_agent
> > Mozilla/5.0 (Twiceler-0.9http://www.cuill.com/twiceler/robot.html)
>
> > problem what is the exact name to put in the text file as this in not
> > always the same.
>
> > Is their a another server variable to acompany
> > $_SERVER['HTTP_USER_AGENT'];
>
> > Desmond.
>
> I don't restrict by user agent, I just do it on a page or folder basis.
> You could go kuku trying to account for every bot.
>
> --
> Adrienne Boswell at Home
> Arbpen Web Site Design Serviceshttp://www.cavalcade-of-coding.info
> Please respond to the group so others can share

I do do it on a page or folder basis. but I would like

[Back to original message]