You are here: Re: simple robots.txt question « HTML « IT news, forums, messages
Re: simple robots.txt question

Posted by Nikita the Spider on 07/24/06 23:50

In article <1153779031.148382.38490@75g2000cwc.googlegroups.com>,
"CRON" <cronoklee@hotmail.com> wrote:

> Found this:
>
> <meta name="robots" content="noindex, nofollow">
>
> Apparantly only a few robots support it. Anyone know which ones?

Hi Cron,
What makes you say that only a few robots support it? I had always
assumed the opposite; that most robots support it. (Most decent ones,
anyway -- the same that would respect robots.txt.)

Just for the record, Nikita the Spider supports it. =)

--
Philip
http://NikitaTheSpider.com/
Whole-site HTML validation, link checking and more

 

Navigation:

[Reply to this message]


Удаленная работа для программистов  •  Как заработать на Google AdSense  •  England, UK  •  статьи на английском  •  PHP MySQL CMS Apache Oscommerce  •  Online Business Knowledge Base  •  DVD MP3 AVI MP4 players codecs conversion help
Home  •  Search  •  Site Map  •  Set as Homepage  •  Add to Favourites

Copyright © 2005-2006 Powered by Custom PHP Programming

Сайт изготовлен в Студии Валентина Петручека
изготовление и поддержка веб-сайтов, разработка программного обеспечения, поисковая оптимизация