Reply to Re: uses of robots.txt

Your name:

Reply:


Posted by Newsgroups on 10/08/07 18:10

> That's where it gets tricky. =) But consider this -- if you obey
> robots.txt 100% from the start, you'll always be doing the right thing
> no matter how many people use your aggregator.

I agree ; but, If I obeyrobots.txt, my aggregator won't aggregate lots
of RSS. Who want to use an aggregator which do not aggregate :-)

For information : There is currently about 70 users that use my
aggregator... It's difficult for me to recruite :-) But i really wants
to be 100% conform with rules and standards...

Thanks for your help and opinion.

[Back to original message]


Удаленная работа для программистов  •  Как заработать на Google AdSense  •  England, UK  •  статьи на английском  •  PHP MySQL CMS Apache Oscommerce  •  Online Business Knowledge Base  •  DVD MP3 AVI MP4 players codecs conversion help
Home  •  Search  •  Site Map  •  Set as Homepage  •  Add to Favourites

Copyright © 2005-2006 Powered by Custom PHP Programming

Сайт изготовлен в Студии Валентина Петручека
изготовление и поддержка веб-сайтов, разработка программного обеспечения, поисковая оптимизация