Posted by Steve on 09/07/05 11:39

> Can you tell me how to use robots.txt though? That sounds like a good
> idea anyway.

See <http://www.searchengineworld.com/robots/robots_tutorial.htm> and
<http://www.robotstxt.org/wc/robots.html> for details. Note that only
compliant robots would be excluded; spam harvesters and the like will
ignore the file.

---
Steve

[Back to original message]