|
|
Posted by Brian Dunning on 08/12/05 01:46
On Aug 11, 2005, at 3:44 PM, Evert | Collab wrote:
> Use robots.txt
> 'evil' searchengines will spoof the user-agent string anyway
Can you be more specific about what you mean by "use robots.txt"?
I just want to cloak for Google, MSN, and Yahoo. I couldn't care less
about what any other search engine (evil or not) does or sees.
Navigation:
[Reply to this message]
|