|
|
Posted by Andy Dingley on 08/08/06 14:37
softwarelabus@yahoo.com wrote:
> I wanted to warn all website owners that some evil web hosts like
> vistapages will periodically place a robots.txt file on your site that
> disallows all search engines. It happened to me.
OK, so that's pretty evil. Not quite sharks with frickin' laser beams
on their heads, but it's more evil than you want from people you're
giving money to.
What did they say about this? How abject was their grovelling apology?
You're not still _with_ these people are you?!
> You should also look for this file when you ftp to your site in case
> your web host places a sneaky server script to make robots.txt
> invisible only to you.
If I were evil (Mwwaa ha ha ha) I wouldn't place a robots.txt in
anyone's web root, I'd use some config to serve a standard robots.txt
for HTTP requests for it, without you even having a file to see. As
easy for the evil admin to do, and less obvious.
Navigation:
[Reply to this message]
|