|
Posted by Jukka K. Korpela on 09/11/07 22:12
Scripsit dorayme:
> What is the simplest and most effective way of stopping robots
> searching a particular html pages on a server.
Put the following into the head part of each of those pages:
<meta name="robots" content="noindex">
Replace "noindex" by "noindex, nofollow" if you also want to stop robots
from following any links on the page (i.e. from finding new indexable pages
through it).
This follows the de-facto standard (Robots Exclusion Standard) that has long
been obeyed by any well-behaving indexing robots. And there's not much you
can do to the ill-behaving robots.
--
Jukka K. Korpela ("Yucca")
http://www.cs.tut.fi/~jkorpela/
Navigation:
[Reply to this message]
|