|
|
Posted by linda on 11/03/06 00:44
"knottulf" <knottulf@yahoo.no> wrote in message
news:454a5c25@news.broadpark.no...
> flamer die.spam@hotmail.com wrote:
>> Not all search engines listen to your robots file. google does, forums
>> arent on a subdomain are they?
>>
> Anyway google doesnt listen to this robots-file either.
> The forum isnt in the root folder of the site, but the robots-file is in
> the
> forums folder. Is this wrong? Maybe it should reside in the root folder,
> and specify that the forums folder should be no-go-area for the search
> machines?
>
> Knottulf
Your robots.txt needs to be in the root, otherwise it wont work. They start
at idexing from your root outwards. So if you place it inside the sub
folder it is already inside that folder by the time it's found the
robots.txt and it will take no notice of it. Spiders etc look for the
robots.txt file in your root, so as not enter and index the sub folder/s and
files concerned.
If you move it to your root ie www. folder then it will find it.
Best wishes,
Linda
Navigation:
[Reply to this message]
|