Posted by Jochem Maas on 08/12/05 04:04

Jasper Bryant-Greene wrote:
> Jochem Maas wrote:
>
>> Jasper Bryant-Greene wrote:
>>
>>> robots.txt will not do what you want it to.
>>>
>>> Just sniff for those robots' User-Agents (Google, MSN and Yahoo all
>>> publish their UA strings on their websites, AFAIK) and send different
>>> content if it's one of those.
>>
>>
>>
>> they will hammer you for it eventually - AFAICT all major SEs send out
>> their
>> spiders occasionally with faked user-agent strings - to catch out crap
>> like
>> this.
>>
>> oh and the guy that invented php is a really bigcheese down at yahoo...
>> and he reads this list :-) though I doubt he has the time or desire to
>> chase
>> you personally.
>>
>> I would recommend you don't go down this road. it's bad for your
>> business in the
>> longer term and its bad for the web because your filling it with shite.
>
>
> Of course it is, but in his original post he said that he realised that
> it was bad, and he didn't want to hear reasons not to do it.

I know - I only really replied to voice my total disdain for idiots
who are filling the search engines with shite. thats bad for all of us
(well those of us that use search engines - you get the impression that
some people here don't know what one is ;-)

if he didn't want to hear this stuf he should have googled - there is
tons of code that does this - he didn't really need the list at all to
figure out how to do it.

>
> I would never even attempt to do something like this on a website of my
> own -- as I said in an off-list email to this guy (it was OT for the

you have to go pretty far to be off topic for this list ;-)

> list) it's going to harm his website more than help it. It's not exactly
> hard for the search engines to detect cloaking.

I concur.

>
> Jasper
>

[Back to original message]