|
Posted by fletch on 04/12/06 21:36
Honestly, this is designed to disallow google from indexing pages
ending with PHPSESSID, but wont work anyway because robots.txt doesn't
support globbing
>From http://www.robotstxt.org/wc/exclusion-admin.html
> Note also that regular expression are not supported in either
> the User-agent or Disallow lines. The '*' in the User-agent field
> is a special value meaning "any robot". Specifically, you cannot
> have lines like "Disallow: /tmp/*" or "Disallow: *.gif".
Navigation:
[Reply to this message]
|