|
|
Posted by Scott on 11/22/05 21:24
Steve Pugh wrote:
>
> Scott wrote:
> > > Luigi Donatello Asero wrote:
> > > > "Scott" <golden@uslink.net> skrev i meddelandet
> > > > news:437FD314.421C1B77@uslink.net...
> > > > >
> > > > > Is there a tag that I can put on a page that will prevent search
> > > > > engines from indexing the page?
> > > >
> > > > As far as I know you could insert the adress of the page into a file
> > > > called robots.txt and indicate which search engine you do not want to
> > > > index it.
> >
> > OK, I figured out what to write in robots.txt. What I'm wondering is exactly
> > where to place that file on the host server.
>
> At the root of your site.
>
> If a spider wants to visit http://www.example.com/foo/bar/page.html
> then it will look for http://www.example.com/foo/bar/robots.txt,
> http://www.example.com/foo/robots.txt and
> http://www.example.com/robots.txt and apply all the rules it finds.
> >From your point of view having a single robots.txt in your root folder
> makes for easy maintenance.
>
> Steve
Steve,
So, you're saying I can just upload the robots.txt file to the same place I
upload all my website files? In my case, my web account on the server is
"public_html". And I should configure robots.txt to exclude the one
particular url that I wish not to be indexed?
Thanks!
Scott
Navigation:
[Reply to this message]
|