exclude a folder from search bot

  • Thread starter Thread starter Guest
  • Start date Start date
Is there a way to exlude a root directory sub-folder from the FP search bot?
Click to expand...
You can do it with a robots.txt file. Search Google for robots.txt for the
format, etc.
 
igor said:
You can do it with a robots.txt file. Search Google for robots.txt for the
format, etc.
Click to expand...
Hi, Igor -
I know that the robots.txt file will affect the EXTERNAL search engines, but
I was trying to exclude the folder from the FRONTPAGE search bot. does the
robots.txt file do that, too? I wasn't aware of this.
 
Sorry. I misread your post. When I read "search bot" (as in "robot") --
an automated ongoing search at certain intervals -- I missed the FP part.
-- Igor
 
Kathleen,

Thank you so much for your help. It did EXACTLY what I needed it to do!

Kind regards,
Dena
 
I've tried this (by putting folders/files that I want excluded from the FP
webot search in _private) but they still appear in the results list. Have I
missed something simple here?

Thanks, Steve
 
Do not place them in _private, instead rename the folder to have a underscore as the first character
or convert the folder to a subweb. Then run Tools | Recalculate Hyperlinks.

If you are hosted on a Windows IIS server with Index Server, the host may have to configure Index
Server to exclude the specific folder(s).

--
==============================================
Thomas A. Rowe (Microsoft MVP - FrontPage)
==============================================
If you feel your current issue is a results of installing
a Service Pack or security update, please contact
Microsoft Product Support Services:
http://support.microsoft.com
If the problem can be shown to have been caused by a
security update, then there is usually no charge for the call.
==============================================
 
Thank you Thomas - this solved my problem, although not straight away.

The basis of my first attempt was to create a folder called _frames directly
on the web server (via FP) and move the files/folders from _private to this
new hidden folder. That went smoothly enough and FP renamed the links etc.
Fired the site up in IE, saw the URL refs were to _frames now but the search
webbot still returned links to files in the _frames folder. I then decided to
amend the local (to FP) site on my desktop and re-publish the whole site to
the server. Did that and it now works. Got there in the end but unsure why my
first attempt did not work.

Thanks again for your help.

Steve
 
This is on the question of robots text file.
Is it necessary? My Folders have the underscore that should not be crawled
by the bots. Do I need the robots file. I would not have anything to exclude.
Do the bots "need" this to crawl a web site correctly?
 
The folders with an underscore as the first character, is only ignore by the FP search component, do
not apply to search engines.

--
==============================================
Thomas A. Rowe (Microsoft MVP - FrontPage)
==============================================
If you feel your current issue is a results of installing
a Service Pack or security update, please contact
Microsoft Product Support Services:
http://support.microsoft.com
If the problem can be shown to have been caused by a
security update, then there is usually no charge for the call.
==============================================
 
Back
Top