Keep page private

  • Thread starter Thread starter Guest
  • Start date Start date
G

Guest

Hi. I am looking to post a page to the www but I would like to keep it
somewhat private without password protection and keep it from getting picked
up by the crawelers and webots. Any ideas
 
Without password protection you are limited to security by obfuscation.
In other words, place it in a folder in your web, and do NOT make any
links to it, from anywhere, not even from other websites, emails,
newsgroups or forums.

--
Ron Symonds - Microsoft MVP (FrontPage)
Reply only to group - emails will be deleted unread.

http://www.rxs-enterprises.org/fp

FrontPage Support: http://www.frontpagemvps.com/
 
However, if you list the page in robots.txt file you are providing the path to content you don't
want found, for any one that read the robots.txt file.

--
==============================================
Thomas A. Rowe
Microsoft MVP - FrontPage

http://www.Ecom-Data.com
==============================================
 
Hm, I didn't realize that. Still, I'm pretty sure that the net effect of a
proper robots.txt file would be to prevent many more people from reaching
the page than it allows via the circuitous route you describe.
 
The robots.txt is really only for "good" SE spiders to inform them of what content you do not want
to have indexed, however the "bad" spiders read it to see what content a site is trying to hide.

Just do a internet search for robots.txt and see what various web sites are attempting to hide from
being indexed.

--
==============================================
Thomas A. Rowe
Microsoft MVP - FrontPage

http://www.Ecom-Data.com
==============================================
 
Bad Spiders are those that are looking to grab email addresses, sensitive content, etc. they do not
obey the robots.txt file, but will read it.

--
==============================================
Thomas A. Rowe
Microsoft MVP - FrontPage

http://www.Ecom-Data.com
==============================================
 
More search engines are using sitemap files
(https://www.google.com/webmasters/tools/docs/en/protocol.html) to index all
the pages in your site. You can either submit it to all the search engines
(which would take a while) or you can have it's location in your robots.txt
file which is a good way to make sure all your pages get indexed (all the
ones that you want). The line looks like this:

Sitemap: <sitemap_location>

See https://www.google.com/support/webmasters/bin/topic.py?topic=8477 or
https://www.google.com/support/webmasters/bin/answer.py?answer=34654&topic=8514
form more information.
 
Back
Top