robots .txt file

  • Thread starter Thread starter Guest
  • Start date Start date
G

Guest

Looks so simple yet when I check to see if it validates I get some messages
wrong syntax etc.
User-Agent: *
Disallow: /cgi-bin/
What could possibly be wrong with the above? Am using Linux server and fp
2003.
Thanks
 
Any robots .txt syntax checker. They all seem to give me something wrong and
not a way that I can understand how to fix it.
 
ok will ignore it but can I use any upper case using Linux server? example
User-agent: *
Disallow: /cgi-bin/
Disallow: /maui-jap-g/
Disallow: /image_event/
I don't need to enter the full url for the disallow do I?
 
Back
Top