M
muscha
Hello all,
Does anyone know an open source implementation of robots.txt parser in c#
(or java)?
thanks,
/m
Does anyone know an open source implementation of robots.txt parser in c#
(or java)?
thanks,
/m
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
V
Vincent Lascaux
Does anyone know an open source implementation of robots.txt parser in c#
what is robots.txt ?
(or java)?
what is robots.txt ?
J
Jon Skeet [C# MVP]
Vincent Lascaux said:what is robots.txt ?
See http://www.searchengineworld.com/robots/robots_tutorial.htm