Iterating and downloading a URL that lists files?

  • Thread starter Thread starter Jim Bancroft
  • Start date Start date
J

Jim Bancroft

Is there a way to open a URL, one that shows nothing other than a directory
listing, and iteratively go through the list to download the files?
 
Is there a way to open a URL, one that shows nothing other than a directory
listing, and iteratively go through the list to download the files?
Click to expand...

Hi Jim,

you can use a HttpWebRequest with a StreamReader to get the page from
the URL, then the content of the page can be parsed with a regular
expression to get the list of all links and then again use
HttpWebRequest to get the files...

Hope it helps
 
Hi,

Alexey said:
Hi Jim,

you can use a HttpWebRequest with a StreamReader to get the page from
the URL, then the content of the page can be parsed with a regular
expression to get the list of all links and then again use
HttpWebRequest to get the files...

Hope it helps
Click to expand...

I did that a long time ago with a Java applet. I then discovered that
the page format was different in Netscape (back then) and IE. I hope
things are more standard now!

HTH,
Laurent
 
the page format was different in Netscape (back then) and IE. I hope
things are more standard now!
Click to expand...

Laurent, do you mean that the default directory listing depends on the
client browser? It depends on a web server and not on a browser. I
have no example for IIS, but here's the one for Apache

Default settings:
http://tomcat.apache.org/tomcat-6.0-doc/default-servlet.html

Example how it looks on the website:
http://cohred.org/cohred/content/

When directory browsing in enabled on a web site and there isn't a
default document available, IIS will display the contents of the
directory as a similar page and it looks the same in IE, and other
browsers
 
Hi,

Alexey said:
Laurent, do you mean that the default directory listing depends on the
client browser? It depends on a web server and not on a browser. I
have no example for IIS, but here's the one for Apache
Click to expand...

Back then, the client browser used to add content to the directory
listing page. I tested again right now in Firefox and IE, and apparently
this is not the case anymore, so you're lucky!

I am talking about Netspace 4 and IE4, that was back in... not sure...
97 maybe?

Anyway, good luck with your project!
Laurent
 
Back
Top