Multi-file selection and download using HTTP?

  • Thread starter Thread starter Phred
  • Start date Start date
P

Phred

G'day mates,

One of my common needs is to download several data files from a web
page. Using MSIE 6 I seem to be limited to selecting one file at a
time. This is a bit of a nuisance as it requires interrupting other
work to go back to the page to select the next in line, etc etc.

It occurs to me that there *should* be some simple way to do this --
maybe even within MSIE itself (but, if so, it escapes me 8-).

Does anyone know an application or script (run from a batch file or
command line would be *very* convenient :) that can do this sort of
thing, please?

Only needs to be very simple. Bells and whistles not required; just
the ability to point to a page, select the files, and download to a
selected directory.

Something that works under Windows 9x/ME/2000/XP would be great!
Failing that, just ME/XP would be good. And, failing that, just XP
would be acceptable.

Thanks for your time.

Cheers, Phred.
 
There's an extension for Firefox called Magpie that will allow you to
download all files of certain extensions on a page to a directory. Look it
up on extensionsmirror.nl.

XemonerdX
 
Phred said:
One of my common needs is to download several data files from a web
page. Using MSIE 6 I seem to be limited to selecting one file at a
time. This is a bit of a nuisance as it requires interrupting other
work to go back to the page to select the next in line, etc etc.
Click to expand...
It occurs to me that there *should* be some simple way to do this --
maybe even within MSIE itself (but, if so, it escapes me 8-).

Phred.
Click to expand...

*** I would suggest that you upgrade to Opera. It's a far superior
browser.

Richard
 
Several such as wGrabber, NetSpider, and Black Widow.

wGrabber so far is the best and the easyiest.
 
Phred said:
G'day mates,

One of my common needs is to download several data files from a web
page. Using MSIE 6 I seem to be limited to selecting one file at a
time. This is a bit of a nuisance as it requires interrupting other
work to go back to the page to select the next in line, etc etc.
Click to expand...


You have to select them individually but you can do so consecutively; i.e.,
select one, it's d/l window opens...select the next, another d/l window
opens...etc.

Or, you could use any of many download managers that let you browse the page
and make multiple selections. You could do the same thing with an FTP
program.

--
dadiOH
_____________________________

dadiOH's dandies v3.0...
....a help file of info about MP3s, recording from
LP/cassette and tips & tricks on this and that.
Get it at http://mysite.verizon.net/xico
____________________________
 
Phred said:
It occurs to me that there *should* be some simple way to do this --
maybe even within MSIE itself (but, if so, it escapes me 8-).
Click to expand...

It can be done fairly easily. If you know VBScript you can go to
http://paulsadowski.com/WSH/ and look at the script for getting binary
files. You'd have to modify it to open a file with the URLs and get each.

I actually wrote a GUI bsed file grabber in VB for someone. It limits what
extension it will grab though. It's designed for vids and pix. But it's
fairly easy to write applications like that that use IE in a hidden window.
Does anyone know an application or script (run from a batch file or
command line would be *very* convenient :) that can do this sort of
thing, please?

Only needs to be very simple. Bells and whistles not required; just
the ability to point to a page, select the files, and download to a
selected directory.
Click to expand...

In this case you'd probably do best getting a copy of wget for windows. You
can then make a batch file that sequentially calls wget with the desired
URLs.
http://www.interlog.com/~tcharron/wgetwin.html
http://www.google.com/search?sourceid=navclient&ie=UTF-8&q=wget+windows
 
Phred said:
Does anyone know an application or script (run from a batch file or
command line would be *very* convenient :) that can do this sort of
thing, please?
Click to expand...

If you install FreshDownload the option 'Download All with FD' is
added to the context menu. It's still there even if FD isn't defined
as your default downloader.
 
G'day mates,

One of my common needs is to download several data files from a web
page. Using MSIE 6 I seem to be limited to selecting one file at a
time. This is a bit of a nuisance as it requires interrupting other
work to go back to the page to select the next in line, etc etc.

It occurs to me that there *should* be some simple way to do this --
maybe even within MSIE itself (but, if so, it escapes me 8-).

Does anyone know an application or script (run from a batch file or
command line would be *very* convenient :) that can do this sort of
thing, please?

Only needs to be very simple. Bells and whistles not required; just
the ability to point to a page, select the files, and download to a
selected directory.

Something that works under Windows 9x/ME/2000/XP would be great!
Failing that, just ME/XP would be good. And, failing that, just XP
would be acceptable.

Thanks for your time.
Click to expand...

KybIE GetEmAll (www.1keytools.com) allows you to do exactly this with
IE.

Go to the page with the links in IE.
Select the links.
Press F7

The selected files will be saved in a cache directory.
You can view them by selecting from a list -or- pressing F8 to see
each one in turn in IE.

The lite version (free) will allow up to 2 sets of pages with 20 pages
each. The pro version (not free) is unlimited.

David. 1keytools.

Software author. (please edit my email addr. to prove you're not a dumb 'bot)
Kybie GetEmAll - Make IE an offline browser http://www.1keytools.com/offline_browser.htm
Kyboma CSS tutor by example - find stylesheet examples http://www.1keytools.com/html_tutor.htm
 
One of my common needs is to download several data files from a web
page. Using MSIE 6 I seem to be limited to selecting one file at a
time. This is a bit of a nuisance as it requires interrupting other
work to go back to the page to select the next in line, etc etc.
Click to expand...


Internet Explorer limits the number of concurrent sessions to a web
server, to comply with the HTTP 1.1 Specification (RFC 2068) ...

9.1.2)
<snip>
Clients that use persistent connections SHOULD limit the number of
simultaneous connections that they maintain to a given server. A
single-user client SHOULD maintain AT MOST 2 connections with any
server or proxy.
(from http://www.w3.org/Protocols/rfc2068/rfc2068)

This means you can download at most two files at a time, when using IE.

Some other browsers allow you to have more concurrent sessions; however
these browsers are devaiting from W3C and RFC standards.

There are two main ways you can work around this:

1) use a client-side download manager. The download manager (at least,
good ones) will serialise the file requests automatically for you, so
that while you're still only downloading max 2 files at a time, it
remembers what files you want and sends further requests as sessions
become available. This is the most "correct" way to solve the problem.

There are a number of commercial, shareware and free download managers,
out there on the web.

2) If you don't mind being a hog, you can increase the number of
simultaneous HTTP sessions that Internet Explorer will provide. You can
do this by editing the registry. The steps are described in Knowledgebase
article 183110:

http://support.microsoft.com/default.aspx?scid=kb;en-us;183110


There might be other reasons to consider using an alternative browser to
Internet Explorer, but the download limit is not one of them! ;-)

Hope it helps,
Andrew
 
One of my common needs is to download several data files from a web
page. Using MSIE 6 I seem to be limited to selecting one file at a
time. This is a bit of a nuisance as it requires interrupting other
work to go back to the page to select the next in line, etc etc.
Click to expand...


Internet Explorer limits the number of concurrent sessions to a web
server, to comply with the HTTP 1.1 Specification (RFC 2068) ...

9.1.2)
<snip>
Clients that use persistent connections SHOULD limit the number of
simultaneous connections that they maintain to a given server. A
single-user client SHOULD maintain AT MOST 2 connections with any
server or proxy.
(from http://www.w3.org/Protocols/rfc2068/rfc2068)

This means you can download at most two files at a time, when using IE.

Some other browsers allow you to have more concurrent sessions; however
these browsers are devaiting from W3C and RFC standards.

There are two main ways you can work around this:

1) use a client-side download manager. The download manager (at least,
good ones) will serialise the file requests automatically for you, so
that while you're still only downloading max 2 files at a time, it
remembers what files you want and sends further requests as sessions
become available. This is the most "correct" way to solve the problem.

There are a number of commercial, shareware and free download managers,
out there on the web.

2) If you don't mind being a hog, you can increase the number of
simultaneous HTTP sessions that Internet Explorer will provide. You can
do this by editing the registry. The steps are described in Knowledgebase
article 183110:

http://support.microsoft.com/default.aspx?scid=kb;en-us;183110


There might be other reasons to consider using an alternative browser to
Internet Explorer, but the download limit is not one of them! ;-)

Hope it helps,
Andrew
 
Andrew McLaren <baz.com> wrote ...

<snip>

Arrgh, apologies for the cross-posting. It was operator error (aka "dumb-
arse user problem"). I'll be more careful, next time.
 
G'day mates,

One of my common needs is to download several data files from a web
page. Using MSIE 6 I seem to be limited to selecting one file at a
time. This is a bit of a nuisance as it requires interrupting other
work to go back to the page to select the next in line, etc etc.

It occurs to me that there *should* be some simple way to do this --
maybe even within MSIE itself (but, if so, it escapes me 8-).

Does anyone know an application or script (run from a batch file or
command line would be *very* convenient :) that can do this sort of
thing, please?

Only needs to be very simple. Bells and whistles not required; just
the ability to point to a page, select the files, and download to a
selected directory.

Something that works under Windows 9x/ME/2000/XP would be great!
Failing that, just ME/XP would be good. And, failing that, just XP
would be acceptable.
Click to expand...

In my experience, multiple simultaneous downloads from the same server
tend to take longer than automated sequential downloads. Firefox
allows Ctrl click to activate a link in a background tab, but the
cross platform standard tool - if you know the URLs or want all the
files on a page - for this sort of thing is wget. wget retrieves the
files sequentially and is very configurable (and free).
 
Richard Bonner said:
*** I would suggest that you upgrade to Opera. It's a far superior
browser.
Click to expand...

Thats not a very helpful answer. You havent even indicated whether his
request can be accommodated in Opera
 
Phred said:
G'day mates,

One of my common needs is to download several data files from a web
page. Using MSIE 6 I seem to be limited to selecting one file at a
time. This is a bit of a nuisance as it requires interrupting other
work to go back to the page to select the next in line, etc etc.

It occurs to me that there *should* be some simple way to do this --
maybe even within MSIE itself (but, if so, it escapes me 8-).

Does anyone know an application or script (run from a batch file or
command line would be *very* convenient :) that can do this sort of
thing, please?
Click to expand...


wget.exe


write a batch file that runs through and downloads the files by url..

simple.
 
That's not a very helpful answer.
Click to expand...

*** I realised that after I posted, but was in a rush and didn't
go back to expand on it.

You havent even indicated whether his request can be accommodated in
Opera
Click to expand...

*** No, but the inference would be that it can.

I don't have Opera but have had it demoed to me and use it from time
to time at friend's homes. One of the things showed to me was multiple
clicking on various links and the ability to do various things with those
links afterwards.

Richard Bonner
http://www.chebucto.ns.ca/~ak621/DOS/
 
Andrew McLaren said:
Internet Explorer limits the number of concurrent sessions to a web
server, to comply with the HTTP 1.1 Specification (RFC 2068) ...

9.1.2)
<snip>
Clients that use persistent connections SHOULD limit the number of
simultaneous connections that they maintain to a given server. A
single-user client SHOULD maintain AT MOST 2 connections with any
server or proxy.
(from http://www.w3.org/Protocols/rfc2068/rfc2068)

This means you can download at most two files at a time, when using IE.
Click to expand...

Although this is the wrong group to discuss this, I have IE6 and it does
download more than just two files for me. I think the OP's point was that
each download has to be initiated manually. ISTM, IE6 also has an
installation option for offline viewing which can be employed to download
pages x levels deep.
 
Hey Phred, here's how you do it in Opera!!

Get to the page that has all the "downloadable" linked files ( e.g.
links to pdf docs)

TYpe CTRL-J

The new special page that pops up has all the links listed from the
prev page. You can refine this list by ( for example) typing ".pdf" in
the search field that is available. Only the links with .pdf will
remain in the window pane. Select some or all of the links and right
click. Then choose "Quick download" to save all the linked files to
your default download directory. Or choose "Save as" and you will be
presented with a download prompt for each downloadable linked file.

Once you get used to it, the process is very fast.
 
wget.exe


write a batch file that runs through and downloads the files by url..

simple.
Click to expand...

That's the third recommendation I have seen for wget. Could I add a
fourth?

One short batch file fetched a complete archive of my public web site.
Actually, only the last line was really needed but the first few lines
set the path for me, changed the video palette to one readable on
my dim monitor and cleared the screen to a readable colour combination
so I could see wget's progress without straining my eyes:

@echo off
call c:\WINDOWS\p.bat
c:\utils\vga pal c:\utils\win2.pal
call cl
@echo on
G:\utils\Win\wget -r -I /~af380 -nH --cut-dirs=1 http://www.chebucto.ns.ca/~af380/index.html
 
Back
Top