Command-line file downloader

  • Thread starter Thread starter Rich Pasco
  • Start date Start date
R

Rich Pasco

I am looking for a Win32 command-line (console) utility, suitable for
being driven from a BAT script, which accepts as input a URL and then
downloads the referenced web page (or other file) to my local disk.

The effect that I want to produce is exactly that I would get by right-
clicking on a link in a web browser and choosing from the popup menu
"save link target as..." except this one would be driven entirely from
the command line, without graphics, popups, or point-and-click.

I began to write my own, but it's harder than it looks; it involves
parsing the URL from the command line, to get the host name, issuing a
DNS query to get its IP address, opening a socket and speaking HTTP
to request the file, then interpreting the HTTP reply to reconstruct
the file and save it locally.

Might be easier if someone else has already done it.

Any suggestions?

- Rich
 
I am looking for a Win32 command-line (console) utility, suitable for
being driven from a BAT script, which accepts as input a URL and then
downloads the referenced web page (or other file) to my local disk.
Click to expand...

WGET
 
Rich Pasco said:
I am looking for a Win32 command-line (console) utility, suitable for
being driven from a BAT script, which accepts as input a URL and then
downloads the referenced web page (or other file) to my local disk.
Click to expand...

Microsoft's BITSAdmin.exe controls the built-in BITS, but is restricted
to http.
If you want a more versatile utility, then get cURL http://curl.haxx.se/;
it's "better" than WGET.

Stefan
 
Mark said:
There's a feature comparison of several command-line downloading tools
including cURL and wget at
http://curl.haxx.se/docs/comparison-table.html
Click to expand...

The lack of recursive downloading is a pretty significant hit against
curl. FWIW, the comparison table also barely scratches the features of
wget.

It appears the main advantage of curl is support for a small handful of
additional protocols. IMO to issue a blanket statement that it is carte
blanch better than wget is pretty bold. That said, curl is a pretty
great choice.

Maybe more important is the related curl library for programming projects :)

It also might be worth taking a look at the ncftp package. In addition
to the interactive commandline ftp client, there is also the scriptable
ncftpput and ncftpget included (of course it is ftp only).

Good luck in all automated file transfer projects...

~Jason
 
Jason Gurtz said:
It appears the main advantage of curl is support for a small handful
of additional protocols. IMO to issue a blanket statement that it is
carte blanch better than wget is pretty bold...
Click to expand...

For the record Stefan said that, not me. You're correct that the
comparison table I suggested is incomplete, but it's a good place to start.
 
Address munging IS HARMFUL!
The lack of recursive downloading is a pretty significant hit against
curl. FWIW, the comparison table also barely scratches the features of
wget.
Click to expand...

It barely scratches the features of curl too.-)
While curl itself doesn't perform recursive up/downloads, there are libcurl
based programs which have this capability.
And besides downloads, curl/libcurl does uploads too.
Has wget an accompanying libwget?!
Does it support large files?
OK, thats now rhetoric, but LFS support was introduced just recently.
Look at the history of both tools: wget was written to fetch web pages,
curl was written to "grok URLs".
It appears the main advantage of curl is support for a small handful of
additional protocols.
Click to expand...

And uploads too, and a library, ...
IMO to issue a blanket statement that it is carte
blanch better than wget is pretty bold. That said, curl is a pretty
great choice.
Click to expand...

I wrote "better" not without reason enclosed in quotes.
Maybe more important is the related curl library for programming projects :)
Right.

It also might be worth taking a look at the ncftp package. In addition
to the interactive commandline ftp client, there is also the scriptable
ncftpput and ncftpget included (of course it is ftp only).

Good luck in all automated file transfer projects...
Click to expand...

But on the first hand: why should I "install" a third party solution at
all and not use BITS instead (which has the advantage to use the Internet
settings of Windows). Unfortunately BITSAdmin.exe needs to be installed
from the support tools package too.

Stefan
 
Stefan said:
Address munging IS HARMFUL!
Click to expand...

Concrete reasons for NOT munging on public groups (particularly ones
mirrored on the web) will be entertained if anyone cares to offer them.

Religious rants....well, I'm a spambot hating atheist sorry :)

~Jason

--
 
Back
Top