(e-mail address removed) (billyNuts) wrote in
I need to crawl a client's website, and generate a list of html
pages, and what images (.jpg's mostly - some .gif/.swf/etc.)
I don't need a graphical map of it, I use Visio for that.
Is there an application that can do this for me??
FtpList 1.14
Starting here, scroll down to FtpList and the download:
<
http://www.etplanet.com/download/application.shtm>
FtpList 1.13 (108KB)
-An application to list all files and directories in a FTP server or
a local computer. The result can be a tree or a URL list.
Homepage: stronghorse.yeah.net
<
http://stronghorse.yeah.net/>
direct download:
<
http://www.etplanet.com/download/application/FtpList 1.13.zip>
If you go to the homepage (in Chinese) you eventually get to:
<
http://www.comicer.com/stronghorse/software/index.htm>
Where it says the version number is 1.14 and the download for the
English version is:
<
http://www.comicer.com/stronghorse/software/exe/FtpList.zip>
Note that the Chinese servers are SLOW, but the download is small
and worth the effort. I used the program on a site and it is very
fast. The output I used was a text "tree" and it gave me the
directories, the filenames and file sizes.
The output looked like this (very much abbreviated):
ftp://XXX.XXX.XXX.XXX (20 folders, 30 files, 231.09 KB, 16.69 MB
in total.)
| .htaccess 333 bytes
| _vti_inf.html 1.71 KB
| about.html 3.98 KB
| cabinet-calcs.html 3.13 KB
| calc1.html 4.66 KB
| calc2.html 4.02 KB
| calc3.html 4.11 KB
| calc4.html 4.51 KB
| calc5.html 4.51 KB
| calc6.html 4.48 KB
| crossref.html 3.08 KB
| downloads.html 8.46 KB
| favicon.ico 2.19 KB
| index.html 6.40 KB
| info.html 5.39 KB
| links.html 2.86 KB
| netload.chk 30 bytes
| new.html 4.25 KB
| parts.html 55.88 KB
| partsclass01.html 24.96 KB
| partsclass02.html 12.85 KB
| partsclass03.html 8.75 KB
| partsclass04.html 6.35 KB
| partsclass05.html 12.50 KB
| partsclass06.html 6.99 KB
| partsclass07.html 5.93 KB
| salesrep.html 19.05 KB
| search.html 3.32 KB
| xxx.xml 1.09 KB
| tandc.html 5.33 KB
+---_private (0 folders, 1 files, 212 bytes, 212 bytes in
total.)
| .htaccess 212 bytes
+---_vti_bin (2 folders, 1 files, 160 bytes, 576 bytes in
total.)
| | .htaccess 160 bytes
| +---_vti_adm (0 folders, 1 files, 204 bytes, 204 bytes in
total.)
| | .htaccess 204 bytes
| +---_vti_aut (0 folders, 1 files, 212 bytes, 212 bytes in
total.)
| .htaccess 212 bytes
+---_vti_cnf (0 folders, 21 files, 100.01 KB, 100.01 KB in
total.)
| .htaccess 97 bytes
| about.html 2.64 KB
| home.htm 2.76 KB
| home.html 2.83 KB
| info.html 1.76 KB
| links.html 2.54 KB
| new.html 5.34 KB
| order.html 10.84 KB
| parts.html 20.95 KB
| partsclass01.html 11.55 KB
| partsclass02.html 5.56 KB
| partsclass03.html 4.36 KB
| partsclass04.html 3.02 KB
| partsclass05.html 5.62 KB
| partsclass06.html 3.45 KB
| partsclass07.html 4.26 KB
| product.html 2.93 KB
| puzoff1.gif 437 bytes
| puzoff2.gif 435 bytes
| salesrep.html 6.13 KB
| tandc.html 2.53 KB
+---_vti_log (0 folders, 1 files, 97 bytes, 97 bytes in total.)
| .htaccess 97 bytes
+---_vti_pvt (0 folders, 17 files, 86.30 KB, 86.30 KB in
total.)
| .htaccess 97 bytes
| .roles 383 bytes
and so forth...
HTH,
Tony
