HTTP Listener??

  • Thread starter Thread starter Rusty Hill
  • Start date Start date
R

Rusty Hill

Is there some tool that I can use to profile all the activity HTML, cookies,
headers, etc. that are sent back and forth to a site when I access it using
my web browser? Essentially I want to navigate a site manually in my
browser and get enough information so I can program something to crawl the
site in an automated fashion but need to know what headers and stuff to send
in my request. Thank you.
 
Rusty,

There are a few programs out there that will let you do this (I'm sure
there is even a firefox extension if that is your browser of choice).

For any kind of web browser though, you can use something like Fiddler
(which is free). All you have to do is run the program, and then set your
proxy settings on your web browser to point to the port and ip that the
interceptor is running on (usually the local host).
 
Charles will record an HTTP session (set of requests) and then replay
them. That sounds like what you're asking for, right?

http://www.xk72.com/charles/

I haven't used that aspec of Charles but in general it's a wonderful
app.

Sam
 
Rusty said:
Is there some tool that I can use to profile all the activity HTML, cookies,
headers, etc. that are sent back and forth to a site when I access it using
my web browser? Essentially I want to navigate a site manually in my
browser and get enough information so I can program something to crawl the
site in an automated fashion but need to know what headers and stuff to send
in my request. Thank you.
Click to expand...

Wireshark?
 
Back
Top