Capture Web Page Source Code

  • Thread starter Thread starter oitbso
  • Start date Start date
O

oitbso

I was trying to collect some real estate data from a website and found thatI was unable to capture the source code using the usual techniques. For example,

my_url = "http://www.coloproperty.com/index.c...ame=Eisenhower&City=61&ResultType=quickReport"

With ie
.Visible = True
.Navigate my_url
.Top = 50
.Left = 530
.Height = 400
.Width = 400

Do Until .ReadyState = 4
DoEvents
Loop
End With

my_var = ie.document.body.innerhtml

opens the desired web page but does not capture the source code. If I right-click the web page and select "View Source", I can see the source code I'd like to capture, but the variable contains just a few lines of something different. Any help on capturing the actual source code behind this web page would be much appreciated...TIA, Ron
 
How about an external query? correct word wrap
MLS# Price SQFT Beds Baths Address Subdivision Locale Photo
Pano
672274 $279,900 1728 3 2 376 Eisenhower Dr Hunters Ridge
Louisville
665060 $499,000 3076 3 3 1806 Eisenhower Dr PONDEROSA
Louisville
 
Thanks Don, what code did you use? When I use

With ActiveSheet.QueryTables.Add(Connection:= _
"URL;" & my_url, _
Destination:=Range("A1"))
.BackgroundQuery = True
.TablesOnlyFromHTML = False
.Refresh BackgroundQuery:=False
.SaveData = True
End With

nothing is imported into the activesheet. Also, do you know of anyway to capture the source code and not import anything directly into the workbook?..Ron
 
For anyone who follows, turns out the info I wanted was in a frame and

my_var = my_var = ie.Document.frames("content").Document.DocumentElement.innerhtml

does the trick.

Thanks again to Don...Ron
 
Thanks Don, what code did you use?  When I use

    With ActiveSheet.QueryTables.Add(Connection:= _
      "URL;" & my_url, _
      Destination:=Range("A1"))
        .BackgroundQuery = True
        .TablesOnlyFromHTML = False
        .Refresh BackgroundQuery:=False
        .SaveData = True
    End With

nothing is imported into the activesheet.  Also, do you know of anyway to capture the source code and not import anything directly into the workbook?..Ron
Click to expand...

I clicked on the url>it changed to which I used for the query.

With ActiveSheet.QueryTables.Add(Connection:= _
"URL;http://www.coloproperty.com/Listings/index.cfm?
&Action=DoAddressSearch&ResultType=quickReport&City=61&StreetName=Eisenhower"
_
, Destination:=Range("A5"))
 
Don...Thanks for pointing that out. With your url, the information is not in a frame so I can simply use
my_var = ie.document.body.innerhtml

Better yet, I don't even need to open IE. With your url the "GET" method works just fine.
Set my_obj = CreateObject("MSXML2.XMLHTTP")
my_obj.Open "GET", my_url, False
my_obj.send
my_var = my_obj.responsetext
Set my_obj = Nothing

....Ron
 
Back
Top