Scraping data from a web form submit

[Brent]

Hi,
I want to build an app (C#, windows app, or web app, shouldn't make a
difference) that submits an address to the canada post website's address
lookup and then scrape the postal code out of the resulting page. Here is
their page. http://www.canadapost.ca/tools/pcl/bin/advanced-e.asp It uses a
form post.

I would like a basic rundown of how to submit my own info into their results
page (post), and then scrape out the info I need.
Thanks,
Brent

 

[bruce barker]

1) use webclient class to call the website.
2) hit the page and view source to see what the input control names are
3) post the page with the values you want.
4) write code to parse the returning html for the values you want. a few reg
expressions ought to work.

-- bruce (sqlwork.com)

 

[Scott Allen]

in addition to Bruce's info, there is a tool (Fiddler) which is a
debugging proxy - it will capture all the HTTP traffic and allow you
to analyze it. This can be a great help when figuring out what you
need to POST.

http://www.fiddlertool.com/fiddler/

 

[Brent]

Thanks for the help, I got it now. Here is the full thing of code that I
used...
System.Collections.Specialized.NameValueCollection values = new System.

Collections.Specialized.NameValueCollection();

values.Add("street_number","113");

values.Add("street_name","MCINNES");

values.Add("prov","AB");

WebClient webclient = new WebClient();

webclient.Headers.Add("Content-Type","application/x-www-form-urlencoded");

byte[] responseArray = webclient.UploadValues(

"http://www.canadapost.ca/tools/pcl/bin/cp_search_response-e.asp","POST",

values);

string response = System.Text.Encoding.ASCII.GetString(responseArray);

this.message_textBox.Text=
response.Substring(response.IndexOf("AB",response.

IndexOf("PCLFormattedAddress"))+4,7);

 

[clintonG]

Aside from your current approach if I were you I would also ask
canadapost.ca if they intend to expose the data using web services
as every time their page changes you may have to rebuild your scraper.