Web Crawling Spidering Question

  • Thread starter Thread starter Rusty Hill
  • Start date Start date
R

Rusty Hill

My boss just gave me a project to create a spider that crawls a series of
web pages and harvests certain pieces of data they contain and store that
data in a database. Of course he needs all this done sooner than may be
realistic.

Can anyone point me to some samples, links or articles that might help me
significantly jump start my efforts on this? Thanks.
 
Rusty,

Take a look at the HttpWebRequest and HttpWebResponse classes. These
will help you get the contents of the page.

As for placing the data in a database, if you are using Sql Server, take
a look at the System.Data.SqlClient namespace, as they will help you with
getting the data into the database.
 
u will need httpwebrequest and hhtpwebresponse class in system.net
namespace and also an html parser to parse what you want.i have an
project about this.
 
Back
Top