HttpWebRequest Timeout problem in DNS resolution

  • Thread starter Thread starter Morgan Cheng
  • Start date Start date
M

Morgan Cheng

In the doc http://msdn2.microsoft.com/en-us/library/system.net.httpwebrequest.timeout.aspx,
it reads,
"A Domain Name System (DNS) query may take up to 15 seconds to return
or time out. If your request contains a host name that requires
resolution and you set Timeout to a value less than 15 seconds, it may
take 15 seconds or more before a WebException is thrown to indicate a
timeout on your request."

The words doesn't appear in .Net 1.1 doc, only apppears since 2.0. Our
system is migrating from 1.1 to 2.0. Unfortunately, we met such issue,
since DNS servier is different in in dev & test environment, it is not
discovered till production release.

For non-exist domain like "www.abcdefaj12343jdafad.abd",
HttpWebRequest.GetResponse() doesn't return after 20 seconds hang,
even we set timeout 5 seconds to HttpWebRequest.Timeout.

This is un-tolerable. My current solution is to use
Dns.BeginGetHostEntry like asynchronous method to detect DNS with 1
second timeout, if it doesn't get result successfuly in 1 second. The
code doesn't proceed to call HttpWebRequest.GetResponse().

I know there is HttpWebRequest.BeginGetResponse(), but i don't want a
non-existent domain cost 5 seconds delay in crawling.

Anyone has any better idea?
Is it possible to config the DNS timeout without code change?
And, I really don't understand why .Net 2.0 bring such limitation.

Thanks.
 
Back
Top