Quote extraction from Yahoo Historical Quotes

  • Thread starter Thread starter Pete Davis
  • Start date Start date
P

Pete Davis

I have the following method that retrieves quotes from the Yahoo historical
quotes web page.

According to their FAQ, there is no limitation on the number of quotes one
may download, and because the problem seems to come and go, I'm concerned it
may be a problem with my code.

In particular, the problem seems to occur when I retrieve quotes over a long
period of time. Normally the routine runs every night and downloads the
previous days quotes. Occasionally, however, I'll download one or more years
worth of quotes for several stocks. It's on these occasions when the code
seems to fail. It doesn't fail the first time, though, and it won't
necessarily fail on the stock that I'm retrieving multiple quotes from.

The exception that is thrown is: "System.Net.WebException: The underlying
connection was closed: The server committed an HTTP protocol violation." The
exception is thrown at wrq.GetResponse().

I originally thought it was because I was going through a machine using
Internet Connection Sharing, but then I did it directly to the internet
without a proxy, and the same problem occurred.

Anyone have any ideas? Code follows:

public QuoteList GetQuoteHistory(Stock StockOb, DateTime StartDate, DateTime
EndDate)
{
if (StartDate.Year < 1970)
{
StartDate = new DateTime(1997, 1, 1);
}
bool done = false;
int tryCount = 0;
QuoteList list = new QuoteList();
while (!done)
{
// Build url query string
string url = @"http://" + yahooIP + @"/table.csv?a=" +
((int)(StartDate.Month - 1)).ToString() + @"&b=" +
StartDate.Day.ToString() +
@"&c=" + StartDate.Year.ToString() + @"&d=" +
((int)(EndDate.Month - 1)).ToString() + @"&e=" + EndDate.Day.ToString() +
@"&f=" + EndDate.Year.ToString() + @"&s=" + StockOb.Ticker +
@"&y=0&g=d&ignore=.csv";

WebRequest wrq = null;
Stream strm = null;
StreamReader sr = null;
try
{
wrq = WebRequest.Create(url);
wrq.Timeout = 30000;
wrq.Proxy = WebProxy.GetDefaultProxy();
strm = wrq.GetResponse().GetResponseStream();
sr = new StreamReader(strm);
}
catch(System.Exception ex)
{
if (null != sr)
{
sr.Close();
}
if (null != strm)
{
strm.Close();
}
// Check to see if the operation timed out. If it did,
// retry a couple of times.
if ("The operation has timed-out." == ex.Message)
{
tryCount++;
if (tryCount > 2)
{
return null;
}
continue;
}
else
{
return null;
}
}
string line;
try
{
// If there's no day, close out.
if (null == (line = sr.ReadLine()))
{
return null;
}
while (null != (line = sr.ReadLine()))
{
// Parse quote data nad add the quote if it's not already in the
// list of quotes
char[] splitCh = new Char[1];
splitCh[0] = ',';
string[] items = line.Split(splitCh);
Quote quote = new Quote();
quote.QuoteDate = Convert.ToDateTime(items[0]);
quote.OpenPrice = (float)Convert.ToDouble(items[1]);
quote.HighPrice = (float)Convert.ToDouble(items[2]);
quote.LowPrice = (float)Convert.ToDouble(items[3]);
quote.ClosePrice = (float)Convert.ToDouble(items[4]);
quote.Volume = Convert.ToInt32(items[5]);
if (false == list.ContainsKey(quote.QuoteDate))
{
list.Add(quote.QuoteDate, quote);
}
}
}
catch(System.Exception ex)
{
Trace.WriteLine("Unknown exception processing request: " +
ex.ToString());
}
finally
{
// Close everything nice and clean
sr.Close();
strm.Close();
done = true;
}
}
return list;
}
 
My apologies. I have no idea what possessed me to post this to the Winforms
group.

Pete

Pete Davis said:
I have the following method that retrieves quotes from the Yahoo historical
quotes web page.

According to their FAQ, there is no limitation on the number of quotes one
may download, and because the problem seems to come and go, I'm concerned it
may be a problem with my code.

In particular, the problem seems to occur when I retrieve quotes over a long
period of time. Normally the routine runs every night and downloads the
previous days quotes. Occasionally, however, I'll download one or more years
worth of quotes for several stocks. It's on these occasions when the code
seems to fail. It doesn't fail the first time, though, and it won't
necessarily fail on the stock that I'm retrieving multiple quotes from.

The exception that is thrown is: "System.Net.WebException: The underlying
connection was closed: The server committed an HTTP protocol violation." The
exception is thrown at wrq.GetResponse().

I originally thought it was because I was going through a machine using
Internet Connection Sharing, but then I did it directly to the internet
without a proxy, and the same problem occurred.

Anyone have any ideas? Code follows:

public QuoteList GetQuoteHistory(Stock StockOb, DateTime StartDate, DateTime
EndDate)
{
if (StartDate.Year < 1970)
{
StartDate = new DateTime(1997, 1, 1);
}
bool done = false;
int tryCount = 0;
QuoteList list = new QuoteList();
while (!done)
{
// Build url query string
string url = @"http://" + yahooIP + @"/table.csv?a=" +
((int)(StartDate.Month - 1)).ToString() + @"&b=" +
StartDate.Day.ToString() +
@"&c=" + StartDate.Year.ToString() + @"&d=" +
((int)(EndDate.Month - 1)).ToString() + @"&e=" + EndDate.Day.ToString() +
@"&f=" + EndDate.Year.ToString() + @"&s=" + StockOb.Ticker +
@"&y=0&g=d&ignore=.csv";

WebRequest wrq = null;
Stream strm = null;
StreamReader sr = null;
try
{
wrq = WebRequest.Create(url);
wrq.Timeout = 30000;
wrq.Proxy = WebProxy.GetDefaultProxy();
strm = wrq.GetResponse().GetResponseStream();
sr = new StreamReader(strm);
}
catch(System.Exception ex)
{
if (null != sr)
{
sr.Close();
}
if (null != strm)
{
strm.Close();
}
// Check to see if the operation timed out. If it did,
// retry a couple of times.
if ("The operation has timed-out." == ex.Message)
{
tryCount++;
if (tryCount > 2)
{
return null;
}
continue;
}
else
{
return null;
}
}
string line;
try
{
// If there's no day, close out.
if (null == (line = sr.ReadLine()))
{
return null;
}
while (null != (line = sr.ReadLine()))
{
// Parse quote data nad add the quote if it's not already in the
// list of quotes
char[] splitCh = new Char[1];
splitCh[0] = ',';
string[] items = line.Split(splitCh);
Quote quote = new Quote();
quote.QuoteDate = Convert.ToDateTime(items[0]);
quote.OpenPrice = (float)Convert.ToDouble(items[1]);
quote.HighPrice = (float)Convert.ToDouble(items[2]);
quote.LowPrice = (float)Convert.ToDouble(items[3]);
quote.ClosePrice = (float)Convert.ToDouble(items[4]);
quote.Volume = Convert.ToInt32(items[5]);
if (false == list.ContainsKey(quote.QuoteDate))
{
list.Add(quote.QuoteDate, quote);
}
}
}
catch(System.Exception ex)
{
Trace.WriteLine("Unknown exception processing request: " +
ex.ToString());
}
finally
{
// Close everything nice and clean
sr.Close();
strm.Close();
done = true;
}
}
return list;
}
Click to expand...
 
Back
Top