Web service - Large dataset problem

  • Thread starter Thread starter kasi.uh
  • Start date Start date
K

kasi.uh

I have a web service that is up and running. I query a central
database on the client side and pass the DataSet to the webservice. It
works fine for small datasets. When I try to call a webmethod on that
webservice for a large dataset (A table with 300,000 rows), I get the
following error message:
The underlying connection was closed: An unexpected error occurred on
a receive.
at
System.Web.Services.Protocols.WebClientProtocol.GetWebResponse(WebRequest
request)
at
System.Web.Services.Protocols.HttpWebClientProtocol.GetWebResponse(WebRequest
request)
at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String
methodName, Object[] parameters)

I have tried all those options in the KB page, none seem to work.
http://support.microsoft.com/kb/915599#top

Btw, am running VS2005 and IIS 6.0 on Win XP Pro. Thanks in advance!
 
Why do you want to use a webservice to deliver 300,000 records? This is not
a knock on your attempt, but so you can think through the architecture and
determine if this is the best approach. I know of no human who can go
through 300,000 records in one meal, so I would separate it into different
hits, based on what the user is actually doing.

The idea behind SOA is to work in messages, not small bits of data, but this
does not mean it should be used for huge datasets (the other end of the
spectrum).

--
Gregory A. Beamer
MVP; MCP: +I, SE, SD, DBA
http://gregorybeamer.spaces.live.com
Co-author: Microsoft Expression Web Bible (upcoming)

************************************************
Think outside the box!
************************************************
 
Hello,

by default, the request limit of a Webservice is limited to 4MB. See
http://support.microsoft.com/kb/910436/en-us for more infos on that topic
and how it can be changed.

That said, It might be a good idea to split the request into multiple
smaller requests and to the processing asynchronous on the server....

Kind regards,
Henning Krause
 
Rather than passing a dataset, pass the string which results from

1) Writing the dataset to xml
2) Compressing the xml string
3) Base64Encodeing the compressed result.

You will discover that the amount of data passed over the wire is reduced by
86% which means, in your case, it is equivalent to sending just .14 * 300000
rows or just 42,000 rows.

Jim
 
Back
Top