G
Guest
I have been working through an issue with a client for some time, and would
like to see if anyone can confirm my observations of behavior experienced
when using the HttpWebRequest class.
Here is the scenario:
The client application is written in C# and is making a large HTTP POST
request to a backend web server, using SSL. The payload contains MIME
attachments (this is a Soap with Attachments client), of which the last
attachment is a large, binary, blob (the original file that the blob is
constructed from can be pretty big – 8, 10 even 25 MB).
We are able to send these large requests, when SSL is turned off, but as
soon as SSL is turned back on, the largest POST we can make contains an
attachment of about 4MB. Any payloads that contain attachments greater than
4MB end up timing out on the server.
My observations:
From what I have read, it appears that the HttpWebRequest class sends a POST
by breaking up the headers and the body and sending them seperately (in the
same connection). Thus, the headers are sent to the server and then the body
follows in a different transmission. My backend server seems to reflect this
as well – in the cases where the server times out, the headers are processed
correctly, as noted by the server logs. But the time out occurs as the server
waits for the body to be sent to it.
I have verified with the server vendor that the server software does indeed
have a 10 second timeout, where if no data has come across in that 10 second
interval, the server aborts the handling of the request and shuts down the
connection. On the client, I have done some rudimentary sniffing and it
appears that indeed there is no traffic happening during this 10 second
interval (after sending the headers). I am making an assumption that with a
large request body like this, under SSL, the client/framework needs to do
some work to prepare the body (like applying the appropriate encryption) and
this is taking a period of time greater than 10 seconds to complete. I have
somewhat proven this by sending the request to a proxy first, and the proxy
takes a good 30secs – 2 mins to fully receive the request from the client,
before being ready to transmit it to the server. Once the proxy receives the
full request, I am able to forward it to the backend server successfully –
which tells me that the format of the payload is correct.
What I am trying to figure out is the following:
- Is the HttpWebRequest misbehaving?
- Is something coded incorrectly in the client that is causing this delay in
transmission?
- Is the backend server in the wrong for timing out an active request in
such a short period of time?
If this is simply the normal behavior of HttpWebRequest, I am inclined to
recommend that the developer look at using the SSLStream and TcpClient
classes in the 2.0 Framework, which would give him complete control over
everything happening at the socket level. But I’d like to confirm that there
is nothing that we can change in the current HttpWebRequest client, before
heading down that path.
Thanks in advance for your response – I’d love to determine some sort of
resolution on this topic.
like to see if anyone can confirm my observations of behavior experienced
when using the HttpWebRequest class.
Here is the scenario:
The client application is written in C# and is making a large HTTP POST
request to a backend web server, using SSL. The payload contains MIME
attachments (this is a Soap with Attachments client), of which the last
attachment is a large, binary, blob (the original file that the blob is
constructed from can be pretty big – 8, 10 even 25 MB).
We are able to send these large requests, when SSL is turned off, but as
soon as SSL is turned back on, the largest POST we can make contains an
attachment of about 4MB. Any payloads that contain attachments greater than
4MB end up timing out on the server.
My observations:
From what I have read, it appears that the HttpWebRequest class sends a POST
by breaking up the headers and the body and sending them seperately (in the
same connection). Thus, the headers are sent to the server and then the body
follows in a different transmission. My backend server seems to reflect this
as well – in the cases where the server times out, the headers are processed
correctly, as noted by the server logs. But the time out occurs as the server
waits for the body to be sent to it.
I have verified with the server vendor that the server software does indeed
have a 10 second timeout, where if no data has come across in that 10 second
interval, the server aborts the handling of the request and shuts down the
connection. On the client, I have done some rudimentary sniffing and it
appears that indeed there is no traffic happening during this 10 second
interval (after sending the headers). I am making an assumption that with a
large request body like this, under SSL, the client/framework needs to do
some work to prepare the body (like applying the appropriate encryption) and
this is taking a period of time greater than 10 seconds to complete. I have
somewhat proven this by sending the request to a proxy first, and the proxy
takes a good 30secs – 2 mins to fully receive the request from the client,
before being ready to transmit it to the server. Once the proxy receives the
full request, I am able to forward it to the backend server successfully –
which tells me that the format of the payload is correct.
What I am trying to figure out is the following:
- Is the HttpWebRequest misbehaving?
- Is something coded incorrectly in the client that is causing this delay in
transmission?
- Is the backend server in the wrong for timing out an active request in
such a short period of time?
If this is simply the normal behavior of HttpWebRequest, I am inclined to
recommend that the developer look at using the SSLStream and TcpClient
classes in the 2.0 Framework, which would give him complete control over
everything happening at the socket level. But I’d like to confirm that there
is nothing that we can change in the current HttpWebRequest client, before
heading down that path.
Thanks in advance for your response – I’d love to determine some sort of
resolution on this topic.