Issue with FileStream on network with transfers larger than 64 megabyte?

  • Thread starter Thread starter Jeroen X. Pluimers
  • Start date Start date
J

Jeroen X. Pluimers

I've come accross a .NET 2.0 issue when writing large MemoryStreams to
FileStreams.

The issue is with this code:

private static void WriteMemoryStreamToFile(string filename,
System.IO.MemoryStream memory)
{
using (System.IO.Stream
file = new FileStream(filename, FileMode.OpenOrCreate,
FileAccess.Write),
buffer = new System.IO.BufferedStream(file)
)
{
buffer.Write(memory.GetBuffer(), 0, (int)memory.Length);
buffer.Close(); // temporary to see if the resource problem
is a handle problem
file.Close(); // temporary to see if the resource problem is
a handle problem
}
}

The code consistently fails when:
- the FileStream is on a network
- the MemoryStream is 64 megabytes or larger

The code succeeds when:
- the MemoryStream is smaller than 64 megabytes
- or the FileStream is not on a network

Client OS is Windows XP with .NET 2.0, and all service packs and fully
patched.
Server OS is Windows 2003 server with all service packs and fully patched.
Application is a command-line app (the code used to be WinForms but it has
been trimmed down pretty well; I can create an even more trimmed down
version if needed).

Example of the error message:

C:\Program Files\silentwavgenerator>silentwavgenerator 446.6
\\myserver\myshare\myfile.wav.WAV
Generating a WAV file of 446.6 seconds in \\myserver\myshare\myfile.wav.WAV
DataBlockCount = 19695060
error during generation the 446.6 second WAV file
\\myserver\myshare\myfile.wav.WAV
System.IO.IOException: Insufficient system resources exist to complete the
requested service.

at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.WriteCore(Byte[] buffer, Int32 offset, Int32
count)
at System.IO.FileStream.Write(Byte[] array, Int32 offset, Int32 count)
at System.IO.BufferedStream.Write(Byte[] array, Int32 offset, Int32
count)
at My.Library.MultiMedia.SilentWav.WriteMemoryStreamToFile(String
filename, MemoryStream memory)
at My.Library.MultiMedia.SilentWav.GenerateSilentWavFile(String filename,
Decimal seconds)
at My.Application.SilentWavGenerator.Program.logic(String[] args)

(a 446 second WAV file is about 78 megabytes large).


Where can I file a bugreport?
What extra information is needed for a bugreport?

Please CC my e-mail address when replying.

--jeroen
 
JXP> The code consistently fails when:
JXP> - the FileStream is on a network
JXP> - the MemoryStream is 64 megabytes or larger

Did you try to perform write operation with chunks of the file?
Click to expand...

Yup. I should have mentioned that I tried that workaround already, and it
works.
But the since the Write accepts ints as length parameter, and there is no
documentation about limitations, it is supposed to work until about 2
gigabyte chunks (and do any necessary splitting itself).

--jeroen
 
Hello, Jeroen!

[skipped]

JXP> But the since the Write accepts ints as length parameter, and there is
JXP> no documentation about limitations, it is supposed to work until about
JXP> 2 gigabyte chunks (and do any necessary splitting itself).

From the API point of view you're right.

The cause of the expception maybe following issue. When doing such kind of I/O ( on network file )
network redirector is involved, and IMO the error message you get comes from that layer.

--
Regards, Vadym Stetsyak
www: http://vadmyst.blogspot.com
 
JXP> But the since the Write accepts ints as length parameter, and there
is
JXP> no documentation about limitations, it is supposed to work until
about
JXP> 2 gigabyte chunks (and do any necessary splitting itself).
Click to expand...
From the API point of view you're right.
Thx!

The cause of the expception maybe following issue. When doing such kind of
I/O ( on network file )
network redirector is involved, and IMO the error message you get comes
from that layer.
Click to expand...

I've tried finding info about that, but even with heavy searching, I
couldn't.
Any suggestions for further digging?

--jeroen
 
Are we just trying to find the source for personal interest. As you know,
the solution is to send smaller blocks. I don't think it is ever a good
idea to load and send such large buffer sizes.

--
William Stacey [MVP]

|
| |
| > JXP> But the since the Write accepts ints as length parameter, and there
| > is
| > JXP> no documentation about limitations, it is supposed to work until
| > about
| > JXP> 2 gigabyte chunks (and do any necessary splitting itself).
|
| > From the API point of view you're right.
|
| Thx!
|
| > The cause of the expception maybe following issue. When doing such kind
of
| > I/O ( on network file )
| > network redirector is involved, and IMO the error message you get comes
| > from that layer.
|
| I've tried finding info about that, but even with heavy searching, I
| couldn't.
| Any suggestions for further digging?
|
| --jeroen
|
|
 
William Stacey said:
Are we just trying to find the source for personal interest. As you know,
the solution is to send smaller blocks. I don't think it is ever a good
idea to load and send such large buffer sizes.
Click to expand...

It is not only personal interest. From an API perspective, it is good to
have documentation about circumstances where the API could not work so other
users are prevented from falling in the same pit as I did.

--jeroen
 
Back
Top