Slow binary mode fwrite() in server 2003

  • Thread starter Thread starter Bjørn Fossum
  • Start date Start date
B

Bjørn Fossum

Hello,
I have a problem writing a bitmap image efficiently to disk, using fwrite()
under Windows Server 2003.

My test image is ~37MB and uses 2.5sec to be written to file on the server.

The same operation with the same image only takes 1.2sec to on my XP
development computer. The opposite relation was expected...

The server in question is a dual cpu, quad core dell server running Server
2003 Web Edition SP2, 2GB RAM etc.

Another strange thing is, when I experimented with this and set the file
mode to text, the time required to get the data to disk fell to 350ms!


This is the code snippet in question:
FILE *fp = fopen(file, "wb");
int size = fwrite(pbits, 1, pixmapsize, fp);
fclose(fp);

Hope someone here can give me a pointer to where this strange behaviour is
originated...
 
Bjørn Fossum said:
Hello,
I have a problem writing a bitmap image efficiently to disk, using
fwrite() under Windows Server 2003.

My test image is ~37MB and uses 2.5sec to be written to file on the
server.

The same operation with the same image only takes 1.2sec to on my XP
development computer. The opposite relation was expected...

The server in question is a dual cpu, quad core dell server running
Server 2003 Web Edition SP2, 2GB RAM etc.

Another strange thing is, when I experimented with this and set the
file mode to text, the time required to get the data to disk fell to
350ms!


This is the code snippet in question:
FILE *fp = fopen(file, "wb");
int size = fwrite(pbits, 1, pixmapsize, fp);
fclose(fp);

Hope someone here can give me a pointer to where this strange
behaviour is originated...
Click to expand...

What kind of disk are you writing to? If it's marked as removeable (and
even hard disks can be if your controller supports hot swapping) then the
call may not return until the data is written through to the disk.

XP has a habit of lying and telling you that the I/O is finished when it's
really not. This was fixed in Vista to prevent data corruption on e.g.
surprise removal of USB flash drives, and MS public image took a beating
because things looked like they took so much longer.

If you have other useful work to do while the file is being written, use
overlapped I/O. If not, then just understand that the data is probably
reaching the disk in the same amount of time, but the function isn't
returning early like it would on XP.
 
Ben Voigt said:
What kind of disk are you writing to? If it's marked as removeable (and
even hard disks can be if your controller supports hot swapping) then the
call may not return until the data is written through to the disk.

XP has a habit of lying and telling you that the I/O is finished when it's
really not. This was fixed in Vista to prevent data corruption on e.g.
surprise removal of USB flash drives, and MS public image took a beating
because things looked like they took so much longer.

If you have other useful work to do while the file is being written, use
overlapped I/O. If not, then just understand that the data is probably
reaching the disk in the same amount of time, but the function isn't
returning early like it would on XP.
Click to expand...

Hello Mr. Voigt,
I can not see that the disk is marked as removable, it is not of the "Hot
swap"-able kind.
It is interesting that XP might be fooling us on the time used for writing
the file. Then it may not be that big difference to the server 2003
installation as it looks.

Overlapped /asynchronous file save is probably not the path we want to
follow, we depend on getting the data down to file in order to free the
associated memory. We have a more or less continous flow of data which we
must save to file preferrably faster than it's being produced...

On the other hand, which difference should this make for the fact that in
text mode, I have the performance I expect? Shouldn't a removable disk behave
the same way for binary and text files?
 
Bjørn Fossum said:
Hello Mr. Voigt,
I can not see that the disk is marked as removable, it is not of the "Hot
swap"-able kind.
It is interesting that XP might be fooling us on the time used for writing
the file. Then it may not be that big difference to the server 2003
installation as it looks.

Overlapped /asynchronous file save is probably not the path we want to
follow, we depend on getting the data down to file in order to free the
associated memory. We have a more or less continous flow of data which we
must save to file preferrably faster than it's being produced...

On the other hand, which difference should this make for the fact that in
text mode, I have the performance I expect? Shouldn't a removable disk behave
the same way for binary and text files?
Click to expand...

Could you try with WriteFile method ( WriteFileEx) ? That would bypass
stdio methods and should be faster.

ismo
 
Bjørn Fossum said:
Hello Mr. Voigt,
I can not see that the disk is marked as removable, it is not of the "Hot
swap"-able kind.
It is interesting that XP might be fooling us on the time used for writing
the file. Then it may not be that big difference to the server 2003
installation as it looks.

Overlapped /asynchronous file save is probably not the path we want to
follow, we depend on getting the data down to file in order to free the
associated memory. We have a more or less continous flow of data which we
must save to file preferrably faster than it's being produced...
Click to expand...

Yes, but if you have multiple overlapped write requests then the OS or drive
electronics can complete them in what order is most efficient. Things like
elevator sorting can come into play. When using overlapped mode you'll have
an event handle you can wait on to ensure that the write reached the disk.
On the other hand, which difference should this make for the fact that in
text mode, I have the performance I expect? Shouldn't a removable disk
behave
the same way for binary and text files?
Click to expand...

Well, the issue actually is the presence of write-caching. XP enabled write
caching for all files. Newer OSes don't. But the stdio layer can still do
its own caching, and the mode (binary vs text) may and probably does affect
that.
 
Back
Top