Writing flat file to network is slow

  • Thread starter Thread starter Barry Flynn
  • Start date Start date
B

Barry Flynn

Hi

I am working with a VB 2005 program which has been converted from VB6.
It writes data out to a flat file, with code like the following line

WriteLine(riFileNo, "Hist", lsAssetID, lsRecordType, lsXNbr, lsFiscYr,
"Beg", CStr(H.BegBalAccDepn), CStr(H.BegBalCost), CStr(H.BegBalCostReval),
CStr(H.BegBalDepCost), CStr(H.BegBalDepnReval))

The program is running from within a Virtual PC
If it is writing to a file on drive c:, the job takes about 30 seconds.
If it is writing to a file on a network drive, it takes about 20 minutes.

If it writes to drive C:, then I copy the file to the network drive, the
copy operation takes just a couple of seconds.
There is very little traffic on the network.

What are possible reasons for it running so slowly when my code writes to
the network drive?

Thanks

Barry
 
Barry said:
I am working with a VB 2005 program which has been converted from VB6.
It writes data out to a flat file, with code like the following line

WriteLine(riFileNo, "Hist", lsAssetID, lsRecordType, lsXNbr, lsFiscYr,
"Beg", CStr(H.BegBalAccDepn), CStr(H.BegBalCost),
CStr(H.BegBalCostReval), CStr(H.BegBalDepCost),
CStr(H.BegBalDepnReval))
The program is running from within a Virtual PC
If it is writing to a file on drive c:, the job takes about 30
seconds. If it is writing to a file on a network drive, it takes
about 20 minutes.
If it writes to drive C:, then I copy the file to the network drive,
the copy operation takes just a couple of seconds.
There is very little traffic on the network.

What are possible reasons for it running so slowly when my code
writes to the network drive?
Click to expand...

For a start, you could try running it on a non-virtual PC to see if that's
creating a bottleneck.

Also, is the file so large that you can't put all the data in a
stringbuilder and then write that in one go to the [network] drive?

Andrew
 
Andrew Morton said:
Barry said:
I am working with a VB 2005 program which has been converted from VB6.
It writes data out to a flat file, with code like the following line

WriteLine(riFileNo, "Hist", lsAssetID, lsRecordType, lsXNbr, lsFiscYr,
"Beg", CStr(H.BegBalAccDepn), CStr(H.BegBalCost),
CStr(H.BegBalCostReval), CStr(H.BegBalDepCost),
CStr(H.BegBalDepnReval))
The program is running from within a Virtual PC
If it is writing to a file on drive c:, the job takes about 30
seconds. If it is writing to a file on a network drive, it takes
about 20 minutes.
If it writes to drive C:, then I copy the file to the network drive,
the copy operation takes just a couple of seconds.
There is very little traffic on the network.

What are possible reasons for it running so slowly when my code
writes to the network drive?
Click to expand...

For a start, you could try running it on a non-virtual PC to see if that's
creating a bottleneck.

Also, is the file so large that you can't put all the data in a
stringbuilder and then write that in one go to the [network] drive?

Andrew
Click to expand...

Andrew - thanks for the suggesrions.
For a start, you could try running it on a non-virtual PC to see if that's
creating a bottleneck.
Click to expand...

Unfortunately, that's a little difficult.
It runs in the Microsoft Dynamics SL environment, and the "real PC" has a
conflicting (VB6) version of that environment.
However, I thought that (a) the fact that it runs in a few seconds when
writng to drive c:, coupled with (b) the fact that I can copy the file to
the network in a few seconds, sort-of suggested that the virtual pc wasn't
the guilty party.
I recognze that a virtual pc is likely to be slower than a "real one", but
this job takes about 40 times as long when writing to the network.
Also, is the file so large that you can't put all the data in a
stringbuilder and then write that in one go to the [network] drive?
Click to expand...

Yes I agree that it would be better to use string builders, and probably
streams.
If I was writimg the program from scratch, I probably would do that.
This prog was converted from a VB6 version, and contains a lot of these
WriteLine statements as it writes different data from several different
tables.
So there's a bit of work involved in changing it.

I do intend to do some testing to see if different ways of writing the data
make a difference.
However, I thought that the fact that it runs in a few seconds when writing
to a local drive indicated that use of stringbuilders etc. might not make a
huge difference.

Thanks

Barry
 
Barry said:
I do intend to do some testing to see if different ways of writing
the data make a difference.
However, I thought that the fact that it runs in a few seconds when
writing to a local drive indicated that use of stringbuilders etc.
might not make a huge difference.
Click to expand...

Ah, but then you could write the whole file in one go so buffers might have
more efficient use made of them [sorry about that grammar]. The local file
system will use 32KB buffers (I think) whereas sending it out from a virtual
PC through real hardware and over a network, well, I can only make guesses.

Andrew
 
Back
Top