large string manipulation

  • Thread starter Thread starter douglas wittner
  • Start date Start date
D

douglas wittner

morning all,
can someone help me find the most efficient way of manipulating a large
file.

i need to replace special characters in a large file and multiple
string.replace functions are causing memory errors.

doug
 
douglas wittner said:
can someone help me find the most efficient way of manipulating a large
file.
Click to expand...

An efficient way for modifications that will change the length of the
file (i.e. possibly insert or delete in the middle) is to stream from
one file to another, iteratively using a buffer in memory, and working
on the buffer before streaming it out to the second file. It requires
careful buffer management to be sure that the boundaries between buffers
don't affect the outcome. For example, if searching for "foo", be
careful that one buffer doesn't end with "f" and the next one start with
"oo".

For modifications which don't change the length of the file, iteratively
reading into a smallish buffer, searching, and then using
FileStream.Seek & Write to perform updates will work reasonably well -
but not as well as a memory-mapped file, but that would be more awkward
in C#, since there is no memory-mapped file primitive.

-- Barry
 
i have the reading down...while reading into a buffer, how do i perrform
the search for a character?

thanks for your help
 
douglas wittner said:
i have the reading down...while reading into a buffer, how do i perrform
the search for a character?
Click to expand...

Either load it up into a string or character array (with an Encoding
class - and watch for split lead and trail bytes for UTF8 encoding, for
example) or scan through it with a loop, or write a more sophisticated
search such as Boyer-Moore etc.

-- Barry
 
Back
Top