StreamReader Excessive Memory Use

  • Thread starter Thread starter Jim Davis
  • Start date Start date
J

Jim Davis

I have an 8 MB text file that I open with a StreamReader and the call
ReadToEnd(). TaskManager shows the exe memory use jumping from 14 MB to 58
MB. How could this be when the file is only 8 MB??? This seems grossly
inefficient. The code in my test form is very simple


StreamReader sr = new StreamReader(txtPath.Text;);
string buffer = sr.ReadToEnd();
sr.Close()

I would love an explanation for this.

Thanks,

Jim
 
Jim Davis said:
I have an 8 MB text file that I open with a StreamReader and the call
ReadToEnd(). TaskManager shows the exe memory use jumping from 14 MB to
58
MB. How could this be when the file is only 8 MB??? This seems grossly
inefficient. The code in my test form is very simple


StreamReader sr = new StreamReader(txtPath.Text;);
string buffer = sr.ReadToEnd();
sr.Close()

I would love an explanation for this.
Click to expand...


ReadToEnd is a convenience method, and it's not supposed to be terribly
efficient. For large files where you care about the amount of memory
allocation, then you just have to read the file more carefully.

Here's a better implementation of what ReadToEnd does

public static string ReadTextFile(string path)
{
FileStream fs = new FileStream(path, FileMode.Open);
StreamReader s= new StreamReader(fs);
//this is the key: pre-allocate the StringBuilder
System.Text.StringBuilder sb = new
System.Text.StringBuilder((int)fs.Length );
char[] buf = new char[1024*8];
int br = s.Read(buf,0,buf.Length);
while (br > 0)
{
sb.Append(buf,0,br);
br = s.Read(buf,0,buf.Length);
}
return sb.ToString();
}


David
 
David Browne said:
Jim Davis said:
I have an 8 MB text file that I open with a StreamReader and the call
ReadToEnd(). TaskManager shows the exe memory use jumping from 14 MB to
58
MB. How could this be when the file is only 8 MB??? This seems grossly
inefficient. The code in my test form is very simple


StreamReader sr = new StreamReader(txtPath.Text;);
string buffer = sr.ReadToEnd();
sr.Close()

I would love an explanation for this.
Click to expand...


ReadToEnd is a convenience method, and it's not supposed to be terribly
efficient. For large files where you care about the amount of memory
allocation, then you just have to read the file more carefully.

Here's a better implementation of what ReadToEnd does

public static string ReadTextFile(string path)
{
FileStream fs = new FileStream(path, FileMode.Open);
StreamReader s= new StreamReader(fs);
//this is the key: pre-allocate the StringBuilder
System.Text.StringBuilder sb = new
System.Text.StringBuilder((int)fs.Length );
char[] buf = new char[1024*8];
int br = s.Read(buf,0,buf.Length);
while (br > 0)
{
sb.Append(buf,0,br);
br = s.Read(buf,0,buf.Length);
}
return sb.ToString();
}
Click to expand...

Yikes, forgot to close the file.

David

public static string ReadTextFile(string path)
{
using (FileStream fs = new FileStream(path, FileMode.Open))
{
StreamReader s= new StreamReader(fs);
//this is the key: pre-allocate the StringBuilder
System.Text.StringBuilder sb = new
System.Text.StringBuilder((int)fs.Length );
char[] buf = new char[1024*8];
int br = s.Read(buf,0,buf.Length);
while (br > 0)
{
sb.Append(buf,0,br);
br = s.Read(buf,0,buf.Length);
}
return sb.ToString();
}

}
 
Back
Top