MD5 Large Files

  • Thread starter Thread starter chis2k
  • Start date Start date
C

chis2k

I need help figuring out how to get the MD5 hash of large files.
I cannot go about the traditional way of reading the entire file into a byte
array, because then the huge file is placed in memory, and bye bye system
resources! I know there might be way using streams, but I am unsure how to
implement it. Can anyone give me an example?

I've tried this but it didn't work, it still loaded the entire blasted thing
into memory!.

public byte[] GetMD5HashFromFile( string file_name) {
FileStream file = new FileStream( file_name, FileMode.Open );
MD5 md5 = new MD5CryptoServiceProvider();
byte[] retVal = md5.ComputeHash(file);
file.Close();
return retVal;
}
 
I need help figuring out how to get the MD5 hash of large files.
I cannot go about the traditional way of reading the entire file into a byte
array, because then the huge file is placed in memory, and bye bye system
resources! I know there might be way using streams, but I am unsure how to
implement it. Can anyone give me an example?

I've tried this but it didn't work, it still loaded the entire blasted thing
into memory!.

public byte[] GetMD5HashFromFile( string file_name) {
FileStream file = new FileStream( file_name, FileMode.Open );
MD5 md5 = new MD5CryptoServiceProvider();
byte[] retVal = md5.ComputeHash(file);
file.Close();
return retVal;
}
Click to expand...

The ComputeHash method on the MD5 class can take a stream. My code is
this:

StringBuilder sb=new StringBuilder();
FileStream fs=new FileStream(filename,FileMode.Open);
MD5 md5=new MD5CryptoServiceProvider();
byte[] hash=md5.ComputeHash(fs);
fs.Close();
foreach (byte hex in hash)
sb.Add(hex.ToString("x2"));
string md5sum=sb.ToString();
 
I need help figuring out how to get the MD5 hash of large files.
I cannot go about the traditional way of reading the entire file into a byte
array, because then the huge file is placed in memory, and bye bye system
resources! I know there might be way using streams, but I am unsure how to
implement it. Can anyone give me an example?

I've tried this but it didn't work, it still loaded the entire blasted thing
into memory!.

public byte[] GetMD5HashFromFile( string file_name) {
FileStream file = new FileStream( file_name, FileMode.Open );
MD5 md5 = new MD5CryptoServiceProvider();
byte[] retVal = md5.ComputeHash(file);
file.Close();
return retVal;
}
Click to expand...

The ComputeHash method on the MD5 class can take a stream. My code is
this:

StringBuilder sb=new StringBuilder();
FileStream fs=new FileStream(filename,FileMode.Open);
MD5 md5=new MD5CryptoServiceProvider();
byte[] hash=md5.ComputeHash(fs);
fs.Close();
foreach (byte hex in hash)
sb.Add(hex.ToString("x2"));
string md5sum=sb.ToString();
Click to expand...

I just noticed we have the same code. I think you're mistaken when
you say that it reads the entire file into memory. A quick test I did
had my MD5 winform using only 19MB of memory when processing a 140MB
file.

Austin Ehlers
 
Back
Top