GZipStream catch-22

  • Thread starter Thread starter bamelyan
  • Start date Start date
B

bamelyan

GZipStream unzip = new GZipStream(stream, CompressionMode.Decompress, false);

In order to read from unzip.Read(buffer, 0, unknownLength), I need to
allocate enough buffer length.
In order to know how many bytes to allocate I need to call
unzip.Read(buffer, 0, unknownLength) that returns bytes.

How do i get unknownLength to allocate buffer before the Read?
 
bamelyan said:
GZipStream unzip = new GZipStream(stream, CompressionMode.Decompress, false);

In order to read from unzip.Read(buffer, 0, unknownLength), I need to
allocate enough buffer length.
In order to know how many bytes to allocate I need to call
unzip.Read(buffer, 0, unknownLength) that returns bytes.

How do i get unknownLength to allocate buffer before the Read?
Click to expand...

unknownLength is the size of your buffer. It is not really "unknown".
The bytes returned will be less than or equal to the value sent into the
call. It seems as though you may be interpreting the call is asking for
the number of bytes in the compressed stream, which is incorrect.
 
bamelyan said:
GZipStream unzip = new GZipStream(stream, CompressionMode.Decompress, false);

In order to read from unzip.Read(buffer, 0, unknownLength), I need to
allocate enough buffer length.
In order to know how many bytes to allocate I need to call
unzip.Read(buffer, 0, unknownLength) that returns bytes.
Click to expand...
Not at all. Any buffer size that's not 0 will do. The Read() method returns
the number of bytes actually read. If it's 0, you're at the end of the
stream. If not, your buffer contains valid data from index 0 up to whatever
it returned (this may not fill your buffer entirely, even when not at the
end of the stream). Process this data in whatever way you want (for example,
by appending it to a MemoryStream you're going to read in its entirety
later), rinse, repeat.

All streams work this way, not just GZipStream.
How do i get unknownLength to allocate buffer before the Read?
Click to expand...

Pick any length. Let's say 1024 because it's a nice round number, and it's
what BufferedStream uses by default. (Using BufferedStream and .ReadByte()
is another option -- not as efficient as calling .Read() yourself, but by
far more efficient than calling .ReadByte() on GZipStream directly.)
 
bamelyan said:
GZipStream unzip = new GZipStream(stream, CompressionMode.Decompress, false);

In order to read from unzip.Read(buffer, 0, unknownLength), I need to
allocate enough buffer length.
In order to know how many bytes to allocate I need to call
unzip.Read(buffer, 0, unknownLength) that returns bytes.

How do i get unknownLength to allocate buffer before the Read?
Click to expand...

Just pick a buffer size (like 51200) and read in a while
loop and process the data as they come in.

If you need to keep all the data in a buffer in memory, then
you will need to either know the maximum size or copy over
to a larger buffer when needed.

You can know how many bytes if the protocol somehow sends
the uncompressed length before the gzipped data.

If the data is "normal", then creating a buffer 10 times as
big as the compressed data and copy to a new larger buffer
inside the while loop if necessary should be fine in the sense
that the chance of having to copy data would be very small.

Arne
 
Back
Top