Using a MemoryStream with GZipStream

This is a very subtle yet annoying bug. And it's hard to find.
Fortunately, I found help on the BCL Team Blog.

Just in case (I've had posts that I linked to disappear in the past) I'll copy here what they say:

We’ve seen cases where our customers have run into issues when using a MemoryStream with GZip compression. The problem can be frustrating to debug and I thought I’ll blog about it in the hope that others would avoid a similar issue. The code for this looks like this;



Byte[] compressedBuffer;

MemoryStream stream = new MemoryStream();

using (GZipStream zip = new GZipStream(stream, CompressionMode.Compress))

{

//compress data

...



//Dont get the MemoryStream data before the GZipStream is closed since it doesn’t yet contain complete compressed data.

//GZipStream writes additional data including footer information when its been disposed

compressedBuffer = stream.ToArray(); //WRONG

}

// CORRECT CODE: call compressedBuffer = stream.ToArray() here after the GZipStream is disposed



The problem arises because the data in MemoryStream is not complete when ToArray is called before the GZipStream is closed. We will write any remaining compressed data and footer information to GZipStream when its being closed. The data in the MemoryStream is still accessible even after its been closed. Both ToArray and GetBuffer methods will return valid data after the MemoryStream has being disposed. This is not so much an issue when another stream like FileStream is used in compression since there is generally time before decompression when a file is used and its ok for the file to be re-opened when that happens.

posted @ Thursday, January 24, 2008 6:44 PM

Print

Comments on this entry:

# re: Using a MemoryStream with GZipStream

Left by Russ C. at 1/29/2008 11:38 AM
Gravatar
Hi, Thanks for the post.

Do you think this problem might also occur when using Deflate ?

# re: this problem might also occur when using Deflate

Left by Angel Eyes at 1/29/2008 12:01 PM
Gravatar
That would be a safe assumption, as the Deflate class basically works in the same manner.

The only question is, whether or not the Deflate format needs "closing" information. I don't know enough about it to answer this question, though.

# re: Using a MemoryStream with GZipStream

Left by David Mellow at 6/1/2008 2:34 PM
Gravatar
Oddly I've just blogged on this.

The constructor of the GZipStream should use a third parameter "true" to keep the underlying stream open when it (the compression stream) is closed. Then immediately before compressedBuffer = stream.ToArray();
close the GZipStream object. The memory stream is still open and able to be accessed.

# To: David Mellow

Left by Angel Eyes at 6/1/2008 2:41 PM
Gravatar
Hey David, how about a link to your blog?

# re: Using a MemoryStream with GZipStream

Left by David Mellow at 6/1/2008 2:50 PM
Gravatar
It's my first endeavour at blogging, and is on this topic. I kinda did it as a test. I only intstalled the blog engine yesterday so ironing out some bumps along the way too.

# To: David Mellow

Left by Angel Eyes at 6/1/2008 3:01 PM
Gravatar
OK, then. After you get your blog going, why don't check back with me? Thanks.

# re: Using a MemoryStream with GZipStream

Left by David Mellow at 6/1/2008 3:29 PM
Gravatar
Hi again Angel, my name should now be url to the blog, blog.davidmellow.com. It is running, I'm just figuring out a bit of stuff along the way.

BTW, regarding this subject in general, I cannot believe how much trouble I had originally figuring something that now looks so simple. Examples I found ommitted simple yet crucial detail. I hope I've managed to explain it in a way that saves someone else the angst.

The sample code definitely does work, I use it in apps.

# re: Using a MemoryStream with GZipStream

Left by David Mellow at 6/2/2008 4:31 PM
Gravatar
Actually, I say very humbly, you are correct Angel. I have always used that third parameter (=True) to keep the underlying stream open, and mistakenly thought its default value was false. So I have said something quite wrong. Your example does work.

Many apologies for going off half cocked. I shall quietly sneak out and say no more. Does a 5 star rating make ammends? Smile.

# David Mellow

Left by Angel Eyes at 6/3/2008 7:32 AM
Gravatar
Not a problem, David. Almost everything I write in this blog is learned the hard way, and usually is already on production servers.

# re: Using a MemoryStream with GZipStream

Left by SMF Destek at 3/26/2009 10:29 PM
Gravatar
Hı :) Thank you post... good :P

SMF Destek

# re: Using a MemoryStream with GZipStream

Left by web development company at 8/18/2009 10:52 AM
Gravatar
Interesting,

it was a well written post, it's a good Idea to post thing that there is a chance that we forget

Anyway, thanks for the post

Your comment:



 (will not be displayed)


 
 
 
 

Live Comment Preview:

 
«November»
SunMonTueWedThuFriSat
25262728293031
1234567
891011121314
15161718192021
22232425262728
293012345