WCF Message Compression - Gzip, 7zip

argot — Tue, 27 Oct 2009 05:20:00 GMT

I was using Gzip Encoder to compress wcf message, it surprised me that sometimes the compression message size is even bigger than the original size, so I looked the code, I found within GZipMessageEncoderFactory.cs, the method "CompressBuffer" in the GZipMessageEncoderFactory class is not quite right. it was like this originally:

private static ArraySegment<byte> CompressBuffer(ArraySegment<byte> buffer, BufferManager bufferManager, int messageOffset)
{
....
var byteArray = new ArraySegment<byte>(bufferedBytes, messageOffset,
bufferedBytes.Length - messageOffset);

return byteArray;
}

if we use the method above, the count of the new ArraySegment<T> will be the bufferedBytes, as BufferManager using TakeBuffer method to grab a buffer which is the nearest bigger 2^N byte, e.g. if input param is 522 bytes for TakeBuffer method, it will grab a buffer which has 1024 bytes. And the constructor of ArraySegment will take the third parameter(bold above) to reflects the actual size of the ArraySegment, if size of the bufferedBytes(first params) is less than third parameter, it will fill byte0 to the rest. that is why sometimes the size is even bigger than the original size.

it should be like this:

return byteArray;
}

Also I found the Gzip compression doesn't have high compression ratio, I decided to use 7zip and gzip combination then, I found a good article about 7zip: 7Zip (LZMA) In-Memory Compression with C#, after adding it to my project, I can use it straight forward:

private static ArraySegment<byte> CompressBuffer(ArraySegment<byte> buffer, BufferManager bufferManager, int messageOffset)
{
using (var memoryStream = new MemoryStream())
{
var zipperStream = new GZipStream(memoryStream, CompressionMode.Compress, true);

  using (zipperStream)
   zipperStream.Write(buffer.Array,
      buffer.Offset, buffer.Count);

  byte[] compressedBytes1 = memoryStream.ToArray();
  byte[] compressedBytes = SevenZip.Compression.LZMA.SevenZipHelper.Compress(compressedBytes1);
  byte[] bufferedBytes = bufferManager.TakeBuffer(
     compressedBytes.Length + messageOffset);

Array.Copy(compressedBytes, 0, bufferedBytes, messageOffset,
compressedBytes.Length);

  bufferManager.ReturnBuffer(buffer.Array);
  var byteArray = new ArraySegment<byte>(bufferedBytes, messageOffset,
     compressedBytes.Length);

return byteArray;
}
}

//Helper method to decompress an array of bytes
private static ArraySegment<byte> DecompressBuffer(ArraySegment<byte> buffer, BufferManager bufferManager)
{
var memoryStream1 = new MemoryStream(buffer.Array, buffer.Offset, buffer.Count - buffer.Offset);
var memoryStream=new MemoryStream(SevenZip.Compression.LZMA.SevenZipHelper.Decompress(memoryStream1.ToArray()));

var decompressedStream = new MemoryStream();
int totalRead = 0;
int blockSize = 1024;
byte[] tempBuffer = bufferManager.TakeBuffer(blockSize);
using (var gzStream = new GZipStream(memoryStream, CompressionMode.Decompress))
{
  while (true)
  {
   int bytesRead = gzStream.Read(tempBuffer, 0, blockSize);
   if (bytesRead == 0)
    break;
   decompressedStream.Write(tempBuffer, 0, bytesRead);
   totalRead += bytesRead;
  }
}
bufferManager.ReturnBuffer(tempBuffer);

byte[] decompressedBytes = decompressedStream.ToArray();
byte[] bufferManagerBuffer = bufferManager.TakeBuffer(decompressedBytes.Length + buffer.Offset);
Array.Copy(buffer.Array, 0, bufferManagerBuffer, 0, buffer.Offset);
Array.Copy(decompressedBytes, 0, bufferManagerBuffer, buffer.Offset, decompressedBytes.Length);

var byteArray = new ArraySegment<byte>(bufferManagerBuffer, buffer.Offset, decompressedBytes.Length);
bufferManager.ReturnBuffer(buffer.Array);

return byteArray;
}

WCF ReliableSession Retry Interval

argot — Tue, 27 Oct 2009 04:45:50 GMT

Been using ReliableSessionBindingElement within WCF, I found we can not set the retry interval, as we only can set MaxRetryCount. WCF uses an internal algorithm to determine when to retransmit, based on a computed average round-trip time. The initial retry time is computed based on the measured roundtrip time of establishing the session. The retransmission algorithm doubles the delay with every attempt, for example, if we set the MaxRetryCount to 8(default), if WCF determines the initial retry time to be 2 seconds, then 8th retry will set the retry ackonwledgement time to be 2^8 which is 512 seconds. We can not change the acknowledgement interval manually. so if you see from Fiddler, WCF Visualizer or Trace Viewer, it is not surprise to see some messages been sent several times if you use ReliableSession. There is no other way to avoid this except making server stateless (disable ReliableSession)