Geekswithblogs.net

El Grego

BizTalk blog


News

My Stats

  • Posts - 28
  • Comments - 32
  • Trackbacks - 55

Twitter












Tag Cloud


Recent Comments


Recent Posts


Archives


Post Categories


Monday, August 03, 2009

Question marks in your flatfile output?



Or how to fix a bug while introducing another one that's a lot nastier. Let's start by describing the original bug: the BizTalk flatfile assembler has issues with custom target (output) encodings.

When compiling a custom pipeline with the FF assembler and configuring the 'target charset' in the pipeline designer everything works as expected and you will get your messages in the desired encoding.

When you want to dynamically control the encoding, according to the docs, you should also be able to do this by writing the XMLNORM.TargetCharSet property onto the message context.

In my case the desired output was 'Windows-1252'.

I was able to verify that this technique indeed works using a default XmlTransmit assembler pipeline.
With a custom FF assembling pipeline though I got a question mark in the flatfile output for every special character.
 
Since the only differentiator being the pipeline I am quite confident that it's a FFASM assembler bug.

Now to come to the second problem: apparently this bug was already discovered over a year ago and I guess it was never reported to Microsoft (since it is still not fixed in BTS 2009, the RC I checked). It might be a regression since people reported that it only happens from version 2006 R2.

Instead, the author decided the shortest path to success was to develop a custom encoding pipeline component that takes care of the output encoding. The issue showed up a few more times since then and it looks like other people were inspired by his workaround.

Now take a good look again at how the author implemented the custom encoding. There is something terribly wrong with it.

HINT: the input encoding is UTF-8, a variable byte length encoding. Inside the loop a fixed number of bytes is read from the input. Got it?
  • Share This Post:
  • Share on Twitter
  • Share on Facebook
  • Share on Technorati