BizTalk

Our first case study on Microsoft patterns & practices!

Chris Han — Fri, 02 Jun 2006 13:49:00 GMT

http://geekswithblogs.net/chrishan/articles/80454.aspx

Although the story only mentions the external apparel we did with ASP.NET 1.1/2.0, the biggest challenge in terms of architecture design we are constantly facing is to balance between the desire of SOA's flexibility/maintainability/reusability and the performance demand by such as a high volume system.

Our client is in fact a typical Application Service Provider in financial circle.(one of those actually survived ASPs.) Their business model has born with a demand of integration. Both integrate with different partners and with a whole spectrum of technologies from legacies to cutting ages, you name it we got it. That’s where the BizTalk gets in the play.

Due to the size of the messages and the processing time restriction, we are trying to use parallel processing as much as possible. The top utilized integration patterns in this project are:

· Scatter-Gather

· Splitter

· Aggregator / Convey

The management patterns implemented are primarily:

· Wire Tap

· Control Bus

I find to implement Wire Tap in pipeline component is very interesting. The challenge here is that we need some information on the incoming message even before it hits the message box, meanwhile you are suppose to process the message here in a streaming manner. A pipeline component is a perfect fit because you don’t have to stop the processing of business messages. All you need to do is to get a copy of the incoming message by pInMsg.BodyPart.Data. From there, it’s all old world you used to.

Run-time Correlation Set

Chris Han — Thu, 01 Jun 2006 11:31:00 GMT

Almost all the time, I'm trying to use correlation to design some sort of “framework“for business process control. BizTalk gives me a big headache not being flexible. Fortunately I found these undocumented techniques in Charles Young's article BizTalk Server 2004: Flexible Correlation .

Populate multiple tables in a parent-child relationship in one roundtrip (Inserting Data by Using XML Updategrams)

Chris Han — Mon, 10 Apr 2006 19:20:00 GMT

The other great post by Stephen Kaufman is this indeep 'Using SQLXML's at-identity with BizTalk'

http://blogs.msdn.com/skaufman/archive/2005/01/31/363920.aspx

To me that's not only explain a way to do this thing, but also some inside information about SQL adapter. So that I know how to cmpare the implementation between using SQL adapter or a custom ADO.NET DAC to insert data to the SQL server.

SQL Adapter, Update Multiple Tables At One Time

Chris Han — Mon, 10 Apr 2006 19:12:00 GMT

You can sure do it by creating a facade stored proc. But Stephen Kaufman gives out the other way by using Updategram. Just like what he said in his blog: http://blogs.msdn.com/skaufman/archive/2005/01/27/361385.aspx

“Just because BizTalk doesn't do this by default doesn't mean that you can't do it or that BizTalk won't be able to handle it.” I guess the wizard guy is the one to blame for misleading us :-)

Throw a customer exception may get an error: May only rethrow in a catch block

Chris Han — Thu, 02 Mar 2006 14:58:00 GMT

I got this error myself when I try to design my own exception handlling strategy in the orchestration.

The BizTalk 2006 document is better. There is a “caution” section says:

Do not select GeneralException in the Throw Exception shape. This type should only be used for rethrowing exceptions in a Catch Exception block.

The error I got is because I'm trying to do a quick and dirty test with a GeneralException.

What should you do? It's simple.

Create a variable.
Give it a .NET type System.Exception. (You should find it in mscorlib assembly)
Assign this variable to your Throw Exception's Exception Object property.
Enjoy it.

How to use the request-response type port with a Fault message defined

Chris Han — Thu, 23 Feb 2006 13:37:00 GMT

I could never find a detailed document talks about this feature. The MSDN implies that you could send the Fault message directly from within the exception block. But no! You will encounter one of these two messages depends on what you did:

must receive before sending a fault message on an implemented port.
must receive before sending a message whose messagetype corresponds to a requestresponse operation on an implemented port.

Thanks for Arch Hacker's article. We have a solution that works for me. The basic idea is to put the send regular response message and send Faul message under a decision shape!

Check out the original here.

http://thearchhacker.blogspot.com/2004/08/multiple-activate-receives-and.html

BTW, the document comes with BizTalk 2006 bete2 remains mysterious. That makes Arch your only “official” instruction writer on this topic.

BizTalk 2006 Flat File Wizard: How to model a complicated flat file schema

Chris Han — Fri, 10 Feb 2006 17:37:00 GMT

I'm recently working on a BizTalk project for a financial industrial ASP. The BizTalk 2006 is used to exchange data with the partners.

As the legacy of 2 decades, all the data being exchanged are in flat file format. BizTalk 2006 Flat File Wizard is very handy when modeling the xml schema off the flat files. In most of the cases, it's just several clicks. The best part of this Flat File Wizard is to give you a visually design experience.

Flat File Wizard is great, it helps a lot in simplifying the trivial jobs. But it's by no mean Harry Potter's magic wand. In fact like all other higher level tools, it trades off flexibility for simplicity. You do have to understand XML and the old dirty way of doing flat file modeling when facing a complex file.

One of the challenges I'm facing is files from Merrill Lynch. It contains a header line at the beginning and a trailer line at the end. Both header and trailer lines has a tag identifier at the beginning of the line. Between header and trailer lines, there are multiple data lines. Each of them belongs to a certain type indicated by a tag identifier at position 8. Header, trailer, and each data line in between are delimited by a line feed symbol “\n”. See the example below:

HEADER RECORD ACI 20050114 200501150459 28B00143

11510N43B115PARIS CEDEX 16 112 AVENUE KLEBER BP 2002 16

11510N430CC FICMA 54

12245N43B115Jersey City 08 201 AVENUE addf45 lm 2006 44

11510N43CPA 75761 FRANCE 33-15-365-5756

14518H31P00661009158106APD 000000000001801SN20020408000000004993064+00000000576900400000000008

22222H31P12121229158106APD 000000000001801SN20020408000000004993064-00000000576900400000000008

14518H31MN28B00145NUVEEN/RITT ASSET MGMT. BAL

22222H31MM19560011Modle X Block K

33333H31MM22222022Modle X Block D

11510N43NN1MR PASPASCAL DETEMMERMAN

67688N43NL1FirstName SecondName

11510N43AL2HEWLETT-PACKARD ROUTE DU NANT-D'AVRIL 150

11510N43RLEER34434-4232434 ROUTE DU NANT-D'AVRIL 150

11510N43X34EEEEEER3333KARD ROUTE DU NANT-D'AVRIL 150

TRAILER RECORD 000011824 000000170

The challenges here are:

The tag identifier for each data line is not at the position 0 but 8.
Each type of data line can appear arbitrary number of time and
in arbitrary order !

You can download the video here, I’ll show you how to model the xml schema for this flat file by using BizTalk 2006 Flat File Wizard.

http://www.urooj.net/archive/ffw in action.zip

How Microsoft's new security strategy SD3 affect BizTalk 2006

Chris Han — Thu, 26 Jan 2006 19:15:00 GMT

A small incident today brought up my thought on this topic. One of my friends is learning BizTalk with 2006 beta 2 following the BizTalk Server 2006 Tutorials in the online document.

While after doing BizTalk Server 2006 Tutorials > Tutorial 2: Purchase Order Process > Lesson 5: Build and Deploy the Projects, he dropped the test message into the file folder and expected the file to be sucked in. But nothing happened in about half hour before he asked me. Then I discovered that unlike BizTalk 2004, the receive location is disabled by default in BizTalk 2006, and it never mentioned in the tutorials.

SD3 is Microsoft's new strategy to satisfy high-end market clients and response to those who making anti-human engineering software but claiming they are much more secure. (how do you like Lotus Notes?) Of course it has to compromise its usability in exchange.

Secure by Design
Secure by Default
Secure in Deployment

This sounds like great idea but I can feel the pains suffered by both Microsoft's tech writer and individual developer like my friend.

The published message could not be routed because no subscribers were found - No double quotation marks in send port filter please!

Chris Han — Tue, 24 Jan 2006 16:32:00 GMT

I was doing a simples subscribtion by using a send port with filter like this BTS.ReceivePortName == "recPort1". I kept getting “The published message could not be routed because no subscribers were found “ error.

For some reason I strongly believed that double quotation marks should be around the filter value. After 2 hours non-productive trying. I did the last pre-bug-report-try with a filter value without double quotation marks, and it worked.

Then I found this note in BizTalk 2006 document: expressions in send ports should not include double quotation marks around the value. This is unlike filters in orchestrations, which require the value to have double quotation marks around it....

There always some one asks me what you do and don't like BizTalk. I can tell something like all people agreed such as debugging or deployment. But now I realize small inconsistant usablility behavior like this is in fact most annoying me. It stops me from applying knowledge I know to what I don't but similar.

Use the right ID for a message: MessageID, WorkID, InterchangeID

Chris Han — Tue, 17 Jan 2006 16:25:00 GMT

This is what I read off BizTalk 2006 document. I feel it's quite helpful but hard to dig. So I copy it here:

BizTalk Server provides a number of different IDs for a message and you should be careful to use the correct ID at the correct time.

A MessageID is specific for an instance of a message in memory, which means that a retry of a message comes with a different MessageID.

A WorkID stays constant across message retries.

An InterchangeID also stays constant across retries but could be shared with other messages in the system. If you need an ID which will stay constant across message retries, do not use the Interchange ID, because it is common to every message that results from a disassembly in the receive pipeline. This ID is used if a target system depends upon duplicate ID detection to enforce once-only delivery.