Geeks With Blogs
ErwinAI - Enterprise Application Integration (EAI) blog on BizTalk 2004, Visual Studio, and application integration

An existing orchestration, that reads Siebel-messages from an MSMQC-queue, maps them, and posts them to a remote http-server had to be upgraded. Together with the upgrade, the performance had to be reconsidered, as the orchestration would be dealing with an increased number of messages.

We did some stress tests and found out that it was easy to get the remote http server into trouble. For a test, we dropped ~20 or more message files in a directory on the file system. This directory is read by a test orchestration that posts the messages in an MSMQC-queue. As of that moment, the upgraded orchestration receives the messages out of the queue, processes them and posts them. Because we drop many files in a directory, and because BizTalk launches separate threads for each message, all messages are processed in parallel threads. It appears as if they are being processed in parallel.

At the end of the orchestration instance's life, the target message is posted to the remote http-server. An interesting thing happens here: on the socket level, the http-adapater only can have one connection (per from-address:from-port to-addres:to-port used) active at the same time. Thanks to http and the underlying network protocols, the transfer of multiple messages to the other side is “multiplexed” in a single http-connection. This was clearly too much for the remote http-server, when we tried ~20 messages.

Because the capacity of the remote http-server could not be extended easily (license issues), we decided to solve it on our side. The idea was to change our orchestration into a “single-instance orchestration”. This is an orchestration (subscribing to the same message type as usual) that can only have one active instance at the same time. This achieved by:

  • Adding an additional receive shape. Based on the “activating“ receive shape at the beginning of the orchestration, a “non-activating“ receive shape is added, that receives the same message, and is connected to the same port. The new receive shape is added as the last step in the orchestration.
  • Because we have two receive shapes for a single message, we need to add a correlation set. The first (original receive shape initializes the correlation set, the second (added) one follows the same correlation set.
  • As correlation type, we take a trivial (i.e. constant value) element out of the message type, that we promoted.
  • A loop shape is added to the orchestration. The argument is set to true, meaning that it is an endless loop. All shapes in the orchestration, except the first receive shape, are moved to the inside of the loop.

This is what happens:

  • After the orchestration has been started, the BizTalk messaging engine waits for the messages it subscribed to, as usual.
  • A new message fires up a new instance of the orchestration. The message is processed as usual.
  • Due to the newly introduced receive and loop shapes, the orchestration instance does not end after processing the first message. Instead, it will let the second receive shape wait for a new message that has the same (trivial) promoted property value.
  • When a next message comes in, the loop shape takes care that the process is run again.

Please note the following:

  • When, for some reason, the started orchestration instance is terminated, a new instance will automatically be started once a new message comes in. The correlation mechanism will again take care that only one instance is active at the same time.
  • This mechanism is a form of serialization, i.e. instead of a multithreaded, parallel form of processing, the messages are processed one at a time. This also affects the behaviour of the whole process when the orchestration “hangs“. No more messages will be processed.
  • Even though there is serialization, no assumptions can be made about the exact sequence of the messages being processed. For that, you would require an “ordered delivery“ mechanism implemented. This is usually done with MSMQT.
  • A delay-shape somewhere whithin the loop-shape will assure that in between delivery of two consecutive messages to the outbound port, there is a desginated, minimal time interval.
  • With multiple destinations (like three different url's to post to, depending on three different customer numbers), you could use the customer number in the correlation type. This will lead to each customer number having its own orchestration instance. This means that per customer number (and per url) the messages are processed one-by-one. Still, the three orchestration instances operate in parallel, and are not blocking each other's progress.
  • When you consider applying this “serialization“ method to an orchestration that already applies correlation sets, some complications might arise. BizTalk will not allow you (design time) to enclose the receive shapes in a loop shape. The reason is that a receive shape that initializes a correlation set is only allowed to be executed once (per orchestration instance). Because the loop shape will change the orchestration instantiation from “once per message“ to “single instance“, you have an issue here.
  • To avoid this potential problem, you can decide to leave your original orchestration unchanged (“one instance per message“). The outbound messages should be written to some sort of a queue. This can be an msmq-queue, or a folder on the file system. From that queue, a dedicated serialization orchestration reads the messages and sends them out one-by-one. Example: see below.


Posted on Friday, February 17, 2006 1:52 PM BizTalk 2004 Hints, Tips & Tricks | Back to top

Comments on this post: BizTalk 2004 - Serialization of send actions

No comments posted yet.
Your comment:
 (will show your gravatar)

Copyright © Erwin Homan | Powered by: