- Architecture

BizTalk: Ordered Delivery

Leonid Ganeline — Wed, 11 Jan 2012 22:34:36 GMT

It is one more description of the Ordered Delivery (OD) in BizTalk.

The main article about it is in MSDN.

Here I am discussing the BizTalk Ordered Delivery “implementation details”.

OD Considerations

Ordered Delivery (sequential) mode is opposite of the “Parallel Delivery” mode. Parallel Delivery is the most productive mode; the Ordered Delivery is less productive mode.
Transports such MSMQ and protocols, supporting the WS-ReliableMessaging, are the protocols supporting OD. Other protocols as FTP, File, SQL, SMTP etc. do not have notion of the “order”.
BizTalk application usually is a part of the whole message workflow.
There are two approaches in the OD implementation:
- All steps of the message workflow independently support OD.
- A Destination System implements the re-sequencing and manages lost and duplicate messages.
Order is relative. Sequences can be ordered regards one or several parameters. For example, OD for the Company or for the Company + Department.

OD and the BizTalk Architecture

MessageBox is an implementation of the Message Queue. OD is an intrinsic feature of the MessageBox.
The BizTalk Server works in the Parallel Delivery mode by default.
There are three parts in the BizTalk message exchange outside of the MessageBox in relation to OD: Receive Locations; Orchestrations; Send ports.
- Receive Locations support OD on the Transport level (say, MSMQ and some WCF adapters).
- OD in Orchestrations is implemented by the sequential convoy pattern.
- Send ports support OD for all static adapters.
The BizTalk Pipelines (part of Receive and Send Ports) always process messages in order using streams.

OD and Ports

To force Send Ports work in order we set up a flag the “Ordered Delivery” in Send Ports, then the BizTalk MessageBox takes care of implementing OD.

To force Receive Locations work in order we set up flag the “Ordered Delivery” option in the Receive Location Transports, whenever is possible. Then the BizTalk Transport Adapter takes care of implementing OD.

Ordered Delivery Send Port instance works as a singleton service. Since start it stays in Running state. It will not recycle if we restart its Host Instance. We could manually terminate it, if we want.

OD and Orchestrations

MessageBox implements the convoy message exchange pattern [See Using Correlations in Orchestrations]. See the detail convoy description in the BizTalk Server 2004 Convoy Deep Dive article.
It is not just a design pattern that developer should follow. There are special mechanisms inside MessageBox responsible for implementing OD.

OD and Orchestration: Sample

Imagine four Orchestrations which implement four approaches to the sequencing.

The first is the ProcessNoOrder Orchestration. It processes all messages without any order. One ProcessNoOrder Orchestration instance will be created for each inbound message.

The ProcessInOrder Orchestration processes all messages in one sequence. Only one ProcessInOrder Orchestration instance will be created.

The ProcessInOrderByCompany Orchestration processes messages in sequences correlated by the Company value (A, B, C, D, etc.). The separate queue is created for each new value of the Company. Messages inside queues are processed in order. Queues for different Companies are independent. A separate ProcessInOrderByCompany Orchestration instance will be created for each new Company value.

The ProcessInOrderByProduct Orchestration works exactly as the ProcessInOrderByCompany Orchestration but correlated by the Product value (xx, yy, etc.).

Discussions

Performance

By default all Orchestration and Messaging Service instances works in the Parallel Delivery mode with maximum performance.

If we check up the Ordered Delivery option in Send Port, BizTalk will initiate the Send Port instance as a singleton service. It is always a single instance. We don’t have the flexibility of the Orchestration where we could tune up “the proportion of the sequencing” and could control the recycling of the service instance.

Send Port OD could be in two states, on and off:

OD is off: a service instance per message, one message per queue, maximum performance.
OD is on: one infinite service instance, all messages in one queue, minimum performance.

Orchestration OD could be in two states also, on and off:

OD is off: a service instance per one activating message, one activating message per queue, maximum performance.
OD is on: one or several service instances, one per new correlation set value; all correlated messages per queue; performance is from min to max, depends ot the correlation set..

Carefully designing the correlation sets we could tune up the performance of the Orchestration. For example, if we only care of the document order for each separate Company, we include the Company to the Correlation set. If we had thousand documents related to hundreds companies, the performance will be near maximum. If there are only two companies, the performance will be near minimum, and we should consider improving the correlation with one more parameter.

Orchestrations and Zombies

Zombies are big problem of Convoy Orchestrations. See BizTalk: Instance Subscription and Convoys: Details article with description of this problem. This problem could be mitigated but could not be completely removed. We are waiting a new version of the BizTalk Server where this issue will be targeted.

BizTalk Server version Next, Ordered Delivery and Zombies

It is possible the BizTalk Server version Next will implement the automatic Ordered Delivery for Orchestrations, with pattern similar to the Ordered Delivery in Send Ports.

Three new Orchestration parameters are shown up there: Ordered Delivery, Stop on Exception, and Recycle Interval (in seconds).

Ordered Delivery parameter works in the same way as the Ordered Delivery parameter of the Send Port. Now we don’t have to build the Convoy Orchestration manually. No more Loop inside Orchestration.

If the Ordered Delivery parameter set up to True, the Orchestration is working as a Singleton. The first Receive shape receives all correlated messages in sequence. Correlation set is created implicitly regards of the Activation Subscription expression.

There are several limitations for this kind of Orchestration. The most important is: only one Receive shape is permitted here.

There are two big advantages of this new feature:

Simplified Orchestration design for the Ordered Delivery.
No more Zombies. The Orchestration instance is recycled in controllable way, when no messages, matched the Orchestration Subscription, are placed in the MessageBox.

Conclusion

We discussed the Ordered Delivery implemented in the BizTalk Server and ways to improve it.

BizTalk: Advanced Questions

Leonid Ganeline — Tue, 10 Jan 2012 17:12:21 GMT

This article is a Part 6 of the Interview Questions series.

Part 1: BizTalk: Questions for interview without answers
Part 2: BizTalk: Interview questions and principles
Part 3: WCF: Questions for studying and interview
Part 4: WCF: Questions for studying and interview: Discovery
Part 5: WCF: Questions for studying and interview: Routing Service
Part 6: BizTalk: Advanced Questions

It is time for fun!

These questions are for an Advanced level of the BizTalk Developers. I hope you don’t have a single chance to answer these questions, if you don’t have a real life experience in BizTalk Server projects.

I would appreciate, if you write the answers in comments to this article. Thanks!
I will mark the questions which got a right/good answer.

If you have more advanced questions, please, comment them down and I will add them to this list under your name.

Advanced level

· The error in BizTalk usually creates a suspended message. But routing error creates two messages. Why so? How we could utilize this feature?

· Send ports have an “Ordered delivery” option. How it works for a File adapter, if the File protocol does not have a notion of file order?

· Send ports have an “Ordered delivery” option but Receive ports do not have it. Why so?

· You set up an “Ordered delivery” option in Send Port. What is special about Service Instance of this Port?

· How to force an Orchestration do the ordered delivery? Is there a special “Ordered delivery” option in Orchestration? If it is not, why?

· Send ports have the “Priority” options, but Receive ports and Receive Locations do not have them. Why so?

· How does the Priority feature of the Send Port work?

· Why the Priority option can be set up only for the Send Port?

· Send ports have the “Retry” options, but Receive ports and Receive Locations do not have them. Why so?

· How does the Retry feature of the Send Port work? In what scenarios this feature is helpful, in

· Send ports have a “Backup Transport” but Receive ports do not have it. Why so?

· Send ports have a “Filter” but Receive ports do not have it. Why so?

· Send ports have the “Maps” and “Transport”; Receive ports have “Maps” but “Transport” is associated with Receive Locations. Why so?

· What is the difference between “Send Port Group – Send Port” relation and “Receive Port – Receive Location” relation?

· Receive Location have the Schedule parameters for date and time but Send Ports have the Schedule parameters only time. Why so?

· Filter expression for the Send Port is defined at run-time, but Filter expression for the Orchestrations is defined only at design-time. Why so?

· We created a custom pipeline component. Do we have to place it into the “<BizTalkFolder>\Pipeline Component” or install it to GAC or both?

· How does an Orchestration make sure the message was sent successfully by one-way Send port? How does it changed for a two-way Send port?

· What is it the “CodeList Database” property of the Schema? How to use it?

· What is the “Property Schema Base” property of the Property schema element? It has the values: MessageDataProperyBase, MessageContextPropertyBase, and PartContextPropertyBase. What is the difference between using MessageDataProperyBase and MessageContextPropertyBase?

· We bind the Orchestration with Send Port. What does it do regards of the subscription?

· We bind the Orchestration with Receive Port. What does it do regards of the subscription?

· How do Filters of the Send Ports and Send Port Groups work together? Say, a Send Port belongs to a Send Port Group. If this Send Port and this Send Port Group have the same Filter expression, does it create two subscriptions and, as a result, two messages are sent to the Send?

· Why do we need a special Property schema for Promoted properties? What is the purpose of this schema? Distinguished properties do not use this kind of schema. Why so?

· Properties of some BizTalk artifacts have the Tracking option. Right-click menu of some BizTalk artifacts have the “Tracking…” command. Why there is such a difference between artifacts? Why Maps do not have any Tracking options? Why Schemas have only Tracking property? Why Send Port Groups do not have any Tracking options? Why Receive Locations do not have any Tracking options? Why BizTalk Applications do not have any Tracking options?

· When the BizTalk Assembly is deployed, is it stored in the BizTalk database or it is just referenced in the BizTalk database?

· Can you describe at least three scenarios where smart Orchestration Engine finds the errors in the messages patterns at design time (while build the BizTalk project)?

· Host has an "Authentication Trusted" option. What does it mean? What implication it has to the message processing? How do we have to use it / don't use?

· Could you create an XLANGMessage in the C#/VB code? Please, provide code.

· You can use an XmlDocument class as a Message type for receive untyped Xml messages in Orchestration. What .NET classes we have to use for receive untyped text or binary messages in Orchestration. Why so?

· Is the Publish-Subscribe BizTalk Engine API well defined? Where? If not, why so?

· Does BizTalk use the strategy “First received, first processed” or “First received, last processed” of “First received, first processed, first sent” of something else? Why so?
Describe scenarios, when this strategy works better, and scenarios, when this strategy works badly.

· Are Orchestration Instances processed in order? Why so? Describe scenarios.

· Is it possible to force Hydration / Dehydration of the Orchestration instances, Messaging instances? How to do this? Why do this?

· In what scenarios you will use the Compensations in Orchestrations? Why so?

· Atomic scopes in Orchestrations have a Retry feature. How it works? Describe scenarios when it is used.

· What serializer does BizTalk use, XmlSerializer or DataContractSerializer? What implication it has for the real life scenarios?

· What transform engine is BizTalk using, XslTransform or XslCompiledTransform? What implication it has for the real life scenarios?

· BizTalk Business Rule Engine uses a Rete engine inside. In what scenarios this Rete engine is the more efficient / less efficient?

· Let’s classify the applications as Front-end, Middle-level and Back-end. Where are the best places to use BizTalk? Please, provide scenarios. What are the most important features we have to take in account?

· What BizTalk features are the differentiators, when you compare BizTalk Server with other alternatives? What are those alternatives?

BizTalk: Samples: Error Handling, Email Notification

Leonid Ganeline — Fri, 06 Jan 2012 17:49:21 GMT

One problem with email notifications is it usually sends too many emails. Errors frequently happen in series. For example, a temporarily issue with network can result in hundreds repetitive emails. In result an email notification designed as a “VIP” notification is downgraded to the “annoying” status.

This email flooding could be mitigated with this solution.

A Sample demonstrates using the emails for sending notification in case of failed messages.

It compounds several similar errors in one email notification.

Recently I've uploaded this sample

BizTalk: Samples: ErrorHandling: Notification emails

to MSDN Gallery.

It has all solution artifacts and can be used in real-life projects.

BizTalk: Sample: Error Handling

Leonid Ganeline — Sun, 01 Jan 2012 19:57:03 GMT

Recently I uploaded a "BizTalk: Sample: Error Handling" in MSDN Gallery

This sample shows the BizTalk features:

Routing Failed Messages in ports
Catching Failed Messages with Send Port
Catching Failed Messages with Orchestration
Handling Exceptions inside Orchestration
Handling SOAP Fault messages inside Orchestration

BizTalk Orchestration Naming Conventions

Leonid Ganeline — Wed, 28 Dec 2011 20:38:39 GMT

This is a second part of the BizTalk Naming Convention.

Part 1: biztalk: BizTalk solution naming convention

Special Orchestration Objects

<RoleLink> =:
roleLink_ + <Name>

Note: These objects are special BizTalk objects. They are used in different language context and sometime they use different language syntax. Prefixes help to differentiate the objects in the XLang expressions.

where

	Send	Receive	Send-Receive (Solicit-Response)	Receive-Send (Request- Response)
prefix	S_+	R_+	SR_+	RS_+

Notes:

The Port objects are the only objects which can be seen outside orchestration. We see them while bind orchestration with Ports, not the orchestration Ports but Ports created in the BizTalk Administration Console. Sometimes the orchestration Ports are named as Logical Ports and the Ports as the Physical Ports. Here are some considerations about this ambiguity with Port names.
Send-receive prefixes help while Orchestration is binding with Ports. Ports with different Communication pattern are using different prefixes.

Example: S_ OrderAck.

Orchestration Object Types

<ArtifactType> =:
<ArtifactName> + “_type”

Note: We can see orchestration types in the Orchestration View window in Types. They are: Port, Multi-part Message, Correlation and Role Link Types. We can use one suffix the “_type” for all different types because different types are seen only in the different lists and never mixed. For instance, we can never see the port types together with message types.

Controversial:Suffixes for types work better than prefixes, because types are mostly used in the drop-down lists not in the XLang expressions.

Orchestration Workflow Shapes

Problems with orchestration shapes:

Shapes are too small to display long names (only 12-18 characters are visible).
We have to hover a mouse over a shape or click shape to see Properties window and to "understand" this shape, to understand which message is processed.

Useful features:

Shapes have names, but names are not the “variable names”, not identifiers. They are descriptions (excluding the Port shapes names, which have not Name parameter but Identifier and Description parameters). Shape names are used only for description not as XLang variable identifiers. Shape names can be long and can include any symbols as spaces, dots, etc.
Icons of shapes give us the useful information. Do not repeat the “icon information” by words. For example, we can a name a Construction shape as “Construct OrderAck” or as “OrderAck”. Last variant give us more clear and short definition because we have the Construction icon + name.
Shape names are used mainly in Orchestration Editor (excluding the Port shapes names). We don’t need the “well-sorted” names, so we don’t need to use prefixes, because the main purpose of the prefixes is creating well-sorted lists.
Group shape is a scalable shape. Nesting other shapes to a Group shape can make a long description visible. Group shape will display as much text as you want. Group shapes add a lot of documentation value to the orchestration.

Rules for shapes

Whenever it is possible use the short MessageType as a shape name.
Do not repeat a type of shape icon by word.
Feel free to use spaces and any other symbols inside the shape names.
Feel free to repeat the shape names.

Note: Purpose of the orchestration and the most of the shapes is in processing the messages. We can unambiguously describe the messages by the message type. A message type gives us the main information about a message. That is why in the most cases using the message type as the shape name gives us the main information about message flow, about the whole orchestration processing.
Example: Send shape with name "OrderAck" means “send OrderAck message”.

Controversial:

Orchestration Naming Convetion works not to make easier the administration and troubleshooting but mostly to make orchestration more readable. Most orchestration names are not visible outside the Orchestration Editor. So, do not force the orchestration naming convention to hard! If your developers use differen conventions, it is fine; of course, if they understand each other. Forcing convention can do more harm than help. It can force team to spend more effort for a little.
When exception is thrown from a shape, the error description includes a name of the shape. It is a rare case when names are exposed outside of the Orchestration Editor. When we use MessageType as a name of shapes, many shapes can get the same name. So, if we want to differentiate shape names for debugging purpose, we could use numbers or single letters in the end of names.
Example: “OrderAck 2”

BizTalk: Using context for routing

Leonid Ganeline — Mon, 28 Feb 2011 00:26:18 GMT

[See Sample: Context routing and throttling with orchestration]

Imagine the project where most of the routing happens between orchestrations. I.e. routing is mostly between the MessageBox and orchestration with direct endpoints.

Imagine also the most of the messages are with the same Message type.

Usually in this case messages got the special node only for the routing. For example, the field can be the “Originator” or “Recipient” or “From” or “To”.

What wrong is with this approach, it creates the dependency between the message and the message processing. Message “knows” something about Originator or Recipient.

So what we can do with it? How can we “colorize” the same message to route it to the different places without changing the message itself?

One of the decisions is to use the message context.

BizTalk uses the promoted properties for routing. There are two kinds of the properties: the content properties and the context properties. The content property extracts its value from inside the message, it is a value of the element or attribute. [See MSDN] The context property gets its value from the message environment. It can be the port name that receive this message, it can be the message Id, created by the BizTalk. Context properties look like the headers in the SOAP message. Actually they are not the headers but behave like headers. The context properties are the good match for our case.

First, we don’t have to change the message itself to set or change the routing property. The context is stored outside the message body.

Second, we don’t have to create the property schema to use the context properties. [See MSDN: How to create Property schema] BizTalk has the predefined schema set for the context properties. [See MSDN: Message Context Properties] Use one of them and that's it.

The main purpose of the context properties is working on behalf of the BizTalk internals. But we can read, create and change them. Just do not interfere with BizTalk internals on this way.

BizTalk: Sample: Context routing and Throttling with orchestration

Leonid Ganeline — Sun, 27 Feb 2011 23:04:53 GMT

[See BizTalk: Using context for routing]

The sample demonstrates using orchestration for throttling and using context routing.

Usually throttling is implemented on the host level (in BizTalk 2010 we can also use the host instance level throttling). This sample demonstrates the throttling with orchestration convoy, that slows down a message flow from some customers. Sample implements sort of quality service agreement layer for different kind of customers.

The sample demonstrates the context routing between orchestrations. It has several advantages over the content routing. For example, we don’t have to create the property schema and promote properties on the schemas; we don’t have to change the message content to change routing.

Use case:

The BizTalk application has a main processing orchestration that process all input messages. The application usually works as an OLTP application. Input messages came in random order without peaks, typical scenario for the on-line users.

But sometimes the big data batch payloads come. These batches overload processing orchestrations. All processes, activated by on-line users after the payload, come to the same queue and are processed only after the payload. Result is on-line users can see significant delay in processing. It can be minutes or hours, depending of the batch size.

Requirements:

On-line user’s processing should work without delays. Big batches cannot disturb on-line users. There should be higher priority for the on-line users and the lower priority for the batches.

Design:

Decision is to divide the message flow in two branches, one for on-line users and second for batches. Branch with batches provides messages to the processing line with low priority, and the on-line user’s branch – with high priority.

All messages are provided by one hi-speed receive port.

BTS.ReceivePortName context property is used for routing. The Router orchestration separates messages sent from on-line users and from the batch messages.

But the Router does not use the BizTalk provided value of this property, the Router set up this value by itself. Router uses the content of the messages to decide if it is from on-line users or from batches.

The message context property the BTS.ReceivePortName is changed respectively, its value works as a recipient address, as the “To” address for the next recipient orchestrations. Ideally we should use two receive ports or two receive locations and BizTalk set up the context property for us. If we use the same context for both messages (the same port and location), we need to use orchestration to modify the context property using content of the messages or something else.

Those next orchestrations are the BatchBottleneck and the MainProcess orchestrations.

Messages with context equal “ToBatch” are filtered up by the BatchBottleneck orchestration. It is a unified convoy orchestration and it throttles the message flow, delaying the message delivery to the MainProcess orchestration.

The BatchBottleneck orchestration changes the message context to the “ToProcess” and sends messages one after another with small delay in between. Delay can be configured in the BizTalk config file as:

Of course, messages with context equal “ToProcess” are filtered up by the MainProcess orchestration.

NOTES:

Filters with string values: In Orchestrations (the first Receive shape in orchestration) use string values WITH quotes; in Send Ports use string values WITHOUT quotes.
Filters on the Send Ports are dynamic; we can change them in run-time. Filters on the Orchestrations are static; we can change them only in design-time.
To check the existence of the promoted property inside orchestration use the Expression shape with construction like this:
if (BTS.ReceivePortName exists myMessage) { …; }
It is not possible in the Message Assignment shape because using the “if” statement inside Message Assignment is prohibited.
Several predefined context properties can behave in specific way. Say MessageTracking.OriginatingMessage or XMLNORM.DocumentSpecName, they are required some internal rules should be applied to the format or usage of this properties. MessageTracking.* parameters require you have to use tracking and you can get unexpected run-time errors in some cases. My recommendation is - use very limited set of the predefined context properties.
To “attach” the new promoted property to the message, we have to use correlation. The correlation type should include this property. [Here is a good explanation by Saravana ]
Say we have two correlation types, each with one parameter inside. If we want to correlate to both parameters simultaneously, we cannot use both correlation types, we have to create the correlation type with both parameters and use this type. Otherwise only one correlation type would be chosen for correlation, that means only parameter would be used for correlation.

[The sample source code is here].

BizTalk: Suspend shape and Convoy

Leonid Ganeline — Thu, 10 Feb 2011 06:06:53 GMT

Part 1: BizTalk: Instance Subscription and Convoys: Details

This is a Part 2.

I am discussing the Suspend shape together with Convoys and going to show that using them together is undesirable.

In previous article we investigated the Instance Subscriptions and how they could create situation with dangerous zones in processing.

Let' start with Suspend shape. [See the BizTalk Help]

"You can use the Suspend shape to make an orchestration instance stop running until an administrator explicitly intervenes, perhaps to reflect an error condition that requires attention beyond the scope of the orchestration. All of the state information for the orchestration instance is saved, and will be reinstated when the administrator resumes the orchestration instance.

When an orchestration instance is suspended, an error is raised. You can specify a message string to accompany the error to help the administrator diagnose the situation."

On the Suspend shape the orchestration is stopped in the Suspended (Resumable) state. Next we have two choices, one is to resume and the second is to terminate the orchestration.

Is the orchestration is stopped or unenlisted? You don't find a note about it anywhere. The fact is the Orchestration is stopped and still enlisted. It is very important.

So again, the suspended orchestration can be resumed or terminated. The moment when the operator or the operation script resumes or terminates can be far away. It is also important.

Let's go back to the case from previous article. Make sure you notice the convoy and the dangerous zone after the last Receive shape.

Now we have a Suspend shape inside the orchestration.

The first orchestration instance is suspended. Next messages start new orchestration instance and have been consumed by this orchestration, right? Wrong!

The orchestration is stopped on the Suspend shape but still enlisted. Now the dangerous zone, the "zombie zone" is expanded to the interval between the last receive and the moment of termination or end of the orchestration. The new orchestration instance for this convoy will not start till this moment. How fast operator finds out this suspended orchestration? Maybe hours or days. All this time orchestration is still enlisted and gathering the convoy messages. We can resume the orchestration but we cannot resume these messages together with orchestration.

Seems the name Suspended of the orchestration is misleading. The orchestration can be in the Started (and Enlisted)/Stopped (and Enlisted)/Unenlisted state. The Suspend shape switches orchestration exactly to the Stopped state. The Stop name would describe the shape clearly and unambiguously and the Stopped state would describe the orchestration.

Imagine we can change the BizTalk.

The Orchestration editor can search these situations and returns the compile error. In similar case the Orchestration Editor forces us to use only ordered delivery port with convoys.
The run-time core can force the orchestration with convoy be suspended in Unresumable state, that means the run-time unenlists the orchestration instance subscriptions.
The Suspend shape name should be changed. The "Suspend" name is misleading. The "Stop" name is clear and unambiguous. The same for the orchestration state, it should be “Stopped” not “Suspended (Resumable)”.

Conclusion:

It is not recommended using a Suspend shape together with the convoy orchestrations.

BizTalk: Instance Subscription and Convoys: Details

Leonid Ganeline — Sun, 06 Feb 2011 01:10:28 GMT

This is Part 1.

Part 2. BizTalk: Suspend shape and Convoy

Convoys are frequently used in the in the orchestrations in BizTalk Server. Sometimes we have got contra intuitive behavior in the convoys, when messages and orchestrations get suspended in unpredictable manner. This issue is well-known, the suspended messages get name the "zombie". The name is unofficial, but issue is still there. Here I would describe in details, what, when and why this zombie situations are happen.

An orchestration can be enlisted with many subscriptions. In other word it can have several Receive shapes. Usually the first Receive uses the Activation subscription but other Receives create the Instance subscriptions. [See “Publish and Subscribe Architecture” in MSDN]

Here is a sample process.

This orchestration has two receives. It is a typical Sequential Convoy. [See "BizTalk Server 2004 Convoy Deep Dive" in MSDN by Stephen W. Thomas].

Let's experiment started.

There are three possible scenarios depended of the message sequences.

First scenario: everything is OK

Activation subscription for the Sample message is created when the orchestration the SampleProcess is enlisted.

The Instance subscription is created only when the SampleProcess orchestration instance starts and this subscription is removed when the orchestration instance ends.

So far so good, the Message_2 is delivered exactly in this time interval and consumed.

Second scenario: no consumers

Three Sample_2 messages are delivered. The first one is delivered before the SampleProcess starts and before the instance subscription is created. Second message is delivered in the correct time interval. The third one is delivered after the SampleProcess orchestration ended and the instance subscription was removed.

Note:

· It is not the first Sample_2 consumed. It was first in the queue but it was not waiting, it was suspended when it had been delivered to the Message Box and didn’t have any subscribers at this moment.

The first and the last Sample_2 messages are Suspended (Nonresumable) in the Message Box. For each of them two (!) service instances have created. One service instance has the ServiceClass of Messaging, and its Error Description is:

The second service instance has the ServiceClass of RoutingFailureReport, and its Error Description is:

Third scenario: something goes wrong

Two Sample_2 messages are delivered. Both are delivered in the same interval, while the SampleProcess orchestration is working and the instance subscription has created.

First Sample_2 is consumed. The second Sample_2 has the subscription, but the subscriber, the SampleProcess orchestration, will not consume it. After the SampleProcess orchestration is ended (And only after! I will discuss this in the next article.), it is suspended (Nonresumable). Now only one service instance is suspended. This service instance has the ServiceClass of Orchestration, and its Error Description is:

In the Message tab the Sample_2 message is in the Suspended (Resumable) status.

Notes:

The orchestration consumes the extra message(s) and gets suspended together with these extra messages. These messages are not consumed in term of “processed by orchestration”. But they are consumed in term of the “delivered to the subscriber”. The receive shape in the orchestration does not receive these extra messages. But these messages are routed to the orchestration. The Error information looks ambiguous.
The time zone between the last receive shape and the end of the orchestration is a "dangerous zone". The message delivery pattern should be scrutinized to avoid it.

Unified Sequential convoy

Now one more scenario.

It is a unified sequential convoy. The activation subscription is for the same message type as it for the instance subscription. The Sample_2 message now is the Sample message. For simplicity the SampleProcess orchestration consumes only two Sample messages. Usually the orchestration consumes a lot of messages inside a loop in this scenario, but now there are only two of them.

First message starts the orchestration, the second message goes inside this orchestration. Then the next pair of messages follows, and so on.

But if the input messages follow in shorter intervals we have got the problem.

We lose messages in unpredictable manner.

Conclusion:

Maybe the better behavior would be if the orchestration removes the instance subscription after the message is consumed, not in the end on the orchestration. This behavior looks like a bug. But right now it is a “feature” of the BizTalk subscription mechanism.
The time zone between the last receive shape and the end of the orchestration is a "dangerous zone". The message delivery pattern should be scrutilized to avoid this zone as much as possible.

Note [2011-2-9]:

I saw several times the explanation of the zombies, where the zombie can be created in the time zone between the moment an orchestration is scheduled to dispose and the moment the orchestration is disposed. I.e. the average dangerous time zone is about half of the MessageBox polling interval (by default it is 1 sec), that means 0.5 sec. It is not correct. The dangerous zone is between the last receive (for the message with the instance subscription) and end of orchestration, and this zone could be much bigger than a second.

See more about zombies in BizTalk from the BizTalk Core Engine Blog

SQL Server Management Studio: Dynamic Folders

Leonid Ganeline — Wed, 26 Jan 2011 04:51:30 GMT

One of the problem with SQL Server Management Studio is it shows the objects in one list. For example, see the list of stored procedures for the BizTalkMsgBoxDb database, one of the BizTalk database.

It is a huge list. Smart developers are trying to make this list useful and searchable and use suffixes and prefixes to group the objects together.

See prefixes “adm_” and “bts_”. Following this practice we have got the nested compound prefixes. See the subgroups after the “bts_” prefix.

It is completely and absolutely awful view. We have to scroll down and up this list to find out the right object.

The same lists are for others SQL objects.

But it can be much better with small change.

Let’s create the virtual dynamic folders.

Now the list is short, easy searchable and easy observable. Moreover, the long awful names are folded to something readable. Yes, now the names are readable.

All we can do to this beauty is to set up the parameters for these virtual dynamic folders.

Right-click the “Stored Procedures” folder and choose Properties / Dynamic Folders…

There are only few parameters to set up:

· Objects to fold, more than X (auto by default)

· Characters to fold, more than X (auto by default

· Show full name (unchecked by default)

Folders are smart enough to fold with good taste and intellect. But sometimes it helps to fold objects with unusual name conventions.