Evan Koch

Failed Message Routing

Evan Koch — Sat, 05 Apr 2008 02:13:46 GMT

One of the things Microsoft introduced with BizTalk 2006 is failed message routing. Prior to this, developers had to come up with custom solutions to remove failed messages from the messagebox to prevent them from accumulating. Now all one has to do is enable routing for failed messages on the receive ports and create an orchestration to consume these failed messages. When a message comes into the messagebox, BizTalk tries to find a matching subscription based on the context properties of the message. If no matching subscriptions are found, the message is suspended. To demonstrate this we can create a receive location and drop in a XML file. Checking in the Group Hub we find the following:

These messages can be handled if failed message routing is enabled for the receive port that handled the message.

The other half of this solution is to create an orchestration to handle the failed messages. For this example we’ll write the failed messages out to the file system. The receive port is directly bound to the messagebox.

The properties of the message in this orchestration are shown below. Rather than specifying a schema for the message type, declaring a value of XmlDocument makes this orchestration flexible enough to consume all types of failed messages rather than restricting it to a particular schema.

The receive shape in this orchestration has a filter expression to limit it to messages with an ErrorType context property value of FailedMessage. There are a number of context properties that are included for all messages.

With this in place, messages that do not match any subscriptions are consumed by the orchestration and written to disk.

Inbound Maps

Evan Koch — Sat, 05 Apr 2008 01:58:54 GMT

In one of the previous examples that talked about convoys within BizTalk, I had the following orchestration:In one of the previous examples that talked about convoys within BizTalk, I had the following orchestration:

The purpose of this map within the orchestration was to receive a message that followed the Claim schema, transform it into the ClaimValidation schema and set the Status to CONTRACT VALIDATION COMPLETED. It’ll see a bit contrived, but we can move the map so that it’s called on the receive port if required so that we’ll always be dealing with a ClaimValidation message inside of the orchestration.

We’ll also use the map from the previous example which determined the Pass/Fail Validation response based on whether or not the ClaimID is odd.

Within the orchestration we’ll still construct an outbound message and set the Status.

Since the inbound map (which we’ll discuss in a bit) happens before the orchestration runs, the receive port deals only with messages of the ClaimValidation schema.

After the orchestration is deployed, we can specify the inbound map on the physical receive port in BizTalk Administration Console.

With this in place, we can drop messages of the Claim and ClaimValidation schemas into the receive locations. If the message is of the Claim schema, it will first be transformed into the ClaimValidation schema, and if it’s of the ClaimValidation schema it will pass directly through to the orchestration.

\Source code for this example can be found here.

Parallel Convoys

Evan Koch — Sat, 05 Apr 2008 01:36:52 GMT

Taking decoupled orchestrations one step further, we’ll use it to do multiple concurrent tasks and then associate the output from those tasks. We’ll borrow our scenario for this example from the health care industry. When a claim is submitted, there are multiple types of validation that happens (and by no means is this a comprehensive list): is this a valid contract? Is the hospital part of the network? Is the procedure performed covered? We’ll take two of those for our scenario.

With decoupled orchestrations we can create an orchestration for each one of the steps above. In this case, both the Validate Hospital Information and Validate Contract Information receive shapes will have filters that are looking for messages matching the Claim schema with a status of CLAIM RECEIVED.

It’s worth discussing the messaging engine of BizTalk here. When an orchestration is deployed to the BizTalk database, it is necessary to enlist the orchestration and then start it. Enlisting the orchestration defines the subscription for the orchestration and BizTalk can form a queue of messages for the orchestration to consume. When the orchestration is started, it will then begin consuming the messages that are in the queue. When a message is received, BizTalk examines the message’s context properties and sees what subscriptions match them. Each orchestration with a subscription that matches the context properties, so when the ReceiveClaim orchestration writes a message to the messagebox with a Status of CLAIM RECEIVED, the message is placed in the queue for the ValidateHospitalInformation orchestration and in the queue for the ValidateContractInformation orchestration.

Rather than generating the same output for each claim, I’ve made the output for each of the Validation orchestrations dependant on the ClaimID: if the ClaimID is even, the validation will fail; if the ClaimID is odd, the validation will pass. I’m doing this logic in a map within the orchestration (I’ll use this as an example for inbound maps in a later post). The map takes in a message of the Claim and generates a message of the ClaimValidation schema. The validation orchestrations update the Status according to what was performed (CONTRACT VALIDATION COMPLETED for ValidateContractInformation and HOSPITAL VALIDATION COMPLETED for ValidateHospitalInformation) and proceed to send the outbound messages to the messagebox.

The ProcessClaim orchestration requires input from both the ValidateContractInformation and ValidateHospitalInformation orchestrations and needs to ensure that the results it uses are for the same claim. To accomplish this, we define a correlation set based on ValidationClaimID.

This is what ensures that BizTalk consumes both the contract and hospital validation results for claim 123 together rather than the contract validation result from claim 234 and the hospital validation result from claim 345. Correlation sets can only be based on promoted properties, so the ClaimValidation schema has two promoted properties – ValidationClaimID and ValidationStatus. The ValidationStatus is used for the filter condition of the two receive shapes shown below, and also note that both receive shapes reference the ClaimIDCorrelationSet.

Below are the properties for the two receive shapes.

Another benefit of the correlation set we defined is that should one of the validation orchestrations take longer than the other to run, the output from the completed validation orchestration will be held until it can be paired with the output from the other validation orchestration, though this will not affect other instances of the messages or orchestrations. To demonstrate this, I’ve stopped the ValidateContractInformation orchestration but the ValidateHospitalInformation is still running. After submitting a test claim, we can view the messages.

The output from the ValidateHospitalInformation orchestration is waiting to be paired with output from the ValidateContractInformation orchestration. When the ValidateContractInformation is started and the suspended instance is resumed, the ProcessClaim orchestration will consume both messages and output the result to the file system.

Source code for this example can be found here.

Multi-part Messages

Evan Koch — Wed, 31 Oct 2007 14:41:28 GMT

Using multi-part messages is first in Marty Wasznicky’s article 8 Tips and Tricks For Better BizTalk Programming
so I thought I might demonstrate this technique in more depth. My experience with multi-part messages has been when I want to define a single message to pass among loosely coupled orchestrations that contains all the messages associated with the initial loan request. This post also uses code from some of my previous posts, so some of it should look familiar.

Multi-part message types can be defined within BizTalk so that a message can contain parts associated with various schemas. For this example, we’ll use a multi-part message to contain the initial loan request and the mortgage documents request. Below is a diagram of the process.

In the main orchestration that calls each task we’ll create the multi-part message and pass it into the other orchestrations as they are called.

The multi-part message is defined so that one of the message parts follows the LoanRequest schema while the second part follows the PreparedLoan schema. I’ve defined the LoanRequest part as the message body which will come into play in the ArchiveLoan orchestration.

Each part of the multi-part message can be assigned separately within a Message Assignment shape. In this case I’m setting the LoanRequest equal to the initial message and the PreparedLoan equal to a XML variable that contains an unpopulated MortgageDocuments node.

In the Prepare Loan orchestration we’ll pass the loan request through a policy to generate the documents for the mortgage. Since the message is being passed by reference, we have no need for output parameters – the main orchestration’s copy of the multi-part message is being updated within the Prepare Loan orchestration. To determine which mortgage documents are needed, the loan is passed through a rule policy.

The rule policy needs instances of both the LoanRequest and PreparedLoan schema to run, though we’re able to pass in the various parts of the multi-part message to satisfy this requirement.

In the GenerateLoanDocuments orchestration, we only want to send the contents of the PreparedLoan message part to the SendGenerateLoanDocuments port.

For the purposes of associating the Send shape with a message, we’re not able to consider each part of the multi-part message separately, so we create a message of the PreparedLoan type first and assign it to be the same as the multi-part message’s PreparedLoan part.

And finally we have the ArchiveLoan orchestration. In this instance we send the multi-part message to the Send port, but what is actually written to the file system might be different than what one would expect: while the entire multi-part message is sent, only the message body (in this case, the LoanRequest message part) is written to the file system.

Source code for this example can be found here.

Using Debug Statements in BizTalk

Evan Koch — Wed, 17 Oct 2007 13:41:11 GMT

One of the things I’ve found useful in Biztalk development are debug statements. True, you can use the orchestration debugger and view the values of messages as the orchestration runs, but it can be a good bit easier to have the orchestrations write debug statements when they reach certain points. To accomplish this you can put calls to System.Diagnostics.Debug.WriteLine in the Expression and Message Assignment shapes and watch them using DebugView (which can be downloaded here).

For this example I’ve got a main orchestration that receives a loan request and another orchestration that determines if the loan is approved or not based on the credit score. Throughout both orchestrations we’ll write statements to indicate our current tasks.

In the Debug Statement expression shape below I’ve specified the following line of code:

And this is the Approval orchestration:

In this case the debug statements are embedded into the message assignment shapes.

After deploying these orchestrations, opening DebugView and submitting a few loans the following is displayed:

The statements can be as detailed as you wish to make them. The other thing worth pointing out is that calls to the System.Diagnostics.Debug library may be removed during compilation depending on the current compilation statement mode. If you want to be able to control whether or not the debug statements are written via a configuration file setting to debug something in a production environment, you may consider using System.Diagnostics.Trace.WriteLine instead, as these calls will not be removed during compilation.

Source code for this sample can be found here.

Missing Images

Evan Koch — Wed, 10 Oct 2007 11:57:35 GMT

I checked my blog this morning only to find that the images were missing from the last five or so posts I had made. Not quite sure what happened, but I've uploaded the images again. My apologies to anyone who read a post recently and was unable to view the images, but they should be there now.

Using the BRE outside of BizTalk

Evan Koch — Thu, 04 Oct 2007 20:18:27 GMT

Recently I've wanted to demonstrate using the BizTalk Rules Engine outside of BizTalk. In our post 9/11 world, everyone's probably had some contact with the heightened security and restrictions for air travel and banking transactions. Along the same lines, the Nuclear Regulations Commission often updates the rules concerning access control to nuclear power plants within the United States. Knowing the flexibilty of the BRE and the ease with which new rules can be added and deployed, I thought this was a good premise for my example.

For the purposes of this example, one of the assumptions is that a XML is provided to the system containing the information for the employee following the schema below:

Within the code this XML document is loaded into an XMLDocument variable, used to create a TypedXMLDocument variable, and then passed to Access Control policy within the BRE.

So then let’s look at the rules. The first rule to be executed is “Grant Access.” While probably not the best idea when handling real world situations, for the purposes of this example I grant access to the employee in question and then check criteria to determine if that access should be revoked.

The criteria used for this rule is just a way of making the rule execute every time. The BRE uses the RETE algorithm and other things I don’t fully understand but enjoy the benefits of nonetheless to generate an agenda of which rules to try based on the input. In this case I’ve declared a priority of 1 for the Grant Access rule to ensure that it is run before the other rules. Below is the text displayed within the Business Rule Composer for Priority:

So then we evaluate other things. Is the person requesting access currently employed? If not, deny access and add a <AccessDeniedReason> node.

Did the person pass his or her last drug test?

Was the last drug test performed within the last 60 days? Note here that the BRE calls an external function which takes in an integer represents how long drug tests should be considered valid, so if the NRC were to change it to 45 or 30 days, all one would have to do is update the parameter.

We submit the following access request and are denied access.

The following employee is granted access.

By determining whether or not the employee has access within the BRE, we’re afforded a high degree of flexibility when it comes to implementing new rules. If the NRC were to come out with a list of names that should be denied access, the developer could create a rule that would check first and last names against a database table. Source code for this example can be found here.

Tightly Coupled Orchestrations

Evan Koch — Thu, 04 Oct 2007 19:30:52 GMT

Considering the last post concerned loosely coupled orchestrations, I thought it might be useful to demonstrate tightly coupled orchestrations. This method still can promote reuse as orchestrations can be called from multiple sources. We’ll use the same loan process diagram.

And here’s the layout of the master orchestration. In this case we don’t have a Send shape at the end because that’s being handled in the Archive Loan orchestration.

Orchestrations can have a few types of parameters, though I’ve only used message parameters before. Each message parameter is associated with a direction: the parameter can be passed in by value by the calling orchestration, passed out by reference to the calling orchestration, and there’s a ref option to pass in a parameter by reference.

In the PrepareLoan and GenerateLoanDocuments I create a copy of the message and assign a new status value to indicate that something has happened, since I’m not really doing any other work in the orchestration.

As the messages are being passed by reference between one another, there’s no reason for the messages to be written to the Biztalk messagebox, which can be quite a boon to performance.

Source code for this example can be found here.

Loosely Coupled Orchestrations

Evan Koch — Thu, 04 Oct 2007 19:20:31 GMT

At its heart, BizTalk is based on the publish/subscription model. As messages come in to the message box, BizTalk checks to see what orchestrations have subscriptions that match the criteria of the incoming message. This pub/sub model is opens up a new paradigm for developers in terms of programming. For this example we’ll use the following process diagram for lending:

A developer could create one monolithic orchestration to handle all the tasks shown above. The downside of this would be that the entire orchestration would have to be modified if steps were added to the diagram above, nor would we be able to easily facilitate reuse. Another option would be for the developer to create an orchestration for each of the steps above and then another orchestration to call the others and pass information between them. A more interesting option, however, would be to create separate orchestrations and have them activate as needed.

We can accomplish this by routing messages between the orchestrations. Messages are routed based on their context properties, so we’ll add a Status element to our basic LoanRequest schema and make it a promoted property.

After the task is completed in each orchestration, we’ll create a copy of the message and update the status to indicate that the task has been completed. After the new message has been created, it will be passed into the message box using a port that’s directly bound. The following orchestration will be checking the messagebox looking for messages where the status indicates that the preceding orchestration has been completed.

The following orchestration will have a receive port that’s directly bound to the message box. The receive shape that’s connected to this port will have a filter that’s looking for messages where the Status context property indicates that the previous orchestration has completed. For instance, we’ll look at the code in the Message Assignment block of the Prepare Loan orchestration.

Let’s also take a look at the filter expression for the Receive shape in the GenerateLoanDocuments orchestration.

So going back to the terms used at the beginning of this post, the PrepareLoan orchestration publishes a message to the messagebox which matches the subscription for the GenerateLoanDocuments orchestration and activates an instance of the orchestration. Similarly, the GenerateLoanDocuments orchestration publishes a message to the messagebox with a Status value of “LOAN DOCUMENTS GENERATED” that matches the subscription for the ArchiveLoan orchestration and activates an instance to consume the message.

As mentioned earlier, this type of model can more readily accommodate changes to the process diagram. We’ll use the updated process diagram below.

In order to handle this, we’d create an additional orchestration that handled the specifics of the Generate MIN logic and activate it with a receive shape that looks for messages with a Status of “LOAN PREPARED”. After the MIN logic has been performed, it will create a new message with a Status value of “MIN GENERATED”. Additionally, the GenerateLoanDocuments receive port filter would need to be updated so that the filter expression matched the new Status value. The direct bound ports are only logical ports and have no physical representations that can be updated within the BizTalk Administration Console, so this change can only be made at design time within Visual Studio.

This pub/sub model could also be used to promote reuse in orchestrations. Imagine that there was a new requirement that each step within the loan process be logged. To accomplish this, we could create another orchestration called LogProcess with a receive shape that had a filter on Status values of AUDIT. Each orchestration could be updated to create a message for auditing with a Status value of AUDIT and send it to the messagebox before sending the other message indicating that the orchestration had completed its task.

Source code for this example can be found here.

Signing BizTalk Assemblies

Evan Koch — Tue, 02 Oct 2007 11:12:38 GMT

While this isn’t exclusive to BizTalk assemblies, I can’t say I’ve had much need for strongly named assemblies prior to working with BizTalk. When a BizTalk assembly is deployed, the assemblies are installed into the GAC. For an assembly to be installed into the GAC, however, the assembly must be strongly named. It’s easy enough to create a .snk file and open up the project properties and browse to the location of the .snk file, though it’s a recommended practice to specify the assembly key file with a relative path rather than using an absolute path. This practice can help reduce any frustrations on a project with multiple developers who may have varying preferences for source code location, etc. There’s another way to share assembly keys among assemblies that might prove useful at some point.

Using the sn.exe to create assembly keys, I’d imagine most are reasonably familiar with the –k option. One of the other options is –i which will install the specified key into a container.

When modifying the project properties to specify an assembly key, you can specify this key container rather than pointing to the assembly key file.

I’ve found this approach useful when there are multiple projects within one solution that share one assembly key – for instance, one BizTalk solution that had different projects for schemas, orchestrations, utilities, etc. Using relative paths you could store the assembly key file in the solution folder and use relative paths, or you could install the assembly key into a container and specify the name of the container.