Here is a list of the more meaningful tools and extensions which I have downloaded and enjoyed this year:

• Google Search Bar + Bookmarks
• XML Notepad 2007 (Microsoft, free)
• Tail 4.2.12
• NUnit for Windows
• WebServiceStudio/SoapBits
• BizTalk ScheduleAdapter  (Scheduled Task Adapter) http://www.codeplex.com/BizTalkScheduledTask
• Microsoft Enterprise Library 3.x
• Camtasia Studio

System Tools

• Unlocker http://ccollomb.free.fr/unlocker/
• .Net Reflector http://www.red-gate.com/products/reflector/index.htm
• Process Explorer (SysInternals/Microsoft)
• Kapersky Internet Security

Tripped on this one again -- the idea is that the orchestration posts a message directly to a message box and from there it is picked up by subscribers. This wasn't working, then when I saved the message out of the HAT, I realized the routing properties weren't being promoted. Sigh. Luckily this excellent blog article popped up. Right, use a correlation set. I can't believe I always forget that. LOL. And I also needed the tip from the comment -- declare the correlation set instance inside a scope (Transaction type=none) so that it will work in a loop.

"Property Promotion inside Orchestration" http://www.digitaldeposit.net/saravana/post/2007/01/08/Property-Promotion-inside-Orchestration.aspx

Here's my added tip:

I like to declare the correlation type in the same project as the schema and the property schema. I also like to define the ports here. This makes reuse easiler and forces correct naming scheme -- gone are the days of outPrt.Request (PortType1) =WTF?!?  (Since you must declare the type within an orchestration, so often I'll add a "dummy" orchestration in the messaging projects. )

1) Make sure you declare the type as "Public" (default is internal) and 2) After re-compiling the messaging project, you must Unload/Reload the project which has the orchestration (or just reboot LOL!). This allows the new type to be visible/ used.

Also, when one of my developers asked "Are there any other ways to promote properties", the answer is yes:

First realize, the promotion has to happen *before* the message hits the message box.

1. Run the message through a pipeline -- technically can do this in an orchestration, but this sounds like overkill to me.

2. Publish the message to a file folder, have another receive port pick it up using XMLPipeline. This is fragile and overly complex.

3. Write some C# code which can call in an Assign shape from your orchestration. You will need this if you want to do custom promotion (i.e. you cannot automatically promote because of loops,etc) or property Demotion as well, this is likely your only option.  This is too complex and time consuming for ordinary stuff -- consider using a map to manually copy THE needed properties out of the looping body to a header/envelope and promote from that instead.

Note for later:

Bug in biztalk mapper with multiple messages.

BizTalk mapper does not support multiple input messages where 2 (or more) use the same namespace. I get a compile error on the btm.cs file -- a file I cannot even edit.

Workaround: Use two maps. In second one, take the result of the first one, mass copy, plus add the second message.

And a *somewhat* related bug report: http://support.microsoft.com/kb/883376

BizTalk & BRE - sigh - This error seems to happen upon compile when your business rule policy isn't working/is broken and you try to compile your orchestration.

update: When adding helper classes to a policy, make sure it and ALL of its dependencies are in the GAC. If at a later date you have even one missing dependency, it will halt your development, make you unable to view the policy, etc.

Today I edited a file from our online document repository without first downloading it. I had just clicked "Open" when IE7 prompted me instead of "Save"...   I worked on the document using Office 2007 Powerpoint for a couple of hours, clicked Save and then closed it.  Next I tried to find it to re-upload it to the repository. OOOPS!  I hadn't done a "Save As..." silly me and my file was GONE! How's that for a Friday night horror story. 

But instead of crying... I was tempted, I began to think like a geek (rather than a document-bound architect): This file is SOMEWHERE, I mean the bits were not just in the ether.  I looked in the obvious place "Temporary Internet Files" but there was basically nothing there.  Suspicious. Googling a bit, I confirmed this -- a shell extension displays a special view which masks the real contents for this folder. Note: This type of hiding is different that the "normal" one which you can easily disable (and probably already have).

How did I find my file you ask? You could disable the shell extension, download special hard drive analysis tools, or the easiest way (Hard drive space permitting):

How to find a file in the hidden internet explorer cache:

1. Go find your temporary internet files: C:\Documents and Settings\userName\Local Settings\Temporary Internet Files (parent folder is hidden, so you have to go to your explorer options to view these hidden/system files). 

2. Next Copy-Paste the entire folder to somewhere else. 

3. After that finally completes, you can browse the contents normally. **Search** for your filename/extension since there is a lot of stuff nested in that folder.

Despite the happy ending, I am really upset that Office actually let me do that. I can't believe that I was allowed to save a file to a spot that I am not allowed to browse.  So I'll just say it AAAAAAAAAAARRRRRRRRRRRRRRGGGGGGGGGGGHHHHHHH!!!

I finally got an example working where I am using a pure XAML (no code) workflow definition and also using a base class. This took a bit of research and filling in the blanks. I was surprised that Visual Studio doesn't seem to support the XAML-only stuff... sigh. And when I got is working, the designer refused to display it, so I have to stick to the XML Editor going forwards. (I haven't checked the 3.5 fixes yet, xfingers!)

First I created the workflow custom base class as a csharp class, I created my custom activity. Next, in a different project, I let Visual Studio create a XAML file for me (it is important to do the actual workflow definition in a different project, do not optimize this). Then I edited it, and deleted the code behind. Here is the summary of the manual editing steps:

• Remove the x:Class attribute
• Change the initial tag from the current workflow name to the name of your base workflow class. Ensure you add a namespace tag i.e. "ns0"
• define the namespace instead of using http instead using clr-namespace (see below) and after that provide the fully qualified namespace and assembly for the workflow base class. You need not apply a strong name, but that is what I usually do.

<ns0:MyBaseWorkflow x:Name="workflow1" InProp1="{x:Null}"
  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" 
  xmlns:ns0="clr-namespace:My.Workflow.Activities;Assembly=My.LeadImport.Workflow.Activities, Version=1.0.0.0, Culture=neutral, PublicKeyToken=12345">
  <ns0:MyCustomActivity Prop1="Validation" Prop2="ValidateStep" 
	x:Name="myActivity1" 
	xmlns:p3="http://schemas.microsoft.com/winfx/2006/xaml/workflow" />
</ns0:MyBaseWorkflow>

Having worked so much with BizTalk, haven't had the chance to create a pure .NET 2.0 web service (automaticly generated WSDL). Much to my surprise, the concept of a "Required Field" is not explicitly supported -- i.e. setting minOccurs = 1, etc.  For future reference, some mitigating suggestions are provided by Microsoft here: http://msdn2.microsoft.com/en-us/library/zds0b35c.aspx. Feels like going back to the stone ages. Apparently this is much better with WCF bindings....

The message box viewer gives an excellent overview of BizTalk solution including custom tweaks against some "less" documented features. This helped me diagnose an exceptionally high load after we created additional host instances where someone had actually increased biztalk polling interval (but hadn't documented this so it was unknown to me).

http://blogs.technet.com/jpierauc/archive/2007/11/13/msgboxviewer-9-20.aspx

-Jennifer Zouak

Installing a BizTalk App can be quite a challenge. You have options of

1) Manual, lots of documentation needed. But then you were going to create docs anyway, right? This is good when you only need to install it once.

2)  BizTalk-generated MSI. This is simple and quick to produce. My main issue is that this is not customizable with regard to the application name and product version (think Add/Remove programs) nor can you specific the installation folder. Another huge problem is that *if* the deployment BizTalk is configured differently with regard to adapters, hosts, etc than the build machine, then the installation completely fails.

3) Completely automated, wizards or build scripts.  This is obviously preferrable, but the tools given are not up to the task. Hence this method requires development time to encode the bindings, build the install script, test, etc.  There are some frameworks out there too. 

Here are the challenges

The main hurdle here is the bindings. For custom integrations, typically, we do not know the Production bindings up front and they also are very subject to change. And they differ from the pre-Prod bindings and the QA bindings, etc. Another issue is that the passwords are not preserved by the bindings, so this needs to be addressed somehow.

We also usually need application settings in our biztalk config file(s), helper classes into the GAC, registry keys, COM components registered, etc.

I spent some time to try to use Visual Studio's basic setup Wizard project to create a nice installer for one of our common BizTalk projects. I created a super dooper custom action which would invoke "BTSTask.exe" and do stuff like export bindings, install assemblies, etc.

This worked well for me, in test and for the first client. For the second client, it mysteriously failed. It turns out that sometimes when you install and you aren't on a console (i.e. physically at the machine) then the installer will run the "Custom Action" using the local system credentials, rather than the logged in user. Of course, for BizTalk you need to be in the BizTalk admin group in order to configure it.  I did a bit of poking around and came upon the installation attribute "TSAware" (TerminalServerAware) which *if set* can apparently prevent this change in user account. Unfortunately in Visual Studio setup project, this attribute is not anywhere to be found. At this point, I'm kinda stuck since I can't go changing the security setup at client sites -- so we are back to my "favorite" manual method.

Next I will look into the available extensions to msbuild scripts. These can have the TSAware attribute set, so should be able to overcome the problem...

 -Jennifer Zouak

This month I have had to lookup BizTalk 2006 behaviour for 64 bit architectures for two different clients. It doesn't seem to be well known, but this should be more of an issue in future.

Firstly, Microsoft documents BizTalk as supporting 64 bit, and then follows up with a few exclusions: ia64 bit for one, and many of your favorite adapters for another. http://technet.microsoft.com/en-us/library/aa560166.aspx.

So if you are going to target 64 bit architecture, be careful with your deployment design i.e. the host specifications. By default you will only have 32 bit hosts which run in a compatibility mode. You will have to create specific hosts for 64 bit support, otherwise your solution will likely run *slower* than on a 32 bit machine. Also, don't create too many host instances as this can pummel your message box(es). I recommend creating SendRcv host for each of 32 and 64 bit, then creating your Processing hosts as 64 bit.  Be sure to check with vendors of 3rd party adapters to see if they are compatible.

In terms of installing your apps, you will not need to recompile .NET code. The installation is simlilar, but be aware that there is a second .net config file which you should be awaare of if you are using AppSettings in the normal one "btsNtSvc.exe.config" and "btsNtSvc64.exe.config". Also, if you are using DTD's then you will likely need to copy them to both system32 and WOW64 (depending on the adapter used).

For fun, I'll also mention clustering design. With BizTalk, the best way to cluster is to use the "Group" which means you install BizTalk on multiple servers and point them to the same Message box. You then simply declare a host instance for each server and you have automatic load-balancing clustering.  Yay! The exception is that some adapters don't work well when you allow them to run multiple times. Here is MS on this issue: http://www.microsoft.com/technet/prodtechnol/biztalk/2006/library/bts06clustering/

The solution is to have a combination of Windows clustering configuration and also use a BizTalk group. Most items can run on multiple servers for good performance, and Adapters such as SQL Receive (schedule) and FTP Receive need to run in one instance of a Clustered host so they don't publish two copies of the same info.

 -Jennifer Zouak

I have the pleasure recently, after working with BizTalk since 2001, of actually implementing a solution using the business rules engine aka BRE as hosted by a BizTalk Orchestration. While the "insight" is fresh, I though I'd jot down my observations. This may be of interest to you if you are evaluating this for use within your solution/organization.

• BRE uses a different paradigm than typical coding. Now if you are considering BizTalk, then you are probably aware of this already. But even so, although Messaging and Orchestration are two new paradigms for many of us, BRE represents a third way to think about executing program logic. This means there is a significant learning curve for expressing even the most trivial of ideas.
• Source Control: The Business Rules Composer does not integrate with source control. You need to enforce a manual process of export and check-in. (This cannot be scripted, see below).
• Deployment: There is a Business Rules Deployment Wizard. First problem -- this has no command line/ scripting interface. Get used to click-click-clicking your way through deployments. Second problem -- if your policy is dependant upon a certain version of a vocabulary which is not yet deployed, it fails to deploy.
• Vocabulary: In terms of developer experience, each change of a vocabulary requires you to copy-paste, save and publish a new version before you can use the terms. Second problem -- after you create and use a new version, the existing rules will refer to the old version and the updated rules will refer to the new version. This means you need to deploy both vocabulary versions to production. The Acumen tool (below) can automatically update *most* of your references which mitigates this issue. Third problem -- (I may be incorrect here) the vocabulary hard-codes things such as paths to XSD files and these paths need to be manually updated before deploy to production.
• Versioning: In development in order to test out a policy or use a vocabulary, you need to publish (and deploy). This makes the artifact immutable, so to enact further changes you must copy-paste to create a new version. After significant development, you can have many versions. This is analogous to "build" numbers. The problem occurs when you want to use a "release" number. This is an intense manual process to a) keep track of the "release" numbers and b) update the XML files before deployment.
• Debugging: There is no way to debug. You can only perform unit tests.
• Testing: I have big issues with testing. You have 3 options.
  • First you can test within the Business Rules Composer. Problem one: If you use .NET facts with configuration strings this won't work. For some reason the Composer is NOT a .NET application and you cannot feed it any .NET config files. (Hint: abstract your configuration and create default values, be sure to log any use of default values in case you forget to set your config in production). Problem two: I get different results from within the Composer versus the same policy and messages when hosted by an Orchestration!!! (This occured when I was playing with Naf and the "exists" predicate. Looks like an optimization on empty nodes may occur within BizTalk.) As the test executes, you can view a trace of the activity. This trace isn't very helpful, but is better than nothing.
  • Second you can create a test harness orchestration. This is a good idea. But you don't get much information.
  • Third you can write a .NET application which hosts the rules engine, feeds in your test data, applies custom tracing, and even checks the results. But my problem with this approach is that you cannot guarantee the same behaviour as within BizTalk unless you test within BizTalk (see point 1).
• NaF: Negative assertion as Failure. Ok, this is a big term, google it. Basically the BRE *does not* perform any rule where the fact has not been asserted. That means if you have a rule such as "IF A == "1234", THEN B" and the fact A is missing/empty/blank then that rule will simply not fire. You *cannot* use the BRE to validate messages such as "IF C is missing THEN D". And finally if you have a rule "IF E/F is true then G" (an XML fact) and the slot F is missing, then the rule will throw an exception (BizTalk will report XLANG/s in event log). Also if you have a rule with multiple conditions of which a single fact is missing, then the entire rule will not fire.
• Parameters: What is the interface for my policy? You may ask yourself, "Hey, I just took this existing vocabulary and dragged in some terms and made my rules. I wonder what my parameters are?" This is a good question. You have to be very aware of which data structures/databases your rules (via the vocabulary) are using and then be sure to supply one (or more) instances of each at runtime. Second point: Although the documentation seems to indicate you can send in variables or messages, AFAIK, you can *only* send in messages from an orchestration. If you send in two instances for a given Type, by default, the rules will execute twice, one for each copy. Third point: If you forget to pass in an expected parameter at runtime, then you will not get a warning or error. There is no concept of a required parameter.
• Messages in Orchestration: The "Call Rules" shape breaks a fundamental design principle in orchestration: The message is immutable. The BRE modifies messages in-memory and returns them to you. So in the orchestration, you pass in msgA, the BRE sets the approved flag to true and returns to you msgA. But actually it is a new message msgA' that has been assigned to the same variable. If you need your original message context, you should pass the BRE a copy of the original message.
• Documentation: There is *no* ability to annotate the conditions or actions within rules. You can use the rule and policy name as a basic handle on what is happening. The Acumen tool produced a cool PDF report which documents a given policy and does so in a way that other people can understand it.
• Prefix: The rule conditions are expressed as prefix notation. This is a different paradigm from the "normal" infix style coders and business people use (think Excel). Only a math junkie can love this. Personally, I find it very hard to sight-read the rule.
• Short circuit: Most code languages (and also conditions in infix notation) use short circuit logic. This mean that if the first part of the AND statement evaluates to FALSE, then the second part is skipped. We typically use this as follows "IF A exists AND A==1234 THEN action B". But in this case both sides will be evaluated causing the equivalent of a pointer exception when the value of A is attempted to be accessed. Actually, if you view the traces after "Test Policy" you may notice that all your conditions are evaluated and only then are the rules asserted.
• IF-THEN-ELSE? Again, I am biased as a coder, but where is the "else" statement? I chalk this up to "different paradigm". Each rule is expressed as CONDITIONS-ACTION, basically an IF-THEN logic. To get the else, you would have to copy paste the rule, apply a "NOT" to the begining of the conditions and update the action. Even this may not yield the expected results because neither rule may actually fire (see NaF above).

My advice:

Some recommendations:

• Acumen Rules Manager. If you are going to do BRE development, I highly recommend buying this tool.
• Rule Name. I always prefix the rule name with the priority, i.e. -20. If you use negative numbers, then your rules will be listed in execution order.
• Vocabulary: I highly recommend using a vocabulary. Organize them by type, so have one vocabulary for each message type. When a change is made, rev the entire vocabulary and then update your rules using Acumen tool.
• Develop consistent, documented, repeatable procedures to document, version control, regression test and deploy the business rules.
• In your orchestration, place the "Call Rules" shape in a retry loop.  Exit the loop if the rules succeeded, (i.e. check a message property to ensure it has been updated), but Suspend the orchestration if the rules have failed. You then can update the rules and resume the suspended orchestrations.

BizTalk error messages are notoriously opaque. I decided to start recording my translations and solutions for these messages because... well because I keep encountering the same ones time and again, my mental buffer is full, and flipping through loads of notes is no longer fun.

Error: Unexpected event ("document_start") in state "processing_empty_document".

Scenario: This advice applies if you receive this error during receive pipeline xml disassembly processing, and you are using an envelope schema.

Explanation: The error appears when your interchange (input document) has xml element in the form <foo attr="value"/><foo attr="value"/> which doesn't work -- instead of <foo attr="value"/> <foo attr="value"/>, which works..

Solutions: This type of XML is generated by the SQL Adapter, so you can try changing your stored procedure to use "XML EXPLICIT" to get elements instead of attributes.

To debug, use the tool: C:\Program Files\Microsoft BizTalk Server 2006\SDK\Utilities\PipelineTools\xmldasm.exe.

Example: XmlDasm.exe samplefile.xml -ds docspec.xsd -es envelope.xsd

Workaround Solution: I have now confirmed that this is a BizTalk 2006 issue and am waiting for a response from the dev team.

Update: I received a hotfix KB927741 from Microsoft in December. This resolved the issue (Note: have to manually GAC the new pipeline DLL). http://support.microsoft.com/kb/927741