Paul's Petrov Whiteboard

Generally, it is not recommended to store passwords in the binding file and by default BizTalk administration UI clears it out when exporting binding to the xml file. But in development environment where you have automated continuous integration build and deployment not having password in the binding file will prevent application from starting and CI process fail. It can become tedious in staging deployments (TEST, UAT) as well, especially if you have multiple endpoints that use accounts with passwords. In such cases, having password for development account (hopefully not the same as production) in binding file can make life easier.

To do that, export binding file after application is fully configured. Open it and locate section(s) with configuration for the ports/locations of interest. For example for the WCF receive location it will be in:

Recently, while debugging desktop client - WCF service application I came across this error:

System.ServiceModel.Security.MessageSecurityException was caught
Message=”An unsecured or incorrectly secured fault was received from the other party. See the inner FaultException for the fault code and detail.”
Source=”mscorlib”
StackTrace:
Server stack trace:
at System.ServiceModel.Channels.SecurityChannelFactory`1.SecurityRequestChannel.ProcessReply(Message reply, SecurityProtocolCorrelationState correlationState, TimeSpan timeout)
at System.ServiceModel.Channels.SecurityChannelFactory`1.SecurityRequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)

InnerException: System.ServiceModel.FaultException
Message=”An error occurred when verifying security for the message.”

Client was connecting basic http binding with security mode TransportWithMessageCredential. All other desktop clients were able to successfully call WCF service but not this particular one. Thanks to this post the issue was quickly understood and resolved. Apparently, the client machine was a brand new netbook just out of the box and clock was way off. Interesting thing that Windows automatic clock synchronization did not work.

Recently during migration BizTalk project from SQL to WCF adapter had some rather confusing moment. After deploying to test environment WCF ports started generate errors like this:

Failed to open Microsoft.ServiceModel.Channels.Common.Channels.AdapterRequestChannel, Faulted Microsoft.ServiceModel.Channels.Common.Channels.AdapterRequestChannel, Faulted System.ServiceModel.Channels.ServiceChannel, Failed to open System.ServiceModel.Channels.ServiceChannel

 After few frustrating troubleshooting sessions it was found that adapter binding was missing identifier for the default database instance. Here's what we had:

Both adapters worked in development, but in test WCF did not. What was missing is "//" between server and database part of the URI. This is not required for SQL adapter configuration but apparently is critical for the WCF adapter. So the correct URI should have been: mssql://TestServer//Database.

I just installed Microsoft Enterprise Service Bus Guidance framework (November 2007 version for BizTalk 2006 R2). Although lengthy installation process is well documented in the accompanying help file I ran into multiple issues and it took a couple of days to sort everything out. I'd like to mention here major problems/solutions in case somebody decides to go through the same. First, if you install on other than default location (C:/%PROGRAM_FILES% and C:/Projects) make sure to update all scripts and command files as well as bindings. Second, I had issue with UDDI service configuration. It wouldn't install until I enabled NTLM authentication for IIS. It is done by executing IIS utility script per this MS article. But this command worked only when I dropped quotes around Negotiate,NTLM contrary to MS KB article. Also, make sure the default web site ID is "1". Some of my computers had it set to different numbers so I had to change command line appropriately.  In the end command that worked looks like this:

cscript adsutil.vbs get w3svc/<Default Web Site ID>/root/NTAuthenticationProviders Negotiate,NTLM

Another problem is that assemblies in MSI packages are signed with different key comparing to source projects. So even if you install all ESB components from source you will run into this issue because some of the projects  reference assemblies from packaged installation. Solution is to go through all project references, remove them and add new to ones generated from source code. The same applies to sample applications. Don't install them from MSI if you chose source code based ESB installation. Open project, update references, rebuild and deploy.

Uploaded new version of SSO Config Tool including fix for the browsing SSO Config Store application and some usability/UI improvements. Download source code.

Choice of configuration settings storage is an important topic when it comes to enterprise BizTalk application planning. One of the many options is to use regular .Net configuration files. Some prefer this way over the Enterprise SSO database option for reasons of simplicity and familiarity. I wanted to show how it can be done with Microsoft Enterprise Library configuration application block.

In this case Enterprise Library configuration section is placed in the BTNTSvc.exe.config file while application settings are stored in a separate configuration file. The problem here is how do we make EntLib to load required settings file at runtime.  Its done by simple helper class CustomSettings that looks up registry entry for the location and name of the application configuration file and creates FileConfigurationSource with it. The registry entry can be created  by MSI installation package.

The initialization method of this static helper class looks like this (thread synchronization code omited for brevity):

          RegistryKey regKey = null;

          try
          {
             regKey = Registry.LocalMachine.OpenSubKey(@"Software\MyCompany\MyApplication");
             configurationFile = Path.Combine(
             (string)regKey.GetValue("ConfigDir"),
             (string)regKey.GetValue("ConfigFile"));

              ExeConfigurationFileMap fileMap = new ExeConfigurationFileMap();
              fileMap.ExeConfigFilename = configurationFile;
              configuration = ConfigurationManager.OpenMappedExeConfiguration(fileMap, ConfigurationUserLevel.None);
              configurationSource = new FileConfigurationSource(configurationFile);

              //- store configPath in the settings
              configuration.AppSettings.Settings.Add(
              new KeyValueConfigurationElement("configDir", (string)regKey.GetValue("ConfigDir")));
           }
           catch (Exception ex)
           {
              Debug.WriteLine("Exception while initializing Settings:" + ex.ToString());
              throw ex;
           }
           finally
           {
              if (regKey != null) regKey.Close();
           }

 Then it has method to access properties by name:

        public static string GetValue(string name)
        {
            KeyValueConfigurationElement entry = Configuration.AppSettings.Settings[name];

            if (entry == null)
                throw new ConfigurationErrorsException("Key '" + name + "' is not found in the configuration file.");

            return Configuration.AppSettings.Settings[name].Value;
        }

 Which is used as in:

string propertyValue = CustomSettings.GetValue("propertyName");

If you store BizTalk application settings in Enterprise SSO database and adapt continuous integration you'll find this MS Build task useful. DeploySSOConfigStore task reads settings from XML configuration file and saves them to the SSO database. The XML can be created (exported) using Richard Seroter's SSO tool which I modified to support this operation. So, if you change your configuration settings, just update XML file in the source control and build process will pick it up and propagate changes to your target environment. Using this task is very simple, it accepts just one parameter - location of the XML settings file:

    <UsingTask AssemblyFile="CustomMSBuildTasks.BizTalk.dll" TaskName="CustomMSBuildTasks.BizTalk.DeploySSOConfigStore"/>

    <Target Name="DeploySSOConfigStore" DependsOnTargets="BizTalkDeploy">
        <CustomMSBuildTasks.BizTalk.DeploySSOConfigStore XmlConfigurationUrl="$(SolutionRoot)\$(BuildBranch)\Source\BizTalkSettings\BizTalk.Configuration.xml"/>
    </Target>

I also added another useful task for publishing WCF services: PublishBizTalkWcfServices. It's very simple but does all that stuff: setting up virtual directories (if needed), publishing contracts, creating receive locations. It has comprehensive logging that helps tracking down deployment issues quickly.

    public class PublishBizTalkWcfServices : Task
    {
        private string serviceDescriptionUrl;

        /// <summary>
        /// The URL to the description file.
        /// </summary>
        [Required]
        public string ServiceDescriptionUrl
        {
            get { return serviceDescriptionUrl; }
            set { serviceDescriptionUrl = value; }
        }

        /// <summary>
        /// Publishes BizTalk schemas as WCF services.
        /// </summary>
        /// <returns></returns>
        public override bool Execute()
        {
            WcfServiceDescription description = WcfServiceDescription.LoadXml(ServiceDescriptionUrl);

            Publisher publisher = new Publisher();
            publisher.BackgroundWorker = new System.ComponentModel.BackgroundWorker();
            publisher.BackgroundWorker.WorkerReportsProgress = true;
            publisher.BackgroundWorker.ProgressChanged += new System.ComponentModel.ProgressChangedEventHandler(BackgroundWorker_ProgressChanged);

            try
            {
                PublishingResults results = publisher.Publish(description);

                Log.LogMessage(results.Message);
            }
            catch (Exception ex)
            {
                Log.LogError(ex.ToString(), null);
                return false;
            }
            return true;
        }

        public void BackgroundWorker_ProgressChanged(object sender, System.ComponentModel.ProgressChangedEventArgs e)
        {
            Log.LogMessage(e.UserState.ToString(), null);
        }
    }

Complete Visual Studio project is available here.

Richard Seroter published useful utility to store/retrieve BizTalk configuration settings in Enterprise SSO database. I added some more functionality to it.

First, I added Export/Import functions allowing to actually save values in the external XML file (Richard's implementation allowed storing only field names, calling for extra work should you delete and restore SSO aplication). I kept file format the same but added values stored as text nodes:

<sso>
  <application name="eCommerce.BizTalk.Configuration">
    <description>Configuration storage for eCommerce BizTalk applications</description>
    <appUserAccount>BizTalk Application Users</appUserAccount>
    <appAdminAccount>BizTalk Server Administrators</appAdminAccount>
    <contactInfo>http://geekswithblogs.net/paulp</contactInfo>
    <field ordinal="0" label="App1.SiteId" masked="no">2</field>
    <field ordinal="1" label="App1.UserId" masked="no">8</field>
    <field ordinal="2" label="App2.Timeout" masked="no">5000</field>
    <flags configStoreApp="yes" allowLocalAccounts="yes" enableApp="yes" />
  </application>
</sso>

New usage scenarios:

1. Saving configuration:

1.1 Create application; 
1.2 Save to XML; 
1.3 Manage application - Retrieve settings; 
1.4 Export to XML file created in (1.2).

2. Restoring previously saved configuration: 

2.1 Create application;
2.2 Manage application - Import configuration from file created in (1.4) ; 
2.3 Save Changes.

Accordingly, menu items Import/Export available when the Manage Application tab is selected only.

The code can be downloaded here.

Fellow pilot and talented IT professional, David Megginson, put out neat web site http://www.ourairports.com that allows to leave comments about airports visited (as pilot or passenger), and share personal maps. I added those where I landed as a pilot from the logbook and it shows how I definitely need some real "cross country" trips :)

This time we'll combine loosely-typed BizTalk messaging concept with dynamic mapping to create generic transformation orchestration. Then we'll add it to the message broker created in the first article to make it even more powerful.

Quite often message dispatching task must be combined with some content adaptation: data mapping, filtering, calculation, etc. Generally, maps are used in BizTalk to achieve that. Since maps are based on source and destination schemas we need to find a way to abstract transformation process from specific schema type to common XmlDocument. What makes it possible is XLANGs transform command. This command accepts output message as argument and uses System.Type of BizTalk map. It allows us dynamically load and apply any map type to any message type.

First, we have to author schemas and maps. I created two input schemas for this project: NewEmployee.xsd, NewOrder.xml; two output schemas: Employee.xsd, Order.xsd; and two corresponding maps: NewEmployee_to_Employee, NewOrder_to_Order.I did some changes in DispatchMessage orchestration to integrate with TransformMessage.odx. I added new message variable msgTransformed which is an output from transformation orchestration. Then I simply added Decide shape to check if transformation required for incoming fie. If yes, we pass map key and input message to the TransformMessage.odx using CallOrchestration shape. If no mapping provided then we simply copy incoming message content to msgTransformed:

Again, to retrieve required map name at runtime, we're going to use the same key-value pairs in the configuration file. In our example we used file name as the key so let's follow this trend. If we're gonna store these keys in the same configuration file we need to modify them to keep unique, for example by adding ".MAP" prefix, i.e. <add key="NewEmployee.xml.MAP" value="GenericBiztalkPatterns.Maps.NewEmployee_to_Employee" />.

To keep things clean and reusable I factored out mapping functionality in separate TransformMessage.odx. orchestration. This orchestration contains three parameters: msgInput, mapKey, out msgOutput which are self explanatory. Notice, all message parameters are of XmlDocument type again. The orchestration id very simple itself. Read fully qualified map name by passed in key and get Type of this map:

mapTypeQName = System.Configuration.ConfigurationManager.AppSettings[mapKey];
mapType = System.Type.GetType(mapTypeQName);

Then in ConstructMessage shape just apply transformation:

transform(msgOutput) = mapType(msgInput);

That's how it looks in designer:

I did some changes in DispatchMessage.odx to integrate with TransformMessage.odx. There is new message variable msgTransformed which is going to be an output from transformation. Also, I added Decide shape to check if mapping required. If yes, then input message and map key is passed into TransformMessage.odx through CallOrchestration shape. Ohterwise, input message simply copied to msgTransformed:

Now, all we need is to make sure there're proper entries in the configuration file:

  <add key="NewEmployee.xml" value="file://C:\Projects\Design Patterns\GenericBizTalkPatterns\GenericBizTalkPatterns.MessageBroker\Ports\File.out\employee.xml" />
  <add key="NewEmployee.xml.MAP" value="GenericBizTalkPatterns.MessageBroker.Maps.NewEmployee_to_Employee, GenericBizTalkPatterns.MessageBroker, Version=1.0.0.0, Culture=neutral, PublicKeyToken=44e796208f4e4812" />
  <add key="NewOrder.xml" value="ftp://[Your FTP server path here]/order.xml" />
  <add key="NewOrder.xml.MAP" value="GenericBizTalkPatterns.MessageBroker.Maps.NewOrder_to_Order, GenericBizTalkPatterns.MessageBroker, Version=1.0.0.0, Culture=neutral, PublicKeyToken=44e796208f4e4812" />
  <add key="FTPUserName" value="[Your FTP username]" />
  <add key="FTPPassword" value="[You FTP password]" />
  <add key="FTPMode" value="true" />
  <add key="FTPRepresentationType" value="Binary" />
  <add key="FTPCommandLog" value="C:\Projects\Design Patterns\GenericBizTalkPatterns\GenericBizTalkPatterns.MessageBroker\ftp.log" />
  <add key="FTPBeforePut_NewOrder.xml" value="" />
  <add key="NotRoutedDestination" value="file://C:\Projects\Design Patterns\GenericBizTalkPatterns\GenericBizTalkPatterns.MessageBroker\Ports\backup" />
  <add key="NotSuportedProtocolDestination" value="file://C:\Projects\Design Patterns\GenericBizTalkPatterns\GenericBizTalkPatterns.MessageBroker\Ports\backup" />

At some point, configuration file can get quite large, that's why I prefer to keep group of settings in individual configuration files, like destinations.config, maps.config, etc., but that's a topic for another article. So far, we've seen how generic messaging concept can help creating flexible agile BizTalk applications. [Source code is available]

Although current BizTalk release does not support .NET generics it does support concept of genericity at the message level. It is possible, of course, through "untyped" messages or messages that don't have specific type attached to their context. Such messages are represented as System.Xml.XmlDocument type in orchestrations. To read more on message typing aspect of BizTalk please refer to this excellent post by Charles Young. I'd like show practical examples of applying generic programming to typical BizTalk tasks. These examples will help to make your BizTalk solutions leaner, more flexible, and easier to maintain.

Let's start with very common pattern where BizTalk is used as merely message dispatcher - Message Broker. The goal of our generic message broker will be to dispatch any type of message to its destination decided at runtime. So, by avoiding type dependency and early routing binding (basically, hardcoding) we would get single very flexible orchestration which can easily handle requirement changes - one of the top goals of good application design.

Some things to consider before creating BizTalk orchestration. First, we need to figure out how and where to store input to output destination map. Since, it's just key-value pairs, application configuration file will work fine. In my work, I prefer to keep collections of homogenous setting like this in separate xml config files and read them in custom XmlSerializable map (objects like NameValueCollection or Dictionary being non XmlSerializable won't work in orchestration unless placed inside atomic transaction scope). I'll place them in application configuration file now to keep it simple. Next thing, is to decide what to use as a key. Since messages all of the same type to orchestration, we should select something that uniquely maps message to destination. Let's assume that per our requirements it's a input file name, although it can be anything inside message content. The destination will be expressed as complete URL, i.e. [protocol]://[path]/[fileName], for example ftp://myserver/files/dest.dat. So, the map entry can look like this: <add key="emp.dat" value=file://myserver/incoming/employees.dat />. I use .dat extension here just to emphasize it's applicable to any files and not only XML.

Once questions have been answered, the orchestration comes together fairly quickly. On the global level it has one input port, expression to read routing configuration, Decide shape to branch on routing availability condition, and dynamic send port. Receive shape receives input message of XmlDocument type. Output message of the same type is sent through dynamic send port after routing address and all properties have been set:

Inside Decide_IfRoutingAvailable shape there are two branches of execution. If routing address is found then it will read destination URL from the configuration and select transport protocol. Otherwise, destination will be set to the backup location from configuration file in order for the message to be preserved:

Once protocol is identified, we can create output message and set its properties and destination address. In case if protocol is not supported, we route message to the backup location using the same dynamic send port. That's how it looks inside Choose Protocole decide shape:

Let's dive into ConstructFtpMessage shape, to see how properties set:

First line simply copies our generic input message content to the output message. Susequent lines set FTP properties from configuration file. Then, all that left is to set destination URL on dynamic port and send output message through it:

As the result, we have a single orchestration that can handle hundreds of different message schemas and multiple protocols. Another positive outcome is that deployment greatly simplified. The application has one orchestration, two ports, but only one of them is bound at deployment time. Also, note no schemas, no maps at this time, we will add them later when we augment application with content transformation functionality.

Happy New Year! It's good time to summarize flying experience as year past since getting pilot certificate. So, here's some dry statistics:

Airplanes I've flown: Cessna C152, Cessna C172M, Cessna C172SP, Cessna C177RG, Piper PA28-161, Piper PA28-181, Evektor Sportstar, Vans RV-7A, Wheeler Express.

Airports I landed at: PAE (Everett, WA), AWO (Arlington, WA), BLI (Bellingham, WA), OLM (Olympia, WA), RNT (Renton, WA), CLM (Port Angeles, WA), S43 (Snohomish, WA), W16 (Monroe, WA), GPM (Grand Prairie, TX), ADS (Addison, TX), RBD (Dallas, TX), TKI (McKinney, TX), XBP (Bridgeport, TX), SEP (Stephenville, TX), FTW (Fort Worth, TX), CRS (Corsicana, TX), JWY (Midlothian, TX), LNC (Lancaster, TX), GKY (Arlington, TX), UTS (Huntsville, TX), GLS (Galveston, TX), JXI (Gilmer, TX), T31 (McKinney, TX), SWI (Sherman, TX), DTO (Denton, TX), GYI (Sherman/Denison, TX).

One thing I don't want to add up though is the amount of money spent on flying :)

I started instrument training and logged some simulated time under the hood. So, the new year flying resolutions will be to finish instrument training and get that rating, log at least 60 hours total time, get tailwheel endorsement, get high performance and/or complex endorsements. This should keep me pretty busy, I guess. :) Happy flying in New Year!