Todd Uhl

Installing the new '06 SharePoint Adapter

2006-06-06T10:59:00-05:00:00

The new SharePoint Adapter is now included with the install of BizTalk. Typically, in a production environment, the SharePoint site you will be interfacing with is not the same as any of the BizTalk servers. When reading through the documentation, installing the adapter webservice on a remote machine is not very apparent. Easy enough, just run the BizTalk install on the SharePoint box you might say. Well, it's almost that easy, but not quite. The first road block is the install wizard won't even run if SP1 (if on a Win 2003 box) is not detected. This is a bug in my opinion. What if I were just wanting to install the documentation, does that require SP1? Anyway, there appears to be no getting around this point. Don't waste your time finding the files on the CD and copying them to a virtual directory. So, fair enough, install SP1 if not currently installed, then run the install. The SharePoint Adapter web service is listed under Other Software. Logically, we would just select that, it would appear to install succesfully, and we're all set. Nope. You actually need to select the adapter web service, AND ALSO Business Activity Services. Another bug in my opinion. Anyway, select both, complete the install and you should be fine. You may have to go into IIS and create your own virtual directory, depending on if you went through the configuration on the BizTalk server. Don't forget to make sure the site the web service is running under is marked to run under ASP.NET 2.0. And, don't forget to edit the web.config.tmpl.

Cheers,

Todd

Installing BTS 2006 - Sharepoint 1057 system error

2006-04-20T10:52:00-05:00:00

When setting up WSS so you can do a full install of BizTalk, you may get an error similar to “System Error 1057 while trying to query service SPTimer”. For me, this occured while using the Configure Administrative Virtual Server section of Central Administration. I was using a local (non AD) service account that I had created. The problem is corrected if you make sure you enter “domain\user name” or “machine name\user name” format. Not easy to correlate this error description to the actual issue.

Cheers,

Todd

BAM Export error in Excel 2003

2006-04-12T13:31:00-05:00:00

So, I figured it was finally time I broke down and started to really figure out what BAM was all about, partly inspired by Richard Seroter's posting on the subject (everyone should read his blog by the way.... fantastic stuff!!). So, I picked a sample project I've been playing around with for some other POC stuff, followed along with the handy little “walk-throughs” in the help file, and wouldn't you know Excel puked on what appears to be a very important step - exporting the BAM Observation Model (all the KPI's, etc). While trying to export the excel file to XML using the BAM Export XML menu, I received a “Invalid procedure call or argument” error. After a quick google I ran across some information about a possible security patch to Excel that might cause this error. I don't recall installing a security patch, but who knows.

Anyway, the good news is that you can use the XSL file without exporting to XML. I believe the XML file would contain some additional information, like the server where the BAMPrimaryImport database is running, etc., but you can also provide this information when you deploy the observation model with the BAM Management utility (bm.exe). You can use the following syntax at the command line to accomplish this (replacing MyWorkbook.xls with your actual excel file name) -

bm deploy-all -DefinitionFile:MyWorkbook.xls

Using this syntax will default to the local server for SQL Server, and the BAMPrimaryImport database. You can optionally supply parameters to specify a different server or database. To view help on deploying with bm, type the following command -

bm help -Command:deploy-all

Ok, I'm off to go BAMing now.

Cheers,

Todd

More on MSI Deployments - Launch Conditions....

2005-10-11T14:39:00-05:00:00

In talking with others, it appears that the NAnt vs. MSI deployment camps are pretty much split down the middle. I tend to lean towards MSI deployments for several reasons.

When dealing with MSI packages, it is often necessary to go beyond the somewhat basic BTSInstaller project that comes with BizTalk. I have posted before on custom actions to fill in the gaps that come with using the BTSInstaller. Another thing to think about when using MSI deployment is what happens if BizTalk is not installed to the default install directory (it's very common for enterprise clients to install software to a drive other than C:\). If you are deploying pipeline components, this is important as they have to be copied to the Pipeline Components directory of the BTS install directory. Another thing to keep in mind is if you create custom adapters. The tricky part with custom adapters is that they require registry settings. You can include a reg file for the installer to run, but it would be nicer to handle that for them. Additionally, one of those registry settings is the path to the custom adapter component. Again, you can't be sure of the install directory of your components.

Luckily, it's pretty easy to handle both of this situations following the steps below -

Determining the BizTalk install directory

This one turns out to be pretty easy as the BTS install directory gets stored in the registry. And, the MSI install project gives us the ability to search the registry, so we just need to know what to search for.

In the install project, right click and choose View|Launch Conditions.
Name the launch condition BizTalk Install Path.
Enter BIZTALKINSTALLPATH for the Property property. We will use this property later.
For the RegKey property, enter SOFTWARE\Microsoft\BizTalk Server\3.0.
For the Root property, choose vsdrrHKLM from the dropdown.
For the Value property, enter InstallPath.

Now that we have set up a launch condition to find the BizTalk install path, we can make use of that value in our install process. For pipeline components, I like to add a new folder to the File System section of the install project and name it Pipeline Components. In the DefaultLocation property of that folder, enter [BIZTALKINSTALLPATH]\Pipeline Components as the value. You will notice [BIZTALKINSTALLPATH] is how we get to our properties of the install project. It's that easy.

Determining install path chosen during install process

For this section, I'm going to leave out most of the steps you need to take to add the registry settings to register your custom adapters. Contact me if you need that information. The easiest way to handle this is to run the wizard that creates the .reg file for you (you can find this tool on the web), and make sure you add all those keys in the Registry section of the install project.

Two properties to take particular note in are the AdapterMgmtAssemblyPath and the InboundAssemblyPath. These properties point to specific paths of your components (generally the same component). So, we need to somehow capture where the installer has chosen to install our components, and then make use of this value.

Right click on the installer project, and choose View|File System.
Select the Application Folder folder on the left side. Notice that the property (which is read only in this section) is set to TARGETDIR. This is similar to our BIZTALKINSTALLPATH property that we created.
Right click on the installer project, and choose View|Registry. This is where you will enter all the keys and values you need for your custom adapters. When you get to the AdapterMgmtAssemblyPath and the InboundAssemblyPath properties, you will prepend you assembly name with [TARGETDIR] and enter that for the value of the key.

Remember, when creating install packages, we want to make the install process as “painless” for the installer as possible. These folks are typically not BizTalk people, and even worse, the process is usually automated. Hopefully these little tips will make your MSI package creation a little easier.

Cheers,

Todd

Where does WSE 2.0 fit into BizTalk 2004?

2005-09-04T15:39:00-05:00:00

Over the next few posts, I'm going to explore where, and how, WSE 2.0 fits in with BizTalk 2004. I know some of you are thinking, well, we have this wonderful WSE adapter that allows us to retreive SOAP messages and have all the (or at least some) of the benefits of WSE (WS-Addressing, WS-Policy, etc). That's true, we do... BUT, what happens if we want to receive messages over a transport other than HTTP? Many times, in a SOA environment it is necessary to support guaranteed delivery. This is not possible over HTTP. WSE gives us the ability to define our own custom transports. Therefore, from a WSE standpoint we could receive messages via MQ to support our guaranteed delivery requirement. If we do that then we can't use the WSE adapter.

On my current project, I'm exploring ways of dealing with WSE compliant messages, without using the WSE adapter. This of course will require development of a custom pipeline to deal with some of the things the adapter takes care of like handling the WS-Addressing headers, digital certificates, etc. Another thing I'm exploring is implementing WSE pipelines inside of a BizTalk pipeline. This allows you to add custom filters to the WSE pipeline through configuration...pretty powerful. I'm not sure what the implications, if any, are however.

Stay tuned as I post my findings!!

Cheers,

Todd

Our friend BTSSubscriptionViewer.exe.....

2005-03-09T03:46:00-06:00:00

Anyone who has done anything with BizTalk has run into the “could not find a matching subscription error” at one time or another. Keep in mind that under the covers, BizTalk operates under a Publish/Subscibe architecture. What this message is telling us is that nothing has subscribed to our message with it's current properties (namespace, root element, port, etc). This is where the BTSSubscriptionViewer comes in.

This handy little, often forgot about tool can be found buried deep within the /Program Files/Microsoft BizTalk Server 2004/SDK/Utilities directory. Once you launch the tool, select Load under the Main menu to load all the current subscriptions (remember, subscriptions can constantly change due to Convoy patterns, dehydrated orchestrations, etc).

The top of the window will display the current subscriptions, and bottom pane will display detailed information about the specific subscription. So, how is this going to help us? Well, if you find your orchestration in the top pane, chances are there is an issue with the subscriptions information. For instance, perhaps your message has the wrong namespace, or root node, so it doesn't match the subscription in the format of http://YourNameSpace#MessageType . Keep this in mind when dealing with Multi-part messages. I have seen people have the parts reversed, so when the subscription is created the body of the message type is pointing to the wrong body part. Also, if you are using Correlation, this will be helpful to see what the correlation value has been set to (don't forget to set your follow correlation set!!!).

So, remember, when your subscriptions are not found, launch the BTSSubscriptionViewer tool to find out what's going on under the covers.

Exception occurred when persisting state to the database.....

2005-06-07T14:30:00-05:00:00

The chance that you receive this error for the same reason as I may be slim, but I thought I would offer up anyway. If you receive this error, and read the entire error message in the event viewer, it should give an indication of the cause. In my case, this was “no subscribers were found”. Of course I headed to our fried the Subscription Viewer. What I discovered is that when you enlist a dynamic send port, it creates a subscription for every adapter you have registered. In my case I had gotten a little ahead of myself and had deployed a project with an orchestration that made use of a custom adapter that I had registered AFTER deploying the orchestration project. To be safe, I undeployed the orchestration project, registered the adapter, and redeployed the orchestration project. All is good.

Cheers,

Todd

I'm still here.....

2005-05-11T04:51:00-05:00:00

So, I'm looking at my blog today, and I see I haven't posted anything in 2 months and a day. Shameful!!! Anyway, I'm climbing back in the saddle.

In what little free time I had before, I was working on a whitepaper for MSDN for the BizTalk Adapter for SQL Server. I'm glad to say the paper is complete, and should be on the MSDN site very shortly. I will post some examples from the paper here very shortly.

The question came up at work today about debatching in Orchestrations. This is a very common scenario in business processing, or at least the issue of dealing with batch style messages is. Stephen Thomas has done a great job in writing up some examples of how to deal with this. So good, that I won't bother to recap. Anyway, there are a couple of important things to keep in mind when considering your options.

Is it important that “good“ messages must be processesed, and “bad“ messages be dealt with?
Are the file sizes significant?
Does the service level agreement have specific time constraints?
Does the trailer and header (if flat file messages) of the message have to be dealt with?

Many times you can't let one single bad record cause an entire file not to be processed. If this is the case, you will have to deal with your messages in another fashion. One option that is common is to debatch, or “shred“ your messages in the orchestration using XPATH. You will notice from Stephen's examples that when using XPATH to debatch messages inside your orchestration, peformance starts to degrade as the file size grows. I suspect what is happening is that as the XPATH iteration gets deeper in the nest, it is still starting at the beginning of the file, so it slows over time.

If you have larger files and have a time constraint, there are still a few options available to you. Since XPATH seems to be slow with large files due to it's apparent scanning of the entire message each time, you may want to try Streaming XPATH. This option has all the benefits of XPATH, whithout the negative side effects. Look at this information to see if it might work for you.

There are still a couple of other options that may be available to you. I will start to post some more thoughts and examples on this topic in the next several weeks.

Cheers,

Todd

Quick Tip - Deploying other's projects....

2004-12-06T22:51:00-06:00:00

Ever get someones project out of source safe (or any other way), go to deploy it and get a 'SQL exception: SQL Server does not exist or access denied' error?

Go to the project properties, configuration properties, deploy, and make sure it's pointing to your database (typically (local) ).

Todd

Atomic Scope "Batch" property...

2004-11-24T03:31:00-06:00:00

Charles Young has a great post on transactions within orchestrations. In it he mentions the Batch property of the ATOMIC scope. I concur with him that the documentation on this is very light, as well as any information out in the communities. Doing a little testing on my current project with Stephen Thomas (be sure to check out his blog), it has become apparent that the property acts just as Charles suspects.

Our particular scenario -

We had an orchestration processing a large sum of messages. At one point in the orchestration, we were calling a component to insert the records into the database. While that orchestration was processing the large batch of messages, we sent several very small batches of messages that quickly reached the same point in their instance of the orchestration (inserting the records through the component). It turned out that the smaller messages would not complete processing thier messages until the large batch completed.

After we changed the property to be False (the proprety for some reason defaults to True, which seems a bit dangerous), redeployed and rean the test again, the much smaller messages completed ahead of the slower, larger batch of messages (our expected outcome originally).

So, keep in mind when using the Atomic transaction that this property defaults to True, and may have ill effects on your message processing.

Cheers!!

More on deployment...

2004-11-09T02:26:00-06:00:00

Ok, sorry for the short hiatus….very busy times on my current project. Anyway, I previously mentioned some points around deployment. It turns out that using BTS Installer doesn’t work well when utilizing roles and parties. This doesn’t mean we have to scrap the BTSInstaller all together, we just have to do a little extra work in creating scripts that do the deployment as well as the undeployment, and then call those scripts from our MSI package.

In the zip file (get it here), I’ve included some sample scripts that may give you a head start in the deployment process. Keep in mind that if you are NOT using roles and parties, BTSInstaller will get you about 80% of the way there, and you will not need all of the scripts. You will still need some scripts to go ahead and enlist and start everything, and some work is needed to automagically undeploy everything using Add or Remove Programs. With everything included in the zip file, you should be able to gleam something that will help.

A few key points –

Binding files – everything is driven by the binding file. Make sure it is correct, or you will have issues. One thing to watch out for is your binding file contains the PublicTokenKey of the assembly it is derived from. If during multi-developer development these somehow get out of sync, when you deploy you won’t get any errors, but nothing gets deployed either.
Deleting parties – if you are implementing parties and roles, you will have to un-enlist the parties and delete them prior to un-deploying the assemblies that contain the orchestrations. Included in the SDK are 2 sample projects that when compiled create one exe to un-enlist the parties, and one exe to delete the parties. These are coded assuming that the SQL database where your management database resides is local to BizTalk. This is most likely not the case, so you will have to edit this code to determine what server the database is on, and supply that in the connection string. It’s quite simple using MSI, and if you need some code snippets just send me a comment.
Removing orchestrations – running the MSI to delete orchestrations (if using the default BTSInstaller) won’t remove your Orchestrations since they are in the running state. Chances are if you’re testing they are even in the active state. In the supplied StopOrchestrations script, there is a flag you can pass into the Unenlist method that will terminate any active instances. Pay attention to this if this is not your intended actions.
Write to the event log – you will notice in my scripts I have added some code to shell out and use the LogEvent method to write errors to the Application Log. With the nature of VBScript, if you don’t do this and run into errors, you will never figure out where to start looking.

Use the scripts at your own risk. They are commented, so you should be able to read along and see what’s going on. Remember, if you are not using roles, you won’t need half the stuff in the Install.bat file, and some of the stuff in Cleanup.bat, but read through those to see the order things are called, and you will need some of the other scripts regardless.

Cheers!!

BTSInstaller "bug"

2004-10-13T04:40:00-05:00:00

After hours of fighting it, I have concluded that deploying a solution that utilizes Roles and Party's can't be installed using an MSI created with the BTSInstaller. You can use the same binding file and assemblies and use BTSDeploy and it all works fine.

I plan on posting all my deployment scripts here in the next few days as the BTSInstaller gets you about 85% there, and you're required to script the rest.

Take a look at Scott Colestock's blog for a different option (Nant) to the BTSInstaller.

Cheers,
Todd

Shared Config Files........

2004-10-08T15:41:00-05:00:00

BizTalk server relies on a config file to store certain application information. This config file is located in \Program Files\Microsoft BizTalk Server 2004 and is named BTSNTSvc.Exe.Config. View your file here (DO NOT CHANGE THIS FILE WITHOUT BACKING UP!!!).

Let's say that you have some .NET components that you want to consume from your BizTalk solution. Let's also assume that these components are possibly shared with another solution, and make use of a config file for things such as connection strings, directory locations, etc., a fairly common situation. Now, keep in mind when you call a component, and that component makes use of a config file, it uses the config file of the calling application (at least for EXE's). So, we have a couple of options. We can take what's in the shared components config file and paste it into the BTSNTSvc.Exe.Config file. But, this means we now have 2 places to maintain the information. What if we then have a multi server configuration, we now have 2 config files to maintain in more than one location. Not good. Wouldn't it be nice if we could somehow have the BizTalk config file "point" to the components config file? Then, we could maintain the other applications config file as we normally would. Well, the good news is you can.!!!

Take a look at my sample config file to follow along. I'll explain a couple of the sections. It is important to remember to always take a copy of you current BTSNTSvc.Exe.Config file before making any changes as changing it incorrectly will prevent BizTalk from running.

The key to making this work is the section. To make this section "available", you will have to add the following under the section of your config file -

Now that we have this in the config file, we can add the section . I won't copy it here to save space, so look at the example. This section is where all the magic happens. Some of the attributes like AssembliesPerDomain, etc you can find information about in the help documentation. The key sections to be aware of are the section, section, and the section.

This section defines a new domain for your particular situation to run in. *** define this a little better. You will give your domain a name, and then you will use this name later to correlate the calling piece of your BizTalk solution with the called components config file.

This section defines where the called components config file resides. You can use absolute paths, or I believe you can use relative paths also.

In this section you are going to link your AppDomain (which contains the ConfigurationFile section) to the part of you BizTalk solution that calls the shared component. Let me clarify "part of your BizTalk solution". When you create your orchestrations, you give them a namespace. They probably run under the same namespace, or at least the same root namespace. So, in the example you see "Source.Test.Orchestrations.*" as the pattern assignment rule. What this is saying is that any orchestration that is under this namespace would be associated with that particular app domain, and would then be able to make use of the configuration file listed under that app domain.

Hopefully with the example and the explanation, this will make some sense. There is some information in the help file about adding the new sections, but they don't go into much detail of how all the pieces fit together, and what the pattern assigment rule is really supposed to point to, etc.

DON'T FORGET...BACK UP YOUR CURRENT CONFIG FILE BEFORE MAKING CHANGES. This will save a lot of headaches.

One last point to remember...and I won't go into a lot of detail, but remember that if you are calling components from BizTalk, they must be GAC'd. It is possible that the shared components you are calling didn't require to be GAC'd prior to calling them from BizTalk. Now that they are in the GAC however, you will need to change your shared components config file slightly. When you are defining your section for that component, you will have to change the attributes a bit so that you are including the following -

The values for component name, version, culture and token you can get out of .NET Framework Configuration admin tool. Just look for you component, and make note of the values listed. The only one that should change would be the version.

Role Link Sample.....better late than never.....

2004-10-05T04:23:00-05:00:00

I applogize for being anything but prompt on this.

The zip file can be found here.

Steps for setting up Role Link Sample -

1. Create a receive port in BTS Explorer
2. Create 2 send ports - RoleLinkTest_Approver1 and RoleLinkTest_Approver2
3. Set your transports, send or receive locations, etc for your ports
4. Create a party called Approver1. Choose RoleLinkTest_Approver1 as their send port.
5. Create a party called Approver2. Choose RoleLinkTest_Approver2 as their send port.

At this point you should be able to deploy the project. Once the project has been successfully

deployed, you should now have a new Role listed in BTS Explorer called

RoleLinkSample.RoleLinkTestOrchestration.Purchasing.Approver. This is from the role link type we

created earlier.

6. Right click on the new Role and select "Enlist Party". Choose are 2 parties. From this point

forward, if you wanted to add new parties (for new trading partners perhaps), you would just repeat

steps 4,5 and 6...it's that easy!!!

Now you can run the test files through. Read back through the previous post on role links, and

follow along this example (pay attention to the expression shape that initializes the ports).

Hopefully it will make a little more sense.

Please let me know if you have any questions, or if the setup instructions are lacking.

ooops, almost forgot..take a look at the 2 files you'll be submitting, and don't forget to point the project to a keyfile.

Todd

Hosts and Host Instances

2004-10-02T15:59:00-05:00:00

Before I begin, I promise I am working on getting my role link example up here. There was some confusion with server space to upload the example to.

Ok....I know when I first started with BTS'04, it was slightly confusing to me what the whole host/host instance thing was. A fairly new concept from the previous versions. But, once the concept sinks in (things take longer for me ;-) ), it's actually extremely useful.

So, we know that the Host is a “logical container”, and the Instance is the “physical container” for all your adapters, orchestrations, etc. What can we do armed with this knowledge? When we're developing, everything runs under the default BizTalkServerApplication, and life is good. But, we need to start thinking about production, and how this solution is going to run efficiently, be highly available, and be easily scalable. This is were hosts and host instances come in.

It is best to decide on your hosts in the very beginning when planning your production environment. Take a look at the design of your solution, look at the expected message volume, the different methods of receiving and sending messages, service level agreements, and hopefully by now you will have tested some of your design to get an idea of performance. I find it is much better to create hosts down to the granular level. By this I mean if you are receiving messages real-time, some as batch, some as file transports, some as HTTP, break your hosts down accordingly. If you have some orchestrations that perform worse than others (a convoy perhaps), take that into account. It doesn't necessarily mean you have to create a host for each one, but lets say you your batch messages take a lot of processing power...put those in their own host so you can isolate them and not affect your realtime messages. Breaking your hosts down at a granular level will make it much easier to reconfigure your system once you are stress testing and you find you need to re-shuffle your configuration to increase performance. This is because once you have your hosts created, and you have configured your adapters, orchestrations, etc to run in those hosts, all you need to do is create a new instance through BizTalk Admin console on whatever server you want it to run on. Let's say that you have one instance of your host for batch files on a server (which you wouldn't do because you want to account for failover also....), and that server is heavily taxed, slowing throughput of all your batch files. It is very easy to put another server out there, and simply configure a new host instance on it. If we had not separated our hosts initially, it would be much more difficult to now split out that functionality.

Another good reason to create different hosts is for security reason. I won't go into detail here, but remember that hosts can be associated with different windows groups, and host instances can be configured to run as specific accounts.

As you can see, hosts and host instances provide us with load balancing. BizTalk internally handles load balancing through instances of the host. If you have instances of a host running on 3 separate servers, then BTS will handle dividing the load amongst the 3 boxes. If one box were to fail, then the 2 remaining boxes would pick up the slack. Similarly, if you need to add another box to decrease the load on the current 3, you can just configure a new instance of the host on a server, and BTS will now balance the load between the 4 boxes.

So, remember to seriously think about your hosts early on. Think in terms of performance, functionality and scalability. Read the paper on High Availability for BizTalk '04.