BizTalk Build & Deployment

BizTalk Build Generator - New Release for BizTalk 2009

Michael Stephenson — Tue, 18 Aug 2009 23:24:50 GMT

Just to let people know ive released a new version of the BizTalk Build Generator which can be used with BizTalk 2009 and also contains a few new features and addresses a couple of bits of feedback provided.

Details are on the following link:

http://biztalkmsbuildtool.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=31725

Using the BizTalk Build Generator SSO Tasks

Michael Stephenson — Mon, 17 Aug 2009 22:37:23 GMT

In the BizTalk Build Generator there is a tasks library containing tasks which will allow you to easily extend the generated build scripts with some of the common scenarios. Previously I have discussed using the BAM and BRE tasks and this post will show how to use the SSO tasks.

Within the library there are the following tasks which are aimed at configuring SSO (primarily for the configuration application scenario).

Task	Description
Delete Application	Deletes an SSO application
CreateConfigStoreApplication	Creates an SSO application configured to act as a config store with a delimited list of the required columns
PopulateApplicationProperty	Populates a column value within an SSO application
PopulateApplicationPropertyFromFile	Populates a columns value in an SSO application from data in a file. This is useful if you want to store xml in the field for configuration as discussed in many SSO configuration articles.
EnableApplication	Enables the SSO application

The tasks cover most of the common uses. The below picture taken from a test for the tasks demonstrates the use of the tasks.

Normally you would use the SSO tasks in the custom build files and do the delete application in the clean up target and then create and enable to application within the setup targets.

Hopefully this post shows that it is easy to use these tasks.

Using the Build Generators BRE Tasks for MsBuild

Michael Stephenson — Mon, 17 Aug 2009 22:22:31 GMT

In the BizTalk Build Generator, in addition to the normal build scripts which you can produce there is also a library of MsBuild tasks which help you to implement some of the common scenarios for extending your build process. In this post I will discuss how you can extend the build to include the management of BRE artefacts within the build.

In the tasks library you will find the following tasks:

Task Name	Task Description
BizTalk.BuildGenerator.Tasks.BRE.BackupBRE	This task is used to backup all of the current artefacts within BRE before you build goes making changes. This means if you make some kind of mistake you can easily restore it to its previous state.
BizTalk.BuildGenerator.Tasks.BRE.UndeployPolicy	This task will undeploy a policy from BRE. The task supports undeploying all or specific versions of the policy.
BizTalk.BuildGenerator.Tasks.BRE.RemovePolicy	This task will remove a policy from BRE. The task supports removing a specific version or all versions of a policy.
BizTalk.BuildGenerator.Tasks.BRE.RemoveVocabulary	This task will remove a vocabulary from BRE. The task supports removing a specific version or all versions of a vocabulary.
BizTalk.BuildGenerator.Tasks.BRE.ImportVocabulary	This task will import a BRE vocabulary
BizTalk.BuildGenerator.Tasks.BRE.ImportPolicy	This task will import a BRE policy
BizTalk.BuildGenerator.Tasks.BRE.DeployPolicy	This task will deploy a policy to BRE. The task supports deploying a specific version of the policy or all versions of a policy.

The tasks basically just leverage the underlying assemblies which are used by the rules deployment wizard.

The below sample taken from some of the build generators tests demonstrates how to use some of the tasks.

<?xml version="1.0" encoding="utf-8"?>

<BizTalk.BuildGenerator.Tasks.BRE.BackupBRE FolderPath="."/>

<BizTalk.BuildGenerator.Tasks.BRE.UndeployPolicy PolicyName="BizTalkBuildGeneratorTestPolicy"/>

<BizTalk.BuildGenerator.Tasks.BRE.RemovePolicy PolicyName="BizTalkBuildGeneratorTestPolicy"/>

<BizTalk.BuildGenerator.Tasks.BRE.RemoveVocabulary VocabularyName="BizTalkBuildGeneratorTestVocabulary"/>

<BizTalk.BuildGenerator.Tasks.BRE.ImportVocabulary FilePath="TestVocabulary.xml"/>

<BizTalk.BuildGenerator.Tasks.BRE.ImportPolicy FilePath="TestPolicy.xml"/>

<BizTalk.BuildGenerator.Tasks.BRE.DeployPolicy PolicyName="BizTalkBuildGeneratorTestPolicy"/>

</Target>

</Project>

Creating your own build script provider for the BizTalk Build Generator

Michael Stephenson — Mon, 17 Aug 2009 22:22:10 GMT

One of the new features of the BizTalk Build Generator is a provider pattern for the build script generation. This means if you prefer to use a different format/structure in your build scripts or even don't like MsBuild and would rather use NANT, Powershell or traditional batch scripts then you could create your own provider which would take advantage of the metadata used by the BizTalk Build Generator and then produce scripts in your own format.

In order to create your own provider you must create a class that implements the BizTalk.BuildGenerator.IBuildProvider interface shown in the below picture.

In your build provider you will need to implement the CreateBuild method which will write out all of your build scripts. The GenerationArgs class which is passed in will contain all of the metadata about the build you have described through the Build Generator interface.

If you would like to see how the default build provider has been implemented the source code for the Build Generator tool can be downloaded from the CodePlex site.

Id be really interested to hear from anyone who ends up implementing their own provider and maybe you would like to contribute this to the community project.

Solution file version issue when migrating your Build Script to BizTalk 2009

Michael Stephenson — Sat, 04 Jul 2009 23:20:10 GMT

Ive just been working on upgrading the build generator to support BizTalk 2009 projects. While doing this one of the things i wish to do is to change the compile step to use MsBuild rather than dev env.

I found a little issue which others who use MsBuild may come across when migrating custom scripts.

Previously for BizTalk 2006R2 scripts I used to use the below command:

<Exec Command='"$(DevEnvPath)" $(SolutionName) /Build $(ConfigurationName)' />

This would compile the whole solution using devenv.

I changed the script to the following:

<MSBuild Projects ="$(SolutionPath)" Properties="Configuration=$(ConfigurationName);" />

and also changed the property on the project to use ToolsVersion="3.5"

This is how I expected the build to need to be and it would mean our script would take advantage of using the MsBuild task inline. When I ran the script I get the following error:

"Solution file error MSB5014: File format version is not recognized. MSBuild can only read solution files between versions 7.0 and 9.0, inclusive"

My initial thoughts made me think this was more complicated than it actually was. I was looking at using the Exec command rather than the MsBuild task for the compile which I could get to work. Then suddenly the answer hit me.

The problem was that I forgot that the .cmd file I use to call the MsBuild file was configured to use the v2.0 version of MsBuild rather than 3.5. As soon as I corrected this it worked fine.

So if you have the above error remember to double check the original call to start your build script wether it be from TFS/Cruise Control or a custom file etc.

Dynamic BizTalk Versioning Strategy with Cruise Control

Michael Stephenson — Thu, 18 Dec 2008 22:03:04 GMT

Article Source: http://geekswithblogs.net/michaelstephenson

Versioning is one of the more difficult aspects of a BizTalk project mainly because of the ability to deploy side by side versions of the components within a BizTalk application and the fact that often a BizTalk project would require long running processes making it more difficult to do deployments. There are a couple of useful resources available on this area:

In his article Versioning and Re-Deploying BizTalk Server 2006 Applications Richard discusses some different scenarios about when you deploy side by side versions of BizTalk and makes some interesting observations. If you are new to versioning for BizTalk then this is good background reading.
In the MSDN documentation for BizTalk there is a page which discusses versioning and the 2 main strategies for it. This is useful but from my experience I think it misses a couple of things and is wrong in relation to the BAM files
This web cast includes a discussion around BizTalk versioning (mainly focusing on how to partition your BizTalk artefacts)
Nick Heppleson wrote an article about how you can use assembly redirection with BizTalk
There is also a new article on MSDN which provides some information similar to this article called Developing Integration Solutions using BizTalk Server 2006 and Team Foundation Server. Although again they explain how you could do versioning but don't seem to walk through their approach.

While all of these articles are great there is a gap in that nothing really talks about how to implement a versioning approach in your development process. In the rest of this article I will discuss some of the different considerations and how we implemented dynamic versioning for our BizTalk projects at my current project. I will also provide links to a sample which shows this.

Considerations for Versioning with BizTalk

There are a number of different things you should consider for your versioning strategy which are discussed below.

Type of Versioning

There are two main types of versioning which are discussed in the MSDN article above:

Static Versioning

In static versioning you would normally keep the Assembly Version Number the same all of the time and only increment the Assembly File Version with each build. This is the most common type of versioning approach I have seen used.

Dynamic Versioning

In dynamic versioning you would increment both the Assembly Version and Assembly File Version with each build. The most common way I have seen this implemented is to manually change these version numbers (this subsequently requires a number of changes to files within the project)

Deployment Pattern

The way you plan to deploy your application can have an affect on the versioning approach you might use.

Replace application every time

Some BizTalk applications implement processes where there are clear windows where a deployment can be done. There are no significantly long running processes and it is easy to drain the system of in process messages. In this case when you deploy a new version of the application you would be likely to completely remove the existing BizTalk application and then add the new version.

This approach is well suited to either of the 2 types of versioning.

Upgrade application

If you have a BizTalk project where you can not get a clear deployment window and you can not drain the system then your deployment approach needs to be able to support running multiple versions of processes at the same time. In this case you are essentially applying updates to your existing application and when the older instances have completed you may remove the old assemblies.

Latest Version for Development

One of the challenges is how within your code base you will implement these side by side versions and develop/test against them. When we deploy an official version we will take those assemblies and add them to an external libraries folder within our code base. When we build the code we will also deploy these older versions of the code and then run tests against this combined solution. One key thing we do is that we version the BizTalk project files in the code base to be version 999.999.999.999. When we do a local developer build we do not change the version numbers we only do this on our build server when the build is ran by Cruise Control. We need to ensure that the assemblies built from the code will always be the latest version when doing a developer build so we chose this approach.

What types of files do I need to care about for versioning?

There are a number of different file types you need to care about when applying versioning. In my opinion the above article on MSDN doesn't cover everything it needs to so here are my thoughts below:

Binding Files

We already use an approach of tagging the binding files to support changing the configuration for different environments based on stuff I have discussed before in other blog posts. In a binding file any references to assemblies will typically use the strong name for that assembly. When we apply dynamic versioning in the development process we will need to update parts of the binding file in each build to ensure the binding file continues to be valid.

If this is not done correctly the we could get problems importing a binding file.

Web Service Description Files

If you publish any web services from your solution then you may choose to save the web service description files and then regenerate the web services as part of your build. We save these files and have MsBuild tasks to support regenerating these web services with each build for WCF/WSE2 and standard SOAP web services. When we apply version updates we need to amend the WebServiceDescription.xml file that is published before we do the build.

Map Files

BizTalk Map files may contain references to assemblies if you execute an external assembly using the scripting functoid. We need to ensure these maps are updated if the assembly version is changed. If we do not do this we may get errors while executing the map.

Pipeline Files

BizTalk Pipeline files may have references to external pipeline components or schemas or other assembly references in custom configuration. If we apply dynamic versioning we need to be aware of the impact on these files and make appropriate updates to ensure the pipeline still works when it is compiled.

BAM Files

If you use BAM on your project then your solution may contain Tracking Profiles or BAM Observation Models. The BAM Observation Model does not contain information relating to assemblies, but if you use any Tracking Profiles (.btt files) then they may contain references to assemblies which may need to change as you change the version number. In the MSDN documentation it indicates that .btt files are in binary format which Im not sure is right as the ones on our project are clearly XML files. Based on our experience you can just update these files to change the version numbers.

One side comment is that you can also use the Typed BAM API project on CodePlex which will mean you don't have to worry about Tracking Profiles.

BRE Policies

If you manage your BRE policy as an XML file which you import and export then you may also need to update version numbers in it if you reference assemblies.

Configuration Files

Some configuration files in your solution may have assembly references in them. You may need to update the version numbers here too.

Our Versioning MsBuild Tasks

As you have read so far there are a number of different tasks which need to be done to implement versioning. We have developed the following list of MsBuild tasks which help us to do this in our MsBuild process. Note the code for these tasks is available in the accompanying sample.

BizTalkProjectAssemblyInfo

This task will look for any .btproj files within your solution directory (including sub directories) and then update the version attributes of the project file to take the value from Cruise Control

AssemblyVersion

This task will create a file called VersionNumber.cs which contains the C# attributes for versioning an assembly. All of the C# projects within our solution reference this file and pick up its version number when they are built.

ApplyVersionToAdhocFiles

This task is given a list of files which need to be inspected for versions to be updated. The task will use the assembly name regular expression, assembly public key token, assembly culture and assembly version regular expression to identify strong names within the file that need to be updated. Once a strong name is found the version number part of it is updated to the version passed into the task by Cruise Control.

Note that this task is also used to update any BRE policies which are stored as XML files in the solution.

ApplyVersionToWebServiceDescription

This task does the same as the apply version to adhoc files task however this task needs to force the file to be rewritten in a the right encoding for the web service publishing wizard to use.

ApplyVersionToBindingFiles

This task is given a list of paths to binding files which it needs to update. In the binding files it will look for strong names in the same way as the adhoc files task and update them. In binding files there are also some other formats in which assembly version numbers are stored so this task will also handle these to ensure that the binding file is correctly updated.

ApplyVersionToBizTalkArtefacts

This task will start at the root directory of the solution and look for all BizTalk artefacts which may require updating (Maps, Pipelines, Tracking Profiles). For each of these file types it will look for assembly references based on the strong name patterns provided and apply the updates to the version numbers which come from Cruise Control.

Walk Through

Setting up a new solution to use dynamic versioning

When we are starting with a new solution and we want to be able to implement the dynamic versioning straightaway there are a couple of standards we use.

C# Project Files

In our solution it is common to have C# projects to compliment our BizTalk projects. We take the approach of versioning these assemblies by adding an existing file to the project which includes the below code snippet (we usually call this file VersionNumber.cs and it is located in the root folder of the solution). This means that the assemblies are all versioned to the standard 999.999.999.999 during development

When our MsBuild process executes it will overwrite this AssemblyVersion.cs file and replace the version numbers in it to the version label taken from cruise control. We use our AssemblyVersion custom MsBuild task. There are a number of different tasks in the community and Microsoft.Sdc tasks which can do versioning of C# projects and using any of these should be sufficient.

BizTalk project version numbers

On all of the BizTalk projects within the solution we will amend the project properties (see pic below) to set the version attributes to the fixed 999.999.999.999. This will mean during a local developer machine build all of the BizTalk assemblies will be versioned as 999.999.999.999. When Cruise Control runs a build the extra build tasks will run and execute our BizTalkProjectAssemblyInfo custom MsBuild task. This will update the version numbers of all of the BizTalk project files within our solution to apply the version number from Cruise Control.

Other Artefacts

As we will have applied the 999.999.999.999 version number to our projects before we start adding artefacts to the projects then any files we add will correctly pick up these versions and we will not manually need to update them.

The Cruise Control Integration Bit

In the below picture you can see the Cruise Control configuration used for the Continuous Integration of this solution. In the tasks bit you can see we basically call 3 MsBuild scripts as follows:

Apply versioning to the files in the solution
Perform a debug/development build of the solution (including executing tests)
Perform a release/deployment build of the solution (including executing tests)

The bit we have added is the first step where we will call ccnet.targets MsBuild script. This file is in the solution and is where we will call the tasks to apply versioning.

Setting up the CCNet.targets File

The below picture shows the contents of the ccnet.targets file. This file will basically call the list of MsBuild tasks in the ApplyVersion target which will apply versioning to all of the appropriate files. Each of these tasks was discussed in detail earlier in this article.

In the property group at the top of this file you can see that we have defined some common attributes which each of the tasks uses to do its job. These properties are used as follows:

Property	Notes
AssemblyNamePattern	This is a pattern used to identify the name of assemblies. In our case the assemblies will usually be called something like Acme.Something.BizTalk…. We use the regular expression Acme.Something.BizTalk.?\w* which will find assemblies with the right names for us for example: Acme.Something.BizTalk Acme.Something.BizTalk.Orchestrations Acme.Something.BizTalk.Schemas
AssemblyCulture	All of our assemblies have the culture neutral so we use this as a static value in the script
AssemblyPublicKeyToken	All of our assemblies have the same strong name applied usually so we can just keep this as a static value of the public key
AssemblyDynamicVersionPattern	This property is a regular expression which you will use to identify version numbers that need to be changed. In our case because we use the fixed version 999.999.999.999 to indicate it needs to be dynamically changed we just use this value. Having this as a regular expression however allows some flexibility on how you might use this

As you can see from the order of calls to our tasks we version the files in the following order:

BizTalk project file
C# project file
Adhoc Files
Web Service Description Files
Binding Files
BizTalk Artefacts

The order here isn't really that important as long as all of these activities are done before compiling the solution.

Converting an existing solution to use dynamic versioning

This walk through will explain how to configure an existing BizTalk solution to use dynamic versioning rather than static version approaches described above. One key point to reiterate here is that on a developers machine we always do builds as our fixed 999.999.999.999 version number and it is only on the continuous integration build server that we run the additional tasks to do full versioning as described below.

Reconfiguring parts of the solution

C# & BizTalk Project Files

We will need to update the BizTalk and C# projects to reference the 999.999.999.999 version number. We can do this the same as how is described in the section about setting up a new solution for dynamic versioning.

Updating Map Files

Maps are slightly strange. In the map if you reference an external assembly and execute it via the scripting functoid then you need to be aware that the strong name for executing assembly may need to change. In the MSDN documentation explaining this it indicates you need to change maps, but if you disassemble an assembly containing maps and the C# project was referenced as a project assembly then the compiler seems to change the xslt within the map for you. However as this is a little unclear we decided to change the maps anyway.

To do this we needed to open the map using the XML Editor in Visual Studio. We then would change the version numbers highlighted below as 1.0.0.0 (in the pic below) to 999.999.999.999 for all of the assemblies we know we are going to dynamically version.

We would do this for every map in the solution. When Cruise Control runs the build it will modify and strong names in .btm files which need to be versioned.

Pipeline files

Pipelines are similar to maps in that under the .btp file is XML which may contain assembly strong names (see picture). Again we would open the file using the Visual Studio XML Editor and change the version numbers from 1.0.0.0 to 999.999.999.999 for any references to assemblies we are going to dynamically version.

When the build is ran by Cruise Control our additional tasks will find all of the details we need to change and modify the .btp file before it is compiled.

Binding file template

In the solution we have a template binding file which contains tags which we use to configure the binding file for different environments. In a binding file there are lots of references to assemblies by their strong name. If we have any assemblies which we know are going to be dynamically configured then we will replace their version with the fixed 999.999.999.999. In the below picture you can see an example binding file template which shows the version numbers before they have been amended.

When Cruise Control runs our build then an additional step is ran which will execute our ApplyVersioningToBindingFiles task. This will modify the binding file correctly to replace the version numbers with the appropriate label from Cruise Control.

Configuration Files

In some of our solutions we include configuration files which may have assembly version numbers in which need to be dynamically versioned. We follow the same approach to make the version number 999.999.999.999 if it needs to be dynamically configured.

In the Cruise Control build a task called ApplyVersionToAdhocFiles is called passing in a list of these adhoc files.

Web service description Files

In some of our solutions where we publish web services from BizTalk we use a couple of MsBuild tasks in our build process which will support regenerating these web services each time the build is ran by using the WebServiceDescription.xml which is produced by the Web Service Generator Wizards. We have MsBuild tasks which support WSE2, SOAP and WCF.

When we use this technique we also need to apply updates to any version numbers which may need to change in the WebServiceDescription .xml. To do this we use the same approach of replacing the versions with 999.999.999.999 for the developer build and then Cruise Control will run the ApplyVersionToWebServiceDescription task to do the dynamic updates.

BAM Tracking Profiles

The BAM tracking profile is an XML file under the hood and we again replace the version number of any of our assembly strong names with 999.999.999.999.

When the Cruise Control build runs it will execute a task which will apply the Cruise Control version label in place of our version number and then when the build deploys BAM artefacts they will be already versioned correctly.

(For details of how we integrate BAM into our MsBuild process please refer to: http://geekswithblogs.net/michaelstephenson/archive/2008/06/15/122870.aspx)

BRE Policy Files

When we use BRE we usually take the approach of managing the policy as an artifact in the solution. We use custom MsBuild tasks to import and export the policy from BRE and store it in the solution as an XML file. If we reference any assemblies in the policy that need to be versioned then we will amend the xml that is exported from BRE and make the version numbers the usual 999.999.999.999. In terms of Cruise Control we treat the BRE exported XML file the same as an adhoc file which requires version updates. We then use the ApplyVersionToAdhocFile task to update this

If you would like to know more about how we implement the BRE features of our build process please refer to the following article: ???

!!!!!! vWhere is BRE post !!!!!

Check local build

Finally when we have done all of the above steps we will run a local developer build to see that the solution is fully versioned, compiles correctly and deploys. We also ensure that the build has ran our BizUnit tests and they are all successful.

The next stage is to get Cruise Control to so some extra stuff.

The Cruise Control Integration Bit

When modifying an existing solution to use dynamic versioning, the requirements for the ccnet.targets file and Cruise Control to implement the version updates is exactly the same as when you setup a brand new solution for dynamic versioning. For details please refer back to that section.

Summary

At this point you can see it is a fair about of work to modify an existing solution to use this versioning technique. I think the earlier you do it in your project life cycle the better. You can use things like Replace All to modify some of the version numbers manually when doing the above tasks but be careful when doing that as you may not want to change all version numbers.

Frequently Asked Questions

In relation to this topic the following questions have occasionally came up.

How do I do side by side versioning in development for a new major release?

The approach described in this article is entirely intended to support development of side by side versions of the same solution. Imagine you have version 1.1.0.0 of your application deployed and we know you want to now work on version 2.0.n.n. You know from your understanding of this topic that you will employ a deployment model which will mean you will have multiple versions of some of your assemblies deployed.

The way you could implement this is to add external Library folder within your solution store the version 1.1.0.0 versions of all of the deployed assemblies. You could also store the appropriate development binding files and any other artefacts to support this update. In your build script you might setup and deploy the existing application (v1.1.0.0) and then modify your build script to apply updates to the application by deploying some or all of your newly compiled code.

In development the newly compiled assemblies always have a very high version number they will always be the latest version and the versioning rules will apply correctly. You will probably also do things like modify binding files to be able to work with two versions of the assemblies. In the binding file remember that the way we do this only our assemblies which are versioned 999.999.999.999 are dynamically changed so if you add details of an older version of the assembly to the binding file we will not touch that instance of a string name as the version number within the string name will not be 999.999.999.999.

When your code is checked in and build on your build server Cruise Control would apply the correct 2.0.n.n (provided you have updated the major number in its config) version number and your build will allow you to have a correctly configured and working side by side versions of the application.

What's the 999.999.999.999 thing all about?

In applying dynamic versioning to the solution we need to be able to ensure that in our development environment a developer can work on a solution which may have side by side versions. We have chosen to use 999.999.999.999 to indicate this is a developer machine only version of the assembly and it is given a high version number to ensure that when it is build it will always be the latest. Although the build tasks accept a regular expression for the version number which needs to be replaced we find it keeps things simpler by always using this fixed number.

Because we use source control we can not apply dynamic versions on a developers machine local build because the files we need to modify will be read only. A developer will always build version 999.999.999.999 of the solution. When the developer checks code in Cruise Control will replace all of these developer 999.999.999.999 versions with the build label from Cruise Control.

BizTalk 2009 Update

Following the release of BizTalk 2009 CTP I have had a brief look at how the new version would affect this versioning approach. In my opinion the approach stays very much the same with the exception that now BizTalk project files are MsBuild based and you can use the AssemblyInfo file to version the assembly rather than the old BizTalk project file. Based on this the change to our implementation would be to not use the old VersionBizTalkProjectFile task and just use the C# one for BizTalk projects as well.

Summary

As you can see from this article this isn't a trivial undertaking. If you do it right from the start of the project then hopefully you will find things a lot easier.

Hopefully this article will plug the gap I identified at the start and there is now a detailed walk through of applying effective dynamic versioning to BizTalk projects which use continuous integration. If you would like to look at the code for the MsBuild tasks we used to help us implement this approach then please refer to the following location:

http://www.box.net/shared/7ebfbdjau0

(Note some companies block access to box.net so if you are unable to get the sample please email me via the blog)

Introducing the Configuration Management Tool

Michael Stephenson — Thu, 18 Dec 2008 20:27:02 GMT

A while back I discussed the approach we use on some projects to configuration management and how we solve the problem of configuring binding and configuration (and any other) files for different environments. I had originally written the linked article on my blog. While it was fairly successful the main project I've been working on has a number of BizTalk projects within it and I felt that the configuration dictionary approach we were using was becoming a bit of a pain as the XML dictionaries were awkward to maintain and you couldn't get a clear view of your configuration.

To help tackle this problem I wanted to create a tool to help with this, the aims of the tool were:

Provide a nice simple to use UI which would allow me to manage configuration settings for multiple code bases on large projects
Provide an easy way to integrate a build process into the use of this configuration store
Try to enhance the previous approach by allowing you to import configuration as the XML files which people may already be using.

The tool is available on codeplex: http://www.codeplex.com/ConfigSettingsTool

Here are a few points to describe the model for the configuration data:

The system will store information for multiple projects
Each project can have multiple environments associated within
Each project will have multiple solutions (code bases) associated with it
Each solution will have a list of settings which can have default values
For each solution within a project you can specify setting values for each environment, or inherit the default setting

In the rest of this article I will describe the use of the tool, and how we implemented it for our BizTalk projects. Note that nothing about this tool is BizTalk specific so you could easily use it for .net projects etc. Also in this article I have used screen shots from a real project but have had to censor some information from the screen shot.

Prerequisites

In order to run the tool the prerequisites are as follows:

The database should be setup on a SQL Server 2005 database (ensure you configure backups for your database)
The client application requires .net 3.5
The MsBuild tasks need to run on a machine which has .net 3.5

Getting Started

To set up the application you should take the following steps:

Setting up the database

In the zip file you will download there is a folder containing the database script. You will first need to create a database (which I'm assuming you can do already) and then use the CreateDatabaseObjects.sql script to setup all of the database objects by running it in query analyser.

Setting up the application

In the zip file there is an application folder. In this folder the ConfigurationManagement.Administration.exe.config file contains settings for the application. You will need to edit the connection string to ensure it points to your new database.

Once you have done this and saved the file you can now run the application by double clicking the ConfigurationManagement.Administration.exe file.

Using the Application

Now you are setup if you open the application you can begin setting up your configuration. If you open the Tools à Management menu you will see options for managing the different aspects described above.

Managing Projects

In the picture below you can see the screen for managing your projects. This is pretty simple, you can add/update/delete projects in the grid. Note any changes are not saved until you close the form.

I normally use 1 project per customer or major project of work. In this project you can configure multiple solutions and I normally relate each configuration solution to a visual studio solution.

Managing Environments

Within a project you can also configure multiple environments. As you can see in the below picture I have configured only a couple of environments. Examples of environments you might configure include (System Test, UAT, Production). The environments screen again is fairly simple in that it allows you to add/update and delete environments. Note again changes aren't saved until you close the form.

Managing Solutions

Within the Project you can also define the details of multiple solutions. The solutions form allows you to add/update and delete solutions. The changes are not saved until you close the form. The below picture shows an example of the solutions within this project.

Manage the Solution Dictionary

Once your solution is configured you can now set up its dictionary. The below picture shows an example of this. Within the solution you will create a list of setting tags. These tags are things you will put in files to indicate a special key that you will replace. We typically use the format $(-------) to indicate a key. For each setting you can define a default value which will be used if no specific setting for that environment is configured.

In the below picture if we ran this solution dictionary over a file we would look for each setting and replace it with the environment specific or default value. If a setting key could not be found then we will move onto the next tag.

Again the form follows the usual format for adding/updating and deleting settings and as before the settings are only saved when the form is closed.

Manage Environment Settings

Now we come to managing the settings for each environment. We use the below form which allows you to choose a project and solution. This will then execute a cross tab query to create a grid listing the solutions configuration tags in the first column and then create additional columns for each environment as shown in the picture below.

The first column is read-only, but the other columns allow you to define a specific value for each environment. If you leave a cell empty then the default value is used instead.

This form is different to the others by the nature of what it does. You can not add of remove rows, only edit existing cells. Also when you make a change the details are saved when you finish editing the cell.

At this point you have now configured your solution and are well under way to effectively managing your configuration for your development effort.

Template Files to Configure

As mentioned previously you will typically have a template file which you will run the configuration dictionary over to produce the correctly configured file for a given environment. Some examples of files we typically configure on our BizTalk projects are:

A template of BizTalk Binding Files
A template of Web.config and policy cache files for published web services
A template of BTSNTSVC.exe.config

In the below picture which is a snippet of a BizTalk Binding File Template you can see that I have circled in red the setting tags which are in the $(------) format (note that I have had to censor the tag named). We maintain this template as an artefact within the solution and then during our build process we will run the dictionary over this template file to produce the real correctly configured files for each environment.

Integrating the Build Tasks into Your Script

Along with the Configuration Management application you will also find a folder with a dll containing some MsBuild tasks. There are 3 tasks:

Task Name	Description
ConfigureFileBasedOnDictionary	This task will look up the configuration settings from the database and configure a template file.
ConfigureFileBasedOnDictionaryFile	This task will use the old dictionary file format to configure the template file. This will be discussed later.
ProduceSettingsReport	This task will produce an HTML report for the configuration. We use this task to deliver a report of the settings we want them to use to our deployment team.

The below picture shows the MsBuild UsingTask statements you would need to use to reference the Configuration Management build tasks. You should also see a ConfigurationManagement.tasks file in the folder containing the MsBuild tasks assembly. You can import this .tasks file into your build script using the Import MsBuild statement.

In the next picture you can see a snippet from our build script. In this case we are configuring two files (a web.config and a policycache file) based on information in the Configuration Management database. You need to tell the task the Project, Solution and Environment which you want information from and also the connection string to your database.

In the bottom part of the above snippet you can see how we use the ProduceSettingsReport task to produce the HTML report I discussed earlier. A sample of this report is below:

Working with the old Dictionary File Based Model

While the new application is very useful and we are able to access configuration directly from the database, some teams may still prefer to store the configuration as a dictionary file like in my original blog post. This has the benefit that you can easily keep the dictionary in a source control system and have a full version history for it. Based on this I wanted to allow teams to make this choice and if they still wanted to use the file based approach then could still take advantage of the tool anyway. This is described below.

Importing Configuration from File

You can begin by importing your dictionary file into the Configuration Management application. The below picture shows how you can choose the Project, Solution and path to the dictionary file. When you import this information any existing setting information is updated, and any new data is added.

Exporting Configuration to File

When you have edited your configuration you may wish to export it again to a file to update the file in your solution. Like with the import you can choose the Solution, Project and path then just export it.

Summary

Hopefully you will find this tool very useful and time saving. There are a couple of limitations and future enhancements you may wish to be aware of if you are considering using this.

At present there is no additional security other than if you can access the database. In future releases I plan to add the ability to configure reader and editor roles for each solution/environment/project so this can be locked down more. This will make the tool more usable for bigger projects when you want multiple teams to work on this.
At present I do not audit any changes made to the configuration. This would be a useful enhancement.
Any changes made to configuration will not trigger a build in your Continuous Integration environment. This is one reason why a team may wish to combine using the tool with the old file based dictionary approach because when the updated dictionary file is checked in a build would be triggered. If you do scheduled nightly builds this may mitigate this.

First Look at BizTalk 2009 Build and Developer Experience

Michael Stephenson — Sun, 14 Dec 2008 05:09:34 GMT

We had a very useful brown bag session with the BizTalk Product Team this week and went through some of the new developer improvements. Fortunately not much of this was under NDA so I thought id share some of what I learnt and my thoughts on this. Ill just keep this in the simple format below:

BizTalk Project Files are now MsBuild

A new BizTalk project file is basically an extension of a C# project file. It looks very similar to a normal MsBuild based project file see the Picture below.

From this picture you can see lots of familiar stuff. You can see from the following picture from the project file that a BizTalk project file imports the C# MsBuild targets as well as some new BizTalk ones by default.

I believe BizTalk projects have their own project type guid in the project file system which indicates them as a certain type of project file. You can see this above in the MsBuild script where the Project type Guids indicate this project contains C# and BizTalk stuff. I would assume here you can also add the project type guid for MsTest projects and then start putting tests in this project also. In practice you probably wouldn't do this, but it illustrated whats possible with project files now being in MsBuild.

BizTalk Projects Can have non BizTalk Artefacts in them

That's right you can now add other artefacts than BizTalk ones. Some examples of this might be you have a C# class, a text or xml file in your project. Because of what I said above about the C# targets this means if you include a .cs file in your BizTalk project then it will be compiled and made available just like any other normal class. There are a couple of questions and limitations relating to this however:

In the current CTP you can not Add New Item and add a C# file etc to the project. This is expected to be in future releases, but for now you will need to add the item to the file system then add it as an existing item.
Because the BizTalk project file extends on C# if you add a VB file it will not by default be compiled. If however you add the VB MsBuild targets it should be possible to compile these files also. Essentially its just an MsBuild script so you can do what ever you can do with .net projects.

Assembly Info

BizTalk projects now have AssemblyInfo.cs type files like any normal C# project. This makes things simpler for manipulating these kind of files. See the pic below, if you notice there is a new assembly attribute called BizTalkAssemblyAttribute which I guess indicates the assembly is a BizTalk assembly.

No more Deployment/Development

One thing which makes me very happy is that the pointless configuration difference between C# and BizTalk projects is now gone. You can see this below.

It would be nice if the Deploy tick box wasn't checked by default also. This is a useful setting for small demo type projects but in real projects you are unlikely to deploy your BizTalk assemblies through the Visual Studio IDE. I guess now we are based on MsBuild it might be possible to create a new project template where this is not checked (possibly a small community project here).

Compile BizTalk Solutions without Visual Studio

In the installation of BizTalk there is an option to install Project Build components. This means you can use these components to build your solution without Visual Studio. This is what a number of people have been asking for to help them implement the good practice on a Build server of compiling code without needing Visual Studio.

Strong Naming Change

The way BizTalk projects are strong named has changed and is now the same as for C# projects. From the picture below you can see this is set through the project property editor and it sets information in the MsBuild project file. Again another handy improvement in that it is just the same as C#.

Common Project Settings

As you can see from the below picture you can see the BizTalk project also has the same settings which are commonly used in C# projects. You can set the highlighted sections so you can set warnings and constants etc.

One of the more interesting things however though is that you can setup the XML documentation for the project. At present this will only do documentation for things like C# files within the project, but I guess in the future this is something that either Microsoft or the community may look at as a way of incorporating BizTalk Documenter style functionality into the compilation process.

Project Migration

When you migrate your project to BizTalk 2009 there is a migration wizard similar to the one for migrating .net code. You can use this or you can also call DEVENV from the command line with the /UPGRADE switch to migrate your code. You seem to get the normal features of the Visual Studio migration wizards. One limitation here is I believe the project migration does not change the project configurations from the old Development/Deployment modes so you will have to do this manually. And also apparently you can migrate from BizTalk 2006 or BizTalk 2006 R2 to BizTalk 2009.

Unit Tests

As I have mentioned in some other recent posts there is a new Unit Tests property as shown in the picture below. This will make the compilation process generate some additional code to help you unit test various BizTalk artefacts. If you would like some more information about how to use these features please refer to my updated BizTalk Testing Guidance on the following link:

http://geekswithblogs.net/michaelstephenson/archive/2008/12/12/127828.aspx

Improved Build Performance

Apparently when BizTalk projects are compiled now the MsBuild will be able to tell if things have changed and only compile what has changed. Maps and Schemas seem to be complied to a .cs file before being compiled into the assembly a bit like Orchestrations could be if you made that tweak in earlier versions of BizTalk. This means MsBuild is able to detect the changes to these files and only recompile them as needed. This should in theory improve the overall solution compilation time. This could make a big difference on those projects where you end up with lots of assemblies.

This is most of what I remember from the session which was really useful, thanks to the guys who put this together.

BizTalk MsBuild Generator Update

michael stephenson — Sun, 04 May 2008 21:17:28 GMT

Last week I published a new release of the BizTalk MsBuild Generator.

The enhancements/fixes are:

- A new section for management of hosts. It can set your build script up to create hosts and adapter handlers for your BizTalk application

- A fix to the resources are added. there was a bug in some cases where the resources would be listed in the script where they would not be added in the correct order

- A new fix to only generate the call to MsTest when unit test containers have been added

- An enhancement which sets the base path of the source code. Making it easier to move. Not all paths for msbuild tasks can be relative particularly when extending your build with the SDC tasks. This sets a build parameter which you can use across your build scripts

If is available from the following site: http://www.codeplex.com/BizTalkMsBuildTool

%BTAD_InstallDir% not as expected for the destination path

michael stephenson — Wed, 16 Apr 2008 01:34:27 GMT

I made a little mistake the other day which maybe some of you might experience.

I have an MsBuild script which calls BTSTask to add resources to a BizTalk application. For the destination path I was supplying the value below:

%BTAD_InstallDir%\Acme.BizTalk.Orchestrations.dll

The result was that when the application was exported to an Msi and then installed on a different server the assemblies were always installed to as follows:

c:\Acme.BizTalk.Orchestrations.dll

I hadnt noticed this at first but the problem was that in the MsBuild script the %BTAD_InstallDir% tag was being read as environment variable so the path being passed to BTSTask was:

\Acme.BizTalk.Orchestrations.dll

The fix to this was simple to replace %BTAD_InstallDir% with %%BTAD_InstallDir%% which resolved the problem.