MsBuild

I hate days like this

Michael Stephenson — Tue, 12 Aug 2008 18:38:08 GMT

Had one of those days yesterday and today, you know what its like when you find a bug/problem and you dont really get a chance to properly look at it when your doing a million things at once.

It is so often that the cause of the problem is really obvious but you just cant see the wood for the trees.

This is a reminder for myself as much as anything.

I was writing an msbuild task to wrap cruise controls ICruiseManager so I can get the latest build label of a project that has been built.

The problem was that when I tested the task using MsTest from Visual Studio it worked fine, but when I ran it from an MsBuild script it kept failing complaining about not being able to find the ThoughtWorks......Remote assembly. I ensured that the assembly was where my tasks assembly way and frustratingly it wasnt working

When my brain returned I spotted that the problem was that the assembly is being searched for in the directory which the process started (the .net framework directory where MsBuild.exe is) so by copying the assembly there it now works.

BizTalk Documenter in my build

Michael Stephenson — Mon, 04 Aug 2008 23:17:43 GMT

From previous posts you may have read how I integrated the Microsoft BizTalk documenter into my MsBuild process.

In general this has been working fine, but every now and again I kept getting a build failure when trying to generate the documentation as follows:

System.IO.DirectoryNotFoundException: Could not find a part of the path 'C:\Documents and Settings\<UserAccount>\Local Settings\Temp\BTS2K4Doc\Application\fb7b6ba7-016c-4a1f-9c7d-ec9037911456.html'

If you get this it seems to be caused by an hhc.exe (Microsoft Help Compiler) process still being alive from a previous run. Just kill it and the builds will work again.

MsBuild Task for consuming WCF Services

Michael Stephenson — Tue, 29 Jul 2008 00:16:04 GMT

Article Source: http://geekswithblogs.net/michaelstephenson

Recently at the UK SOA/BPM User Group Yossi Dahan and I chatted briefly about using MsBuild and BizTalk, more specifically Yossi mentioned that he wished it was possible to regenerate the schemas for consuming a service automatically.

Following this discussion and a few other things I had a blast at seeing how difficult this would be to do. I think there is a lot of value in having this approach as it fits well with a contract based development approach and continuous integration. I want to be able to regenerate my schemas every time I build the solution, and if a breaking change has been introduced I want the solution to break so it is corrected.

This fits with what I believe is a good practice to identify these kinds of problems as early as possible in the development cycle.

The aims of the task are:

Act in a similar way to the Generate Schema feature for BizTalk/Visual Studio except that it will be driven by an MsBuild task rather than a GUI
Just deal with the schemas, I wasn't too bothered about regenerating the orchestration and binding file samples

(The sample is available for download at the bottom of the article)

The Sample

In the sample the quickest way to demonstrate this is as follows:

Modify the BuildProcess.xml file to point to your url for the MexEndpoint
Double click the RunBuild.cmd file
Check the Acme.BizTalk.Schemas.WCFService folder and you should see your schemas have been updated

It is expected that you would include the schemas in your project, but exclude them from source control so they can be rebuild every time you build the solution.

Digging Deeper

I managed to work out how to do this by inspecting the Microsoft.BizTalk.Adapter.Wcf.Consuming with Reflector. Upon investigation it would have been nice if a few of the classes were externally available from the component. As it was I needed to disassemble the component to progress this idea. In the disassembled component I created a new version of the Consumer class in which I removed some of the functionality and changed the Consume method interface so it wasn't dependant on being passed a Visual Studio Project object.

With a couple of other tweaks I was able to get this working

Conclusion

Although this task seems to do the job, it has not been extensively tested, and it also only currently works with the MEXEndpoint and not yet with meta data files.

Hopefully this idea will help a few people, and the idea might get back to the product team as a possible future enhancement as this kind of thing can certainly help your development process be more effective.

BizTalk MsBuild Generator

michael stephenson — Sat, 16 Feb 2008 01:40:07 GMT

I created this tool ages ago and have been using it on projects for a while. Ive eventually got around to cleaning it up and doing some documentation for it so I have now made it available on CodePlex.

Basically the tool allows you to generate an MsBuild script which will let you do all of the build and deployment actions to compile and deploy your BizTalk solution.

It also produces a script which is well structured and easy to extend.

If you want to know more check out: http://www.codeplex.com/BizTalkMsBuildTool

MsBuild Task for WCF Service Publishing for BizTalk

michael stephenson — Sun, 03 Feb 2008 18:12:48 GMT

A number of months ago I did a couple of posts about how we integrated the Soap and WSE web service publishing wizards into our build process as MsBuild tasks. These posts are located at the following url's:

Soap Adapter: http://geekswithblogs.net/michaelstephenson/archive/2006/09/16/91369.aspx

WSE 2 Adapter: http://geekswithblogs.net/michaelstephenson/archive/2006/09/16/91371.aspx

Now that BizTalk 2006 R2 is around I want to have the same approach for the WCF Publishing Wizard.

Before I get into the details of this, it is important to note that WCF provides a hosting model that means you do not need to host services in IIS and they can be hosted by In Process BizTalk hosts. Although this is cool, there are still a number of reasons you may still want to host the services in IIS.

Aim

The aim of the task is to allow you to regenerate the services which are published as part of your build script. This means if for example your schema changes then the published service is automatically updated for you each time you build the whole solution. This also fits well with a continuous integration strategy.

Process

The development process or approach I use in these situations is as follows:

1. The first time you produce the wcf service you will manually use the WCF Publishing Wizard and generate the service as desired.

2. In the published service there is a temp file called WcfServiceDescription.xml. Take a copy of this and add it as an item to your solution.

3. In your build script call the custom task we will create and pass in the path to the WcfServiceDescription.xml file in your solution.

In this process it is important to note that if you need to add any new schema or orchestrations to the published service you will need to generate the service again by hand to get in there new items, and then simply overwrite the description file in your solution.

Using the task

The following picture shows the sample of the build script using the task.

As you can see above the call to the task is very simple.

Inside the Task

The task itself is very simple and basically references the Microsoft.BizTalk.Adapter.Wcf.Publishing assembly and then executes the following code in the Execute method of the task.

Again the code is very simple

Things to watch for

There are a couple of things to watch for when you use this approach:

Ensure in the service description file you have the overwrite property set to true. If not then if the service is already published you will get an error
I would also recomment setting up the application pool and virtual directory in IIS yourself as part of the build script. This gives you full control over how it is created where as if you let the Wcf Service Publisher do it then it will use some default settings which you probably wouldnt want to use in practice

Summary

I hope you file this task useful, it is a little simpler that the previous ones, particularly the WSE 2 one which unfortunately had that bug we had to work around. If you would like to check out the sample then see below where the source code is available.

Disclaimer

I have noticed a few sites that seem to copy the content of blog articles and display them in their own site. It is a bit annoying that they do not clearly reference or acknowledge the author so I have decided to put this note on the bottom of all of my posts from now so it is clear who wrote it.

This article was written by: Michael Stephenson

The source of this article is: http://www.geekswithblogs.net/michaelstephenson

Integrating the BizTalk Orchestration Profiler into your MsBuild Process

michael stephenson — Tue, 29 Jan 2008 21:22:01 GMT

I have been wanting to include the BizTalk orchestration profiler into our build process for a while and have eventually got around to doing it.

Like some of the other recent posts ive produced a simple example demonstrating this. I will begin by discussing the example on how to use the task and then follow this with some comments on the task itself.

Using the BizTalk Orchestration Profiler MsBuild Task

The idea of the BizTalk Orchestration Profiler Task is that when my build script has ran the BizUnit tests for my BizTalk solution I want to run the Profiler to extract a report giving details of the code coverage within my orchestrations.

The orchestration profiler can be ran from the command line but im not sure that you can specify more specifically which orchestrations you are interested in like you could from the Forms interface. In addition to this it integrates nicely into my build script if I wrap the Profiler inside a custom MsBuild task.

To use the task I need to know the date and time for the start date of the report. This is important to ensure I dont pick up coverage information from previous test runs.

The following demonstrates my sample build script:

At the start of my build script I call the GetDate task to get the date and time when the build starts. These are saved in parameters for use later in the script. In the middle of the script I have a message which is where the calling of the BizUnit tests would be.

After the tests are finished I call the Orchestration Profiler task. In the example case I am running the report for the BizTalk.System application. This will profile all orchestrations within this application.

When the task has ran it will create the report in the place indicated by the output path and the report title. In the above example the report produced will be placed at the following path C:\BizTalk.System Coverage Report.chm

The DelayMinites attribute allows you to specify a pause before running the report to ensure all of the tracking data will be transferred to the tracking database.

Implementing the BizTalk Orchestration Profiler MsBuild Task

In implementing the task I have basically called the Microsoft.Sdc.OrchestrationProfiler.Core assembly in the same way that the Orchestration Profiler interface does. In addition to this I have some code which uses the Microsoft.Sdc.BizTalkOM assembly to get a list of the orchestrations belonging to the application which has been supplied.

The other key point is that the profiler has a dependancy on a key in its config file indicating the path to the html help workshop. This was going to cause a problem running this as an MsBuild task as the config setting would need to be in the MsBuild.exe.config file.

To get around this I have a method that will check for the setting and if its not there it will write the setting to the config file and refresh the section using the System.Configuration assembly.

There are really the highlights of the implementation, but please refer to the sample below which contains all of the code.

Disclaimer

This article was written by: Michael Stephenson

The source of this article is: http://www.geekswithblogs.net/michaelstephenson

Configuring Binding Files for BizTalk

michael stephenson — Sun, 27 Jan 2008 17:31:58 GMT

Problem

In all BizTalk projects you come across the old problem of configuration and how to manage the environmental settings when taking your solution from development to other environments. The common scenario is, I work with my binding file and get my solution build and tested etc and then I take it to the testing environments or production and I need to have a new binding file or I need to manually change the settings for ports etc. Obviously any manual steps are usually error prone so this never fully works.

On our project it is not just the binding files we need to configure between environments. The following table summarises the files that need to be configured:

File	Reason
BTSNTSVC.exe.config	This file contains a few simple settings which are not held within SSO. These need to be configured by environment
PolicyCache file used by BTSNTSVC.exe	The policy cache contains endpoints and SPN values which are different by environment. These are configured
Web.config in any web services that are published	Again there may be some values that need to be configured by environment
PolicyCache for any published web services	The SPN will be different by environment so this typically needs to be configured
Binding Files	We configure the binding files so that any settings which change by environment are configured. These settings usually include: Hosts for the ports Endpoints Adapter specific information Credentials Etc

In solving this problem i wanted to address the following requirements:

Needs to be available as an MsBuild task to integrate into the build process
Xml will be used to hold the configuration data. This would make it easy to maintain and I could extend it by creating a tool to manage the configuration source if I wanted.
Needs to be able to produce files for multiple environments based on a template file
Needs to be able to deal with different file types such as those listed above

Solution

So the problem is defined above and I will explain how our solution works, and a small sample is available which demonstrates its use.

The solution I developed is not BizTalk specific it can be used to configure any file. The process that we use is as follows:

Maintain an xml file which is the source of the configuration data. We refer to this file as the configuration dictionary. This file defines the tags that are to be looked for, and also the real value for each environment.
For each file that is to be configured we would have a template copy of it which would contain tags where the real values are to be replaced.
In our build script we would call the MsBuild task passing in the input file and dictionary file. We would indicate which is the desired environment and where to place the output file.

The following sections will explain how we manage each of the types of file.

BizTalk Binding Files

The bindings file is the most awkward to manage typically because it is the most complicated xml. In the solution I keep a template copy which is used in the build script. When the bindings need to be changed I do them manually in BizTalk and then export the bindings to an xml file. I then need to manually merge the changes into the template (this can be awkward) and replace the values that need to be configured with tags which correlate to the dictionary.

Whe the build script runs I configure the template for every environment I am working with and then copy them to a special folder. This includes the binding file for the development environment.

Later in the build script I use the development binding file which I created to import it to my BizTalk Application and setup my ports etc before I run my unit tests.

When I deploy on another environment I can include the various binding files with the msi when it is produced, or I can import them manually.

BTSNTSVC.exe.config & The Policy Cache

As mentioned above these files also need to be configured for our overall solution. We manage these files as a solution on their own. This solution is setup as a project in Cruise Control. Each time a build is produced configured versions of these files are produced for each environment.

The build process for each individual application basically copies the right files to the development machine and uses them for the individual application build.

When we are deploying to a given environment we can just grab the appropriate files from our Cruise Control server.

Published Web Services (web.config & policyCache.config)

Because we do a lot of web service publishing we need to configure the policy cache and web.config as certain details in there files can change by environment. We take a similar approach to the binding files where we keep templates for each as part of the solution. We then create all of the configured versions for each environment.

When we want to run tests we copy the development versions of web.config and policyCache to the right place and then run the tests.

When we come to deploy to other environments we deploy the web services via the msi, but then we simply copy the appropriate environments versions of these files to the right location.

Solution Summary

Because we use Continuous Integration and the build process to create all of the files it allows us to produce versioned instances or all of the files that need to be configured as part of the output package from each build

Example

As mentioned above I have setup a small example which goes over the core details of the solution, and I will discuss these in this section

There are 2 solutions in the sample:

MsBuild.ConfigurationTask = The msbuild task and some tests for it
MsBuild.ConfigurationTask.Example = A sample solution demonstrating its use in the build script

Im not really going to go into the detail of the MsBuild task as the sample contains a test which should demonstrate its use. Im going to focus on the example on how to implement it in your build. If you have any questions about the task however feel free to drop me an email.

In the example solution I have the sample input file which contains the following xml.

<?xml version="1.0" encoding="utf-8"?>
<SampleXml>
<SampleElement>$(BindingFilesSettings-Parameter1)</SampleElement>
</SampleXml>

The input file will have the $(BindingFilesSettings-Parameter1) tag replaced with a value from the dictionary.

The key points about the dictionary file are as follows:

It contains sections to group together similar config tags. This makes the file easier to manage
Each tag contains a default value which is used when you choose an environment which does not have a specific setting

In the build script you will find the following task:

<Acme.MsBuild.Tasks.ConfigureFileBasedOnDictionary
   DictionaryFile = "Acme.ConfigurationDictionary.xml"
   Environment = "SystemTest"
   InputFile = "Acme.SampleBindingFile.xml"
   OutputFile = "ConfiguredOutput\Acme.SystemTestBindings.xml"/>

This will call my custom task passing in the input and output files as required and will ensure a file is produced which is configured for the system test environment.

Summary

Hopefully you will find this solution can help you on your project, if so then feel free to download the sample from below and use or amend it as you wish. This solution has worked very well for us. If you have any feedback or questions please feel free to comment below

Disclaimer

This article was written by: Michael Stephenson

The source of this article is: http://www.geekswithblogs.net/michaelstephenson

Where have the microsoft sdc msbuild tasks gone?

michael stephenson — Fri, 25 Jan 2008 23:10:53 GMT

I was looking for the Microsoft Enterprise Solutions Build Framework or Microsoft.Sdc build tasks as it is sometimes known.

if you google it most of the links point to gotdotnet. I did a search on codeplex and initially didnt find it because i was looking for the wrong name. Eventually i found it, and if anyone needs it it is located at

http://www.codeplex.com/sdctasks

Hope this helps

Mike

Generating the C# to call BizUnit Tests

michael stephenson — Sat, 19 Jan 2008 23:05:46 GMT

I'm not sure about anyone else but when I come across something that is boring and repetitive I usually get around to doing some tool that minimizes the amount of time I spend doing it. In this case the task that is boring me is writing the repetitive C# code to call BizUnit to make it run your tests. I'm sure there are lots of different ways you could do this but one day when I was stuck waiting for a while at the airport I got bored and decided to do something to stop me having to do this.

Aim

The aim of the tool was as follows:

Don't write anymore C# code to call BizUnit tests, I want to generate it

Maintain a simple list of tests which can be easily updated

Include the tool in the build process so that any newly added tests have the appropriate C# code created

Design Decisions

I made the following decisions about the tool I would create:

I decided to make an MsBuild task so it would easily plug into the build process and also regenerate the C# each time

I decided to use an xml list of the tests. I could have looked for files which were BizUnit tests but this would probably need to have some kind of file naming convention.

I wanted to wrap the call to BizUnit with a try/catch block so I could log the error to the event log. This is particularly useful when you run the tests from the command line and you don't see all of the failure details.

The task would be easily configurable so it could use Nunit or MsTest to run the tests.

Using the Task

Before I get into how I built the task I think if I explain how it is used it should seem simple and easily usable. We have used this task on a couple of projects and it has been quite handy.

To begin with we have the below image from the sample project. In this picture you can see the following things:

A folder containing 3 BizUnit definition files

The BizUnitTestDefinitions.xml file which describes the tests within the project and determines the code that will be generated.

The OverrideBuild.targets file which is imported by the project file and contains overrides for some of the build events (more on this later)

The BizUnitTests.cs file which is the code file which gets generated

BizUnitTestDefinitions.xml

As I mentioned this file is the one that describes the tests within the project. The below picture shows an example of this xml.

The key points of the xml are as follows:

The TestGroup element allows you to group related tests. Each TestGroup will produce a class containing tests. The Name attribute will become the name of the test class.

Each Test element will become a method which will run a test. It will be contained within the class from its associated TestGroup. The Name attribute will become the name of the test

The BizUnitTestCasePath element will point to the file containing the details of the BizUnit test. In this case because the example project uses MsTest I have copied the BizUnit tests to the test results directory using the TestRunConfig so I dont need to worry about the path.

OverrideBuild.targets

This file is imported by the project file allowing me to specify MsBuild targets to customise the projects build process. The contents of this file are below:

In this file I have a target which overrides the BeforeBuild target. This is because I want to generate the class containing my tests before the project is compiled. The key points of this file are:

The UsingTask element imports my custom MsBuild task so I can use it

The use of my task in this case is set to use MsTest, but you could easily use Nunit instead by specifying the appropriate values for the TestAssemblyName, TestFixtureAttributeName and TestMethodAttributeName.

This task will basically take these parameters and inspect the xml definition of the tests then produce the output file containing the C# code.

BizUnitTests.cs

This file contains the code which is generated. An example of some generated code is in the picture below:

The couple of key points to note are:

I have made it generate code which will log to the event log any errors to make troubleshooting easier

If you use source control you might want to exclude this file from source control so it can be regenerated each time you build the project.

Summary

If there is any interesting feedback on this post I will probably do a follow up describing how the MsBuild task is build. In the meantime the code for the task, and also the sample project are available at the below location so you should be able to see this for yourself.

http://www.box.net/shared/63g7rjuo0k

Disclaimer

This article was written by: Michael Stephenson

The source of this article is: http://www.geekswithblogs.net/michaelstephenson

Problem resuming an orchestration with WMI

michael stephenson — Tue, 15 Jan 2008 00:44:35 GMT

Problem

I recently had to make a change to one of out orchestrations on a BizTalk project. The change was that we needed to in a certain case suspend the long running orchestration and raise an event which would be handled by MOM. This would cause an alert to an administrator who would check out what we think could be a potential error condition.

When the administrator had checked everything was ok he/she would use the instance id to find the orchestration via the group hub and resume it.

This is straight forward enough but i also needed to amend the BizUnit tests to handle this condition.

I amended the tests to check for the event being logged, and I then made BizUnit simulate the corrective action with a custom step. The final step was to resume the orchestration.

Because I knew this would be the only functional test running I cheated a little and created a step that would resume all suspended orchestration instances for a given host. This would save me having to parse the event log message to find the instance id or having to listen for a WMI event being fired.

When I did the step to resume the orchestration I experienced some strange and unexpected behaviour.

The first thing the step did was to use the WMI MSBTS_ServiceInstance class to iterate the service instances for a given host the next step was to use the MSBTS_HostQueue and call the ResumeServiceInstancesByID which should resume the instance.

When executed we found the following behaviour:

The wmi method executed without errors
The instance when viewed in the group hub was no longer suspended
The instance seemed to stay in a dehydrated state we waited for upto 20 minutes and it didnt become active
From the dehydrated state if we suspended then resumed manually through the group hub the instance would complete successfully

Solution

The fix was easy enough, basically I changed the resume orchestration call so it used the Microsoft.BizTalk.Operations dll rather than WMI. This worked fine and the code snippet is below.

using Microsoft.BizTalk.Operations;

BizTalkOperations ops = new BizTalkOperations();
CompletionStatus status = ops.ResumeInstance(instanceId);
Trace.WriteLine("Status = " + status.ToString());

Disclaimer

This article was written by: Michael Stephenson

The source of this article is: http://www.geekswithblogs.net/michaelstephenson