blogdoc

Using the BizTalk Build Generator SSO Tasks

Michael Stephenson — Mon, 17 Aug 2009 22:37:23 GMT

In the BizTalk Build Generator there is a tasks library containing tasks which will allow you to easily extend the generated build scripts with some of the common scenarios. Previously I have discussed using the BAM and BRE tasks and this post will show how to use the SSO tasks.

Within the library there are the following tasks which are aimed at configuring SSO (primarily for the configuration application scenario).

Task	Description
Delete Application	Deletes an SSO application
CreateConfigStoreApplication	Creates an SSO application configured to act as a config store with a delimited list of the required columns
PopulateApplicationProperty	Populates a column value within an SSO application
PopulateApplicationPropertyFromFile	Populates a columns value in an SSO application from data in a file. This is useful if you want to store xml in the field for configuration as discussed in many SSO configuration articles.
EnableApplication	Enables the SSO application

The tasks cover most of the common uses. The below picture taken from a test for the tasks demonstrates the use of the tasks.

Normally you would use the SSO tasks in the custom build files and do the delete application in the clean up target and then create and enable to application within the setup targets.

Hopefully this post shows that it is easy to use these tasks.

Using the Build Generators BRE Tasks for MsBuild

Michael Stephenson — Mon, 17 Aug 2009 22:22:31 GMT

In the BizTalk Build Generator, in addition to the normal build scripts which you can produce there is also a library of MsBuild tasks which help you to implement some of the common scenarios for extending your build process. In this post I will discuss how you can extend the build to include the management of BRE artefacts within the build.

In the tasks library you will find the following tasks:

Task Name	Task Description
BizTalk.BuildGenerator.Tasks.BRE.BackupBRE	This task is used to backup all of the current artefacts within BRE before you build goes making changes. This means if you make some kind of mistake you can easily restore it to its previous state.
BizTalk.BuildGenerator.Tasks.BRE.UndeployPolicy	This task will undeploy a policy from BRE. The task supports undeploying all or specific versions of the policy.
BizTalk.BuildGenerator.Tasks.BRE.RemovePolicy	This task will remove a policy from BRE. The task supports removing a specific version or all versions of a policy.
BizTalk.BuildGenerator.Tasks.BRE.RemoveVocabulary	This task will remove a vocabulary from BRE. The task supports removing a specific version or all versions of a vocabulary.
BizTalk.BuildGenerator.Tasks.BRE.ImportVocabulary	This task will import a BRE vocabulary
BizTalk.BuildGenerator.Tasks.BRE.ImportPolicy	This task will import a BRE policy
BizTalk.BuildGenerator.Tasks.BRE.DeployPolicy	This task will deploy a policy to BRE. The task supports deploying a specific version of the policy or all versions of a policy.

The tasks basically just leverage the underlying assemblies which are used by the rules deployment wizard.

The below sample taken from some of the build generators tests demonstrates how to use some of the tasks.

<?xml version="1.0" encoding="utf-8"?>

<BizTalk.BuildGenerator.Tasks.BRE.BackupBRE FolderPath="."/>

<BizTalk.BuildGenerator.Tasks.BRE.UndeployPolicy PolicyName="BizTalkBuildGeneratorTestPolicy"/>

<BizTalk.BuildGenerator.Tasks.BRE.RemovePolicy PolicyName="BizTalkBuildGeneratorTestPolicy"/>

<BizTalk.BuildGenerator.Tasks.BRE.RemoveVocabulary VocabularyName="BizTalkBuildGeneratorTestVocabulary"/>

<BizTalk.BuildGenerator.Tasks.BRE.ImportVocabulary FilePath="TestVocabulary.xml"/>

<BizTalk.BuildGenerator.Tasks.BRE.ImportPolicy FilePath="TestPolicy.xml"/>

<BizTalk.BuildGenerator.Tasks.BRE.DeployPolicy PolicyName="BizTalkBuildGeneratorTestPolicy"/>

</Target>

</Project>

Creating your own build script provider for the BizTalk Build Generator

Michael Stephenson — Mon, 17 Aug 2009 22:22:10 GMT

One of the new features of the BizTalk Build Generator is a provider pattern for the build script generation. This means if you prefer to use a different format/structure in your build scripts or even don't like MsBuild and would rather use NANT, Powershell or traditional batch scripts then you could create your own provider which would take advantage of the metadata used by the BizTalk Build Generator and then produce scripts in your own format.

In order to create your own provider you must create a class that implements the BizTalk.BuildGenerator.IBuildProvider interface shown in the below picture.

In your build provider you will need to implement the CreateBuild method which will write out all of your build scripts. The GenerationArgs class which is passed in will contain all of the metadata about the build you have described through the Build Generator interface.

If you would like to see how the default build provider has been implemented the source code for the Build Generator tool can be downloaded from the CodePlex site.

Id be really interested to hear from anyone who ends up implementing their own provider and maybe you would like to contribute this to the community project.

Testing BizTalk Maps with Custom XSLT

Michael Stephenson — Thu, 09 Jul 2009 09:31:34 GMT

A while back I wrote a couple of posts about Testing maps in BizTalk. They are at the following links:

BizTalk 2006 R2

http://geekswithblogs.net/michaelstephenson/archive/2008/03/30/120851.aspx

BizTalk 2009

http://geekswithblogs.net/michaelstephenson/archive/2008/12/12/127826.aspx

One possible extension to this might be if you have a map which rather than using the map designer imports a custom XSLT to perform the mapping. (If you are not familiar with this refer to MSDN).

So if you have a custom XSLT then you really have a couple of choices with regard to testing it. The first choice is that you can just run the testing approaches which are described above, or you could also test the XSLT before you use it in the BizTalk map or do both.

If you wish to test the XSLT in isolation you can use the XslCompiledTransform class within the System.Xml.Xsl namespace. This will allow you to load the xsl and then transform an input file to an output file like in the below example. You could then perform a binary comparison on the output file against what it is expected to look like.

Summary

So you can see above this is easy to do. I would probably in practice do 1 test which would test the BizTalk map to ensure it works correctly when loading the custom XSLT and the rest of my tests will focus on testing the XSLT in isolation and testing different cases within the map.

Testing BizTalk Flat File Schemas

Michael Stephenson — Wed, 08 Jul 2009 20:27:12 GMT

I got some feedback a while ago asking for some advice on how we test flat file schemas and have eventually got around to writing this post.

The aim of testing schemas which have been created to represent a flat file (positional or delimited) is to be able to test the schema in isolation to ensure that it can be used to parse and assemble a message correctly.

The reason testing the schema in isolation is important is that it allows you to ensure it works before the schema is used within BizTalk where problems will take more time to troubleshoot and require deployment to BizTalk to do any testing.

Normally when this is done people will create BizTalk pipelines and test this with BizUnit but we have actually used the Pipeline Component Test Library which actually contains some classes which make this really easy to do.

The Sample

In the below sample you can see I have creates a solution structure containing two projects. One is a BizTalk project containing flat file schemas. One schema represents a positional file and the other represents a delimited file.

In the C# Test project I have creates tests for each schema and there are input files for each schema to support testing of assembly and disassembly with each schema.

There are 4 tests.

The assembly tests will take an xml input file representing the schema and then run the assemble process and produce the appropriate flat file.

The disassembly tests will take a flat file and run the disassembly process to produce an xml file of the parsed input in an xml format.

The below code snippet from the tests shows how we use the SchemaTester which is a generic class which you can type to be relevant to your BizTalk schema type. The SchemaTester then provides methods to do each of the types of testing.

Normally what I would do after SchemaTester has ran and produced the output file is to do some kind of binary comparison against a known expected copy of what the xml should look like.

Summary

Hopefully you can see this is quick and easy to implement and can save you a lot of time in getting these schemas right.

The downloadable sample is available from the following location:

http://code.msdn.microsoft.com/flatfileschemas/Wiki/View.aspx?title=Home

Solution file version issue when migrating your Build Script to BizTalk 2009

Michael Stephenson — Sat, 04 Jul 2009 23:20:10 GMT

Ive just been working on upgrading the build generator to support BizTalk 2009 projects. While doing this one of the things i wish to do is to change the compile step to use MsBuild rather than dev env.

I found a little issue which others who use MsBuild may come across when migrating custom scripts.

Previously for BizTalk 2006R2 scripts I used to use the below command:

<Exec Command='"$(DevEnvPath)" $(SolutionName) /Build $(ConfigurationName)' />

This would compile the whole solution using devenv.

I changed the script to the following:

<MSBuild Projects ="$(SolutionPath)" Properties="Configuration=$(ConfigurationName);" />

and also changed the property on the project to use ToolsVersion="3.5"

This is how I expected the build to need to be and it would mean our script would take advantage of using the MsBuild task inline. When I ran the script I get the following error:

"Solution file error MSB5014: File format version is not recognized. MSBuild can only read solution files between versions 7.0 and 9.0, inclusive"

My initial thoughts made me think this was more complicated than it actually was. I was looking at using the Exec command rather than the MsBuild task for the compile which I could get to work. Then suddenly the answer hit me.

The problem was that I forgot that the .cmd file I use to call the MsBuild file was configured to use the v2.0 version of MsBuild rather than 3.5. As soon as I corrected this it worked fine.

So if you have the above error remember to double check the original call to start your build script wether it be from TFS/Cruise Control or a custom file etc.

BizTalk Testing Guidance - Dealing with external systems/dependancies

Michael Stephenson — Tue, 12 May 2009 17:13:20 GMT

Article source: http://geekswithblogs.net/michaelstephenson

When we are testing our BizTalk solution one of the biggest challenges is how we deal with the dependency on external systems. Some of the common scenarios I have come across are as follows:

We will be integrating with some web services but there are no test instances of these services we can use
We will integrate with a LOB application but there is either no test instance of the LOB system of using the LOB application will make it difficult to perform a lot of our testing
We are working on a larger project and the system we will be integrating with does not yet exist or is in early development and is changing a lot
I'm working with a B2B partner and they can not support my testing

These problems are typical on most projects and are one of the common excuses for poor testing and a resulting low quality solution. In this article I will discuss some of the approaches we take to dealing with these problems.

Probably one of the key things here is that if you are working with BizTalk or some kind of services solution you will hopefully be thinking carefully about a contract first approach. By using a contract first approach you will understand the messages you will be passing between you and external systems and at least how your solution will react to different messages it receives. With this understanding you should be able to use various techniques to simulate how an external system will work in your solution.

Obviously there is a trade off because you don't want to recreate the whole external application but if you can mimic the communication protocol and replicate some of the messaging exchanges that will be involved in some of your testing scenarios then you can make significant improvements in the testing you can do.

Use a Mocking Framework

For completeness I will mention the use of mocking frameworks such as Rhino Mocks. While these frameworks are good and offer a lot of value in the testing of some aspects of BizTalk solutions I think when it comes to BizUnit style testing of an end to end process they don't really help you.

I usually use something like Rhino Mocks to test a custom .net component, pipeline component or possibly a custom adapter. I have mentioned these in previous posts in the testing series.

I will later in this article discuss a community project called BizMock which I feel is slightly different to NMock and Rhino Mocks.

Creating a Stub

One of the simplest ways you can replicate the external system is by creating a simple stub of that system. One of the most common ways to do this if it is a web services enabled application is to create a Visual studio web services project in your solution and then add some web services and web methods that match the external web service. I generate a server side stub of the web service and then add some logic to the generated method to make it meet the test cases. Often the stub will implement some logic to return different responses based on a parameter in the inbound request message or do a different action.

I will provide a couple of follow-up articles to show how we normally create stubs for asmx web services and WCF services.

I have made a list of what I think are some of the main good and bad points about this approach:

Good Points	Not so good points
Its an easy way to simulate an external system providing the same contract and communication protocol	If the contracts are changing regularly then the stub will usually need to change regularly
It is easy to stub things which have a web service or WCF interface	You need to write more code and could end up doing lots of stubs
It fits well with a contract first approach	Some systems can be difficult to stub
The stub can implement simple logic to support your test cases. Eg if a request contains a certain parameter value then you will return a certain response
You can deal with a lot of different types of external system
The stub can be part of your code base
Works well with BizUnit testing

I have an example of a situation where a stub approach really made our life easier. In a project we were developing a web service façade on top of the API for the Cognos Reporting system. The intention was to provide a flexible service which applications could call to execute a report in Cognos and the service would encapsulate all of the logic required to execute the report.

The Cognos API is a large assembly which allows you to do a lot of different things against Cognos. One of these things is to execute a report. The report execution is slightly complex and to get good coverage of our code was going to be slightly difficult. One of the challenges was that we would need an instance of Cognos to be able to connect to. This would be a pain as there were a lot of Cognos projects going on and getting on of their boxes with a set of suitable test reports which they could maintain wouldn't really work because as soon as there is a conflict between team priorities someone is going to be affected by this coupling.

We considered having a local instance of Cognos to support our development/build/test effort but straightaway this requires our team to have a Cognos skill set.

With a little bit of investigation we worked out that the Cognos API used to send HTTP traffic between itself and the Cognos server so we were able to capture some of this traffic and to use this to simulate the HTTP calls so we could then create a .net Web Application which used an aspx page to receive and send the appropriate messages to allow the Cognos client side API to work. The benefits of this stub were:

We didn't need to depend on a Cognos installation for all of our development effort
We were able to test parts of the code which we couldn't without working out how to break Cognos from our test case
We didn't need to create any test reports
We didn't need any Cognos skill

This was probably one of the more extreme cases of stubbing out an application. We had initially depended on a Cognos instance but we found that our Continuous Integration builds would constantly break because someone deleted our reports etc.

Using Mocking Bird

Mocking Bird is a codeplex project that was introduced to me by Santosh Benjamin. It is something I haven't used in anger but it's been on my radar for a while. Basically the project provides a pattern to allow you to implement stubs to simulate external systems. This project can simulate traditional web services and WCF services.

I think it will save you some of the effort of creating and maintaining the stubs discussed above.

Santosh has some good articles about Mocking Bird and I would recommend looking into it. Probably the one key area where it will not help is if your external application is not supported by WCF/Web services. In this scenario you can just drop back to the above stub approach and code something to simulate that system.

For more info refer to: http://www.codeplex.com/mockingbird

Using Alternative Bindings

I was providing some advice to a project once and they were doing some integration with Dynamics AX (I think). Anyway for whatever reason they could not integrate with web services and were using the BizTalk AIF Adapter for Dynamics AX. One of the dependencies of the adapter was that you needed a build and ready deployment of Dynamics AX for the adapter to work with.

On the project at the same time that the BizTalk team were developing an EAI solution, a Dynamics team were also working hard on the Dynamics part of the project. This means that this coupling would cause problems with the BizTalk team not being able to test against Dynamics and it being difficult to stub out the LOB application due to the complexity of the adapter.

One of the advantages of BizTalk is that because you have loose coupling between an orchestration and any ports it uses the BizTalk team in this scenario were able to change their approach to testing in their development environment and use different bindings to change the communication protocol to talk to a stub which was created to simulate the behaviour of Dynamics but it would use a different communication mechanism.

By looking at the message exchange patterns the BizTalk team were able to use File ports to simulate 1 way communications and WCF ports to simulate 2 way communication patterns. This means that in the development tests the team could use BizUnit to effecticely test the solution with these bindings and focus on the logic in their orchestrations and other components.

The one big risk with this approach is that you are not actually testing connectivity to the LOB application and some of the vendor specific situations may result in problems being faced later in testing. There isn't much you can do about this but you have certainly ensured you are doing an effective job at testing your own code.

I think this model would also be suitable for off-shoring situations where your off shore resource may not have access to the LOB applications to help with testing.

In summary this approach basically uses a different type of port in the bindings for different test environments.

Using BizMock

I came across BizMock a few weeks ago and it seems like a really interesting community project that could be a valuable resource for those who develop BizTalk solutions. I asked one of my team to have a look at BizMock and to try to replicate some of our existing BizUnit tests with BizMock to see what it could offer us.

Unfortunately we were getting the same error that has already been logged on the BizMock issue tracker so we haven't been able to complete a full comparison yet. BizMock is a very new Codeplex project in its alpha release so being we can live with this, but the ideas that the project offers are very interesting.

The below blog article provides a good overview of what BizMock will offer.

http://blogs.msdn.com/pierreml/archive/2009/02/13/bizmock.aspx

My opinion on BizMock is that I would use it in situations where I have a lot of complex logic in an orchestration which I need to test and external dependencies make this difficult. BizMock looks to be aimed at testing orchestrations rather than an end to end process so it has a clear differentiator to BizUnit. BizMock also offers schema and map testing capabilities but with the enhancements in BizTalk 2009 I'm not sure what value they would offer.

Once there is a more stable release of BizMock I would use it to compliment the tests I currently do in BizUnit. I think managing another set of bindings to configure the BizMock ports is something which I don't particularly want to do so the trade off of BizMock vs Stubs is just work saved in 1 area to be spent in another. I think the real benefit of BizMock is in being able to concentrate on those test scenarios which are difficult to perform or require lots of manipulation of a stub.

I think in the feature list in the above lined article it discusses the benefits of the tests being written in code vs in xml. Remember in the most recent version of BizUnit you can also write tests in C#, but to be honest I think I actually prefer them in XML where they are much clearer and easier to follow and have a more structured approach and you don't have to worry about some developer who is no longer around who had a coding style that no one can follow.

I'm looking forward to a resolution to the problem logged on Codeplex so we can look into BizMock in more detail after which I will update this section.

More info is available on: http://www.codeplex.com/bizmock

Summary

Hopefully in this article you got a good feeling for some of the approaches that are available to help you with some of the common yet challenging aspects of BizTalk testing. The things I've discussed here are primarily focused around the development cycle where you are trying to perform automated testing as part of a continuous integration approach.

In general my comments were that Stubs are a good way to help you test your solution and deal with the parts of your project that have dependencies to external systems. When you have very tight coupling to an external system you are helped by BizTalk in that you can use a different type of port to reduce this coupling to do some testing with a trade off that you take on some risk of adapter specific implications.

I think Mocking Bird and BizMocks offer some interesting new ideas on how to look at these problems.

One thing that I haven't mentioned is that in addition to this focused development testing it is also very important to get your real systems together on a regular basis and perform some integration testing. Ideally this would be automated but it can be effective even if its just getting a couple of developers from the different teams together on a regular basis. You will always get some problems this is why integration is usually such a pain but with these approaches we are making our best effort to catch problems early and mitigate as much risk as possible.