I've just been looking over Bizmonade which is a testing tool for testing BizTalk orchestrations in isolation from the BizTalk execution environment. It's similar in concept to BizMock however the implementation is different. The simple way to describe how BizMock is implemented is that it uses a custom adapter which allows you to manipulate the messaging around an orchestration which is executing within BizTalk. Bizmonade is different in that it creates a custom version of the XLANG Engine which it then allows you to load an orchestration completely outside of BizTalk for testing.
Both projects allow you to test the orchestration from NUnit or MsTest with a syntax similar to most mocking frameworks.
As I've wrote a bunch of stuff in the past about testing, I'll be keeping an eye on these projects. What I'm really interested in is their positioning within the testing you would normally do. Both projects position themselves in that gap between grey-box testing with BizUnit and the component testing of schemas/maps/etc. There is clearly a gap there but how big of a deal it is I'm not sure.
My current questions and thoughts about where I would use the above two components are as follows:
- Would I really want to extensively test all orchestration paths with Bizmonade and probably have an amount of duplication in my BizUnit testing
- Would I be looking to use Bizmonade for edge or obscure test cases which would be a pain to test with BizUnit
- Would I be looking to trust Bizmonade enough to reduce the amount of BizUnit testing I do
- Could a key place for Bizmonade be where I usually have a chain of orchestrations which could be difficult to test with BizUnit, and I could test the orchestrations individually with Bizmonade
- What risk does running my orchestration outside of BizTalk pose. Might I end up dealing with issues which wouldn't happen in XLANG or missing conditions that would happen in XLANG
- Would Bizmonade offer a better way to ensure good test coverage for orchestrations
- Would I lose the ability to debug orchestrations
- Am I going to have to pay for Bizmonade (longer term)
Hopefully the answers to a lot of these questions will become clear as I get more experience with the component and with future releases (remember this is only a preview release).
As mentioned earlier there is a gap in the testing capabilities between BizUnit and the ability to unit test orchestrations, however this hasn't really caused me too much pain in the past. I think to be successful Bizmonade needs to be able to save me time in my development cycle and improve the quality of any deliverable maybe one of the best things Bizmonade could look to do would be to interpret an orchestration in the same way that Visual Studio Unit Tests can for C# and then generate some code which has tests for most of the common conditions and branches within my orchestration. Now that id really like.