That BizTalk guy from India

Benny's Identity & Integration Journal

Benny Mathew — Thu, 29 Nov 2012 09:24:05 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2012/11/28/151392.aspx

Click here for to get directed to my new blog: http://bennykmathew.wordpress.com/

Intro On AppFabric on EndPoint.tv (Steve & Danny)

Benny Mathew — Thu, 08 Apr 2010 15:24:53 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2010/04/08/139156.aspx

http://channel9.msdn.com/shows/Endpoint/endpointtv-Pro-Windows-Server-AppFabric/

Here is a nice intro to Windows Server Appfabric by a good friend and colleague Steve and Danny.Cutting through all the hype and misunderstandings especially between AppFabric in the Cloud vs. Windows Server AppFabric. Also on when to position BizTalk versus Windows Server AppFabric.

BizTalk Benchmark Wizard

Benny Mathew — Wed, 17 Feb 2010 06:04:07 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2010/02/16/138004.aspx

Check this out:

http://msdn.microsoft.com/en-us/biztalk/ee946766.aspx

Static code analysis for BizTalk using BizTalkCop

Benny Mathew — Sun, 01 Mar 2009 12:03:20 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2009/03/01/biztalkcop.aspx

The Problem

If you have been a BizTalk developer long enough, you will know that application lifecycle management has always been a pain point when it comes to managing BizTalk projects.

The Out-of-Box support available in BizTalk Server is not too great for things like automated builds, continuous integration, code analysis, code coverage, unit testing... I can go on and on, but I am sure you get the picture.

Thankfully, most of these are set to improve with the upcoming release of BizTalk Server i.e. BizTalk Server 2009, but there are still improvement areas that I hope the product group will address soon.

One of the other hard things (IMHO) that one needs to do early-on in a BizTalk project is to decide on the naming convention and projects structure for modular deployment. Sometime this is dictated by your customer and sometimes you just follow what has been decided by your company, simple isn’t it? Wish life was that easyJ.

In reality you will find that customers want you to use their own guidelines that are often not complete or not fully thought-out, and you end up using that and try to fill the gap or modify it by adding your own stuff.

Is this so bad?... not if you have gone through this exercise and got it frozen and signed off before you start your development / coding, else, this can turn out to be the cause for endless frustrations for you and your team.

This is especially true for BizTalk projects, where you know that small changes to things like namespaces or typenames can cause a rippling effect across the project, and you end up rebuilding / reconfiguring all of the artefacts.

I have been on multiple projects in the past where naming conventions changed almost every week while the project was halfway through the development phase, so this is not an uncommon thing; however you need to be aware of this and take measures to avoid getting into such situations.

Another challenge is in sticking to the guidelines or conventions that you have so painstakingly chosen and frozen. If you have a large team, you will often see that not all developers are fully aware of the guidelines and conventions that are being followed in the team, especially when developers are constantly moving in and out of the team.

Solution

So what can be done in such situations? One of the things you can do on your BizTalk project is to run a static code analyser as part of your daily builds. I have always been a huge fan of build automation and the niceties that come along with it like automated testing, static code analysis report, code coverage report etc. There is nothing that makes you happier than getting up in the morning gazing at your PDA / mobile device to see that there were no errors or build breaks on the nightly build, especially when you are the Dev Lead / Architect on the project.

You must be thinking... “all this is great... but isn’t this is a BizTalk Solution?... how do you automate checking naming conventions?... especially on things like orchestration shapes, map names, physical ports etc...”. well you are in luck, you can now do static code analysis on BizTalk solutions too, thanks to Elton Stoneman for this cool new tool/plug-in called BizTalkCop available on codeplex.

Check out the links:

http://geekswithblogs.net/EltonStoneman/archive/2008/11/14/introducing-biztalkcop.aspx

http://www.codeplex.com/BizTalkCop

BizTalkCop is essentially a set of FxCop rules that will allow you to inspect BizTalk assemblies for structure and naming convention of your BizTalk artefacts such as Orchestrations, Pipelines, Schemas, Maps etc. It can even look into the deployed solutions inside BizTalk Server (using BizTalk Object Model and BizTalkMgmtDB) to validate the names of your physical ports and receive locations on your deployed BizTalk Server application.

If you are already familiar with FxCop, BizTalkCop provides you the same experience since it is just an extension of the FxCop ruleset.

BizTalkCop is currently is release 1.0 and contains a set of rules based on Scott Colestock's naming conventions, but they're (mostly) configurable so you can modify them to suit your own standards. You can create your new custom rules as well, since the full source code is provided with base classes and frameworks.

Another thing to note is that since BizTalk (2006/R2) projects are not integrated with Code Analysis tool in Visual Studio 2005, you need to download and install FxCop separately before installing BizTalkCop (even if you are using the Team Suite edition of Visual Studio 2005). I have not tried it with BizTalk 2009 Beta yet since I have it on Professional edition of Visual Studio 2008 installed on my VPC (Code Analysis is available only on Visual Studio Team Suite).

The link above provides instructions on how to install and configure BizTalkCop so I am not going to detail it out here.

Demo – The Bad Project

Let me illustrate the usefulness of BizTalkCop by running it on a badly done solution. I created a very simple BizTalk application a la “Hello World” in BizTalk (I call it the ‘BadProject’) without giving much attention to the project structure or naming convention (just used the default names, like we do most of the time). The application basically receives an inventory replenishment request; the request is transformed into a purchase order request and is sent out. For simplicity sake, both the receive port and the send port use FILE adapters.

So essentially, my BizTalk project consists of a single BizTalk project within which there are two Schemas, a Map and an Orchestration as shown in the solution explorer screenshot below:

The orchestration, transformation and schemas are all very elementary as shown below (notice the naming of the artefacts highlighted).

After I have built and deployed the solutions, I fire up FxCop, load the target BizTalk application assembly and hit the Analyze button. I selected only BizTalkCop rules and unchecked others

Note: You also need to make sure that BizTalkCop is configured to point to the right BizTalk application so that it can validate the application level artifacts like the physical post names etc.

You see from the screenshots that many errors (about 28 of them) are generated for such a simple application.

Here are the errors:

The errors are pretty self explanatory so I don’t want to go through each of them.

1. Port names should be prefixed with their direction - start 'ReceivePort1' with 'Receive.'

2. Port names should be prefixed with their direction - start 'SendPort1' with 'Send.'

3. Receive Location names should be prefixed with their Receive Port name - start 'Receive Location1' with 'ReceivePort1'

4. Receive Location names should be suffixed with their transport type - end 'Receive Location1' with '.FILE'

5. Artifacts should be declared in modules with the correct suffix - consider module name 'BadProject.Schemas'

6. Schema names should end with the data format. Format: 'inventoryRequest' is unknown

7. Schema names should begin with the root node. Start: 'inventoryRequest' with: 'Root'

8. Artifacts should be declared in modules with the correct suffix - consider module name 'BadProject.Transforms'

9. Map names should have the format "SourceSchema_DestinationSchema"

10. Artifacts should be declared in modules with the correct suffix - consider module name 'BadProject.Orchestrations'

11. Orchestration members should be Camel cased. Replace Message name: 'InvReqMsg' with: 'invReqMsg'

12. Orchestration members should be Camel cased. Replace Message name: 'PurchaseOrderMsg' with: 'purchaseOrderMsg'

13. Orchestration members should be Camel cased. Replace Port name: 'Port_1' with: 'port1'

14. Orchestration members should be Camel cased. Replace Port name: 'Port_2' with: 'port2'

15. Orchestration shapes should be correctly named - replace 'ConstructMessage_PurchaseOrder' with 'Construct_PurchaseOrderMsg'

16. Orchestration shapes should be correctly named - replace 'Transform_InvToPurchaseOrder' with 'Transform_inventoryRequest_purchaseOrderSchema'

17. Orchestration Shapes should have the correct prefix - start 'MessageAssignment_1' with 'Assign_'

18. Orchestration Shapes should have the correct prefix - start 'Receive_InvRequest' with 'Rcv_'

19. Orchestration Shapes should have the correct prefix - start 'Send_POMsg' with 'Snd_'

20. Orchestration types should be Pascal cased. Replace Port Type name: 'PortType_1' with: 'PortType1'

21. Orchestration types should be Pascal cased. Replace Port Type name: 'PortType_2' with: 'PortType2'

22. Orchestration Types should have the correct suffix - end 'PortType_1' with 'PortType'

23. Orchestration Types should have the correct suffix - end 'PortType_2' with 'PortType'

24. Orchestration Types should have the correct suffix - end 'Port_1' with 'Port'

25. Orchestration Types should have the correct suffix - end 'Port_2' with 'Port'

26. Artifacts should be declared in modules with the correct suffix - consider module name 'BadProject.Schemas'

27. Schema names should end with the data format. Format: 'purchaseOrderSchema' is unknown

28. Schema names should begin with the root node. Start: 'purchaseOrderSchema' with: 'Root'

You can also save the FxCop project and use it in the command line option to generate a pretty cool report as show below.

Demo – The Good Solution

Now I took the same solution and made changes by following the naming conventions and also restructuring the solution to be suitable for modular deployment. You see from the illustration below, I have split the application into three separate projects so that it is not only easier to maintain and saves time during development, but it is absolutely critical that it is broken in this manner from an administration point of view when the solution is in production. Any seasoned BizTalk developer / administrator will understand what I mean here, it’s a whole different topic and don’t want to get swayed into that.

You also see in the solution explorer above that the name of some of the artefacts have also been changed in accordance with the naming convention. For example, the schema names indicate the data format and the root node name of the schema. I also made sure that the physical ports confirm to the naming standards.

The orchestration, map and the schemas below show that changes have been made in the names of the artefacts.

You can see from the below screenshots that the errors have been reduced to just one error.

Conclusion

To sum it all up, I feel FxCop / BizTalkCop makes a great tool to have in your arsenal for that next BizTalk project you are waiting for or even in your current project. Saves you (or the SQA team) a lot of time from having to check for the naming conventions manually on a routine basis and provides you reports on demand.

It can be used both on the individual machines as well as on the build server. You can integrate it with your build scripts so that your application health report now also includes static code analysis among other things. This way you can be assured and can sleep peacefully knowing that the code drop you just sent to the customer conforms to all the naming conventions without having to manually review it at the eleventh-hour, not to mention the amount of time saved. I hope to see many more new rules and improvements from Elton in future releases of BizTalkCop.

Benny Mathew
Sr.Consultant, MGSI (Microsoft Global Services, India).

While I acknowledge that I am an employee of Microsoft., any & all views expressed in this article are mine and do not necessarily reflect the views of Microsoft.

GUI for BizUnit on codeplex

Benny Mathew — Tue, 25 Mar 2008 22:25:17 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2008/03/25/120738.aspx

A good friend and colleague of mine has started a project on codplex to build a GUI for BizUnit,

Check it out here : http://www.codeplex.com/bud

Did you know? – You can read / write to BAM database directly from outside BizTalk.

Benny Mathew — Fri, 15 Jun 2007 03:42:34 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2007/06/14/113219.aspx

You know that BAM is used to gather statistics from your BizTalk application. What you probably don’t know is that:

· You can collect BAM data from your non-BizTalk applications such as external .NET components that BizTalk calls into.

· Tracking profile editor (TPE) is not the only way to collect data you can use a set of APIs available in the Microsoft.BizTalk.Bam.EventObservation namespace to read and write directly into the BAMPrimaryImport database.

Check out some of the links here:

· http://www.topxml.com/rbnews/BizTalk-BAM-activity/re-29195_Business-Activity-Monitoring--GenerateTypedBAMAPI-Tool.aspx

· http://www.developer.com/net/net/article.php/11087_3587296_2

· http://blogs.msdn.com/keithlim/archive/2006/02/02/522649.aspx

· http://blogs.msdn.com/keithlim/archive/2006/03/08/545851.aspx

Open Source BizTalk Utilities on CodePlex

Benny Mathew — Fri, 15 Jun 2007 03:01:08 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2007/06/14/113218.aspx

I am sure most of you (not talking about the BizTalk gurus out there J) have worked with some of the open source utilities available for BizTalk such as the BizTalk Adapter Wizard or the BizTalk Server Pipeline Component Wizard or BizUnit. However what some of you may not know is the wealth of other open source utilities for BizTalk available on CodePlex.

Here are some of the interesting ones and their descriptions as found on CodePlex :

SDC Tasks Library - . The SDC Tasks are a collection of MSBuild tasks designed to make your life easier. You can use these tasks in your own MSBuild projects. You can use them stand alone and, if all else fails, you can use them as sample code.

BizUnit - Framework for Automated Testing of Distributed Systems - BizUnit enables automated tests to be rapidly developed. BizUnit is a flexible and extensible declarative test framework targeted that rapidly enables the automated testing of distributed systems, for example it is widely used to test BizTalk solutions. BizUnit is fully extensible. Its approach is to enable test cases to be constructed from generic reusable test steps, test cases are defined in XML which allows them to be auto-generated and also enables the ‘fixing up’ of Url’s for different environments, e.g. test, staging and production environments. Defining test cases in XML enables test cases to be auto-generated.

BizTalk Server Pattern Wizard - The BizTalk Pattern Wizard is an extensible tool to help you capture, share, and reuse your orchestration best practices. By using the BizTalk Pattern Wizard, you can capture a best practice, turn it into a generalized and configurable pattern, and share that pattern with the rest of your team or the entire BizTalk community. The wizard comes with over a dozen patterns ready for implementation in your next project.

WCF Adapter for BizTalk Server 2006

BizTalk Adapter Wizard for BizTalk Server 2006 - The BizTalk Adapter Wizard for BizTalk Server 2006 is a Visual Studio 2005 project wizard which creates all of the framework code for your custom BizTalk adapter. The adapter wizard is accessible from the Visual Studio menu: File - New - Project - BizTalk projects.

BizTalk Server 2006 Documenter - Creates compiled help files for a given BTS 2006 installation. This tool can be run on an ad-hoc basis using the UI or from the command line as a post build/deploy task to create a compiled help file describing a BTS 2006 installation. It will compile: BTS Host configuration, Send/Receive port configuration, Orchestration diagrams, Schema and Map content, Pipeline process flow, Adapter configuration, Rule engine vocabularies and policies, More… and publish them as compiled help files. Optionally you can embed custom HTML content and custom descriptions for all BTS artifacts to produce a more customized look and feel to the CHM output

BizTalk Server 2006 Orchestration Profiler - Creates CHM report files illustrating the level of coverage for specified BizTalk orchestrations. This tool can be run to gain a consolidated view of orchestration tracking data for a specified period of time to help developers get an idea of how their orchestrations are being processed and how much coverage they are getting in their testing. In addition to simple coverage information the data presented helps to identify latency and code path exceptions by highlighting long running and error prone orchestration shapes.

PowerShell BizTalk Provider - A full PowerShell provider for exposing BizTalk Server as a filesystem. Administer your BizTalk installation. List all applications / orchestrations / Schemas. Stop an application, enlist an orchestration ... Use the full power of the shell to script away the pain of GUI based mass-management.

MapCop - A program for testing BizTalk map files (.BTM) for a set of best practices.

BizTalk Instance Controller - Limit the number of instances of any BizTalk Service.

Have Fun!

Links to materials on GoF Design Patterns

Benny Mathew — Fri, 15 Jun 2007 00:45:14 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2007/06/14/113217.aspx

As I am into BizTalk consulting / development / training for quite sometime now, I feel that I am missing out on hardcore c# programming and the enchantment of object oriented concepts lately.

For instance, the other day I was trying to recollect the implementation details of the Decorator design pattern, and searched for it on the Internet and in the process, got glued at the wealth of information available on design patterns.

So I thought it would be helpful to have a quick reference list of the materials available so that i don't waste time searching for them again.

Here goes:

Abstract Factory
http://www.dofactory.com/Patterns/PatternAbstract.aspx
http://en.wikipedia.org/wiki/Abstract_factory_pattern
http://www.exciton.cs.rice.edu/JAvaResources/DesignPatterns/FactoryPattern.htm

Factory Method
http://www.dofactory.com/Patterns/PatternFactory.aspx
http://en.wikipedia.org/wiki/Factory_method_pattern
http://gsraj.tripod.com/design/creational/factory/factory.html

Facade
http://www.dofactory.com/Patterns/PatternFacade.aspx
http://en.wikipedia.org/wiki/Facade_pattern
http://aspalliance.com/970_Facade_Design_Pattern

Iterator
http://www.dofactory.com/Patterns/PatternIterator.aspx
http://home.earthlink.net/~huston2/dp/iterator.html
http://www.c-sharpcorner.com/UploadFile/acusis/IteratorPattern07072006023056AM/IteratorPattern.aspx

Observer
http://www.dofactory.com/Patterns/PatternObserver.aspx
http://en.wikipedia.org/wiki/Observer_pattern
http://www.exciton.cs.rice.edu/JAvaResources/DesignPatterns/ObserverObservable.htm
http://sern.ucalgary.ca/courses/SENG/609.04/W98/lamsh/observerLib.html

Singleton
http://www.dofactory.com/Patterns/PatternSingleton.aspx
http://en.wikipedia.org/wiki/Singleton_pattern
http://www.exciton.cs.rice.edu/JAvaResources/DesignPatterns/singleton.htm
http://www.javaworld.com/javaworld/jw-04-2003/jw-0425-designpatterns.html

Adapter
http://www.dofactory.com/Patterns/PatternAdapter.aspx
http://en.wikipedia.org/wiki/Adapter_pattern
http://www.exciton.cs.rice.edu/JAvaResources/DesignPatterns/adapter.htm
http://vico.org/pages/PatronsDisseny/Pattern%20Adapter%20Object/index.html

More to follow soon...

Great BizTalk Tips by Marty Wasznicky and Scott Zimmerman

Benny Mathew — Sat, 14 Apr 2007 12:57:00 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2007/04/14/111628.aspx

Marty Wasznicky and Scott Zimmerman have published a great article on MSDN on some best practices while developing BizTalk application.

Please read the detailed article here http://msdn.microsoft.com/msdnmag/issues/07/05/BizTalk/default.aspx

Here are the main points of the article:

1. Always Use Multi-Part Message Types
2. Always Try to Design Orchestrations with Direct-Bound Ports
3. Always Use Separate Internal and External Schemas
4. Never Expose Your Internal Schemas Directly in WSDL
5. Always Optimize the BizTalk Registry for Web Services
6. Always Set the Assembly Key File with a Relative Path
7. Never Overlook Free Sample Code
8. Debug XSLT in Visual Studio

Great job Guys, realy well written!

Benny Mathew
Seamless Integration Solutions
BizTalk Consulting, Development, Training
Bangalore, India

Website: http://www.seamless.in
Blog: http://GeeksWithBlogs.net/benny
BizTalk Usergroup: http://groups.google.co.in/group/b-bug

Wish you all a happy Easter

Benny Mathew — Sat, 07 Apr 2007 10:58:00 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2007/04/06/111037.aspx

Here's wishing you all a happy Easter!

Latest BizTalk 2006 Documentation, PDF vs CHM, which is better?

Benny Mathew — Fri, 30 Mar 2007 11:35:00 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2007/03/29/110312.aspx

Everybody these days seems to be pretty excited about the new PDF version of BizTalk 2006 documentation (Btw, a latest .CHM version is out as well, but there is no hoopla around that).

So thought I'll download the PDF version and check it out as I have been using only the .CHM version extensively. But I was not very impressed with what I saw.

To start with, the PDF version is really huge file 114 MB (whereas the .CHM version is only 44MB, the contents are the same), there is no way my PDA mobile device will load that kind of thing even with external memory card.

The links in the PDF version are not active (not clickable), so you cannot navigate through the document

Searching through the pdf document is very slow and the interface not as user friendly as the .CHM version.

The CHM version on the other hand (IMHO) is more convenient, and the search is blazing fast, you can do incremental search as well using the index tab.
You can even maintain a list of favorite pages just like keeping shortcuts, and the best part is, when I overwrote the old .CHM file with the latest version, all the old favorite pages (shortcuts) appeared intact.

I am not sure why we have a PDF version in the first place, I am sure missing out on something, someone kindly enlighten me please.

Thanks & Regards
Benny Mathew
Seamless Integration Solutions
BizTalk Consulting, Development, Training
Bangalore, India

Website: http://www.seamless.in/
Blog: http://GeeksWithBlogs.net/benny
BizTalk Usergroup: http://groups.google.co.in/group/b-bug

Why you can’t access some promoted / distinguished properties inside the pipeline.

Benny Mathew — Fri, 30 Mar 2007 01:21:00 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2007/03/29/110226.aspx

Some of us when new to building custom pipeline components would have faced this problem, especially with long messages.

You have a custom pipeline component placed downstream after the disassembler component and expect that the disassembler component would have promoted / written the required schema fields into the context, so that your component can reach out to the context and access them, but you find that it fetches a null even when the message has a value.

This is because the pipeline works in a streaming fashion, which means, all components in the pipeline gets started as the message flows through them. For example, before the disassembler has finished its work, your custom component kicks-in and tries to access the context. Now if the value that you are looking for is somewhere at the end of a lengthy message, it would not yet have got read by the disassembler to be able to promote it into the context, but your custom component already tried fetching it and failed.

One easy solution to this is to read the entire stream from your custom component before accessing the context, this will make sure that all previous components have finished its job and have promoted / written the value into the context. However this is not a good approach as you will end up reading the whole message into memory and may cause an Out-Of-Memory error. You will also anyway need to convert it back to a stream and reset the stream pointers before you can pass it on further since the whole BizTalk architecture is stream based.

Another option (the correct way, but a bit more complex) is to wrap the stream in your own stream and hook to its events to notify when it had finished the reading fully, so that you can go ahead and access the context.

If you are building a custom pipeline component, maybe its worth checking out an undocumented class called XpathMutatorStream, defined within the Microsoft.BizTalk.Streaming namespace (available only in the GAC, Microsoft.BizTalk.Streaming.dll)

Check out the following article by Martijn Hoogendoorn for more on this:
http://martijnh.blogspot.com/2006/03/xpathmutatorstream.html

Thanks & Regards
Benny Mathew
Seamless Integration Solutions
BizTalk Consulting, Development, Training
Bangalore, India

Website: http://www.seamless.in
Blog: http://GeeksWithBlogs.net/benny
BizTalk Usergroup: http://groups.google.co.in/group/b-bug

Did you know? (XMLReceivePipeline, Schema validation, XMLDisassembler)

Benny Mathew — Tue, 23 Jan 2007 08:38:00 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2007/01/22/104208.aspx

Did you know?
The default XMLReceivePipeline does not validate the structure of the input XML message. Based on the MessageType of the input message, it will try to lookup a document specification (schema) and if it finds one, will do all the required property promotions / distinguish fields as specified in the schema and also promote the MessageType.
If there is no schema deployed in BizTalk that match the MessageType of the input message, it simply throws an error as shown below.

Finding the document specification by message type <MessageType of the input message> failed. Verify the schema deployed properly.

On the other hand, if you are using a custom pipeline with "XML disassembler" component and you have added your schema in the document schemas collection, it will check to see if the MessageType of the input message match any of the schemas specified in the Document schemas collection ONLY, if not, it will throw an error a shown below.

No Disassemble stage components can recognize the data.

Note that, this error occurs even if you have a schema deployed that matches the MessageType of the input message, but is not available in the Document schemas collection.
The above is obvious / expected behavior... this is just a refresher.

Benny Mathew
Seamless Integration Solutions
BizTalk Consulting, Development, Training
Bangalore, India
http://www.seamless.in
http://GeeksWithBlogs.net/benny

Clean up your BizTalk databases

Benny Mathew — Thu, 29 Mar 2007 07:29:00 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2007/03/28/110122.aspx

Here are a few scripts / stored procedures that most of the BizTalk newbies would love to know.

These scripts are very useful in a development environment to clean up your MessageBox and Tracking databases.

To clean up the MessageBox

First create the stored procedure bts_CleanupMsgBox by running the sql script found in <BizTalk Installation Folder>\Schema\msgbox_cleanup_logic.sql against your MessageBox database(BizTalkMsgBoxDb).
- Note that your BizTalkMsgBoxDb database will already have a dummy stored procedure by the same name which does nothing, so it is important that you run the above script if this is your first time.
Stop all BizTalk services.
Reset IIS server (run iisreset from start/run) if you are running any webservices
Execute stored procedure bts_CleanupMsgbox on your message box database
Execute stored procedure bts_PurgeSubscriptions on your message box database
Restart all BizTalk services

To clean up the Tracking data

Stop all BizTalk services.
Execute stored procedure dtasp_CleanHMData on your tracking database (BizTalkDTADb)
Restart all BizTalk services

Regards

Benny Mathew
Seamless Integration Solutions
BizTalk Consulting, Development, Training
Bangalore, India

Website: http://www.seamless.in/
Blog: http://GeeksWithBlogs.net/benny
BizTalk Usergroup: http://groups.google.co.in/group/b-bug

Books on BizTalk Server 2006

Benny Mathew — Tue, 13 Mar 2007 13:05:00 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2007/03/13/108581.aspx

Here is a handy list of books available on BizTalk Server 2006. Note that some of them are not published as of this writing:

1. BizTalk 2006 Recipes: A Problem-Solution Approach by Mark Beckner, Ben
Goeltz, Brandon Gross, and Brennan O'Reilly (Paperback - Sep 13, 2006)

2. Pro BizTalk 2006 (Pro) by George Dunphy and Ahmed Metwally (Paperback -
Oct 23, 2006)

3. Foundations of BizTalk Server 2006 (Foundations) by Daniel Woolston
(Paperback - Jan 8, 2007)

Not yet published
4. Microsoft BizTalk Server 2006 Unleashed by Stephen Mohr and Scott
Woodgate (Paperback)

5. Professional BizTalk Server 2006 by Darren Jefford and Kevin T. Smith
(Paperback)

--
Regards
Benny Mathew
Seamless Integration Solutions
BizTalk Consulting, Development, Training
Bangalore, India

http://www.seamless.in/
http://GeeksWithBlogs.net/benny
http://groups.google.co.in/group/b-bug

New in BizTalk 2006

Benny Mathew — Sat, 03 Feb 2007 23:39:00 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2007/02/03/105323.aspx

When I do corporate training, I always get asked this question... "So what is new in BizTalk 2006 when compared to BizTalk 2004?" mostly from people who have been using BizTalk 2004. So here I have tried to summarize some of the things. Note this is by no means an exhaustive listing as there are other small but significant improvements.

SETUP

Automatic installation of redistributable components.
In BizTalk Server 2006, the BizTalk team provide a single file that contains the latest version of all redistributable components required by BizTalk Server during installation, eliminating the need for users to individually search for the components.

Simplified setup experience for first-time users.
BizTalk Server 2006 includes a significantly simpler installation experience for the developer, single machine user, and the first time user. Selecting to install the product by using the default setting will allow all of the product's components to be installed under a single account on one machine, with minimal need of user input.

Flexible setup experience for advanced users.
In BizTalk Server 2006, the setup experience has been redesigned to provide greater flexibility to support the advanced user tasked with deploying solutions across large server farm. Once installation completes, the custom configurations can be exported and imported to ease the installation of settings across server farms or development build labs.

Seamless upgrade experience.
Upgrading from BizTalk Server 2004 to BizTalk Server 2006 is an in-place upgrade process. When a customer executes the BizTalk Server 2006 setup on a machine where BizTalk Server 2004 is already installed, the setup program will automatically upgrade the installation to BizTalk Server 2006. All solutions built for this earlier version will work unchanged in the new version.

OPERATIONS

Introducing the application concept
BizTalk Server 2006 formalizes the concept of a BizTalk application by providing a logical container for housing all the artifacts for a given solution. This allows administrators to work with a complete BizTalk application as a unit, greatly simplifying the management, troubleshooting, and deployment of business processes.

Orchestration Zooming
In BizTalk 2004, working with complex orchestrations in the Orchestration Designer sometimes requires a lot of scrolling on the design surface to view different parts of the orchestration. In BizTalk Server 2006, the designer has been enhanced to provide the ability to zoom in and out on an orchestration, allowing a developer to view only those parts of an orchestration of current interest.

Flat File Schema Wizard
The BizTalk Flat File Schema Wizard is designed to simplify the process of creating flat file schemas by providing the following:

The ability to use flat file instances as input
A visual design surface for working with delimited and positional flat file schemas
An interactive wizard-based process for adding elements to the schema and defining flat file related annotations.

Application packaging
Using the BizTalk Administration Console, an administrator can package entire applications into .msi files by just right-clicking on the application and exporting as .msi file. Once packaged, installing these applications to one or more servers is as easy as running / importing the .msi file.

More Powerful Administration Console
In BizTalk Server 2006, all the management functionality is consolidated into the BizTalk Administration Console. Users now have the ability to create artifacts and messaging components by using the console in addition to configuring, deploying, stopping, and starting applications across multiple servers.

Server health monitoring
Using the BizTalk Administration Console's Group Hub page, an administrator can view at a glance the health of currently running BizTalk applications. Rather than requiring an administrator to search for problems, the Group Hub page uses color-coded indicators to display those problems, allowing the administrators to take a more proactive approach to application monitoring.

ADAPTERS

POP3 Adapter
BizTalk Server 2004 provides support for sending email through the SMTP adapter, however, no out-of-the-box support is provided for receiving email. With the introduction of the POP3 adapter in BizTalk Server 2006, BizTalk Server now supports receiving email messages and their attachments using version three of the Post Office Protocol (POP3) from any POP3 compliant mail server.

Windows SharePoint Services Adapter
BizTalk Server 2006 introduces the Windows SharePoint Services Adapter to provide built-in support for accessing and publishing documents stored in Windows SharePoint Services document libraries. Organizations can now link the Office documents and tools familiar to knowledge workers with the enterprise portal systems that run their businesses.

MQ Series Adapter
For BizTalk Server 2004 the MQSeries adapter was available as a Web download. In BizTalk Server 2006, the MQSeries support comes built-in. With the MQSeries adapter, organizations can use XML-enabled MQSeries investments, making it possible to more securely and reliably integrate disparate applications

New Line-of-Business Adapters
SAP, PeopleSoft, JD Edwards, ODBC Adapter for Oracle, Siebel, TIBCO, IBM DB2

RUN TIME IMPROVEMENT

Recoverable Interchange
In BizTalk, an Interchange can contain two or more messages, such as a batch. In BizTalk Server 2004, a validation error in any message contained in the interchange will result in the entire interchange being suspended. With the introduction of Recoverable Interchange in BizTalk Server 2006, only messages that fail validation are suspended, and the messages can be resumed once the error is corrected.

Failed message routing
In BizTalk Server 2004, when a message fails within a receive pipeline, the message is suspended and cannot be subscribed to by end points. In BizTalk Server 2006, functionality is introduced to allow orchestration and send ports to subscribe to failed messages. When used appropriately, failed message routing can be used for notifying users of failed messages or building rich error handling and message repair capabilities.

Message resume
In BizTalk Server 2004, message resume is only enabled for the send side but not for the received side. In BizTalk Server 2006, nearly all receive-side messages can now be resumed through the Health and Activity Tracking tool.

In-order message delivery
In BizTalk Server 2004, end-to-end, in-order processing is accomplished only when using MSMQT as a transport. In BizTalk Server 2006 this capability is expanded to support any send port that uses the same ordering semantics that outbound MSMQT does in BizTalk Server 2004 today.

Large Message Transformation
In previous releases of BizTalk Server, mapping of documents always occurred in-memory. While in-memory mapping provides the best performance, it can quickly consume resources when large documents are mapped. In BizTalk Server 2006, large messages will be mapped by the new large message transformation engine, which buffers message data to the file system, keeping the memory consumption flat.

Executing Pipelines within Orchestration
A new capability in BizTalk 2006 is the ability to synchronously call a pipeline from within an Orchestration. This enables orchestrations to leverage the message processing encapsulated within a pipeline (either send or receive) against a body of data without having to send that data through the messaging infrastructure.

BAM IMPROVMENTS

BAM portal
BAM has been enhanced in 2006 to include an out-of-the-box BAM portal that allows an information worker to easily examine and configure BAM information. Using the BAM portal, an information worker can select a particular instance of some business process to monitor and then choose a specific BAM view into the process to get a different perspective on the key performance indicators being monitored-sized image

BAM alerts
Through integration with SQL Server Notification Services, BizTalk Server 2006 allows BAM information to be delivered as notifications through e-mail or another communication channel to the business user, enabling them to make real-time business decisions based on this information.

BAM Web service
BAM in BizTalk Server 2006 builds on the existing Web services available in BizTalk Server 2004 and adds new Web service interfaces to expose the query of aggregate and instance data, creation of alerts, and retrieval of BAM configurations. The enhanced Web services interface can be utilized by custom applications to expose BAM functionalities within their user interface.

Benny Mathew
Seamless Integration Solutions
BizTalk Consulting, Development, Training

Bangalore, India
http://www.seamless.in
http://GeeksWithBlogs.net/benny

XLANG/s is not C# (Dos and Don't / Difference between XLANG/s and C#)

Benny Mathew — Thu, 15 Feb 2007 07:42:00 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2007/02/14/106295.aspx

It is normal to find people who are new to BizTalk, writing expressions just like they do in C# programming language, well this works most of the time, it is when it doesn't work that you start wondering what is wrong, and end up wasting a lot of time figuring out the problem and some more time finding the solution.

BizTalk uses a language called XLANG/s which is suitable for expressing the process flow (Just like BPEL) and resembles C# syntax when writing expressions, but it is not C# you are using there.

One does not usually get to learn the complete XLANG/s syntax as we construct the orchestration visually by dropping and configuring shapes on to the orchestration surface. You can however see the XLANG/s if you open the orchestration in a text editor like Notepad.

Here is a nice blog post by Charles Young mentioning list of things that are supported in BizTalk.
http://geekswithblogs.net/cyoung/articles/3820.aspx

Benny Mathew
Seamless Integrations
BizTalk Consulting, Development, Training
Bangalore, India
http://www.seamless.in
http://GeeksWithBlogs.net/benny

Middleware Mania 2007

Benny Mathew — Wed, 10 Jan 2007 04:57:00 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2007/01/09/103139.aspx

Last Saturday we had a one day event called Middleware Mania 2007 at BDotNet (http://groups.msn.com/BDOTNET), India's largest and most active .NET user group in Bangalore with 8750+ members. Held in one of Microsoft's building, which received a good number of people right from junior developers, solutions architects to project managers.

I got invited to present 2 sessions at the event, one was on the New Features in BizTalk Server 2006 and the other was on Adapter Development in BizTalk 2006.

You can download the presentations and code form the link below.

http://www.seamless.in/files/NewFeaturesInBizTalk2006.zip

http://www.seamless.in/files/AdapterDevelopmentinBizTalk2006.zip

For me, this was a good event after a long time... there were other presenters who gave good presentations on RosettaNet Implementation Framework (RNIF), Introduction to BizTalk, Enterprise Integration Patterns etc. Received a lot of good feed back about the presentations.

Over all it was fun…, the audience was quite enthusiastic, they even did a little leg-pulling of one of the presenter and one funny lady who was there to promote her company and was woo-ing everyone to join her company J

Regards

Benny Mathew
Seamless Integration Solutions
BizTalk Consulting, Development, Training
http://www.seamless.in
http://GeeksWithBlogs.net/benny

Mama Pendse (The artificial intelligence based chatterbox)

Benny Mathew — Fri, 12 Jan 2007 00:32:00 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2007/01/11/103353.aspx

Have you tried adding this guy / thing 'mama_pendse' (mama_pendse@yahoo.com) to your yahoo messenger buddy list?

Did you know? (Setting output filename to a field value of the message without any custom code or orchestration)

Benny Mathew — Fri, 09 Mar 2007 19:03:00 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2007/03/09/108323.aspx

I saw this question in the BizTalk newsgroups:

> Hi,
> Is it possible to set the filename of a message with a value inside it
> which is available as a promoted property..
> I know this can be done using a custom pipeline component .But is there a
> way without it?
> I have no orchestrations in my solution..
>
> TIA
>

Yes, it is possible!... here is how.

1. Right click on the schema node you want to promote, select Promote-->show promotions.
2. On the “property fields” tab, click on the folder icon (to add new property
schema)
3. Expand the references, expand Microsoft.BizTalk.GlobalPropertySchemas and select FILE.bts_file_properties.
4. Add your node as promoted and select ReceivedFileName in the property column as the property to which to promote to.
5. In the send port configuration use the macro %SourceFileName%
6. Don’t forget to use XMLReceive pipeline at the receive port.

That’ it! No custom pipeline no orchestration. This can be useful in content based routing scenarios where there are no orchestrations involved.

The XMLReceive pipeline will overwrite the source file name value in the context with the value of the field inside the message.

Regards

Benny Mathew
Seamless Integration Solutions
BizTalk Consulting, Development, Training
Bangalore, India
http://www.seamless.in
http://GeeksWithBlogs.net/benny

Bangalore now has as local BizTalk user group (b-bug)

Benny Mathew — Fri, 23 Feb 2007 09:35:00 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2007/02/22/107014.aspx

Bangalore BizTalk User Group (b-bug)

Check it out here: http://groups.google.co.in/group/b-bug

Benny Mathew
Seamless Integration Solutions
BizTalk Consulting, Development, Training
Bangalore, India
http://www.seamless.in
http://GeeksWithBlogs.net/benny

Back to bloging

Benny Mathew — Fri, 22 Dec 2006 00:53:00 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2006/12/21/101781.aspx

I am back to regular bloging... Year 2006 has been very different and unique, I experienced a lot of things that has made me more mature, faced a lot of ups and downs, learnt new things, even learnt a little people management :)

All this also gave me the strength and courage to quit Two Connect and startup my own company Seamless Integration Solutions (http://www.seamless.in/).

Hope 2007 will be a good year.

Wish you all a Merry Christmas and a Happy and Prosperous New Year.!!!!

Cheers!

Regards
Benny Mathew
Seamless Integration Solutions
BizTalk Consulting, Development, Training
http://www.seamless.in/

Did you know (Mapping multiple schema nodes in one go using the "AutoLink By" property)

Benny Mathew — Wed, 21 Feb 2007 09:34:00 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2007/02/20/106802.aspx

Most of you might have come across situations where you have to map a huge number of elements from the source schema to the target schema using the BizTalk Mapper.

There is an easy way to do this depending on your source and target schema structures.

Actually there are two scenarios where you can make use of this:
1. If your source and target schemas have an identical structure, or even if a part of your source schema and target schema have an identical structure.

2. If your source and target schemas have an identical node names, or even if a part of your source schema and target schema have an identical node names.

Here's how to do it:
1. Bring up your BizTalk mapping editor and load the source and target schemas

To Auto Map by Structure
2. Click on the grid surface of the map and in the properties window and set the "AutoLink By" property to "Structure" (which is actually the default value)

To Auto Map by Node Name
2. Click on the grid surface of the map and in the properties window and set the "AutoLink By" property to "Node Name"

3. Map the nodes by holding down the SHIFT key and drawing a line from the parent node of the source schema to the parent node in the target schema as shown in the figure below:

Note that in our figure above, the node names as well as the structure is the same, so you could set the "AutoLink By" property to either "Structure" or "Node Names" in this case.

Benny Mathew
Seamless Integration Solutions
BizTalk Consulting, Development, Training
Bangalore, India
http://www.seamless.in
http://GeeksWithBlogs.net/benny

Did you know? (you can do XML debatching without custom pipeline)

Benny Mathew — Thu, 15 Feb 2007 12:47:00 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2007/02/15/106329.aspx

Everybody knows that you need to use a custom pipeline to debatch an XML file, where you use a custom XML disassembler pipeline component to specify the Envelop schema and Document schema.

Well, you can do it with the built-in XMLReceive pipeline as well, provided you don't require advanced features like preserving header information, 'Recoverable interchange processing' or validating document structure etc.

Do the following:

1. Create and deploy an envelope schema (Set the Body XPath property of the root node to the appropriate record in the envelope schema)
2. Create and deploy the document schema
3. Make sure the message you are receiving is an instance of the envelope schema
4. Make sure that the namespace and root node name of the document schema aligns with the MessageType of the debatched message (you may have to set the 'Element FormDefault' property of the Envelope schema to 'Qualified')

All of the above are something that you will anyway make sure even when using a custom pipeline, so nothing new.

Benny Mathew
Seamless Integrations
BizTalk Consulting, Development, Training
Bangalore, India
http://www.seamless.in
http://GeeksWithBlogs.net/benny

Debatching a flat file in BizTalk 2006

Benny Mathew — Mon, 06 Feb 2006 11:46:00 GMT

Originally posted on: http://geekswithblogs.net/benny/archive/2006/02/05/68249.aspx

Sometimes you receive a batch file containing multiple records and you have to split each record into a separate message and process them individually.

This kind of splitting of a batch file is also known as debatching.

Depending on your requirement, you can either debatch an incoming message inside the pipeline or inside the orchestration.

The technique used to debatch a message inside the pipeline depends on whether the input is an XML file or a flat file.

For an XML file, you will need to use an XML disassembler, an envelop schema and a document schema, which I will discuss some other time on separate blog post / article.

Here I will show you how you can debatch a flat file.

Let us assume that we need to debatch a flat file that contains the following structure.

Field1,Field2,Field3,Field4,Field5,Field6
Field1,Field2,Field3,Field4,Field5,Field6
Field1,Field2,Field3,Field4,Field5,Field6
Field1,Field2,Field3,Field4,Field5,Field6
Field1,Field2,Field3,Field4,Field5,Field6
Field1,Field2,Field3,Field4,Field5,Field6

First you need to create a flat file schema that looks similar to the one shown in the figure below.

Make sure you have specified the correct child delimiters for the Root (0x0D 0x0A) node as well as the Record (,) node.

Root Node:
Child Delimiter Type = Hexadecimal
Child Delimiter = 0x0d 0x0a
Child Order = Infix or Postfix

Record Node:
Child Delimiter Type = Character
Child Delimiter = ,
Child Order = Infix

Now select the Record node and set its MaxOccurs property to 1. This is actually the key to debatching. The flat file disassembler will automatically separate each record into individual message.

Important: If you have set the Child Order Property of the Root node to Infix, make sure you have also set the property 'Allow Message Breakup At Infix Root' to Yes

Next, as you would normally do, add a receive pipeline to the project and add a flat file disassembler and set the Document schema property to the schema we created.

That’s all there is to it.

Now suppose you have a flat file that also has a header record or a trailer (footer) record or both as shown below.

FirstName,LastName,RegNo,School,City
SubjectName1,Score1
SubjectName2,Score2
SubjectName3,Score3
SubjectName4,Score4
SubjectName5,Score5
SubjectName6,Score6
SubjectName7,Score7
SubjectName8,Score8
TotalScore

You would normally create a flat file schema as shown below to parse as one single message.

But for the purpose of debatching the body records, you need to create 3 separate schemas as shown below. Make sure the MaxOccurs of the Body record is set to 1.

And in the flat file disassembler, you need to set the Header schema, Document schema and Trailer schema properties appropriately.

Note: You might need to set a unique Tag Identifier property for each of the schemas, and also prefix the same tag identifier on the records as shown below, so that the flat file parser can distinguish the header, body and footer records.

HFirstName,LastName,RegNo,School,City
BSubjectName1,Score1
BSubjectName2,Score2
BSubjectName3,Score3
BSubjectName4,Score4
BSubjectName5,Score5
BSubjectName6,Score6
BSubjectName7,Score7
BSubjectName8,Score8
FTotalScore

It is possible to preserve the header record into the message context, by setting the Preserve header property of the flat file disassembler to true, so that you can use it as a header record for each of the debatched message.

To use the preserved header, you need to set the Header schema property of the flat file assembler to the same header schema you used in the disassembler. Here’s how the output will look.

HFirstName,LastName,RegNo,School,City
BSubjectName1,Score1

HFirstName,LastName,RegNo,School,City
BSubjectName2,Score2

HFirstName,LastName,RegNo,School,City
BSubjectName3,Score3

HFirstName,LastName,RegNo,School,City
BSubjectName4,Score4

HFirstName,LastName,RegNo,School,City
BSubjectName5,Score5

HFirstName,LastName,RegNo,School,City
BSubjectName6,Score6

HFirstName,LastName,RegNo,School,City
BSubjectName7,Score7

HFirstName,LastName,RegNo,School,City
BSubjectName8,Score8

Regards
Benny Mathew
Seamless Integration Solutions
BizTalk Consulting, Development, Training
http://www.seamless.in/