Geeks With Blogs

News View Michael Stephenson's profile on BizTalk Blog Doc View Michael Stephenson's profile on LinkedIn
Michael Stephenson keeping your feet on premise while your heads in the cloud

Article Source: http://geekswithblogs.net/michaelstephenson

Following a recent post about the different approaches to caching you might consider when implementing reference data mapping in BizTalk one of the things that stands out most was that the solutions where a team had used a caching approach often resulted in them not using the BizTalk Cross Referencing features. As I've mentioned many times I prefer to use this unless there is good cause not to (there are reasons where you might not want to) but I feel development teams often ignore or don't consider the impact of adding custom databases to a solution without consideration for the extra work this requires in development, testing , deployment and management.

In most cases why do you want to do this when you already have a data store designed for this purpose? One criticism I would make of BizTalk is that the product does not do a very good job of making it easy for people to use the cross referencing featured from a developer experience but these can all be worked around with few problems.

Anyway I have decided that I will produce this sample showing how I have combined the use of NCache and BizTalk Cross Referencing to get a solution which does not need custom databases yet can still have a high performance caching solution which will not increase the BizTalk hosts process memory unnecessarily. The sample can be downloaded from the bottom of the article.

Prerequisites for the sample

You can obviously review the code in this sample, but if you want to run it you will need to do the following things:

  • Install NCache Express Edition

NCache Express Edition is available free from the following link http://www.alachisoft.com/ncache/. I assume you will be installing this to the default location, but if not you might need to modify the msbuild script where I configure NCache.

  • Modify Cross Referencing Setup

In the SetupFiles.xml file in the solution it contains the xml used to setup the cross referencing data in BizTalk. This file requires absolute paths to work so you will need to tweak these to suit your location. The below picture shows what the xml looks like.

Setting up the sample

In the sample you will notice there is a file which is called Setup.cmd. If you run this file it will perform the appropriate actions to configure things for this sample. The actions it will take are as follows:

  • Stop the cache in NCache if it is already running
  • Clear the BizTalk cross referencing tables
  • Stop the NCache windows service
  • Copy the pre-configured NCache config files to their appropriate places to configure NCache with the cache we will use in this sample
  • Start the NCache windows service
  • Start the custom cache
  • Load the BizTalk cross referencing tables using BTSXRefImport

These actions are all done in an msbuild script (picture below) which should make it easy for you to see how this is done.

You should now be able to run the sample.

My Cross Referencing Component

To keep the sample simple I have developed a component which will provide an interface which is the same as that provided by BizTalk cross referencing. I provide a class called CrossReferencingFacade which implements the façade pattern to give you an easy way to obtain the common and application specific id's. The below picture shows this:

There is also a test in the test project which shows how to consume this component. It is as easy to consume as the BizTalk cross referencing dll. If you look in the CrossReferencingManager class you will see there are two key methods which are discussed below:

  • LoadXRefIDData

This will use some data access code to retrieve all of the cross reference data for one specific type of cross reference (xrefId) for example all of the mappings for Product Type. It will then return them to the calling method.

  • GetCrossReferenceIDData

This method will check the cache to see if the data is already there for the requested cross reference data type. If present it will be returned from the cache, and if not the data will be loaded using the LoadXRefIDData, and then placed in the cache.

The result of this means the data is cached once for both the GetAppID and Get CommonID methods.

One interesting bit on this (and there may be better ways to do this) is that to allow you to search for the appropriate mapping data from the same source by both CommonID and AppId I have held the data in a container object which houses a dictionary of the reference data with a unique key for each one, and then I also have 2 dictionaries of the app specific keys and common id keys. This just makes it possible to hold the data just once but search for it in different ways. As mentioned I'm sure if I have a think about this there are better ways to do this but it will do for this sample. (note although this last bit may have sounded overly complicated this is encapsulated so the consumer does not need to care about this)

The NCache bit

So from the above hopefully you can see I have provided a handy way to use BizTalk cross referencing within this sample. The next thing to discuss is NCache. I believe there are a number of additional features which come with the Enterprise version such as security features and tools to manage caches so for any production usage I would definitely recommend that version. For the purposes of this sample the Express edition is more than sufficient.

You can see from the below picture that the code to interact with NCache is very simple.

(Note: In the above picture you can only partly see it, but the cache allows you to insert objects with expiration parameters and also a cache dependency)

With NCache one of the things that I like is I don't need to worry about configuration within my application, so you will notice there isn't any app.config files in the sample which are used by the consumers of the cache. That said there is some configuration for the caching service. You will see in the NCache folder there is some config which controls how your caches are setup. The below picture from the config.ncconfig file shows how I have configured my cache for this sample.

You will notice here that I'm able to control if my cache runs in process or out of process which is how I'm able to move the cached data outside of my Biztalk host process and there are a bunch of other possible settings. This configuration is held along side the caching service.

Plugging it into BizTalk

This component is now very easy to add to a BizTalk implementation by using the call external assembly feature of the scripting functoid to call the component. You will now be able to use the cross referenced features of BizTalk but with out of process caching of the data.

Summary

Hopefully you will see that it is not that difficult to implement a good caching solution which has addresses a combination of the considerations I discussed in my previous article. I quite like this approach and based on my limited experience of the different caching systems available I would probably at present choose NCache over Memcached because it is an established 3rd party system which comes with additional tools and features to support it. That said I will be keeping an eye on the "velocity" project as I think this will definitely be one to watch for the future.

If you have any experiences with this I would be interested to hear your thoughts. The sample is available below:

Posted on Sunday, September 21, 2008 7:48 PM BizTalk , Caching | Back to top


Comments on this post: NCache and BizTalk Cross Referencing Example

# re: NCache and BizTalk Cross Referencing Example
Requesting Gravatar...
This is a great article! well done sir! I'll definitely be looking at something like this going forward.

Where is the download link though?
Left by Ryan CrawCour on Sep 23, 2008 8:57 AM

# re: NCache and BizTalk Cross Referencing Example
Requesting Gravatar...

Hi,

You can downlaod NCache Express by going to http://www.alachisoft.com/rp.php?dest=/ncache/ncache_express.html page.

And, thanks Michael for a very nice article.

Left by Iqbal Khan on Oct 31, 2008 1:49 AM

# re: NCache and BizTalk Cross Referencing Example
Requesting Gravatar...
Hi,
Great article, I have one question:
Suppose that the mechanism is in production and we need to add/modifiate/delete some reference data and mapping from different applications
How can I do that without interupt the production?
Between the purge.sql and import data, some messages coud arrive and crash?
Left by JC Esna on Oct 13, 2010 4:54 AM

# re: NCache and BizTalk Cross Referencing Example
Requesting Gravatar...
They craft real coffeehouse productions with an espresso manufacturer, coffeemakered.com Consider exactly how you like your coffee to taste, the amounts.
Left by deric on Sep 19, 2017 6:13 PM

# smokerify
Requesting Gravatar...
actually emerged as the very best alternative for newbies with smokerify If you have dealt with or presently deal with the trouble.
Left by jarry on Nov 26, 2017 5:06 PM

Your comment:
 (will show your gravatar)


Copyright © Michael Stephenson | Powered by: GeeksWithBlogs.net