News

My Stats

  • Posts - 26
  • Comments - 25
  • Trackbacks - 55

Twitter










Recent Comments


Recent Posts


Archives


Post Categories


November 2004 Entries

The BizTalk SQL Server Adapter isolation level


The isolation setting is fixed and is SERIALIZABLE. So beware that even your most basic receive locations that only execute those very simple SQL statements such as 'select * from tablename' can generate locks. BTW, this is a general misconception: usually people think that select-statements never lock resources. Nothing is less true of course.
 
While having a shared lock on a range of keys, in addition to the keys themselves having locked, no records can be inserted. 
Here's a sample to demonstrate the effects of the SERIALIZABLE isolation setting:
 
Open your SQL Query analyzer
Open 2 seperate query windows to the local Pubs database
Copy & paste these samples statements:
 
Window A:
 
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
BEGIN TRAN
Select * from authors where contract = 1
GO
 
Window B:
 
BEGIN TRAN
INSERT INTO [pubs].[dbo].[authors]([au_id],[au_lname], [au_fname],  [contract])
VALUES('666-66-6666','Grego', 'El', 1)
GO
 
Now, if you try to execute both queries you will see that the second query will always be blocked by the first, irrespective of the order. You can immediately unblock the process by typing, selecting and executing 'ROLLBACK TRAN' in the blocking transaction's window. Now repeat this test and replace 'SERIALIZABLE' with 'READ COMMITTED' and you will see that when you first start batch A, you can still execute Batch B simultaneously (which wasn't so with the SERIALIZABLE level).
 
Here are the isolationlevel-enumerator values from system.data:
 
[Flags]
public enum IsolationLevel
{
      // Fields
      Chaos = 0x10,
      ReadCommitted = 0x1000,
      ReadUncommitted = 0x100,
      RepeatableRead = 0x10000,
      Serializable = 0x100000,
      Unspecified = -1
}
 
The BizTalk SQL adapter always uses 0x100000. You can check this by viewing the requested locks in Enterprise Manager:
 
 
Now, I believe the above will rarely be a problem in real-life. You should only expect performance problems when you have a lot of transactions and a lot of simultaneous lock requests for the same heavy resource while having a bad database-design (having no or having the wrong indexes). You should also know that lock waits are perfectly normal: a simple wait for a lock is different from a deadlock. The waiting process will get the lock anyway when the process that's holding the lock completes.
If you use SERIALIZABLE my best advice is to tune you sql statements for performance (also having correct db-design, normalization, the right indexes,...) in order to make your select statement execute as fast as possible .
 
**UPDATED**
 
Is there a solution?
  • If you try to add the 'SET TRANSACTION ISOLATION LEVEL READ COMMITTED' to a select-statement based receive location then you may have no issues at design time. But when you try to add a RL based on this SQL statement in the  BizTalk Explorer you will get a 'The SQL statement must be either a select or an exec' error.
  • You can add ‘With (Readcommitted)’ to the tables in the select, or data modification statement. 'Select * from authors with (Readcommitted) where contract = 1'. This will override the default Serializeable isolation level, and keep the number of locked records to a minimum. Credits go to Dirk Gubbels from Microsoft...
SQL Server locking experts' comments are very welcome...

posted @ Thursday, November 25, 2004 4:35 PM | Feedback (9) |


About publisher policy assembly chaining


I've just found out the hard way that GAC'ed publisher policy assemblies do not chain. When you have a policy.1.0, redirecting the binding of assembly v1.0.0.0 to assembly v1.1.0.0 and also having a policy.1.1, for redirecting the binding of assembly 1.1 to assembly 1.2, this doesn't result in a binding to the assembly 1.2 when your app requests 1.0 (but rather to 1.1). If you want this kind of binding-behavior I guess you will have to use publisher policy versioning.
 
Why do I tell all this? Well, I've never read anything about this before so I thought it would be nice to mention here.
I also want to thank Alan Shi, who confirmed and explained this binding behavior to me. For more excellent information regarding the GAC and Fusion you should definitely visit his blog here.

posted @ Tuesday, November 23, 2004 11:14 AM | Feedback (3) |


The BizTalk Configuration Dilemma


I’ve watched several times Jurgen Willis’ excellent online presentation about the Business Rule Engine (BRE). One of the BRE usage scenarios demoed by him contains a sample orchestration that uses the BRE to dynamically configure a delay time.
This confused me since it added another option to accomplish dynamic configuration of business processes. We are definitely facing a configuration dilemma now; here are some of the alternatives:
 
  • Config Files
    Use the default .NET config files as the store (BTSNTSvc.exe.config) for your key-value pairs or custom types. You can easily read the settings using the default .net classes from inside your orchestrations.
    This is definitely the easiest option. But it makes your business processes host-instance depended (every host instance can be differently configured). It’s also not easily deployable, when having different environments you will have to manually copy paste your configuration sections, you could easily make mistakes. I guess there are no tools available for the business users to manage the values.
 
  • Business Rule Engine
    Although I have the feeling the BRE and its terminology is not really geared towards this simple functionality (storing key-value pairs). Most of the BizTalk included samples use schema-facts, some of them use class-facts but none addresses the config management purpose which was demoed in the presentation.
    I’ve tested a couple of things myself, including calling the BRE from code inside an orchestration by using several Stringbuilder instances or a Hashtable as the argument(s). This seemed a very strange solution to me (it’s not easy to define the rules/vocabularies when having several instances of the same class).  Another option is to create a custom configuration class which gets and sets the values, this will simplify the vocabulary. Or you could always use the classic approach and create a custom schema to hold your configuration values.
 
 
Finally I emailed Jurgen, who appeared to be a very friendly and helpful man. He pointed me out that the BRE is in fact not specifically targeted at this scenario and that, in general, it focuses more on complex types than on using value types (especially when multiple instances are evaluated in the same policy). 

posted @ Wednesday, November 17, 2004 5:25 AM | Feedback (10) |


Executing pipelines from inside an orchestration: Introducing the LOOPBACK adapter


Let's go over some of the out-of-the-box options for executing a pipeline from inside your orchestration.
 
It struck my mind first that the MSMQT adapter IS in fact the messagebox.
 
  1. MSMQT
 
  • Set up a schedule with 1 MSMQT send port and 1 MSMQT receive port sharing the same MSMQT queue name (for example ' loopback' would make a lot of sense)
  • Create a correlation type and set based on the my MSMQT label property.
  • Assign the MSMQT message-label to a newly created GUID inside your orchestration.
 
Try to do a send and a receive and you will see that your pipelines will be executed. As a test I could successfully parse a message from inside a schedule.
 
Another very simple option is using the HTTP adapter, credits go 100% to Scott Colestock. Scott used this as just a temporary approach to get past this problem, and then switched to the loopback adapter below...
 
  1. HTTP
 
  • Use a solicit-response send port 
  • Use bogus HTTP page that simply echoes the content back.
 
<% @ webhandler language="C#" class="LoopbackHandler" %>
using System;
using System.Web;
using System.IO;
public class LoopbackHandler : IHttpHandler
{
   public bool IsReusable
   { get { return true; } }
  
   public void ProcessRequest(HttpContext context)
   {
      using(StreamReader sr = new StreamReader(context.Request.InputStream, true))
      {
         StreamWriter sw = new StreamWriter(context.Response.OutputStream, sr.CurrentEncoding );
         sw.Write(sr.ReadToEnd());
         sw.Flush();
         sw.Close();
      }
   }
}
 
This can be easily explained because it requires no knowledge of correlation types and correlation sets, since you are using request/response ports the whole way. If simplicity is the main goal then this solution is probably superior to the MSMQT one.
 
  1. Custom LOOPBACK Adapter.
 
I also had another idea. Why not make your own loopback adapter by writing a bogus solicit-response adapter that returns a response BizTalk IBaseMessage that is a copy of the original request message. Recreate the original individual message-parts and copy of the request stream to the response stream. It's very simple - download my sample loopback adapter here.
 
  • It's a custom coded VB.NET non-batched but non-blocking (async) adapter that really doesn't do anything.
  • Uses limited memory-footprint, streams to disk. 
  • To install use the MSI package. Don't forget to add the adapter by using BizTalk administration...
  • It auto generates the transmit location URI (in fact it's a GUID). I also added a Boolean property to specify whether you want the original message and part properties to be copied on to the response-message and parts.
 
Here are some of the benefits:
 
  • Trigger pipeline execution from inside your orchestration. Just define your outbound and inbound pipelines and they will be executed.
  • Send pipeline errors can be catched by adding a deliveryfailure-exception handler. Also adapter exceptions can be catched there, but since this adapter really doesn't do anything this will be very rare (at least I hope so).
  • Receive pipeline errors can be catched by adding a general SOAP-exception handler.
  • Execute mappings from inside your orchestration by defining a map on the inbound and outbound port. When you use the later, mapping failures will be catched by the SOAP-exception handler.
  • It's a black-hole adapter if you want. It can be used to ignore the processing of certain incoming messages. Make a send-port, name it 'VANISH' and subscribe to a given message-type or receive-port name for example.
  • It's very useful for doing in-order processing. You can for example receive messages from MSMQT through a pass-through pipeline (which is recommended BTW) and process messages in-order from start-to-end while doing exception-handled parsing/validation/mapping by implementing a serial convoy.
 
Here's a sample project that demonstrates some of the benefits.
 

Remarks:
 
  • The code can definitely be improved. It has been written very quickly in the scope of a POC. Alternatively you could make a better one using the adapter base classes and the wizard. In fact I'm hoping the community will see the benefit of this loopback adapter and someone else will make a better one. I'm not (yet, ha!) a streaming and multi-threading expert.
  • Apparently it is advised to return a read-only forwarding stream to the receive-pipeline. At least until SP1 arrives. I just used the VirtualStream class from the BizTalk Streaming assembly.
  • Do not forget, and as usual: there's no warranty at all. I haven't tested this very thoroughly yet so…use the adapter at your own risk.

posted @ Wednesday, November 10, 2004 3:24 AM | Feedback (20) |