Introduction
The compensation model in BizTalk Server 2006 provides a versatile mechanism for addressing an extensive range of business process scenarios. It is used in situations where some condition arises that invalidates the outcomes of previously completed units of work associated with the same business activity. In these scenarios, it is generally necessary to revisit the completed units of work, inspecting the state of the system as it existed at the completion of each of those stages, and taking appropriate actions to undo the work, undertake appropriate compensatory actions or, at the very least, maintain an audit log and/or provide notifications.
A text-book example of such a scenario is where, in a retail ordering/delivery process, a customer cancels an order that has already been partially met, but not yet delivered. Cancellations tend to be indeterminate with regards to when they occur. The goods may have been picked. They may have been dispatched and may therefore be en route to the customer premises. Invoices may already have been created and sent, etc. The ordering/delivery process must contain mechanisms for handling cancellation robustly, wherever it occurs within that process.
BizTalk Server 2006 compensation is an enabling mechanism designed to be used within such scenarios. It provides a flexible means for performing back-tracking over discrete units of work that have been completed, whilst automatically ignoring units of work that remain incomplete. It allows compensatory logic to access the process state as it existed at the point a unit of work was deemed complete. In conjunction with exception handling and transaction management, it allows orchestration designers to define paths through the business process that will be invoked when compensatory actions must be performed. It does not address all issues and scenarios, however, and should be considered as just one part of the toolset required to implement failure handling and recovery logic.
Long Running Transactions
BizTalk Server compensation is linked inextricably to the concept of long running (L-R) transactions. L-R transactions are units of work which have high-level (business-orientated) transactional semantics, but which cannot appropriately be undertaken using ‘classic’ ACID (atomic, consistent, isolated and durable) transactions. This is generally (though not always) because the unit of work will take some significant time to complete. ‘Significant’ can mean anything from a few seconds to months or even years. The ACID model is inappropriate because it would require long-term data locks and serialisation of work that must be done concurrently. L-R transactions are often used where the process requires human intervention and where interactions span major organisational boundaries. There may be a significant degree of indeterminacy involved in the sub-actions contained within an L-R transaction. For example, a business process may be subject to a number of external factors which cannot be controlled or anticipated. Indeterminacy sits uncomfortably alongside the traditional ACID transactional model, and requires a less rigid approach to be taken in servicing the overall transactional semantics.
It is common to attempt to define L-R transactions in terms of ‘reduced’ ACID properties. For example, Microsoft states that L-R transactions “possess consistency, and durability, but not atomicity and isolation”. Within the context of BizTalk Server, this statement is broadly correct, but is a little simplistic. It could be taken to imply that L-R transactions are a species of ‘defective’ ACID transaction. This entirely misses the point, and is an incorrect way to view L-R transactions. ACID transactions presuppose the existence and use of transactional resource managers that enforce a set of well-defined constraints. L-R transactions do not make this presupposition, and consequently have a more general-purpose applicability. A relational database applies ACID transactions to data stored within database tables. An orchestration, however, must often manage all kinds of data represented throughout a business process. This data may be internal orchestration data or external data. External data may be managed across a variety of otherwise unrelated systems and resource managers which may, or may not support ACID transactions.
L-R transactions can be defined very simply as a facility for modelling real-world business transactions without conforming to the ACID model. A definition of a business transaction (based on the 1999 US Uniform Electronic Transactions Act) is “an action or set of actions occurring between two or more parties or actors relating to the conduct of business, commercial, or governmental affairs”. An L-R transaction is not constrained in the same way as an ACID transaction, although it may exhibit some of the same characteristics. It may also be decomposable into a number if inner transactions, some or all of which may be ACID transactions.
Sagas
The term ‘saga’ is not often used within the BizTalk community, but is deeply connected to the concept of L-R transactions and has a venerable history. Sagas were defined by Hector Garcia-Molina and Kenneth Salem. Their paper, published some twenty years ago, described Sagas as a mechanism for handling ‘long lived’ transactions within relational databases. The idea was that, in some cases, a long-lived ACID transaction could be decomposed into a sequence of smaller, more discrete transactions which could then be interleaved in real-time with inner transactions from other Sagas. Each inner transaction retains its ACID properties, but becomes a member of an enclosing Saga. The Saga, itself, does not possess ACID properties, but represents the entire long-lived transaction.
Fig. 1: Concurrent Sagas
The most distinctive feature of a Saga is the way in which failure is handled. At the level of each inner transaction, a failed transaction can be rolled back in the normal way. However, if previous transactions in the Saga sequence have already been committed, it is too late to roll these back. Hence, if, in any Saga, a second or subsequent inner transaction failed, the Saga as a whole will be left in an intermediate state in which some work has been completed and some is left undone.
In order to handle this scenario, a Saga employs the concept of compensation handlers. Each inner transaction is provided with its own compensation handler. In the event of a failure, the failed inner transaction is rolled back. The Saga then executes the compensation handler for each of the previous inner transactions within that Saga. It does so in the reverse order of commitment. Hence, if Transaction 4 fails in Saga A, the Saga invokes the compensation handlers for Transactions 3, 2 and 1 in that order.
Fig. 2: Sagas – Backwards Recovery
This approach is termed ‘backwards recovery’. Of course, the term ‘recovery’ is used in a very broad sense. We cannot roll back committed transactions, so each compensation handler must, ideally, take some action that cancels or reverses the effect of its associated transaction. More formally, the ideal is to ensure a commutative relationship between each transaction (T) and its compensating transaction (CT). If f(T,CT) represents the state transition of the system when the first transaction, T, is followed by the second transaction, CT, then f(T,CT) should be identical to f(CT,T). If this cannot be guaranteed, then Sagas are not an appropriate transactional mechanism within a relational database.
In a database system that contains multiple concurrent Sagas, each Saga has visibility of partial changes made by other Sagas. There is no isolation at the Saga level, and a Saga is not atomic. Hence, commutativity is not always achievable, and Sagas are restricted in their applicability. Sagas can safely be used in scenarios where there is no requirement for transaction isolation, or where each inner transaction in each Saga operates on a constrained set of data, or performs a constrained set of operations that effectively isolate its effects from other Sagas.
As well as backward recovery, the Saga model also introduces a mechanism for ‘forward recovery’. Forward recovery depends on the existence, within the flow of the Saga, of well defined ‘save points’ at which the current state of the Saga is persisted to some durable store. Of course, the inner transactions, which conform to the ACID model, will already have the property of durability. Save points, however, ensure that the contextual data of the Saga itself is persisted.
When an inner transaction fails, the Saga can be restarted from the save point. If one or more committed transactions lie between the save point and the point of failure, the Saga must first do partial backwards recovery, executing the compensating transactions for these committed transactions before restarting the Saga from the save point.
Fig. 3: Sagas – Forwards Recovery
An obvious problem with this approach is that the Saga may not be able to complete, even on re-starts. Hence, it could easily enter an endless loop, performing backwards recovery to a save point before attempting forward recovery, only to fail again at the same point. One way to deal with this is to combine compensation handling with exception handling. For example, a system might detect such loops, and, after a suitable number of re-starts, break out of the loop by invoking an exception handler. The exception handler represents out-of-band code which can be invoked to take appropriate action in non-recoverable scenarios. Another way in which this model may be simplified is to add a save point at the beginning of each inner transaction. When the Saga attempts forward recovery, there is now no need to perform any backward recovery. Note, however, that if forward recovery itself fails, the Saga could choose to perform backward recovery.
The Saga Model and Business Process Management
BizTalk Server, like other BPEL4WS-compliant systems, adapts and extends the Saga model for use in the context of automated business processes. It is important to emphasise that this context is really quite different to that of a relational database. Within a relational database system, we generally have only a few categories of data which need to be managed. These include:
· The relational data tuples stored within tables
· Additional application-level data such as declared variables and cursors in stored procedures
· Underlying contextual data managed by the database system at both the inner transaction level and the Saga level.
Of course, this list is not necessarily exhaustive. These days, a stored procedure may well be interacting with external systems, sending and receiving messages etc. However, relational database systems provide constrained environments where most of the data a Saga is likely to affect is managed within and through the database system itself. Complex process workflows are not generally managed within a database system, and the flow of a Saga is likely to be entirely linear with no requirement to combine multiple Sagas to support complex flows.
In BizTalk Server, things are more complex. Consider the data that the BizTalk Server orchestration engine must manage:
· Internal contextual data defined for each scope within the orchestration, including the overall orchestration scope.
· Custom data defined at specific scope levels. This includes immutable messages, orchestration variables and correlation sets, ports and role links. These may be passed by reference to child orchestrations via the ‘Call’ shape at run time.
· .NET objects created within our orchestration.
In addition, it must provide a safe and usable environment for interacting with a range of external data. This includes
· Data with which .NET objects interact. This could be in-memory, or persisted to some store, or managed through external resource managers which may or may not be transactional, and which may share the data with other non-BizTalk processes
· More specifically, data managed via external transactional (DTC-base) resource managers and accessed through classes derived from System.EnterpriseServices.ServicedComponent.
· Services (including web services), applications, etc., to which we may send, or from which we may receive, data asynchronously or synchronously via the message box.
In addition to this, consider that any one orchestration may exhibit a complex process flow internally, and may participate in complex patterns of process flow with other orchestrations, as well as with a myriad of external systems. Compared with a relational database system, the environment is a good deal more challenging. Consider a complex business process as a whole, including all the actor systems (of which BizTalk Server is just one), the different data stores and the flow of messages across entire business activities. BizTalk Server orchestrations are designed to play a pivotal role in the management and orchestration of all this complexity.
Earlier, we discussed how, in the Saga model, a transaction should be commutative with its compensating transaction. This makes sense in the constrained environment of a relational database system where transactions act primarily on well-structured data tuples contained in relational tables. In a business process scenario, however, we may often be able to relax the requirement for commutativity to some extent. Orchestrations model real-life business processes in which there are often several actors including human beings, external systems and organisations, etc. It may not always be necessary to rigorously ensure commutativity in order to exploit the compensation model. In effect, we are saying that we might use the compensation model to undertake a wider range of actions than ‘pure’ compensation. For example, in many scenarios, it may be good enough to use the compensation model to simply alert end-users that some exception has occurred and inform them about which sub-tasks have been completed successfully at the point of cancellation. We may not need to attempt true compensatory activity on the state managed by our business process. Whilst this might sound liberating, it is often the source of great complexity in designing a compensatory strategy. In reality it is often necessary to adopt a hybrid approach where we perform true compensating transactions on some data whilst combining this with a variety other actions. This can be further complicated by complex interdependencies between different compensating transactions which cannot be adequately modelled using the built-in compensation features alone.
BizTalk – The Basic Compensation Model
The basic compensation model in BizTalk consists of the Saga model of backward recovery. This model is extended in a number of ways. We will investigate the basic model first, before moving on to describe the extensions. The model centres on the use of L-R and atomic transactions in association with scopes. An L-R transaction is broadly equivalent to a Saga. Like a Saga it can contain nested atomic (ACID) transactions. Each nested atomic transaction scope can be provided with a ‘Compensation Block’ in which developers can implement orchestration code to provide an associated compensating transaction. An L-R transaction has built in backward recovery. This is called ‘default’ compensation handling in BizTalk. In the event that one of the nested atomic transactions throws an exception, the enclosing L-R transaction can opt to invoke Saga-like backward recovery by using default compensation.
Unlike the invocation of exception handlers, the invocation of default compensation is not automatic. It is always invoked from within the context of an exception handler on an L-R transaction. This is a potential source of confusion on two counts. First, compensation blocks look very similar to exception handlers. Hence, developers naturally expect that compensation will be invoked at the level of Atomic transactions. This is not the case. When an Atomic transaction throws an exception, this will generally be caught at an outer non-atomic scope. Atomic scopes cannot have exception handlers of their own. Compensation can only be invoked if the exception is caught by an outer L-R transaction.
The second point of confusion arises from the fact that, by default, every L-R transaction is silently equipped with a default ‘General Exception’ exception handler. This acts as a catch-all, and is not visible within the orchestration designer, or, indeed, within the underlying XLANG/s. It is equivalent to a ‘General Exception’ exception handler that contains a Compensate shape and a Throw shape. The default exception handler invokes compensation on the L-R transaction and then re-throws the exception.
You can override the default exception handler by explicitly adding an empty exception handler for General Exceptions. This approach can be used to suppress compensation. More generally, however, it is advisable to add exception handlers for explicit .NET exception types. The default exception handler will only be invoked for otherwise uncaught exceptions. A General Exception handler will catch all .NET exceptions including exceptions that are not derived from System.Exception, but is loosely typed and therefore cannot provide access to any methods or properties of the exception object. Most .NET languages, including C#, Visual Basic and XLANG/s, enforce the derivation of exceptions from System.Exception, and it is rare to see any non-System.Exception exception within a BizTalk context. It is therefore generally preferable to use System.Exception as a catch all. If an exception is caught in one of your own exception handlers, you can control the invocation of compensation by using Compensate shapes.
Compensation blocks on Atomic shapes are very different to exception handlers. They are more like nested sub-orchestrations associated with transactional scopes, and act as containers for compensation handling code. The code can only be invoked after the associated atomic scope has been committed, and hence it cannot be invoked from within the context of that associated scope. It is always invoked from an enclosing L-R scope. You can think of this invocation as being similar to invoking a ‘child’ orchestration using the Call shape. However, in this case, you use the Compensate shape. The Compensate shape is typically used within an exception handler in the enclosing L-R shape, although, as we will see in the extended model, it can also be used inside a compensation block on the enclosing scope. Compensating transactions are only ever invoked directly or indirectly as a result of catching and handling exceptions, and BizTalk compensation effectively constitutes a built-in exception handling pattern.
When implementing the Saga-like model, compensation is invoked directly on the enclosing L-R transaction that catches the exception, rather than on any of the nested transactions. Assuming that the ‘Compensation’ property of the enclosing L-R transaction is set to ‘Default’, BizTalk will perform compensation precisely as you would expect, calling compensation handlers of any nested, committed atomic transactions in the reverse order in which they were committed.
Fig. 4: Basic Compensation Model
The model is entirely logical, but not always immediately obvious to the BizTalk developer. In the Saga model, it is the Saga itself that is responsible for performing compensation on the nested atomic transactions, and this is precisely the mechanism of default compensation within BizTalk Server. However, developers familiar with exception handling often find the compensation model unclear. This may be why compensation appears to be under-used in many BizTalk projects.
Forward Recovery in BizTalk
BizTalk provides very limited support for Saga-like forward recovery. There are two forward recovery mechanisms:
· Retrying atomic scopes
Atomic transactions have a ‘Retry’ property. If this is set to true, the atomic transaction will, in certain circumstances, handle failure by attempting to re-execute the logic within the atomic scope.
Atomic transactions are only retried in three scenarios:
· If BizTalk Server attempts to commit the transaction, and the commit fails.
· If BizTalk Server fails to persist state at the end of the atomic transaction. This is really a specific case of the first scenario, as commitment includes persisting state.
· If the code in the atomic transaction scope throws an instance of RetryTransactionException.
The atomic scope performs up to 21 retries. In the first two scenarios, BizTalk uses a 2 second interval between each retry, and these values do not appear to be configurable. If you throw a RetryTransactionException, the retry interval will be set to 0 by default. However, you can, in this case, configure your own interval value using the DelayFor property. If, after 21 retries, the atomic transaction cannot be committed, or the orchestration cannot be persisted, BizTalk will suspend (resumable) the orchestration instance.
This facility provides no support for performing backward recovery, and does not exploit persistence points.
· Resume suspended orchestrations
A suspended orchestration may be resumed either manually via the management console, or through some custom scripted approach. In this case, BizTalk Server will attempt to re-start the suspended orchestration from the most recent persistence point. Each transactional scope introduces a persistence point just after it has committed, ensuring that partial backward recovery is never required when resuming.
Extending the Default Compensation Model
If you have used the compensation features of BizTalk Server, you will be aware that the description of default compensation above fails to describe the full functionality provided by BizTalk. BizTalk Server extends the basic model in a number of ways in order to provide a more versatile approach which is better suited to the complexities of business process management.
In correspondence with extended versions of the Saga model, BizTalk allows the nesting of L-R transactions within outer L-R transactions. Any enclosing L-R transaction can therefore contain a mixture of L-R and atomic transactions. The nested L-R transactions can, themselves, nest further transactions. Atomic transactions, however, cannot nest any further transactions.
This approach provides greater flexibility in designing and implementing a compensation strategy for complex business processes. Each nested L-R transaction can be used to control compensation directly for its immediate child transactions, and provides a compensation target to its enclosing L-R transaction. As we have seen, compensation is always invoked within the context of exception handling, and thanks to the use of structured exception handling within BizTalk orchestrations, this allows compensation to be performed over constrained sub-sets of the business process in accordance with the type of exception that is raised, and where it is handled.
When an outer L-R transaction catches an exception and performs default compensation, BizTalk does not distinguish between nested L-R and atomic transactions. The outer L-R transaction will invoke the compensating transaction on each successfully completed inner transaction in reverse order of completion. Atomic transactions are regarded as successfully completed once they have been committed and the orchestration state has been persisted. In the case of a nested L-R transaction, the transaction is not, of course, committed in the ACID sense. It is marked as successfully completed when it has undertaken all its internal actions without throwing an exception to the enclosing scope or suspending the orchestration.
If a nested L-R transaction has no custom compensation block, it will perform default compensation on its immediate child transactions when invoked by the outer L-R transaction. The outer L-R transaction has no visibility of the transactions nested within the inner L-R transaction. It sees the nested L-R transaction as a single ‘black box’ transaction target.
In scenarios where nested L-R transactions extend the default Saga-like compensation model, and where the only transactions that have compensation blocks are atomic transactions, we can visualise this extended model very simply as a ‘compensation tree’ in which the leaf nodes are atomic scopes with compensation blocks, and the root and branches are composed of L-R transactions.
Fig. 5: Compensation Tree
This model exhibits a subtle characteristic. Consider a simple scenario where we nest an L-R transaction (L-RInner) within an enclosing L-R transaction (L-Router). L-RInner contains an atomic transaction (ACIDTx1). If an exception is thrown within L-RInner at some point after ACIDTx1 has been committed, the exception can be caught by an exception handler on L-ROuter. If L-ROuter invokes its own default compensation handler, BizTalk will proceed to invoke the compensation handler of each completed child transaction in reverse order of completion. However, L-RInner was not completed successfully. It threw an exception. Hence, we might expect that its compensation handler will not be invoked and ACIDTx1 will therefore remain uncompensated.
In reality, L-RInner is automatically equipped with its own default exception handler which, as we have seen, will invoke compensation on L-RInner and will then re-throw the exception to the outer scope. Hence, the ACIDTx1 transaction will be compensated, even though its enclosing L-R transaction has not been completed. L-RInner will automatically intercept the exception to ensure this behaviour. Of course, you can override this behaviour by adding explicit exception handlers to L-RInner. Note how each L-R transaction is directly responsible for managing the compensation of its immediate children. You cannot delegate this responsibility to an outer L-R transaction.
Fig. 6: Basic Flow for Compensation Tree
Atomic scopes cannot nest transactions, and so will always represent leaf nodes in a compensation tree. In addition, they cannot contain exception handlers. This often surprises developers, but is a logical consequence of the close association between exception handling and compensation. Consider, also, that atomic transactions already have built-in exception handling in terms of performing roll-backs and retries. It is difficult to see how custom exception handling could be incorporated at the atomic scope level without introducing a degree of inconsistency and confusion into the orchestration model. Of course, you can always nest non-transactional scopes within an atomic scope in order to handle exceptions within the context of the atomic transaction.
Using Compensation Blocks on L-R Transactions
In the previous section we saw how nested L-R transactions can be used in conjunction with the exception handling mechanism to extend the default Saga-like compensation model. In this section we will see how L-R transactions can be equipped with their own compensation blocks to further extend the model and liberate it from some of the constraints defined by Sagas.
When an L-R transaction is equipped with its own compensation block, it changes its compensation model from the Saga-like ‘default’ model to a ‘custom’ model. In the designer, this can be done either by directly changing the Compensate property to ‘Custom’ or by right-clicking the L-R scope and adding a compensation block. Note that, unlike exception handlers, a scope can only ever have a single compensation block.
In the custom model, you can take control, within certain constraints, of the entire compensation approach. You can compensate completed child transactions in any order you wish. To do so, you add a Compensate shape to the compensation block for each transaction you wish to compensate. You can, of course, add additional logic. For example, you might use Decision shapes to decide exactly what compensation to undertake based on an inspection of current orchestration state.
A potential problem with this extended model is the indeterminacy of the completeness of nested transactions. When a custom compensation handler is invoked on an L-R transaction, you often cannot know for certain exactly which child transactions have successfully completed. This is largely mitigated by the fact that if a Compensate shape attempts to invoke compensation on an incomplete child transaction, nothing happens. This is not treated as an exception, and no compensation code is invoked. Instead, control simple passes to the next part of the L-R transaction compensation handler.
Sometimes, this may not be sufficient. If you need to test a child transaction explicitly to see if it has successfully completed, you can use the little-know ‘succeeded ()’ function in an XLANG/s expression. This takes a single parameter which is the identifier of the transaction you want to test, and returns a Boolean to indicate if the transaction can be compensated. You could use it within a decision shape, for example, to select a branch depending on the ‘completed’ status of one or more child transactions. Indeed, the Compensate shape automatically wraps a call to the XLANG/s ‘compensate <tx> ()’ statement within in ‘if’ statement which tests the transaction status using succeeded (). This is why nothing happens if you attempt to perform compensation on an incomplete transaction using the Compensate shape.
There is a very minor issue here. If you use succeeded() within a conditional branch of a Decision shape, and you then include the Compensate shape within that branch to compensate the same transaction you tested, the Compensate shape will mechanically insert another ‘if’ statement that redundantly tests the completeness of the transaction a second time. You can easily avoid this redundancy by simply calling ‘compensate <tx> ()’ directly within an Expression shape.
Custom compensation has a number of consequences. For example, consider the scenario where you introduce a compensation block for an L-R Transaction which is also catching an exception, and where you want to invoke compensation as a result of handling that exception type. In this case, you will need at least one Compensate shape in your exception handler. Compensation always begins within an exception handler. If you compensate the current L-R transaction, the custom compensation handler will be invoked, rather than the default compensation handler. You will probably, therefore, have additional Compensate shapes within your compensation block.
The following diagram shows this model. In this case, we are using the compensation block to invoke compensation on inner transactions in the order they were completed, rather than reverse order.
Fig. 7: Custom Compensation (A)
This leads directly to another observation. The model shown in the above diagram is unnecessarily complex. BizTalk allows us to invoke compensation directly from within exception handlers, and therefore we can replace the Compensate shape in the exception handler with the contents of the compensation block and set the L-R Transaction back to default compensation in order to remove its compensation block.
Fig. 7: Custom Compensation (B)
If you invoke compensation on the current L-R transaction from within the compensation block of that transaction, BizTalk Server will invoke the default compensation handler. This is often useful if you simply want to add some additional logic around the default compensation behaviour. If an L-R transaction is using custom compensation, there is no other way in which the default compensation can be invoked.
Strangely, you are allowed to call the default compensation handler of an Atomic transaction within a compensation block on that same atomic scope. Behind the scenes, the atomic transaction goes through the motions of attempting to invoke compensation on nested transactions. However, atomic transactions cannot contain any nested transactions, and hence the call is redundant.
You can only directly invoke compensation on the current transaction or its immediate children. This rule applies within both exception handlers and compensation blocks. You cannot directly invoke compensation on transactions nested within an inner transaction. This keeps the model clean, ensuring that custom compensation is always managed directly within a directly enclosing L-R transaction.
One weakness of the model is that you can easily end up attempting to invoke compensation on the same transaction more than once. At run time, BizTalk Server will guarantee that each transaction is compensated at most once. In most cases, any attempt to compensate a transaction more than once will be caught at the compilation stage, but will be reported as a warning only. The orchestration will still compile. At run time a second or subsequent attempt to compensate an already compensated transaction will be ignored in much the same way that attempted compensation of incomplete transactions is ignored. This is not reflected, however, in explicit XLANG/s code.
Transactional scopes can be contained within Loop shapes. This constitutes a special case where multiple instances of the same transaction will be generated at run time as the code iterates through the loop. Each of these instances is associated with a single transaction, and when that transaction is compensated, BizTalk will invoke compensation on each of the instances in turn. This will be done in reverse order of their completion, as for default compensation. This behaviour only applies if the transaction is compensated from an enclosing L-R transaction. If the transaction is an L-R transaction, and performs self-compensation from within an exception handler, only that one instance will be compensated.
Orchestration-level Compensation
An orchestration can be provided with a transaction at the orchestration level. As with any transaction, you can change the compensation model from ‘default’ to ‘custom’. In this case, because there is no corresponding scope shape, the exception block is displayed using a separate tab at the bottom of the orchestration designer.
Orchestration-level compensation sometimes confuses BizTalk developers. Like any compensation handler, the default or custom compensation handler of an orchestration can only be invoked from an exception handler in an enclosing L-R transaction. However, the enclosing transactional scope clearly cannot be part of the same orchestration, as the compensation handler is assigned to the outermost scope of the orchestration. In this case, it must be part of a parent orchestration that invokes the child orchestration using the Call shape. Note that this does not apply when using the Start shape which uses the message box to invoke an orchestration indirectly.
Data Handling within Transactions
An atomic scope supports ACID-like characteristics with regard to the management of orchestration state. It defines a sub-unit of orchestration work which will be done completely or not at all, which results in a consistent transition between two states, which will isolate the relevant orchestration state from changes by other parts of the process and which will ensure that all changes to state are persisted to disk before the transaction is committed.
In addition, atomic transactions can extend their boundaries by automatically enlisting distributed (DTC) transactions of which it is aware. To exploit this feature, you declare and instantiate objects derived from System.EnterpriseServices.ServicedComponent at the level of the atomic scope. BizTalk always treats ServicedComponent classes as non-serialisable, even if they are marked as serialisable (the ServicedComponent bases class is, itself, marked as serialisable). Hence you can only declare variables for ServicedComponent classes at the level of atomic scopes.
In order to support ACID-like characteristics, atomic scopes make copies, as necessary, of the relevant orchestration state, and act on the copied data within the context of the transaction. For example, if an orchestration variable, message or correlation set has been declared in an enclosing scope, and is used within an atomic scope, the atomic scope will act on a copy of that data[1]. If the atomic scope is completed successfully, the original data is updated as part of the atomic transaction, and the changes are committed. For variables that reference .NET objects, atomic transactions perform a deep, or custom, serialisation of the object in order to create a copy.
For each successfully completed transaction (both atomic and L-R) BizTalk stores the orchestration state as it existed at completion. For atomic scopes, this is, of course, the committed state. If a compensation handler is invoked on a completed transaction, it is this data that the handler acts upon. This feature goes some way to supporting the possibility of commutativity between transactions and their compensating transactions. However, it falls well short of enforcing it. However, in scenarios where commutativity is possible, it may sometimes only be possible because of this feature.
Compensating transactions are, themselves, a type of L-R transaction. They do not support ACID properties in regard to the state they manage. This leads to one problem. If you wish to use non-serialisable .NET types in your orchestration code, you must declare orchestration variables for these types within atomic scopes. However, these variables cannot be used in the corresponding compensating transaction. If you try to reference such variables, you will get a compile-time error. This, of course, is unavoidable. BizTalk uses serialisation to capture the state of the committed atomic transaction, and then provides this state to the compensation block. It cannot, therefore, provide data for non-serialisable types to compensation handlers.
There is one further restriction. Compensating transactions are not atomic, and occur after their associated transaction has completed. They, therefore, cannot be enlisted into an atomic transaction and cannot interact with external DTC transactions. They are therefore unable to interact with ServicedComponent types declared in the associated atomic scope. This is actually a formal part of the XLANG/s specification, but is enforced though BizTalk’s treatment of ServicedComponent classes as non-serialisable. Compensating transactions cannot reference non-serialisable objects.
You are free to nest transactional scopes within a compensation block. This includes atomic transactions. When a compensation block throws an exception, it is, as you might expect, passed to the context from which that compensation was invoked. This may be another compensation block, or an exception handler. Ultimately, all compensation begins at an exception handler, so an exception will be passed back through the compensation chain until it reaches the root exception handler from which compensation began. If this is a default exception hander, the exception will simply be re-thrown.
Compensation parameters
As we have seen, compensation handlers act on the copy of orchestration state made at the successful completion of their associated transactions. Sometimes, it is useful to be able to pass other state to the compensation block, including copies of orchestration state belonging to enclosing scopes. XLANG/s supports compensation parameters for this purpose.
Unfortunately, the orchestration designer does not expose this functionality. If you wish to use compensation parameters, you have to edit the XLANG/s code directly, and your changes will not be visible within the graphical designer. This is an unfortunate omission in more complex compensation scenarios. Editing your XLANG/s directly is certainly not recommended on two counts. First, your changes are opaque to users of the orchestration designer. They are not captured with the Orchestration Designer XML (ODX), and are therefore not viewable graphically. Secondly, the orchestration designer regenerates the XLANG/s code from the ODX every time it encounters a compile-time error. This causes all the ‘opaque’ code to be removed, including parameter definitions and arguments. There appears to be no way of preventing this.
An alternative is to create some external storage mechanism to which an enclosing scope can write specific data keyed on some identifier that is unique to the current orchestration instance. The compensation handler can then read back this data using the same key value. You could extend this by using a composite key that includes scope identifiers, allowing compensation code to extract data specific to a given scope. Your store will need to be able to persist state so that it is available on re-hydration of an orchestration instance, and also after a re-start of a host instance. You may need to use an external database system to persist the data, or you may try to design some solution that uses BizTalk’s own persistence mechanisms for this purpose.
Fortunately, the need to parameterise compensation handlers appears fairly rare. If you ever need to parameterise your compensation handlers, and decide to do this by editing the XLANG/s directly, here are examples of the code you might write. The first example shows a parameterised compensation block definition for an atomic transaction
scope atomic transaction AtomicTransaction1
{
body
{
;
}
compensation (message MyMsgSchema msg1, ref System.Boolean var1, correlation MyCorType cs1)
{
;
}
}Just like C#, parameters can be marked as ‘ref’ or ‘out’. Note the use of ‘message’ and ‘correlation’ keywords for the appropriate parameters. Unlike called orchestrations, you cannot, and don’t need to, pass ports or role links.
The second example shows how the above compensation block might be invoked. Note the test to ensure that the compensation has succeeded.
if (succeeded(AtomicTransaction1))
{
compensate atomicTransaction1 (myMsg1, ref myVar1, myCorSet1);
}Conclusion
This article has discussed the model and mechanism of compensation in BizTalk Server. However, it leaves open the wide-ranging questions of incorporating compensation into orchestration design patterns, best practice guidelines, etc. A good introduction to the appropriateness of the compensation model is provided by a paper entitled ‘Compensation is Not Enough’ authored by Paul Greenfield, Alan Fekete, Julian Jang and Dean Kuo. You can download it from http://www.ict.csiro.au/staff/Julian.Jang/EDOC2003.pdf. The paper takes e-procurement as an example, and shows how the BizTalk compensation model is, in itself, sometime insufficient to meet the requirements of real-world scenarios. It should be noted that BizTalk Server (and, no doubt, its rivals) provides other facilities which can be brought to bear on those scenarios where the compensation model proves inadequate.
Bibliography
H. Garcia-Molina and K. Salem, “Sagas,” ACM International Conference on Management of Data (SIGMOD)., pp. 249-259, 1987.
P. Greenfield, A. Fekete, J. Jang and D. Kuo, “Compensation is Not Enough,”a name=top>Seventh International Enterprise Distributed Object Computing Conference (EDOC’03)., p. 232-239, 2003
[1] This is a simplification of the true behaviour of BizTalk Server 2006. In some scenarios, BizTalk Server is able to guarantee atomic semantics without having to copy the orchestration data. In these cases, BizTalk optimises the orchestration by avoiding unnecessary data copying.
