Being known for my interest in rules processing, I quite often get asked to help with problems with MS BRE. A couple of days ago, I was asked to help investigate an issue occurring in production for a BizTalk Server application. Occasionally, in a fairly high throughput system, BizTalk logs an error stating that a problem has been encountered while executing a rule set. That is the only information provided, with no hint of what the problem might be, and because the issue only occurs intermittently under real-world conditions in the production environment, it was not obvious how to obtain further insight without disrupting live operations.
The MS BRE is designed to be hosted within .NET applications as an in-process library. Its approach to handling error conditions during rule set execution is to throw all exceptions back to the application that invoked execution. The first question I was asked was what registry setting could be used to turn on event log tracing for the engine. I had to report that no trace functionality exists at all within the engine, although the Rule Engine Update service does log exceptions. Although emitting exceptions back to the host application makes sense, MS BRE really ought to provide the ability to switch additional event log tracing on for diagnostic purposes. If the host application does not log the details, exceptions become opaque.
We were getting no detailed information on the error because the application-level exception handling code does not recurse through inner exceptions in order to log as much information as possible. The Call Rules shape in BizTalk orchestrations uses the Policy class to invoke a rule set. When a Policy object catches an exception, it wraps it in an outer PolicyExecutionException which it then throws. The information we needed was assigned to the inner exception.
The best approach would be to bite the bullet and deploy a new version of the BizTalk application with improved error logging. If we are lucky, this may only require updating some centralised error logging assembly. However, there is a good chance that we may need to deploy a whole new version of an orchestration. Either way, there will probably be some disruption to the live service because we will probably end up stopping and starting the BizTalk application in order to load new assembly versions. I suppose, thinking about it, we could try using side-by-side assembly deployment as an alternative strategy, but I’m not keen on that idea.
The problem I was posed was to find a way of getting richer error information from the rules engine with minimum disruption to the live system. As we have established, there is no nice feature in the engine that I can simply switch on with a registry setting. However, all is not lost. In this article, I will describe another approach that allows you to control and configure the logging of rule engine exceptions using the straight-forward, but little known, 'compensation' feature of the rule engine. As we will see, this feature can be used this to introduce error logging without stopping the live system.
Given that the issue here would be best resolved by introducing better error logging at the orchestration level, you may well question the relevance of this article. However, it will serve a number of purposes:
a) To demonstrate one way of solving this particular dilemma when supporting existing production environments. That may possibly be of help to someone out there.
b) To demonstrate the general mechanics of the rule engine compensation feature.
c) To build an understanding of where the compensation feature might be used for more advanced purposes.
Along the way, I will also use this as an opportunity to illustrate some additional information concerning a previous article on the various ways in which rule sets are represented in the repository, the Rule Engine Update (REU) service and the engine itself. See:
Rule Engine Compensation
In regard to the rules engine, the term 'compensation' is, frankly, a little misleading. In BizTalk orchestrations, WF workflows and the BPEL specification, compensation refers to built-in support for a set of exception handling patterns based broadly on the 'Saga' model and associated with the concept of transactions. There is no equivalent notion of transactions within the rules engine, and the compensation mechanism is really just a call-back facility that allows rule engine exception handlers to notify custom code of exceptions before re-raising those exceptions. As it happens, the relevant event handlers in the MS BRE are associated with discrete engine functions such as assertion and deletion. However, I cannot fully defend the term 'compensation' in this context, especially as the engine does not pass information about the current engine operation when it performs a callback.
As you might expect, the callback mechanism is implemented through the use of delegates. An application passes a delegate (which may, of course, be a multicast delegate) to the engine in order be notified of exceptions. The delegate is of type RuleEngineCompensationHandler, and takes two arguments. The first is the exception object and the second is a 'user data' object. This can be of any type, and is registered with the rule engine along with the delegate. It is passed back to the delegate by the engine when an exception occurs. The user data object allows a host application to pass arbitrary data to compensation handlers.
Here is an example of a very simplistic logging method that is compatible with the RuleEngineCompensationHandler delegate in which a custom user data object is used to pass an event ID. The code uses the System.Diagnostics.EventLog class:
public bool LogException(Exception ex, object userData)
{
// Initialise data
bool retVal = false;
string sourceName = "MSBusinessRulesEngine";
string message = "An exception occurred when executing a " +
"rule set.\r\n" + ex.ToString();
EventLogEntryType entryType = EventLogEntryType.Error;
// Assume the userData object is a CustomUserData object.
int eventId = ((CustomUserData)userData).EventId;
// Create the source, if it does not already exist.
if (!EventLog.SourceExists(sourceName))
{
// Create an EventLog instance and assign its source.
EventLog newLog = new EventLog();
newLog.Source = sourceName;
}
try
{
EventLog.WriteEntry(sourceName,
string.Format(CultureInfo.CurrentCulture, "{0}: {1}",
DateTime.Now.ToString(), message),
entryType, eventId);
}
catch (Exception)
{
throw;
}
return retVal;
}
Delegates are associated with user data objects. This association is represented by the use of CompensationHandlerInfo objects. These are created by the host application initialised with a delegate and a user data object and then passed to the rule engine in order to hook in the compensation handler code.
After the Call Rules shape in BizTalk orchestrations, the second most common way of invoking the rules engine is via the Policy class. Indeed, as I noted above, the Call Rules shapes uses the Policy class internally. A Policy object represents a version of a rule set associated with a rule engine instance, and is used to execute the rule set against a set of facts. The Policy object, however, does not support the registration of compensation handlers, so to use this feature you must exploit lower-level objects. Microsoft's engine operates in the context of the Microsoft Business Rule Framework which defines a number of classes and interfaces representing various abstractions. These include the RuleEngine class and executor objects that implement the IRuleSetExecutor interface. It is generally assumed that the RuleEngine class explicitly represents Microsoft's engine, but this not the case. It actually represents the abstract concept of a rule engine, but does not implement any rule processing functionality itself. The RuleEngine class is defined as part of the framework, and the framework is not tied to any one specific rule engine technology.
A RuleEngine instance uses an executor object to execute a rule set. Executor classes implement the IRuleSetExecutor interface, and provide access to the specific functionality of a given engine. The RuleEngine class is a text-book example of the adapter design pattern. It wraps the executor, exposing methods that largely correspond to the IRuleSetExecutor interface. However, it does not implement this interface itself.
Compensation handlers can be registered using either a RuleEngine object or an executor. However, the MS BRE Rete executor is an internal class, and is not directly accessible to custom code. When writing custom code to configure compensation, you will therefore use a RuleEngine object. Later in this article we will see that executors can, in fact, be passed to custom code indirectly from within a rule.
Registering compensation from within external custom code doesn’t really make too much sense. Because all exceptions are thorwn back to your code, you might just as well trap any exceptions and handle them at that level. This is probably why the Policy class does not support compensation handler registration. In a addition, when writing code to use the RuleEngine object, you will often want to obtain rule sets via the Rule Engine Update (REU) service in the same way the Policy class does. This is entirely achievable, but with issues.
Here is some simple code that demonstrates how to obtain the latest version of a rule set from the REU service and to configure a rule engine instance to use a compensation handler. The code uses the LogException handler defined above. Given my observation about the limited usefulness of this approach, you might wonder, why I have bothered to provide a code example. One reason (excuse?) is that it helps illustrate a previous article I wrote on the various forms in which a rule set is expressed as it moved from a repository to the engine via the REU Service. The code demonstrates how to obtain rule sets directly from the REU service and how the framework supports conversion of MS BRL into RuleSet object graphs using the IRuleLanguageConverter interface implemented, in this case, on the BusinessRulesLanguageConverter object.
...
using System;
using System.IO;
using System.Xml;
using Microsoft.RuleEngine;
using Microsoft.RuleEngine.RemoteUpdateService;
...
public ExecuteRuleSet()
{
RuleSet rs = null;
// Create a proxy for the REU Service
RemoteUpdateServiceProxy proxy
= new RemoteUpdateServiceProxy();
// Get latest version of rule set
RuleSetInfo rsi = proxy.GetLatest("RuleEngineLoggerExample");
// Create a language converter for MS BRL
BusinessRulesLanguageConverter msbrlConverter
= new BusinessRulesLanguageConverter();
try
{
// Check that we have permission to the rule set
if (proxy.IsRuleSetAccessible(rsi))
{
// Get the raw MS BRL
byte[] msBrl = proxy.GetDefinition(rsi.Name,
rsi.MajorRevision,
rsi.MinorRevision);
// Convert the MS BRL into a ruleset object graph
// NB: There is a close association between MS BRL
// and Microsoft’s Rule DOM, so this really
// constitutes a form of fairly direct de-serialisation
// using the converter.
using (MemoryStream msBrlStream
= new MemoryStream(msBrl))
{
RuleSetDictionary ruleSets;
VocabularyDictionary vocabs;
msbrlConverter.Load(msBrlStream,
out vocabs, out ruleSets);
// Extract the required rule set
rs = ruleSets[rsi.Name];
}
}
}
catch (Exception ex)
{
throw new ApplicationException(
"Could not load rule set.\r\n" + ex.Message, ex);
}
// Check we were successful
if (rs != null)
{
// Create a RuleEngine instance.
// RuleEngine represents the abstract concept of
// a rule engine. The rule set passed to the constructor
// has configuration data which, by default, configures
// the rule set to be translated using the
// RuleSetToReteTranslator. This component creates
// an MS BRE Rete object which is the executor. An
// executor accesses a concrete implementation of a
// rule engine.
RuleEngine re = new RuleEngine(rs, true);
// Create a custom user data object to hand information
// to the RuleEngine_LogException handler
CustomUserData custUserData = new CustomUserData();
custUserData.EventId = 2500;
// Create a CompensationHandlerInfo to bind a delegate
// to the user data
re.CompensationHandlerInfo
= new CompensationHandlerInfo(LogException,
custUserData);
// Create the facts
object[] facts = new object[1];
XmlDocument xDoc = new XmlDocument();
xDoc.Load(@".\ExampleFiles\po.xml");
// NB. Ensure the document type is correct here. This
// can be obtained by inspecting the root element in the
// schema using the Rules Composer.
facts[0] = new TypedXmlDocument("po.PurchaseOrder", xDoc);
try
{
// Execute the rule set without tracking
re.Execute(facts);
// Dispose of the rule engine instance
re.Dispose();
}
catch (Exception ex)
{
// It would generally be better to do exception
// logging here rather than use compensation
throw new ApplicationException (
"The application has received an exception " +
"from the rules engine,\r\nbut can’t be " +
"bothered to provide you with any details!!");
}
// Save Typed XML
xDoc.Save(@".\ExampleFiles\po.xml");
}
}
This code suffers from a specific problem. When you use the Policy class to invoke a rule set, you get the benefit of caching. There are two caches involved. The first stores RuleSet objects in order to avoid repeated calls into the REU service. The second caches RuleEngine instances in order to speed up second and subsequent invocations of a rule set. Unfortunately, the two caches are internal, and not accessible to custom code. Hence, the example code cannot access the caches and is really quite inefficient in comparison to using the Policy object. There are a couple of ways you might remedy this. The first would be to use reflection to access the internal caches. The second would be to extend the above code to support custom caching.
As you can see, using the RuleEngine object in custom code poses some challenges. It would be better to write code that uses the Policy object and use local exception handling to log errors. However, this would be too disruptive for our purposes because it means changes to the orchestration code and re-deployment of those orchestrations to the live environment. Unless side-by-side deployment techniques are used, this would mean taking the production environment down after first ensuring that any long-lived processes are completed. Fortunately, there are better ways that require significantly less disruption.
Using a Fact Retriever
We want to introduce logging in a new version of a policy without the need to redeploy an updated version of the live BizTalk application. There are two ways in which we can achieve this. The first is to use a Fact Retriever component. Fact Retrievers are a really useful feature of MS BRE. They are helper components that can be used to assert facts to the engine each time a rule set is executed. They can be used alongside the more common approaches to asserting facts (e.g., passing an array of facts to the Execute method of the Policy object or directly using the Assert method of the RuleEngine object). Their power derives in part from the ability to configure a Fact Retriever declaratively as part of a rule set without writing any additional code (except, of course, the code that implements the Fact Retriever itself). In addition, Fact Retrievers support a simple handle value. When a RuleEngine object is first instantiated and executed, it calls the UpdateFacts method of the configured Fact Retriever and provides a null handle. Importantly for our purposes, it also passes a reference to the RuleEngine object. In the typical design pattern, the Fact Retriever assigns some custom object as the handle.
When execution of the rule set completes, the engine automatically retracts all remaining facts that have been directly asserted, but retains any facts that have been asserted by the Fact Retriever. This is why these facts are called ‘long-term’ facts. The RuleEngine instance is cached for future re-use. When, at some future time, the RuleEngine instance is retrieved from the cache and re-executed, the long-term facts are still within the working memory. The engine re-invokes the UpdateFacts method of the Fact Retriever, passing back the handle that was assigned in the first invocation. The handle, therefore serves two broad functions. The first is to allow the fact retriever to track executions of the rule set. Typically, this is used to differentiate between the first and subsequent executions. The second function is to round-trip data between executions in order to control fact retrieval. There are all kinds of ways this can be used. For example, a Fact Retriever might decide to re-assert long term facts based on the length of time that has elapsed since the last invocation by recording and updating a timestamp on the handle object.
Each time the engine invokes the UpdateFacts method, the Fact Retriever can decide what facts it wishes to assert, retract or update. However, it can also access other RuleEngine methods and properties, including the CompensationHandlerInfo property we used in the example code earlier. This means that we can initialise a rule engine instance with a compensation handler. A simple example is shown below:
using System;
using System.Globalization;
using System.Diagnostics;
using Microsoft.RuleEngine;
class ExceptionLoggerInitialiser : IFactRetriever
{
public object UpdateFacts(RuleSetInfo rulesetInfo,
Microsoft.RuleEngine.RuleEngine engine,
object factsHandleIn)
{
object factsHandleOut;
if (factsHandleIn == null)
{
// Create a custom user data object to hand information
// to the RuleEngine_LogException handler
CustomUserData custUserData = new CustomUserData();
custUserData.EventId = 2500;
// Create a CompensationHandlerInfo to bind a delegate
// to the user data
engine.CompensationHandlerInfo
= new CompensationHandlerInfo(LogException,
custUserData);
factsHandleOut = custUserData;
}
else
{
factsHandleOut = factsHandleIn;
}
return factsHandleOut;
}
private bool LogException(Exception ex, object userData)
{
// Initialise data
bool retVal = false;
string sourceName = "MSBusinessRulesEngine";
string message = "An exception occurred when executing a " +
"rule set.\r\n" + ex.ToString();
EventLogEntryType entryType = EventLogEntryType.Error;
int eventId = ((CustomUserData)userData).EventId;
// Create the source, if it does not already exist.
if (!EventLog.SourceExists(sourceName))
{
// Create an EventLog instance and assign its source.
EventLog newLog = new EventLog();
newLog.Source = sourceName;
}
try
{
EventLog.WriteEntry(sourceName,
string.Format(CultureInfo.CurrentCulture, "{0}: {1}",
DateTime.Now.ToString(), message),
entryType, eventId);
}
catch (Exception)
{
throw;
}
return retVal;
}
}
public class CustomUserData
{
private int _eventId;
public int EventId
{
get
{
return _eventId;
}
set
{
_eventId = value;
}
}
}
All that is now necessary is to compile the assembly with a strong name, install it into the GAC on the server and configure a new version of the rule set to use the Fact Retriever. Deploy the new version of the rule set and wait for exceptions to be logged!
Using a Rule to configure the Executor
The Fact Retriever approach outline above is, in my opinion, the best way to solve the problem in hand. However, there is a second approach. It’s worth describing this because it leads nicely into understanding more advanced uses of compensation handlers in rule sets. The second approach is a less preferable than the Fact retriever approach because it requires changes to rules in existing rule sets, and introduces a correspondingly greater risk of breaking the application.
We saw earlier that the classes that implement the MS BRE engine are marked as ‘internal’. Specifically, this includes the RuleSetToReteTranslator class (an implementation of IRuleSetTranslator) and the Rete class (an implementation of IRuleSetExecutor) produced by RuleSetToReteTranslator. The interfaces implemented by these two classes are part of the rules framework, and can potentially be used to enable the integration of other engines into the framework. They are, therefore, marked as ‘public’. This means that, although we cannot directly instantiate or manipulate the MS BRE Rete class from custom code, we can certainly access a Rete object via its executor interface, assuming we can somehow obtain a reference to that interface.
Microsoft’s engine implements a number of pre-defined ‘engine operations’. These can be used, as required, within rule conditions and actions, and will be very familiar to anyone who has written MS BRE rules. The easiest way to explore them is via the Rules Composer. Microsoft has defined two vocabularies, one listing friendly names for engine ‘functions’ and the other listing names for engine ‘predicates’ (methods that return a Boolean value). Engine functions include the familiar ‘Assert’, ‘Update’ and ‘Retract’ functions as well as a small library of additional commonly-used functionality. One of the more mysterious engine functions is called ‘Executor’. It returns an instance of the current executor typed as IRuleSetExecutor. Because you will almost certainly be using Microsoft’s built-in engine functionality, this will really be an instance of the internal Rete class.
You typically use the Executor engine function to pass an instance of the executor out to custom code. You can create a method in custom code, and define it with an IRuleSetExecutor parameter. At run-time, you will assert an instance of the method’s class to the engine, even if your method is static. This, of course, assumes you have not set the StaticSupport registry key to a value greater than 0. I strongly recommend that you do not use the StaticSupport feature of MS BRE unless there is a compelling reason to do so. It introduces a high risk of breaking rule sets that were created under the default setting. The resulting run-time failures are very opaque and difficult to diagnose.
You can now invoke your method within your rules. This may be done using a rule action, or alternatively a predicate or argument in a rule condition. You use the Executor engine function to assign a value to the method’s parameter. In your custom code, you can now access the IRuleSetExecutor methods of the executor class, including the SetCompensationHandler method.
We can use the Executor engine function to enable logging. The basic idea is to fire an initial rule that will configure the executor with a compensation handler. To do this, you first need to create some helper classes and methods. The code will be very similar to the equivalent Fact Retriever code. The following provides a simple example.
using System;
using System.Diagnostics;
using System.Globalization;
using Microsoft.RuleEngine;
public class RuleEngineExceptionLogger
{
private bool _isInitialised;
private CompensationHandlerInfo _compHandlerInfo;
public bool IsInitialised
{
get
{
return _isInitialised;
}
}
public void SetCompensationHandler(IRuleSetExecutor executor)
{
if (executor != null)
{
CustomUserDatauserData = new CustomUserData();
userData.EventId = 2500;
RuleEngineCompensationHandler reCompHdlr =
new RuleEngineCompensationHandler(LogException);
_compHandlerInfo =
new CompensationHandlerInfo(reCompHdlr, userData);
executor.SetCompensationHandler(_compHandlerInfo);
_isInitialised = true;
}
}
public bool LogException(Exception ex, object userData)
{
// Initialise data
bool retVal = false;
string sourceName = "MSBusinessRulesEngine"
string message = "An exception occurred when executing a " +
"rule set.\r\n" + ex.toString();
EventLogEntryType entryType = EventLogEntryType.Error;
int