Geeks With Blogs

News
Charles Young
My colleague, the ‘Arch Hacker’, forwarded an email to me this evening from Kenton Price.   Kenton had noticed some peculiar behaviour using the Microsoft Business Rules Engine.   Here are the pertinent extracts from Kenton’s report:
 
“It appears that if you assert a fact in a rule, it hangs around for subsequent instantiations of the rule engine, behaving like a long-term fact.... I solved it by retracting each asserted fact on every exit path (I had a jump-out-early that halts and stops all other rules, so I had to retract the asserted facts here too)....Facts provided in the object[] passed to the BRE call are all retracted automatically. It appears that any others you assert within the rules must be manually retracted.”
 
Kenton is quite correct.   To be precise, the engine, itself, never automatically clears facts from the engine.   If this is done at all, it is done by code external to the engine.   Confusion arises because of the way that the Policy class, which effectively sets up an external context in which the rules engine is executed, handles clean up.   The Policy class has behaviour which can cause problems to the unwary.   I discovered this some time ago when writing some reflection code to inspect what happens in the engine.   I should have blogged about the issue long before now.
 
Most developers execute the rules engine via a Policy object.   When a Policy is instantiated, it retrieves a RuleEngine instance from a cache.   A RuleEngine is a very simple object that wraps an 'Executor' object.   In our case, the Executor is simply the root node of the Rete inference network (an instance of the Rete class).   You can create custom executors if you want, but the only executor provided by Microsoft is the Rete class.   When, at some later time, the Policy object is disposed (which may be when it is finalised), the RuleEngine instance is released back to the cache.
 
When you execute a Policy, the code calls Execute() on the RuleEngine instance, passing in the facts.   This, in turn, calls Execute on the executor object, asserting the facts to the root node of the Rete inference network.   The facts are wrapped in WME (Working Memory Element) objects which trickle down the network and get collected into working memory through the use of linked lists of memory ‘tokens’. Ultimately productions are activated on the agenda and fired.    The engine may forward chain by asserting new facts or reasserting/updating/retracting existing facts.
 
Internally, the Rete executor manages execution via a processing loop that en-queues various operations and manages state transitions.   Whenever the engine arrives at the 'idle' state, this loop de-queues the next engine action (assert, retract, clear, halt, etc.,) and transitions to a new state.   The de-queued action is then invoked.
 
Once the engine reaches an 'idle' state and the queue is empty, nothing more happens.    There is no automatic retraction of any facts.    At that point, the process loop returns, and control passes back to the RuleEngine, and from there to the Policy object.
 
The very next line of the Policy object code retracts the same collection of facts that was asserted on the previous line.   This constitutes clean-up code that is external to the actual engine itself.   The problem is obvious.   If, during forward chaining, you have asserted any additional facts, and these facts are still in the working memory, they are not retracted because the Policy object does not know about them.   Only those facts that you asserted directly via the Policy object are retracted by the Policy object.   When the RulesEngine is returned to the cache, no further clean up is done.   So, on a future invocation, you may well find that your RuleEngine instance already has facts asserted within working memory and these may result in unexpected behaviour.
 
Kenton is absolutely correct in linking this to long term facts asserted via a FactRetriever.   It is precisely because of this feature that the Policy object and RuleEngine cache cannot afford to clear the working memory.   They have no way of knowing what an individual FactRetriever has asserted/retracted, and it would be very, very unwise to try to second-guess the custom logic contained in a FactRetriever.   I suppose, thinking about it, Microsoft could have implemented the Policy class to clear the working memory when no FactRetriever is used.   However, this would mean that its behaviour would change in a very opaque fashion.   The way you should think about it is that, as far as the engine is concerned, you are always responsible for cleaning up after yourself, but that the Policy object helps out by automatically retracting the facts it knows about.
 
There are a few ways you can deal with the problems caused by this behaviour.    The first is to write your rules in such a way that the very last rule to fire invokes the built-in Clear function. You could use a low priority rule that tests "1 is equal to 1" for this purpose.   If you use Halt, then you need to call Clear() just prior to that, or possibly retract facts explicitly, as Kenton did.   It would generally be best to call Clear(). NB. calling Halt does not clear the working memory.   It can, optionally, clear the agenda. When you invoke the Clear function, this clears both working memory and the agenda.   There is also a separate ClearAgenda engine operation.   However, this is not surfaced in the Rules Composer and can therefore only be invoked via custom code.   If absolutely necessary, you can pass the Executor out to some custom code in a rule action and have that code invoke the engine operations directly.
 
Another approach is to write a FactRetriever to clear the working memory.  I wrote about FactRetrievers in an article published at http://geekswithblogs.net/cyoung/articles/118804.aspx.   The FactRetriever has a single Update method that could be implemented to call Clear() on the RuleEngine object passed to it.   Having created and GACd the FactRetriever, you can configure policies to use it via the Rules Composer. At runtime, the Policy class invokes the Update method just before asserting facts to the engine, so this approach will ensure that working memory is cleared each time the RuleEngine is executed via Policy.
 
A third approach would be to dispense with the Policy class (and therefore with the CallRules shape in BizTalk orchestrations) and write custom code to execute the RuleEngine.   I published some code in the same article at http://geekswithblogs.net/cyoung/articles/118804.aspx which may serve as a starting point.   I never intended this code to be used for production.   It doesn't cache RuleEngine instances, and it doesn't clean up.   You would need to extend the code with custom caching.   You could call Clear() directly on the RuleEngine instance to clear the working memory and agenda before returning your RuleEngine objects to your cache.
 
The last thing Microsoft should do is to add automatic retraction to the Rete executor.   There are many valid scenarios in which you want to be able to retain facts in working memory for subsequent execution cycles. That's why the FactRetriever feature is provided.   It is typically used to manage the assertion of long-lived reference data as facts to the engine in order to eliminate unnecessary database lookups or other activities.   Other scenarios might include things like event processing where you need to retain event 'facts' over a given time window and re-execute the engine every time an event arrives.   Of course, MS BRE is not optimised for event processing and provides no help whatsoever with regard to temporal logic.   Few Rete engines do, though this is beginning to change.
 
One final thought.   Microsoft could have designed a more sophisticated approach in which assertion of facts to the RuleEngine is always managed through components similar to the FactRetriever.   As it is, the engine itself has no knowledge of FactRetrievers.   These are only used by the Policy and PolicyTester classes (PolicyTester retracts the facts it knows about, just like Policy).   They could have designed things so that a collection of FactManager objects (I made that up - there is no FactManager type for real) are added to a RuleEngine instance, and could have built some mechanism whereby each FactManager can indicate to the RuleEngine if the facts it is managing should be automatically retracted or not.   The engine could then perhaps have used a special internal instance of the FactManager to handle facts asserted during forward chaining and allow developers to configure its retraction behaviour.   This would have been a better design.   As it is, you either have to write your rules to clean up things, or write your own logic around the engine to manage things the way you want.   That means you sometimes have to dispense with the Policy class and write your own replacement.
 
Posted on Tuesday, April 28, 2009 10:07 PM | Back to top


Comments on this post: MS BRE: Clearing Working Memory

# re: MS BRE: Clearing Working Memory
Requesting Gravatar...
Thanks for this, Charles. As we discussed and agreed last night, it's fine to call Clear if there's no long-term facts from a fact retriever involved, but I couldn't call Clear() upon exit because that would also clear all the long-term database facts I had asserted via my fact retriever that I do actually want to hang around for future instantiations - instead I needed to individually retract everything I'd asserted.
Left by Kenton Price on Apr 29, 2009 8:28 AM

# re: MS BRE: Clearing Working Memory
Requesting Gravatar...
Yes, indeed. The various approach I outlined are not universal solutions. You need to select the strategy that best fits your needs. The FactRetriever approach would only be effective in situations where the only 'long-term' facts in your working memory are those that are asserted by rules. In your case, the issue is that you already have a FactRetriever that is asserting a whole lot of reference data as facts. That really means that you have to avoid using Clear() at all. So, the approach you have taken of building the logic into your rules to retract specific facts is the only option available to you. Remember there is a RetractByType action. If the unwanted facts are all of different types to thise that your factProvider is asserting, this could be really useful.
Left by Charles Young on Apr 29, 2009 8:36 AM

# re: MS BRE: Clearing Working Memory
Requesting Gravatar...
I'm asserting five facts, but they're five different types (they are also different to the FactRetriever-asserted types). Is there any performance advantage or disadvantage to using RetractByType over specific Retracts when there's only one of each type to be retracted?
Left by Kenton Price on Apr 29, 2009 8:42 AM

Your comment:
 (will show your gravatar)


Copyright © Charles Young | Powered by: GeeksWithBlogs.net