I got an email today requesting help in deciding the appropriate selection of rule processing technology for a workflow application. I’ve got requests like this before, so I’ve decided to post a reply publically. Here is the text of the email:
Before anything I must tell that I have no experience using MS BRE or WF. I just have a general idea.
I think I better explain what my problem is:
I'm trying to develop an application using Microsoft .NET in which users should be able to create a workflow and some tracking points on that workflow. Each tracking point will have a Business Rule associated to it that will be used to specify whether the workflow has failed in that point or not. They have to be able to define the workflows, tracking points, and BRs in runtime.
It sounds to me a BizTalk based application. I just want to make sure that I feel right about this. Also I've heard that I can use MS BRE without having BizTalk installed (except its runtime). But I couldn't really find anything about this. Could you please help me a little bit to find the start point and get the right feel?
I would really appreciate your reply.
My reply is as follows:
In reverse order...
BizTalk Server 2006 allows you to install just the Rules Engine, Rules Framework, Rule Repository and other associated tools and assemblies without installing other parts of the product. However, the big issue here is licensing. The Rules Engine is not re-distributable. You still need a full BizTalk licence per-processor even if you only install the rules engine. One of the industry research companies (Forrester) suggested some time ago that, based on the licence costs of the standard and developer editions of BizTalk, this still made MS BRE significantly cheaper than the main rule engine vendors, which is broadly true, unless, perhaps, you intend to use a 32-processor box! However, Microsoft does not provide the same level of sophisticated tooling that you get from the main rule engine vendors. MS BRE tooling (the repository, rule composer, etc.) is relatively basic and doesn’t support features such as decision trees/tables, rule set analysis, etc (Microsoft pulled rule set analysis features from the beta of the framework back in 2003).
Another common point of confusion is that MS BRE and BizTalk Server are quite separate pieces of software, written by different product teams. There is only very light-weight integration between the two. MS BRE offers a programmatic API that allows you to invoke rule sets (policies) from anywhere.
The obvious question to ask is whether you should use BizTalk Server or Windows Workflow Foundation (WF). I’ll try as best as I can to highlight the main points you need to consider here...
BizTalk Server is a licensed, enterprise-level integration and message routing toolset. One of its major features is 'orchestration'. This is primarily designed to handle and execute definitions of automated business processes. As well as all the basics of process flow, process tasks (activities), etc., BizTalk Server orchestrations handle many other concerns such as high-throughput scenarios (handling, say, thousands of concurrent process instances simultaneously), long-running business activities where it is necessary to store serialised process state for long period of time, distributed transactions across different resource managers, etc. Orchestration is hosted and integrated within the wider context of BizTalk as a whole, and therefore interacts with BizTalk's queues (the 'message box') using subscription rules. BizTalk provides built-in tracking features (mainly used for health and diagnostic information) and a technically-related feature called BAM (Business Activity Monitoring) which can be used to track process milestones and collect/aggregate data from process instances. The Business Rules Engine can be invoked from within orchestration code, typically to govern decision points within the process flow or to implement policy-driven process design patterns. BizTalk Server ships with a variety of development tools (mainly hosted in Visual Studio) for defining orchestrations, message types, BAM activity definitions, etc.
WF, by contrast, is not a product. It is provided free-of-charge as part of the .NET 3.0 platform. It is a 'foundation' - i.e., a foundational code framework. The basic idea is that if you are writing a custom application that has a workflow aspect, or if you are an ISV writing some workflow product, you can use WF as the starting point for implementing workflow features. WF does not provide enterprise-level workflow hosting out of the box. WF provides a core runtime to which hosted services can optionally be added using pre-defined interfaces. These services are implemented to address concerns such as persistence, tracking, etc. WF ships with some pre-defined host service implementations (e.g., the SqlTrackingService which stores tracking data in a custom database) However, WF does not provide BizTalk-equivalent enterprise-level host services out of the box. The BizTalk product team plan to use WF in future versions of BizTalk Server as the basis of a new implementation of orchestration. They will write their own services for hosting WF–based orchestrations. The Office team uses WF to support workflow in SharePoint Server (MOSS). In this case, the WF host environment is provided by the underlying Windows SharePoint Services (WSS) which implements a basic range of host services.
WF only addresses workflow concerns. It provides no equivalent to the BizTalk message box, no subscription-based message routing facilities, etc. It can handle things like process state persistence, but only if you create and integrate the repository to store process state. It is not integrated with a tracking system or any equivalent to BAM. Microsoft provides some basic tooling, including the graphical process definition tool hosted in Visual Studio.
WF does have some very interesting features, however. It may not provide a tracking database, but it does support a very flexible approach to tracking (and some very basic tools, such a sample workflow tracking monitor and some pre-built support for storing tracking data in custom SQL Server databases). Tracking can be customised through 'profiles' which govern exactly which events are tracked.
Another important feature is that, unlike the current versions of BizTalk Server, WF has a built-in rule processing facility. In the majority of workflows and processes, the main requirement for rules is to govern decision points within process flows. Generally, rules are simple and straightforward ('if PO total amount > 5000, then discountPercentage = 10, else discountPercentage = 5'). WF rules are ideal for this purpose. MS BRE ultimately offers significantly greater power, and in certain scenarios can scale far better. However, advanced scenarios are not too common in workflows and processes. I’ll give an example, though. I once worked on a BizTalk Server project which should have been (but wasn’t) architected to exploit MS BRE rule processing to the full extent of its ability. This was an application where sets of high-level investment directives were combined with lower-level contextual data about current fund investments in order to infer sets of detailed financial adjustments, transfers, etc., required to implement the directives. Our custom code adopted a procedural, rather than a declarative approach to performing heavy-duty rule-driven inferencing. The resulting logic was dense, opaque and very hard to understand. MS BRE rule sets would have been a far better choice – something I only realised in hindsight as my understanding of business rule engines grew.
Most developers find WF rules easier than MS BRE rules to understand, debug, etc. This is because MS BRE rules are more like SQL Select statements than familiar 'If...Then' statements found in an imperative programming language. They act on data sets rather than discrete data values. Another issue is that Microsoft provides some basic tooling for WF rules, but does not provide a rule repository (see, however, http://wf.netfx3.com/files/folders/rules_samples/entry309.aspx) or a rule deployment framework. Yet another point to consider is that it is technically possible to invoke WF rules within BizTalk orchestrations, though I have yet to adopt this approach.
So, there is a lot to consider here. If your .NET application would not otherwise require BizTalk Server, and if the most natural model would be to handle workflows within application code rather than hand them off to an external system, then WF may well be the better choice. This is especially true if your application does not have to handle large number of concurrent workflow instances, doesn't integrate too heavily with other external systems and doesn't need heavy-duty resilience features. Does the application run on the client side or as a centralised service? If it is centralised, are there only a small number of concurrent users, or a large number? Does it communicate with external systems, organisations, etc? If you select WF, you may need to do some addition development work to create tracking databases, persistence services, etc.
I hope this is of some help in making decisions. Hopefully, the selection and application of appropriate rule processing technology across the Microsoft platform will become much clearer and easier in future. I know that this is a goal for future versions of BizTalk, WF, MOSS, etc.