Geeks With Blogs

Charles Young
After a very pleasant Texan meal last night with Greg, James, Mark and others, a fascinating discussion about evolutionary approaches to rule processing, and a reasonable night’s sleep, it’s now day three of ORF 2009.   Karl arrived this morning, so Microsoft is now represented. We can’t allow the Java crowd have rules processing all to themselves J   Karl has short hair now.   I guess that’s what a couple of years on the Redmond campus does to you.
Dr. Hafedh Mili from Quebec was up first this morning talking on “Agile Business Rule Development: a tautology or an oxymoron?”   He started off by suggesting possible areas of non-correspondence between the Agile Manifest and rule methodology.   For example, the manifesto values working software over comprehensive documentation, but rules methodology values the externalisation of documented rules from process definitions.
Agile methods are used by smaller teams to handle rapid change in turbulent environments.   Plan-driven methods are used by larger teams to ensure stability and high assurance in stable, low-change projects.   Agile methodologies support situations where the team has access to dedicated on-site customers focused on prioritised increments, whereas plan-driven methods may be better in scenarios where customers are not collocated.   Agile methodology provides many degrees of freedom to foster a sense of comfort and empowerment whereas plan-driven approaches use a framework of policies and procedures.   Dr Mili suggested that both approaches are needed, and talked about hybrid methods.   He advocated starting with the simplest methodology needed to get the project started and then building the methodology up as required. Focus on people, values, communication and expectations rather than method.
Dr Mili is working on a book and web site for Agile Business Rule Development (ABRD).   This is an open-source methodology developed by Jerome Boyer of IBM ILOG.   It advocates rule discovery, analysis and validation using active collaboration between different personas, iterative and incremental development of rule sets, etc. The notion of decision points in process definitions is a key concept. Organisations harvest, complement, improve, enhance and maintain rule sets.   Harvesting involves rule discovery and analysis.   This can be extended using prototyping using initial rule authoring.   Rules sets are built by incrementally executing rules on real or near-real data, and involves validation of the rules as they are added.   Valid rule sets can then be integrated into systems.   Dr Mili described enhancement of rule sets as the ‘cruising speed’ cycle of rule set development which continues throughout the life time of the application.
The cycles advocated by ABRD are designed for starter projects.   Organisations then need to mature their methodology in terms of rule governance, rule structure and integration infrastructure.   The rest of the talk looked at various aspects of these elaborations.
Eric Charpentier of Primatek then spoke on “Rule Classification First”.   Eric’s talk centred on the problem of communication where different perspectives exist about the nature of business rules.   The database perspective (constraints on tables) can be quite different to the process modelling perspective (decision points in process definitions), for example. A CMS system might encode rules to govern how content is presented, etc., etc.   Eric concentrated on the use of rules in an ESB environment, and used the SBVR definition of a business rule as “a rule that is under business jurisdiction”.   On this, he defined the following rule categories:
System / IT Rules
·         Data Transformation Rules: Rules that transform data from one format to another.
·         Referential Integrity Rules: Rules that represent and control relationships between entities.
·         Data Validation Rules: Rules that keep the information (data) in the system consistent.
·         Security Rules: Rules that control access to functionality or data based on roles.
·         Presentation Rules: Rules that allow for dynamic content to be presented to users.
Business Rules
·         Business Process Rules: Rules that are business process and workflow related.
·         Decision Rules: Rules used to define the dynamic, business-level building blocks of the policies, directives and initiatives of an enterprise.
·         Rating Engine Rules: Rules that represent a pricing model to be applied to a given transaction.
According to Ron Ross, business rules must be actionable, must be about the business rather than knowledge/data-recording systems that support the business or about a platform used to implement such a system.   They must be expressed in the language of the business. Eric recommended up-front classification of rules as an aid to communication.
Dr Jacob Feldman gave the third presentation of the day.   Dr Feldman spoke on ‘Rules Violations and Over-Constrained Problems’.   He started with a scheduling issue where a manager wants to schedule a single worker to do two jobs, one expected to take three hours and the other two, but the worker is given four hours to do both. If the scheduling engine with a rule that says that the worker cannot be in two places at the same time the manager will have a problem creating the schedule.   One approach to dealing with these problems is to use measured constraints to ‘soften’ rules and define a degree of tolerance.   A soft rule does not have to be fully satisfied, but we would like it to be.   Several CSP approaches exist.  For example, partial CSP maximises the number of satisfied constraints. Weighted CSP associates a weight to each constraint and maximises the weighted sum of satisfied constraints.   A cost-based approach introduces cost variables and uses post meta-constraints on cost variables together with optimisation approaches that turn ‘soft’ problems into ‘hard’ problems.
Dr Feldman went on to describe approaches to the over-constrained problem using the map colouring problem. He explained how by using weightings as multipliers, the problem can be reduced to a simple optimisation approach.   Definitions of constraint violation cost are meta-constraints which can be expressed as business rules.   He described an example involving the balancing of financial portfolios that is driven by rules but which is affected by constant fluctuation in stock prices.   The introduction of ‘soft’ rules allows the system to do a much better job of finding the best approximation to a balanced portfolio in the face of constant change.
The next example was to do with scheduling constraints where the system must deal with late activities, overcapacity, use of overtime, overuse of skills, etc.   Common requirements include minimising travel time, starting jobs as soon as possible, minimising overtime, maximising skill utilisation, etc.   These require a degree of compromise which can be enabled through soft rules.
The third example was solving the Miss Manners problem.   Now, before we go further, can I please remind everyone that the Miss Manners benchmark instantly ceases to be the true Miss Manners benchmark if you change the way the problem is solved.   The actual problem itself (seating guests according to gender and hobby) is of no consequence when using Miss Manners for benchmarking. Just a small point, but one that causes a lot of confusion.  The true Miss Manners benchmark uses a contrived approach in order to find a solution in a deliberately and highly inefficient fashion. The code we were shown is not in any sense an implementation of the Miss Manners benchmark, but it does solve the same problem.   For our purposes this morning, the point was to soften the constraints in order to find the most optimal solution in scenarios where the mix of genders or hobbies is such that the ‘hard’ problem cannot be solved. This has nothing to do with benchmarking, although Dr Feldman did provide some comparative timings between solving the problem using mixtures of soft and hard rules, which is fair enough.
Daniel Brookshier, chief architect of No Magic, then spoke on ‘Generating Rules from UML’.   Modelling is more than software.   How can we model rules in UML with PRR.  Daniel talked about how rule design is about more than rules and described some of the problems with documenting rules, managing requirements, analysing, testing and deploying rules etc. We need ways to support rule modelling that use representations that are not tied to specific tools.   PRR is a neutral representation of production rules from the OMG and is designed to allow rule definitions to be shared between UML tools.   Daniel provided a brief introduction to PRR and then discussed the good and the bad from his direct experience of implementing support for PRR.   The good includes its support for simple rules, visualisation, proven ability to be used for generation, support for generic analysis, improved reliability for un-typed languages and extensibility.   The bad includes lack of support for complex rules, awkwardness in regard to visualisation, lack of an action language (this is being addressed), a degree of complexity of entering rules using UML and general lack of understanding of OCL.   The really good news is that it is easy to extend and interface to other models.   Extensibility can be added to support language-specific features, initialise the working memory, etc. PRR does a good job of separating aspect of rules such as working memory, conditions, actions and rule sets. Daniel showed use some examples of composability with other OMG profiles in which, for example, a test view uses rule conditions only.
Do requirements make better rules? No, but they help with completeness, testing, simulation, etc.   Daniel worked through a ‘Monkeys and Bananas’ example explaining various representations used in defining rules.   I was fascinated in what he had to say about rule simulation, although I think this must be understood in the context of his comments about PRR being suitable for simple, rather than complex rules. Daniel finished by demonstrating a worked example that targets Ruleby (a Ruby rule engine) and Drools.   It can also generate tests, simulations etc.
Lunch was a delicious Italian chicken dish at a local restaurant with Daniel, James, Charles and others talking about British sitcoms and the beach at Brighton, followed by a conversation with Rick and Andrew.   I managed to miss the first few minutes of Paul Vincent’s talk on “What's Different About Rules in CEP? That was a bit silly, seeing as I am speaking on much the same subject on Friday and need to fine-tune my talk to ensure that I don’t just repeat the same information.  Anyway, his talk centred on Complex Event Processing generally. He talked briefly about the support for CEP in various Rete engines.   Paul works for Tibco who support CEP through their Tibco BE rules engine.   He discussed the use of distributed caching and persistence of data in CEP processing and then went on to discuss what rules engines need for CEP.   Chief is the need to handle temporal processing.   He then described the rule types used in Tibco BE.   These include basic condition-action rules, triggers (event-condition-action), timers/schedulers and event lifecycle rules (timeToDie-action).   Paul then moved on to scalability.  For low latency, he explained that event performance is dependent on minimising run-to-completion time. To do this, Tibco uses distributed event processing with blackboard and load-balancing techniques with multiple rule agents.
Paul described a fraud detection example and showed how various aspects of event processing and rule definition are handled in Tibco BE. The example rules utilised event history processing and analytics. He then described the use of state models and machines, and the way they can be mapped onto rule engines.   State modelling for CEP involves management of time-related state / flow transitions and modelling missing events through the use of time-outs.   State models are more complex than rules for a number of reasons. Sates can be defined per instance, by type and can be inherited.   State transitions are defined by events.
Carole Ann Berlioz-Matignon from FICO was up next talking about “Measuring Your Rules' KPIs”.   Taking on business rules and empowering business users can be a painful process.   Allowing business unlimited access to rule in production is a dangerous and risky thing to do, and can lead to sub-optimal operations and contradictory decisions.   The objective of agility is to lead to better systems.   Reality can be rather different, however.   How can business users be empowered in a less risky fashion.   SOA, EDA and mash-ups empower enterprise architects and allow them to build sophisticated applications. However, supervision is needed to ensure that organisations stay on track.
Ways of implementing safeguards include rule verification, rule validation and lifecycle management.   The key is to define business objectives and ensure that the logic produces the expected business results.   Organisations need to identify risks and opportunities to improve their strategies.   Simulation has a central role here, and KPIs can be measured and analysed.   Metrics can be presented to business users via dashboards.   Consider turning dashboards into business forecast tools to predict the bottom line and perform sensitivity analysis.    Performance-driven culture is pervasive, even in production.   Champion / Challenger approaches can be used to test-drive strategies in production, reducing the risk of change.   KPI-enabling architectures depend on a number of things being in place.   One is master data management (MDM).   This acts as a foundation for decision performance management based on monitoring and alerting and supporting advanced approaches such as dynamic adaptation.    KPIs tell you if you are on target. They can be fed back into systems to help improve business performance.
Luke Voss from Mindviews labs, who spoke yesterday on “Graph-Based Knowledgebases & Rules” spoke again about “Designing a System of Rulebased Agents”.    Luke described his association with “Team X” at JPL.   They are responsible for brainstorming and mission design for space missions (how cool is that – can I join the team please?).   The team has an interesting mode of operation involving 1 room, 20 engineers and scientist, workstations for each person, networked spread sheet models.   They meet for three three-hour sessions over a few days and have a facilitator and customer representatives on hand.   The process is complex, but is repeatable which indicates that the complexity has been mastered.   During the sessions, the team identifies science goals, mission constraints and evaluates baseline alternatives. If time permits they explore alternatives. The work results in the drafting of a final report. The team uses various models and tools that are common across different sessions.   Models cover power, thermal, structures, attitude control, telecoms, ground control, science payloads etc.   Tools include CAD together with trajectory and risk tools, mainly implemented as Excel spreadsheets.    Networking is used to pull all the data together into an overall system model.
Team X has been working for a long time, but JPL want to see improvements in terms of time, cost, quality and quantity.   A number of concepts have been considered such as using compositional models, altering the team structure, implementing executable models and using rule-based systems.   The team uses agent-based, networked approaches.   Luke identified four types of agent. These are human agents, coordination agents, domain expert agents and algorithmic agents. Coordination agents know which agents solve which problems and keeps the big picture.   They request clarification, identify solution strategies, delegate activities and return solutions.   Domain expert agents provide standard expert system features to solve unstructured problems.   They request clarification, return estimates and work out solutions.   Algorithmic agents perform efficient problem solving and make use of existing analysis tools. They marshal data and make library calls.   Human intervention and interaction agents answer questions that other agents cannot and supply live experts.   They solve problem requests, supply knowledge and override other agents.
The agent approach provides good features for complexity management compared to a more traditional module-based approach. It helps with inspection and debugging, provides better separation of concerns, is easier to maintain and provides better overall performance. The future holds implementation of more expert systems, refining the message bus and generating executable models.
Edson Tirelli from the JBoss Rules team then spoke about “Extending General Purpose Engines with Domain Specific Resources”.   He asked the question “what do you want to be when you grow up – a solver or an enabler?”   A solver solves problems on behalf of people.   An enabler helps people solve their own problems.   Which role does a rule engine play?   His talk was about using rules engines as enablers.   Users care about ends, not means. We use abstraction to introduce layers of indirection and simplify problems.   Higher abstractions bring problems closer to the human way of thinking and facilitate understanding of the whole picture. Domain specific resources such as DSLs and libraries facilitate the creation of solutions specific to a given domain and perceived expressiveness.   Drools supports function libraries which can be inline or imported.   Drools provides accumulation (set-based) functions.   It provides evaluators to define relationships between two facts.   Evaluators support parameters.   Drools supports domain specific languages (note for BizTalk readers – the approach is essentially the same as vocabularies in MS BRE).   There is also support for domain specific processes using the rule flow features.   Work items (think ‘activities’ are configured to show specific text, icons, etc.
The day ended with the daily Q&A session.   Question were asked about how agile the methodology described by  Dr Mili truly is.   There was discussion about the importance of MDM and its growing importance.   Paul Vincent pointed out that MDM is itself about process with rule-driven decision points.   That sounds familiar. I’ve worked on BizTalk projects that are little more than MDM.    Questions were asked about a number of other issues such as the role of domain-specific in helping with debugging and troubleshooting.   A couple of question were put to Paul Vincent concerned the issue of event mutability. Paul explained that while events are generally immutable, in terms of their specific event semantics, there are situations where a degree of mutability may be useful.   It’s not a black and white issue in terms of real-world processing.   Another question concerned the choice between rules and constraint engines.   The general conclusion is that there is no absolute way of determing the best approach, but that constraints may be more appropriate when you have many alternative possible solutions.   Carole Ann Berlioz-Matignon talked about the use of heuristics as yet another alternative approach.
Posted on Wednesday, October 28, 2009 6:06 PM | Back to top

Comments on this post: October Rules Fest: Day 3

No comments posted yet.
Your comment:
 (will show your gravatar)

Copyright © Charles Young | Powered by: