Building large object oriented systems requires us to be able to think/visualize them on different levels of abstraction. Whatever we want to describe – data or functionality – we want to describe in more or less detail. That´s also true if we want to switch our OO approach to messaging.
Enter nesting.
If flows describe networks of objects communicating through messaging, then we need to be able to nest flows. And in fact that´s what I´ve done all along. Take this flow from my previous posting for example:
That´s a single flow on a certain level of abstraction – but in reality it´s already nested. It´s nested in an encapsulating, yet invisible flow consisting of only one functional unit:
The rules for such nesting are simple:
- each input to a parent functional unit must connect to at least one input of one of its children
- each output from a parent functional unit must be connected to at least one output of its children
Now we can build messaging hierarchies of arbitrary depth. We can design them top-down or bottom-up. We can refactor them: If at some point you think there is too much detail, you can do an “extract flow” and create a child flow to replace a part of some flow. Or you do it the other way around and do an “inline flow”.
So much for the easy part of it. Conceptually nesting is simple. The hard part is in implementing such nested flows. No, not because there are any technical difficulties. It´s just hard to stay true to yet another principle. I call it the Integration Operation Segregation Principle (IOSP).
Let me explain it using the above example. Here´s how one could implement the top-level flow knowing of its structure:
void process(T data) { if (validate(data) save(data); else report_validation_error("..."); }
That might be straightforward to common OO lore – but it´s an invitation to brownfield disaster.
What´s the problem, you might ask. Well, what this implementation does is smearing logic all over the place. There is logic in save and in report_validation_error and in validate. But also there is logic in process. According to the SLA principle there is no validation condition in process, but still there is logic in it. It´s the control structure.
Any if, while, for, repeat, switch statement represents logic like an expression. In fact control structures and expressions are the very definition of program logic. They are the atoms software is made from.
Logic in and of itself of course is not the problem. But how logic is distributed across a code base can be. Because what makes it so hard to change a growing code base are not only physical dependencies (like process knowing about save), but logical depdencies between pieces of scattered logic. Several principles are trying to persuade us to keep logic (responsibility, concern) close together (high cohesion): DRY, SRP, SoC, SLA, ISP, LSP. But it´s hard to follow principles. Principles are no rules.
But what can we do about the detrimental undue distribution of logic? It starts so small, like above – but then it grows and spreads. It´s like metastasizing cancer. And in the end software dies because all energy is sucked up by keeping all the proliferation consistent. Nothing new can be added because the implications are incalculable.
So here´s my take on this problem: Structure messaging based logic according to the IOSP. The very concrete rule is: Subroutines are either integrations or operations. Operations contain logic, integrations not. Integration means aggregating operations and other integrations into a larger whole.
The above implementation of process does not follow this rule. The control structure must be pushed down into an operation, for example like this:
void process(T data) { validate(data, save, report_validation_error ); }
void validate(T data, Action onValid, Action onInvalid) { if (...) onValid(data); else onInvalid("..."); }
Not the sole purpose of process is to “wire up” validate, save, and report_validation_error into a sequence of processing steps (flow) according to its own purpose. It´s ok for process to be dependent on those subroutines. That´s what process is for, that´s its integration task. But it´s not ok for validate to know of save or report_validation_error. That would violate the Principle of Mutual Oblivion (PoMO) described in my previous article.
I know, the temptation is huge to put logic into functional units on all levels of abstraction when implementing a flow. But don´t! Keep any non-leaf nodes in a messaging hierarchy clean of logic. Logic must only reside in leaf nodes:
Interestingly, adheres not only to the PoMO, but also to the ISOP. Here´s some code he sent me via email:
var partMessage = new PartUploadMessage;
var validationPipeline = new Task .Register(Validate.PreValidate) ... .Register(Validate.PostValidate);
var loadPipeline = new Task .Register(ExcelFile.PreSpreadsheetLoad) ... .Register(ExcelFile.PostSpreadsheetLoad);
var importPipeline = new Task .Register(loadPipeline.Execute) .Register(validationPipeline.Execute);
importPipeline.Execute(partMessage);
The point is, his integrating functional units are pipelines. And they are generic. So there is no domain logic in them. Logic is only in the subroutines registered with pipelines, e.h. Validate.PreValidate.
This suggests, flows or integration should not be defined in the same language as operations. Using a DSL without any features for logic is easy to come up with, easy to implement, and greatly helps holding up the IOSP. Here is an example I have implemented myself a while ago:
process: .in, validate validate.onValid, save validate.onInvalid, report_validation_error
But take this just as an outlook. No DSL is needed to code clean IOSP messaging hierarchies. It just takes a little discipline to follow the its simple rule. Very straightforward, I´d say.
This does not make dependencies go away, but they are much more ordered and focused. That way we can get a better handle on large systems.