The Architect´s Napkin

Software Architecture on the Back of a Napkin
posts - 69 , comments - 229 , trackbacks - 0

My Links



Post Categories

Nested Messaging - Flows on Different Levels of Abstraction

Building large object oriented systems requires us to be able to think/visualize them on different levels of abstraction. Whatever we want to describe – data or functionality – we want to describe in more or less detail. That´s also true if we want to switch our OO approach to messaging.

Enter nesting.

If flows describe networks of objects communicating through messaging, then we need to be able to nest flows. And in fact that´s what I´ve done all along. Take this flow from my previous posting for example:


That´s a single flow on a certain level of abstraction – but in reality it´s already nested. It´s nested in an encapsulating, yet invisible flow consisting of only one functional unit:


The rules for such nesting are simple:

  • each input to a parent functional unit must connect to at least one input of one of its children
  • each output from a parent functional unit must be connected to at least one output of its children

Now we can build messaging hierarchies of arbitrary depth. We can design them top-down or bottom-up. We can refactor them: If at some point you think there is too much detail, you can do an “extract flow” and create a child flow to replace a part of some flow. Or you do it the other way around and do an “inline flow”.

So much for the easy part of it. Conceptually nesting is simple. The hard part is in implementing such nested flows. No, not because there are any technical difficulties. It´s just hard to stay true to yet another principle. I call it the Integration Operation Segregation Principle (IOSP).

Let me explain it using the above example. Here´s how one could implement the top-level flow knowing of its structure:

void process(T data) {
  if (validate(data)

That might be straightforward to common OO lore – but it´s an invitation to brownfield disaster.

What´s the problem, you might ask. Well, what this implementation does is smearing logic all over the place. There is logic in save() and in report_validation_error() and in validate(). But also there is logic in process(). According to the SLA principle there is no validation condition in process(), but still there is logic in it. It´s the control structure.

Any if, while, for, repeat, switch statement represents logic like an expression. In fact control structures and expressions are the very definition of program logic. They are the atoms software is made from.

Logic in and of itself of course is not the problem. But how logic is distributed across a code base can be. Because what makes it so hard to change a growing code base are not only physical dependencies (like process() knowing about save()), but logical depdencies between pieces of scattered logic. Several principles are trying to persuade us to keep logic (responsibility, concern) close together (high cohesion): DRY, SRP, SoC, SLA, ISP, LSP. But it´s hard to follow principles. Principles are no rules.

But what can we do about the detrimental undue distribution of logic? It starts so small, like above – but then it grows and spreads. It´s like metastasizing cancer. And in the end software dies because all energy is sucked up by keeping all the proliferation consistent. Nothing new can be added because the implications are incalculable.

So here´s my take on this problem: Structure messaging based logic according to the IOSP. The very concrete rule is: Subroutines are either integrations or operations. Operations contain logic, integrations not. Integration means aggregating operations and other integrations into a larger whole.

The above implementation of process() does not follow this rule. The control structure must be pushed down into an operation, for example like this:

void process(T data) {

void validate(T data, Action<T> onValid, Action<string> onInvalid) {
  if (...)

Not the sole purpose of process() is to “wire up” validate(), save(), and report_validation_error() into a sequence of processing steps (flow) according to its own purpose. It´s ok for process() to be dependent on those subroutines. That´s what process() is for, that´s its integration task. But it´s not ok for validate() to know of save() or report_validation_error(). That would violate the Principle of Mutual Oblivion (PoMO) described in my previous article.

I know, the temptation is huge to put logic into functional units on all levels of abstraction when implementing a flow. But don´t! Keep any non-leaf nodes in a messaging hierarchy clean of logic. Logic must only reside in leaf nodes:


Interestingly, Steve Bate´s implementation of messaging adheres not only to the PoMO, but also to the ISOP. Here´s some code he sent me via email:

var partMessage = new PartUploadMessage();
var validationPipeline = new Task<PartUploadMessage>()
var loadPipeline = new Task<PartUploadMessage>()
var importPipeline = new Task<PartUploadMessage>()

The point is, his integrating functional units are pipelines. And they are generic. So there is no domain logic in them. Logic is only in the subroutines registered with pipelines, e.h. Validate.PreValidate().

This suggests, flows or integration should not be defined in the same language as operations. Using a DSL without any features for logic is easy to come up with, easy to implement, and greatly helps holding up the IOSP. Here is an example I have implemented myself a while ago:

.in, validate
validate.onValid, save
validate.onInvalid, report_validation_error

But take this just as an outlook. No DSL is needed to code clean IOSP messaging hierarchies. It just takes a little discipline to follow the its simple rule. Very straightforward, I´d say.

This does not make dependencies go away, but they are much more ordered and focused. That way we can get a better handle on large systems.

Print | posted on Monday, August 19, 2013 12:25 PM | Filed Under [ OOP as if you meant it ]



# re: Nested Messaging - Flows on Different Levels of Abstraction

So this is much closer to an example I wanted, but how would you fix the validate method?
8/19/2013 3:59 PM | Nick

# re: Nested Messaging - Flows on Different Levels of Abstraction

validate() is fixed. Passing in continuations for onValid/onInvalid fixes it. It´s oblivious of it´s environment. It´s self-sufficient. It does a single job and informs its environment of some result. Who gets to process the results is a matter of the environment: that´s what integration does.

Operations decides, integrations wire-up. That´s clear cut responsibilities.
8/19/2013 5:06 PM | Ralf Westphal

# re: Nested Messaging - Flows on Different Levels of Abstraction

Hi Ralf, in your final example of Process(..) & Validate(..), I understand that Process would be the integration task, but I'm unclear on the validate function. Would validate be the operation task? Thanks, Oli
1/8/2015 2:10 AM | Oliver Tomlinson

# re: Nested Messaging - Flows on Different Levels of Abstraction

Yes, Validate() is an operation. It's a leaf in the function call hierarchy.
1/8/2015 8:29 AM | Ralf Westphal
Post A Comment

Powered by: