The Architect´s Napkin

Software Architecture on the Back of a Napkin
posts - 68 , comments - 222 , trackbacks - 0

My Links



Post Categories

Friday, November 27, 2015

Teaser: Hierarchical Scrum – How to Introduce Scrum Step by Step

Not all practices are created equal, I’d say. That became clear to me again while reading “Scrum: The Art of Doing Twice the Work in Half the Time” by Jeff Sutherland.

Even though I’ve grown somewhat skeptical of the Agile silver bullet Scrum and find some of what Jeff Sutherland wrote an exaggeration and a bit heavy on the side of self-praise, I still like the book. I can recommend it, especially to non-developer people of all trades.

What the book made me think of was a kind of hierarchy of Scrum practices. Some of it to me simply is more important than the other.

[Read on in my new blog…]

New RSS Feeds

Since October 2015 I write all my new blog articles in a consolidated blog right on my homepage at

If your want to be notified automatically about new postings, please subscribe to one of the following RSS feeds or follow me on Twitter.

Posted On Friday, November 27, 2015 4:38 PM | Comments (0) |

Wednesday, July 29, 2015

Who do you want to KISS? - About simplicity in coding

What does KISS really mean? How do you write KISS code like required by TDD for phase two when going from red to green? To answer that we need to know what simplicity is. Wikipedia says

“Simplicity is the state or quality of being simple. Something which is easy to understand or explain seems simple, in contrast to something complicated.”

Well, that helps, doesn’t it? No, I don’t think so. Because it begs the question: What is easy to understand?

I was wandering in the dark for a long time myself. Couldn’t really get a hold on simplicity. I had the feeling I knew what it meant - but then it still was hard to explain. But fortunately those days are over.

What one day cleared my understanding was a simple sentence I read on a page of B.J. Fogg:

“Simplicity is a function of your scarcest resource at that moment.”

Suddenly simplicity was easy to recognize and to apply.

But let me explain what my understanding of this definition is:

  • function of: Simplicity is related to something. Simplicity thus is relative. There is no absolute simplicity. Rather it’s in the eye of some beholder.
  • resource: Simplicity pertains to time, space, or any other resource. If something is simple more of a resource is left than if it was complicated.
  • scarcest: If more than one resource is involved in a matter then effort to create simplicity should focus on the most limited resource or the resource that’s hardest to replenish.
  • at that moment: Simplicity is not only relative in terms of resources but also in time. What is simple today might not be simple anymore tomorrow. And which resource to focus on today to relieve through simplicity might not be the resource to focus on tomorrow.

Now, is this a simple explanation of simplicity? :-)

Taking it seriously the answer has to be: It depends. It depends on the resource it tries to unburden.

What could possibly be resources involved here? How about:

  • Screen real estate
  • Memory on your computer
  • Memory on some server
  • Bandwidth
  • My time writing it
  • Your time reading it
  • Your time applying it

And what’s the scarcest resource in this list? The physical resources are plenty today. Those couple of KB are no burden on any of them. Which leaves us with time.

Is my time the scarcest resource? No. I’m willing to put it in. I’m writing pretty fast and it’s fun. No need to minimize its use - especially compared to your time or combined time of all readers. If I put in 2 hours instead of maybe 1 but thereby save each of a thousand readers just 1 minute in reading or applying the explanation… then that’s an hour well spend as a producer to save 16 hours on the consumer side.

So it’s about your time. But is this supposed to be a simple explanation with regard to your time reading it - or in the end applying it?

It seems a text like this should be succinct to save you time reading it. The average attention span on the web is short anyway. But I think that would be a misunderstanding. Striving for succinctness to make it a quick read is not targeting the really scarce resource. Your time right now might the short - nevertheless you’re investing it in reading. Reading, learning, that’s what you want to do in this moment.

Sure, the more you can suck in during that time the better. Your time for reading surely is limited. But what’s even more limited is your time designing a solution or writing code or setting up infrastructure. And it’s during that time you don’t want to get distracted by complicated definitions of simplicity. You need a simple definition to save your scarcest resource: production time.

If you have 1 hour to accomplish a task in a simple manner then it would be counterproductive to be force to lengthily ponder some definition of simplicity and how to apply it.

It’s certainly worthwhile to meditate over Leonardo da Vinci’s “Simplicity is the ultimate sophistication” - however I doubt this will lead you in a straightforward manner to a simple solution in your day to day work.

Or how about Bruce Lee’s “Simplicity is the key to brilliance”? Does that inform you whether to use recursion or not to achieve simplicity while coding a function? Again, I doubt it.

But what about B.J. Fogg’s “Simplicity is a function of your scarcest resource at that moment”? In my view that’s immensely helpful!

Think of the typical TDD situation. You just wrote a red test. Now you’re about to code a little to get it to green. You want to do the “absolutely simplest thing in the application code in order to get [the] test to run”. But how can you judge which approach to solve the problem posed by the test is the simplest one?

  1. Ask yourself, what the scarcest resource is. Examples:
  2. Do you want simplicity to save time during runtime?
  3. Do you want simplicity to save memory consumption during runtime?
  4. Do you want simplicity to save coding time now?
  5. Do you want simplicity to save reading time in the future when someone needs to understand the code?
  6. Do you want simplicity to save modification time in the future?
  7. Ask yourself which approach would save most of the identified scarcest resource.

I think the answer now is very easy: The scarcest resource right now during TDD is coding time. You need to get feedback quickly about whether you understand the problem and are capable at all to solve it. The simplest code then is the code you can come up with and type fastest.

But then, as we all know, code is more often read than written/changed. In the long run the scarcest resource therefore is reading or modification time.

As the explanation of simplicity made clear, the simplest code with regard to the coding time resource might not be the simplest code when looking at it from the perspective of another resource. What does that mean for the application of the KISS principle during TDD?

It means, you need to find time to optimize for reading/modification later. Interestingly TDD has this time built in. That’s what the refactoring phase is for. During refactoring you apply the KISS principle once more - now creating simplicity for reading/modification time. If thereby the simplicity for coding time is destroyed so be it.

As you can see: There is no such thing as absolute simplicity. You can’t create a simple solution once and for all. What is deemed simple depends on the scarce resource, which changes over the course of a day, across the lifecycle of a software, and varies for team members.

This final point is not to be neglected. Simplicity is not just relative with regard to resources but also depends on the beholder’s competency.

Even if two developers agree on the resource for which simplicity should be created they might disagree on which solution should be called simplest.

A recurring example for this in the .NET community is the use of lambda expressions. Some developers are very familiar with them and find their application to lead to simple solutions - regarding the coding as well as reading time resource. Other developers are not familiar with lambda expression and thus don’t use them at coding time, to keep their code simple. And they don’t like to be confronted with them at reading time; it takes them too long to analyze what’s going on in the code.

A simple solution for one person can be a complicated solution for another person. Sad, but true. The only way out of this is to keep all team members close together in their competencies (as long as there is a chance one member needs to deal with another’s simple solutions).

What does that mean for technologies like Linq, Angular.JS, functional languages, MongoDB, Pubnub etc. etc. which promise to make it easier to code simpler solutions? Well, simplicity has a price. It’s that simple ;-)

There are levels of simplicity that can be reached within a given technological horizon. But once you reach the top level you hit a glass ceiling. If you want more simplicity you need to invest into learning a new technology first.

So before you get lost in arguments whether a given solution is simple or not, first check if the discussion suffers from misunderstandings:

  • Is there a common understanding as to what the scarcest resource is for which simplicity should be created?
  • Are knowledge and capabilities of the participants in the discussion homogenous?

I guess in general you could say, solutions with less detail or more abstraction are simpler than solution with less abstraction or more details. But then… people not familiar with the abstraction may find more detailed solutions simpler.

This is also true for lines of code (LOC). In general solutions with less LOC might be deemed simpler. But then there is always a point where even less LOC make a solution harder to understand.

Sometimes that’s a matter of taste. But more often it seems to be a matter of experience with some technology or the problem domain.

So we’re back at simplicity is relative. But at least you now know what to look out for when trying to apply KISS. Be very clear about the scarcest resource. And once you have made up your mind try to minimize the burden on it.

It’s like a friendship kiss for a dear team member; an expression of sympathy, even empathy. Or if you don’t like to kiss men ;-) think of a manly man hug.



P.S. Just in case you’re still in doubt: In my view the chronically scarcest resource in software development is usage time. That includes the time users actually work with our software. But mainly I mean developers using code because they need to understand and modify it, plus developers using tools and applying processes.

It can hardly be overestimated how well invested an hour writing/setting up a solution is if that helps to make it simpler to use. This pertains to production code as well as screen designs and usability in general and infrastructure.

Simplicity focused on the scarcest resource is not a matter of beauty, but a matter of economics.

Posted On Wednesday, July 29, 2015 3:28 PM | Comments (3) | Filed Under [ Software design ]

Wednesday, July 22, 2015

There is no such thing as technical debt

Ward Cunningham coined the “technical debt” metaphor. It sounds plausible, it might resonate with management. But it’s far from reality.

Let’s think about financial debt for a moment. How do you get indebted?

  1. With a financial debt it’s crystal clear when you get into it. You go to a bank and ask for money. That’s a very clear cut moment in your life.
  2. The amount of money you owe somebody is crystal clear to you and the creditor.
  3. How much you pay for a credit is crystal clear to you and the creditor. If you want to get a loan of $10,000 you’ll have to pay back more than that depending on the interest rate and the time you’ll take for repayment.
  4. Financial debt comes with a very specific plan for how to pay it back, the payback schedule.
  5. The creditor is very keen on getting his money back including full interest payment.

Martin Fowler saw the main problem with technical debt in its “crippling interest payments”:

„The all too common problem is that development organizations let their debt get out of control and spend most of their future development effort paying crippling interest payments.“

But what if there was no debt at all? What if the metaphor is wrong?

Please check the above statements about financial debt against the reality of software projects:

  1. Do you know exactly when you get into “technical debt”? Do you go to some creditor and ask her for a time loan? (Because technical debt is about time. Time you want for doing B now instead of doing A, which would be the “right” thing to do.)
  2. Do you know exactly how much you’re indebted, how big the time loan is you get from some creditor? (Who is the creditor anyway? The customer, the architect, some manager?)
  3. Do you know how much you’re paying for the loan? Say, you got a credit of 2 days to do B now, instead of doing it later and doing A now. How much time will you spent in the future when you go back to A to do what you postponed? Will you just use 2 days or will it be 2.5 or 3 or 5 days then? You need to rebuild mental state, you need to deal with more evolved structures… So how much longer is it going to take?
  4. Do you know exactly how to pay back “technical debt” in the future? Do you know when you will start paying back, do you have a payback schedule?
  5. Does the creditor you got your time loan from have a keen interest in you paying back the debt? Does she come back to you and ask for the payback, like the customer asking for a refactoring you postponed two months ago?

Think about your answers to these questions for a moment…

I can tell you what I see in projects. I just see No for an answer. Each and every question has to be answered with No by those people usually talking about “technical debt”.

Nobody enters “technical debt” consciously, nobody knows its amount, nobody knows the costs for the loan, nobody has a payback schedule, and the creditor is not very keen to get a repayment.

“Technical debt” shares only one attribute with “financial debt”: some kind of interest - and that even being of unknown percentage. But that’s all.

That’s the reality of “technical debt”. And that’s why I say, there is no such thing as “technical debt”, because nobody treats “doing B now instead of doing ‘the right thing’ A” like a debt that has to be payed back.

It’s about addiction, stupid!

With the “technical debt” metaphor gone, what else could be more fitting?

I’m sorry, but to me there seems to be only one explanation left for what’s happening in software projects that constantly leads down the road to the brownfield. We have to face it. How projects are done is similar to how junkies live. We need to talk about addiction.

Here’s my simple layman’s definition of addiction:

A recurring behavior which favors a kick/quick win in the present over long term beneficial behavior.

And here are some attributes of addictive behavior:

  1. Addicts live in an illusion of control about their behavior. Before they engage in it they swear they drink just one glas or play only one Poker game. But then… there is no limit to the addictive behavior. The addictive behavior usually is associated with an inevitability and necessity that borders on a natural force: “I can’t help it, I have to do it…”
  2. Sooner or later addictive behavior leads to some kind of hangover. After the kick, which feels great, addicts suffer. They whine about massive headaches or money lost etc. - and this leads to very seriously meant promises to stop the addictive behavior. But of course those promises are never kept.
  3. Over time the detrimental effects of addictive behavior are not just acute hangovers anymore but serious health/social problems. Addictive behavior is not sustainable.
  4. Addicts strive for easy access to the addictive drug. They fear not having access to it. More and more of their time revolves around the addictive behavior/drug or preparing for it.
  5. Because so much focus is on obtaining or consuming the addictive drug, disposal of its remnants (e.g. bottles) or other garbage or keeping the household clean in general deteriorates. Addicts households are a mess.
  6. Addicts lie to conceal their addiction or its effects or to obtain more of the addictive drug.
  7. Addicts deny the state they are in and any cause-effect relationship between their decisions/behavior and their state.
  8. Any rational insight into the detrimental effects in the long run of the addictive behavior are not strong enough to stop it.
  9. Addicts refuse to accept help to free them from their addiction; they even assume others to have a problem, but not themselves.
  10. The addictive behavior often is not seen as such. The addiction can go undetected for a long time and look like normal behavior - including the hangovers. Because so many others are doing it, too. Think about smoking and drinking, both socially accepted addictions.

Now, what happens in software projects? At some point it’s clear A should be done to keep the development sustainable. But someone says, “No, no, we can’t do A now. We need to do B instead, because that’s what the customer/market needs most now.” A then A is postponed. Indefinitely. Instead B is done.

But is this addiction? Let’s compare software development with the attributes of addiction:

  1. Doesn’t software development lose control over when “the right thing in the long run” is postponed? Isn’t that done not only occasionally, but in a regular manner?
  2. Does software developer feel a hangover? Does postponing “the right thing in the long run” cause pain later on? Do developers seriously promise to not do it again?
  3. Does focusing on quick wins in the present cause long term damage? Does a codebase deteriorate because of a stream of postponements of “the right thing in the long run” up to a point where it hardly can be maintained at all?
  4. Does software project management make it easy to decide in favor of short term customer/market satisfaction instead of long term feasibility?
  5. Does a code base or a repository and a team room contain remnants of previous quick win activities which are of no use anymore today? Does garbage of all sorts pile up in code bases (e.g. misleading comments, dead code, code that no one understands anymore)?
  6. Does software development keep shortcuts and local/temporal optimizations hidden from customers? Does software project management hide how superficial quality is obtained by sacrificing sustainability?
  7. Does software project management deny the causal connection between the focus on short term results and long term problems?
  8. Does software development feel helpless to stop the urge to cater to customers’ short term expectations?
  9. Does software development refuse to accept help from literature/consultants or even inhouse sources to get back on a track towards sustainable development? Does software development accuse others of having no clue how things work and that the current behavior is inevitable?
  10. Do developers point to others in the industry who behave and suffer alike to explain how normal their own behavior is?

Think about your answers to these questions for a moment…

I can tell you what I see in projects. I just see Yes for an answer. Most projects need to answer Yes to the overwhelming majority of the questions.

Well, and that to me unmistakably means: The reason why software projects turn into brownfields, become unmaintainable, become hard to understand and change legacy code is addiction.

That is what the software industry has to face. To accept. To admit.

Without that no improvement is possible. It’s like with any other addiction.

Addictive drug

But what is the addictive drug, what is software development craving for? To speak of “technical addiction” would be wrong.

I think there are two sides to the addiction. Two needs seem to be the main drivers behind the addictive behavior:

  • Certainty: Software development strives for certainty. It wants to tame the beast of uncertainty called customer/market. It still believes that uncertainty can be controlled somehow; you just need to try harder. The means to control uncertainty are buffers of all kinds, e.g. a plugin framework to be ready for all sorts of changes, some validation logic because you never know what data enters the system, an API to be cater to possible desktop/web/mobile clients etc.
  • Connection: Software development longs for connection with the customer/market/upper management. It feels dependent on them and does everything to please them. It wants constant appreciation, a pad on the back. The flow of money must not stagnate. The means to keep the connection up is reactivity. Software development is geared towards immediate reaction to any request uttered by a money source. It gives up self-control/autonomy in order to receive constant appreciation. Software development fears to see a tear in the eye of a money source more than a vampire fears garlic.


“Technical debt” is a romantic metaphor. It feeds the illusion of control. But software development is out of control. It cannot control the self-inflicted deterioration of code bases. Because software development is an addict.

It suffers from “certainty and connection addiction”.

The signs of addiction are clear and here for everyone to see. We just have to overcome delusion and denial and have to admit them.

Then software development has a chance to overcome its addiction.

Posted On Wednesday, July 22, 2015 10:35 AM | Comments (42) | Filed Under [ Thinking outside of the box ]

Thursday, May 21, 2015

More than one order matters

Order matters. In real life when you’re in a library or a city or in your kitchen. And in software development. Order, the right order, helps understanding. Where there is order orientation is easier. Compare a city like Mannheim, Germany:


…to Algier’s traditional quarters, the Casbah:


Where, do you think, it would be easier for you to find the house of a friend?

Order is helpful, even important. But often order is hard to create and/or to maintain. Why’s that? What’s order anyway?

Order’s definition

If you look at Wikipedia it seems order is quite an elusive thing. There is no single page just defining the order I’m talking about and you very likely have in your mind.

Yes, order is about “an arrangement of items”. But it’s more than a “sequence”. The Free Dictionary defines it as:

“A condition of logical or comprehensible arrangement among the separate elements of a group.”


“A condition of methodical or prescribed arrangement among component parts such that proper functioning or appearance is achieved.”

That clears it up, doesn’t it? :-)

No, not really, I guess. So let’s dissect the definitions:

  • “arrangement”: Order is about “things” (elements, items, “component parts”) and their relations. Arranging “things” this way or that changes their relations with each other.
  • “proper functioning”: Order has a purpose. Elements in order serve the purpose better than elements in disorder. Thus order is the result of a conscious effort aiming at creating something “useful” through order.
  • “comprehensible”: Ordering items leaves them in a state which is easier to comprehend than the state they were in before. Reasoning about the items and their relations becomes easier. This requires some logical system consisting of rules and principles, though.

To sum it up I’d say “order” means:

A conscious arrangement of items according to a logical system to best serve a specific purpose.

This explains why Robert C. Martin says in his article “Does Organization Matter”:

“The organization structure that works for a thousand line program, does not work for a ten thousand line program. And the organization structure that works for a ten thousand line program does not work for a hundred thousand line program.”

He speaks of three different purposes which a single order cannot address. A 1K LOC program very likely serves a different purpose than a 10K LOC program or a 100K LOC program. That’s why they need different orders.

And in his article “Screaming Architecture” he argues that even two programs both with the same number of LOC should have different orders if their domain differs, because the order (structure) should mirror the purpose.

Order’s purpose

Since creating order is a conscious effort, it not only needs to enable some purpose (e.g. a motor pulling some weight or a software enabling online shopping), though. It also needs to serve its creators. Creators (and observers) need to understand it, otherwise it would be hard to create it in the first place or change it later on for adaption to a changed purpose (requirements).

An intelligible order is an order where cause and effect relationships can be determined. In software development this means questions like “If I change this here how does that affect the behavior of the software?” and “If I change this here how does that affect the evolvability of the software?” become easier to answer with more/better order.

Take these two arrangements of Mikado sticks as an example: For which arrangement do you find it easier to determine what happens when you move a stick?


To me, the purpose of order itself is to arrange items not just in any way to serve a “behavioral purpose”, but to arrange them in a way so that complexity is minimized.[1]

That’s why the “logical system” is so important behind any order. Because a “logical system” is an abstraction. It lifts a huge number of trees onto the level of a forrest. Causal relationships are easier to recognize if there are less details to take into account.

Multiple orders

By now it should be clear why order is important in general and in software development in particular. Without order, i.e. a conscious arrangement of code items following some “logical system”, a software’s behavior is pretty much accidental. And reasoning about it in order to fix a bug or add a feature might be near to impossible.

However, it should also be clear, a software system cannot be ordered in just one way.

Take a kitchen for example. I’m sure you appreciate an orderly kitchen with cupboards and drawers as containers for silverware and dishes.


The order is high - or to venture into the realm of physics: the entropy is low - because if you know the location of a single fork or plate it’s easy to reason about the location of other pieces of your household. There is a conceptual system behind how all this is arranged in a kitchen. And if not in your own kitchen then at least in a restaurant’s kitchen ;-)

But why arrange kitchen items in an order? Why not pile them up in a corner of the kitchen? That might damage them, it’s bad for hygiene - but above all it makes it harder to find an item when you need it. The specific purpose of the order in a kitchen is to enable efficient cooking and serving.

Now look at this picture:


That’s items taken from a kitchen - but in a different order. The same items can be arranged in different orders for different purposes. Here the purpose is to make eating easier.

And finally this:


Again items taken from a kitchen - but in yet another order.[2] What’s the purpose here? Maybe to ensure all items are cleaned best or maybe maximum load (with less cleanliness).

Anyway, as you see, the same items can be arranged in multiple orders to serve multiple purposes.

And I guess it also has become obvious that order is relative. Order is in the eye of the beholder. You need to at least understand the “logical system” behind an order to appreciate an arrangement. That’s why - if you’re like me - you should visit a gallery of modern art only with some kind of guide. Otherwise you might see chaos where there indeed is order ;-)

Multiple software orders

Now finally on to software. What kind of order does software need? And what is it that needs to be consciously arranged?

The last question is easy to answer. It’s logic and data that needs to be arranged. With logic being programming language statements producing behavior, i.e. transformations, control statements, and hardware access/API calls.

But to what purpose should these language constructs be put in an order?

I suggest to order code to fulfill some requirements. Creating order takes time, i.e. it costs money. So it better serves the source of money, the customer. That means it should be of some use to her. What’s of use to a customer is stated in a requirements document.

Requirements can be divided into two broad and obvious categories:

  • Functional requirements
  • Non-functional requirements

But that’s not the only way to slice requirements. Here’s another perspective:

  • Runtime or behavioral requirements
  • Design time or evolvability requirements

Both categorizations can be brought together in a triple:

  • Functional runtime requirements; example: an online shop should let you search products.
  • Non-functional runtime requirements; example: product search in an online shop should be so fast it shows results while you’re typing.
  • Non-functional design time requirements; example: the online shop should be (quickly) adaptable to all sorts of unforeseen requirements over the next three years.

Maybe you come up with even more requirement categories. But even with just these three at hand the question is: How do you create separate orders for each of the categories?


The order creating the most heated discussions is the one pertaining to non-functional design time requirements. When you do object-oriented programming you do it for this reason.

Classes are modules like libraries and components and functions. Modularization thus is the art of imposing an order on code to increase developer productivity and code evolvability.

Here’s a class design as an example for a module order from a recent article of mine:



The order most desired is a different one, though. It’s the order needed to deliver on non-functional runtime requirements (or efficiencies) like performance, scalability, security etc.

How should logic be arranged to make a system scalable or responsive? And how do you call the containers for code to pack-up logic in for that purpose? I call them hosts as opposed to modules.

Hosts run code on a thread or in a process or on a machine or in a network. It’s the same code as arranged in a module order - but now arranged differently. Like with silverware and plates. Two purposes, two orders.

Here is an actor diagram from the same recent article as an example of a host order:


Don’t be surprised to see classes in this diagram. Classes are used to manifest some hosts in code.


But wait, there is more! What about functional requirements? What’s the order for them? Interestingly there is hardly any talk about order to fulfill functional runtime requirements.

There are some diagrams to depict functional order, e.g. flow chart, Nassi-Shneiderman diagram, sequence diagram, activity diagram. But their use is comparatively rare and not very systematic.

But functionality needs order like performance or evolvability. That’s why I suggest to become more conscious about it and to build skills to design it. The tool for that to me is Flow Design. Yes, I believe functional order is best expressed by declarative flows, that is data flows.

Data flows describe functional behavior as a series of processing steps connected by messaging.

Finally here is an example data flow from the mentioned article as an example of functional order:


The functional units you see are encapsulated by modules and hosts.

Designing orders

The orders for functionality, efficiency, and evolvability are orthogonal to each other. You can change the module order without changing the functional order. You can change the efficiency order without changing the module order. At least in principle, at least to a certain extend.

Because the orders intersect. They intersect at the function level. Functions are the smallest modules and the smallest units of the functional order and the smallest containers to contain logic for some efficiency requirement to be distributed across threads and operating system processes etc.

  1. Since the first requirement to fulfill is the functional requirement, functional order is the first to establish. Start by defining behavioral processes. You need to give the input-output transformation an orderly structure. They will be running whenever a user triggers a software’s behavior.
  2. Once functional units have been arranged in data processing flows start to arrange them in a host order. Distribute them in a way so they fulfill efficiency requirements, e.g. by running them on their own threads or even devices.
  3. Finally arrange the functional units in a module order. Distribute them in a way to maximize evolvability - without sacrificing any of the host order requirements.

Although I listed the order design steps in this order and recommend to advance in this way through software design, sooner or later you’ll change the sequence. Depending on the size of the project you might want to start with a coarse grained host order, then continue with the functional order, and determine modules last.

Anyway, it’s up to you. Just don’t forget to give your code an order in all relevant dimensions. There is always more than one order to code. And all of them matter.

  1. My personal definition of complexity is simple: A system is complex if cause and effect relationships are not clear. Your TV set is not complex; it’s perfectly clear to you what happens when you press a button on its remote control. But an airplane is a complex system. Unless you’re a pilot it’s not clear what’s gonna happen when you press a button or turn a knob on the dashboard in a cockpit. On the other hand a system is only complicated if cause and effect relationships are clear or can be found with some reasonable effort. Now think about relationships with friends and your spouse: Are your social systems complicated or complex? ;-)

  2. Believe me, even if you sometimes might think there is no order to how items are place in your friend’s dishwasher, there is an order. Just try to add another item to an already fully loaded dishwasher and your friend will certainly tell you you’re doing it wrong ;-)

Posted On Thursday, May 21, 2015 3:42 PM | Comments (2) | Filed Under [ Software design ]

Tuesday, May 12, 2015

Actors in a IODA Architecture by Example

The IODA Architecture is supposed to make software design easier and improve code readability for about any problem domain. I’ve tried to show an example of that in my previous article. It was a larger scenario than the usual code kata, but still it was a problem to be solved by synchronous programming.

In this article I want to move beyond that and tackle a problem where multiple threads need to be employed. Here’s the challenge:

Write an application to work as an alarm clock. It displays the current time and the user can set a wakeup time. Once the wakeup time is reached the application sounds an alarm.

Requirements analysis

This scenario needs some user interface design. How should the user interact with the program’s body? How about this:


When the user switches the alarm on, the remaining time until the wakeup time will be displayed. Both current time and remaining time will constantly (every second) be updated.

The user can switch off the count down at any time. The remaining time will then disappear.

When the wakeup time is reached a WAV or MP3 file should be played by the application.

Trying to bridge the requirements-logic-gap I see there is just a single dialog to this application, and within the dialog there are two interactions. Let’s call them “switch on” and “switch off”.

However… behavior is not only triggered by a user putting the switch left or right. The application is supposed to also behave in some way on the tick of a clock. So there is another “invisible user” interacting with the application. It’s the system clock.

From this I derive a picture of the whole application.


The system-environment diagram shows very application specific behavioral aspects separated into their own modules. I don’t know yet how those modules will be working together, but it’s clear to me at least these modules should be present. The portals and the provider each encapsulate an API. And the domain is the home of the very alarm clock logic.

Although “module” sounds technical this to me is still analysis. Because these aspects can readily be found in the requirements. They are so tangible I could and should talk about them with a customer.

But please note: According to the IODA Architecture these behavioral modules are not related to each other in any way. There are no functional dependencies between them whatsoever.

Finally one last step in the requirements analysis phase: finding features. Interaction behavior usually is not monolithic. So what behavioral aspects can be found in each interaction?

  • System clock tick
    • Update display of current time
    • Update display of remaining time while alarm is switched on
    • Check if wakeup time has been reached
    • Sound alarm is wakeup time has been reached
    • Switch off checking for wakeup time if wakeup time has been reached
  • Switch on the alarm
    • Switch on checking for the wakeup time
  • Switch off the alarm
    • Switch off checking for the wakeup time

To me Agile Architecture means, such increments have a 1:1 correspondence to programming language structures. Dialogs become classes, interactions become functions, and features also become functions. So what the analysis phase has brought me is a first rough sketch of my code. It will consist of at least 4 classes (dialog portal, system clock portal, alarm bell provider, domain). And there will be at least 3+7=10 functions for all the interactions and features.

I guess, this is all I need to enter the next phase: designing the solution, i.e. describing the behavior visible on the outside by building it from smaller behaviors in a declarative way.

Solution design

My initial sketch of the data flows in the application looks like this:


It took only maybe 5 minutes to come up with it. I like to draw flow designs with pen and paper. It’s flexible and simple. No other tools needed.

But I’d understand if you have a hard time deciphering my handwriting ;-) So let me go through the parts step by step and redraw them in a legible way.

Let’s start with the most interesting interaction: the system clock tick.


For every tick of the system clock it is checked, if the wakeup time has been reached. Of course this check is only done while the alarm is switched on (see status).

Also, while the alarm is switched on, the remaining time until the wakeup time is calculated. Both current time and remaining time flow to the UI to be displayed.

In case of the wakeup time having been reached this is flagged to the UI and the alarm bell is rung.

After this interaction the rest is easy. Let’s do start and stop in one diagram:


Starting the alarm clock means registering the wakeup time and setting the status to “active”. Stopping it means setting the status to “inactive”.

Taking multithreading into account

That’s an easy design - but unfortunately it would hardly work if implemented just like this. The reason: there are multiple threads at work here. This would at least cause difficulties for displaying the current/remaining time. They flow into the UI from a non-UI thread. You can see this in the following diagram: a red data flow arrow is connected to blue box.


Interestingly the same happens in Check wakeup time: the functional unit accesses the wakeup time and the status from the red thread - but this state is blue, because it first got set by Start which gets its data from the UI.

Multiple threads reading and writing the same resource does not seem like a good idea. So what can we do about it?

To solve the problem of data flowing into the UI from a non-UI thread it’s easy to insert a synchronization point. This functional unit just takes data coming in on some thread and passes it on on another thread, the UI thread.


Technically this is no problem in .NET with a SynchronizationContext.

But how to protect access to the common state of Start and Check wakeup time reached and Stop from multiple threads? That could be done with a lock.

For a long time I’d have gone down that road - but not anymore ;-) I’ve become a fan of the actor model and its .NET implementation Akka.NET.

Before I introduce actors into the design, though, some class design is in order after the flow design.

Class design

So far the design contains only “functional units” (FU) as processing steps in a data flow. Those FUs will be implemented as functions. But which classes should these functions be put in?

This is easy, I’d say. Have a look:


UI, clock and alarm bell all get their own classes. I guess that’s a pretty obvious decision.

And Watchdog? That class encapsulates the state shared by the remaining functional units. Shared state to me is a “hard aspect”. It calls for being wrapped in a module of its own.

Actor design

From the threading point of view the class situation looks like this:


Two classes need their own threads: the UI runs on the UI thread, the clock on some system thread.

The Watchdog is accessed by multiple threads and needs to protect its inner state.

And the Alarmbell pretty much does not care. It receives data on the clock thread, but does not need to do its job on that thread. Depending on how the alarm bell gets implemented it can run on the clock thread - or maybe gets its own for even more decoupling.

Actors help to implement this without resorting to special synchronization mechanisms like SynchronizationContext or locks. In this case I just need to encapsulate each class in its own actor:


Between actors flow messages. They are received by the actors’ “inboxes” and worked upon when a previous message has been processed. All actors work on different threads. But each actor just on one (although that might be a different one for each message). That way there cannot be any conflicts when accessing state. Actors protect any state by sequential message processing.

Message design

Data flowing between two functional units in a Flow Design often does not need a special type. It’s clear what it is and what’s supposed to happen with it. A single processing step representing an activity produces data which is consumed by a single processing step representing an activity.


With actors that’s different. Actors are more like nouns than verbs. They represent entities rather than activities. That’s why they are called actors. Hence actors often receive many different messages to process - but all in a single “inbox”.

In order to find the right message handler it helps to send messages of very concrete types. That could be commands or events. Commands describe, what’s supposed to happen. Events describe what has happened.

Let me take a subset of the actor flow design as an example:


Each actor has a single entry point into which all messages flow.

The UI actor outputs/sends commands because it is the one relating the user intentions to the rest of the program.

The watchdog actor on the other hand outputs/sends events informing other actors about what happened.

Commands and events are “implicit” in the original flow design. They now have to be made explicit in order to have the actors communicate clearly with each other. See the GitHub repo with the implementation for a list of them.


I implemented the application in two phases:

  • First I did it without actors. There were only the classes and synchronization with the UI thread. Not even a lock around the watchdog state. This I did not to get it 100% right, but to see if my idea of an alarm clock flew at all. It was much more than a walking skeleton - but not really the fully fleshed application I wanted. You find the sources for this version here.
  • Only in a second step I wrapped actors around the classes. This proved very easy. I just took what was already working and put in another set of containers. Functions and classes are modules; modules are containers to produce evolvability and productivity. Actors on the other hand are host; that’s how I call containers to produce some efficiency, e.g. a non-functional requirement like performance, scalability, or responsiveness. The latter is required for this application. The UI must stay responsive to be able to start/stop the alarm clock at any time. You can find the sources for this version here.

The implementation did not pose any challenges, except one: Getting the alarm bell running on OSX. I’m working with C#/Mono on a Mac and all’s well including doing graphical user interfaces with WinForms. (WinForms GUI’s on a Mac don’t look that nice, but they serve the basic purpose.) The simplest way to play sound files in .NET/Mono is to use the System.Media.SoundPlayer class. But it simply would not work with a .WAV file on OSX. Bummer. So I had to switch to an out of process call to the OSX utility afplay. This led to an interface for the alarm bell - IAlarmbell - and two implementations: AlarmbellOSX and AlarmbellWin.

But that’s really a minor point. Most important is that switching from “regular” multithreading to actors worked like a charm.

The first actor I wrote was the clock. I call it an actor even though it’s not an Akka.NET actor. Rather it’s a simple class doing its work on its own thread: every second it publishes the current time via a C# event.


I needed it for the first phase and saw no reason to wrap a real actor around it in the second phase.

The other actors I wrapped around an existing class. For example the Watchdog class to check, if the wakeup time has been reached:


Each of the “bubbles” of the design becomes a function. Input flows into these “bubbles” as function parameters. But the output of Check flows out via C# events. I could have implemented it with continuations instead, but wiring up the initial classes in phase #1 was easier with events. Anyway… I hope you agree, this looks pretty straightforward and clean.

This class is free of any multithreading concerns. I can test it easily.

Enter the actor: The WatchdogActor takes a Watchdog instance and wraps it so it becomes thread-safe.


See how the actor itself does not contain domain logic. It just wires its “inbox” to the “input ports” of the domain class:

Receive<StartCommand> (cmd => dog.Start_watching_for (cmd.WakeupTime));

(Please see the Akka.NET documentation on how receiving and sending messages with actors works. Or even better: Join the Akka.NET Bootcamp to get your hands on this awesome framework.)

The same happens for the "output ports" of the domain class. The actor registers handlers for each event, wraps the event data into an event object, and sends it on...

dog.OnRemainingTime += remainingTime => {
var e = new RemainingTimeEvent{RemainingTime = remainingTime};
  onEvent.Tell(e, self);

The domain class does not know what’s done with its output downstream in a data flow. Neither does the actor know what’s done with the event objects it outputs. It’s a matter of the integration in Program.Main() to determine who’s going to listen to whom:


Actors are created through a factory, e.g. sys.ActorOf (dogActorProps, “WatchdogActor”). The factory gets what it needs to produce a Watchdog via a Props structure:

Props.Create<WatchdogActor> (dog, new[]{ "DlgActor", "AlarmbellActor" });
Props.Create<DlgActor> (dlg, dogActor)

The parameters of Create() can be the ones passed on to the constructor. In this case that’s an instance of the domain class, e.g. dog for Watchdog.

In addition I pass in a list of actors who are recipients of the actor’s output. I do that with a list of actor names. This list then feeds an ActorMultiRef which is used to broadcast messages to multiple actors:

var onEvent = new ActorMultiRef (Context, eventReceivers);
onEvent.Tell(e, self);

Akka.NET does not seem to support “multicast events” in any other way. Routers seem too heavyweight for that purpose. So I created my own small multicast actor reference. It looks up the concrete actors by name. That way I can “forward reference” actors not yet instantiated. This lets me integrate actors in a very easy way and as flexibly as ordinary objects.


The next diagram shows the class hierarchy of the solution:


As you can see there are two strata of integration and - again - only one for operations. Only the Operation-classes contain logic. Neither Integration- nor Data-classes contain any. There are no functional dependencies in this solution.

The actors solve an efficiency problem as hosts - but at the same time work as modules integrating domain behavior. This integration is very simple, since there is a 1:1 relationship between actors and the “workhorses” they “ride”. But still… The resulting class diagram is very, very simple. And the “workhorses” (operations) are easy to test.

My bottom line on this project is: Actors fit nicely with the IODA Architecture and Flow Design. And they make solving problems with multiple threads easier.

To me actors look like coarse grained modules. They are a different kind of classes so to speak. Their USP is the much more rigorous encapsulation of internals. They feel quite a lot like objects as Alan Kay envisioned them. They are somewhere in the middle between class and µService with their asynchronicity and message based, but platform dependent contracts.

I guess, I’ll use more of them in the future :-) Not for technology’s sake, but as a means to encapsulate logic even better than with classes - and at the same time improve on performance and scalability.

Posted On Tuesday, May 12, 2015 6:29 PM | Comments (5) | Filed Under [ Software design ]

Saturday, May 2, 2015

IODA Architecture by Example

The IODA Architecture to me is fundamental. But as many fundamental notions it might be a bit abstract. That's why I want to provide an example of how to design and implement software based on it. I keep things easy, but not trivial. So here is a scenario:

Build an application that translates a roman number entered by the user into an arabic number and vice versa.

This sounds easy, doesn't it. It's more than the code kata Roman Numerals requires, but not much.

Requirements analysis

Before I start with the design, a short analysis seems to be in order.

The requirements ask for an application. That means the user should be able to start a program which converts numbers. But how should the user interact with the program? Since the requirements don't tell me much about it I assume the simplest approach: a console application. It could work like this:

$ convertroman XIV
$ convertroman 42

Roman number conversion is no rocket science. I won't go into it here. Look it up on Wikipedia, if you like.

What about validation? How should the application react to negative arabic numbers or invalid characters in a roman number? The requirements are silent about this. So I assume just very, very basic validation.

The main purpose of this analysis is clarification of how data gets into the program and how output is presented to the user.

Solution design

The IODA Architecture does not prescribe any behavioral responsibilities. So what can I start with? I could dig into the requirements and identify candidate classes, if I was doing traditional object orientation. But I'm not.

Objects - at least the traditional ones - are secondary. What is primary is behavior. What's supposed to happen? That's the driving question.

At the beginning I know very little. I just know, "everything" needs to happen upon a trigger from the user.


But then, when I think about it, I do know a bit already. It's just a pattern:

  1. The input data has to be read. It's passed in via the command line.
  2. The data has to be converted.
  3. The conversion result has to be displayed.

Flow Design

The overall behavior is made up of "parts"; there are distinct processing steps. It can be refined. The current functional unit "convert roman" becomes the root of an integration hierarchy.


Now I repeat this: For every functional unit I ask myself, from what even smaller processing steps it could be made up of. If I truly understand the problem I'm sure I'll be able to name at least a few such sub-tasks.

  • How complicated is it to read the input number from the command line? It's trivial. No refinement is needed.
  • How much effort is it to display the result? It's trivial, too. No refinement needed.
  • How does the conversion work? That's not trivial. Refinement seems necessary.

Conversion has at least two sides to it: converting a roman number to its arabic equivalent, and converting an arabic number to its roman equivalent.

Also which conversion to choose needs to be decided.

And how about the simple validation?

Let's put these behavioral pieces together. Here's the "domain process" to do the conversion:


In the former design "convert" was a leaf, an operation. But now it has been opened up and become an integration.

Again I repeat asking myself for every functional unit how difficult it seems to implement it. If I clearly see an implementation before my mental eye, I won't refine it further during design. Otherwise I need to do a bit more thinking. How could the partial behavior possibly be accomplished as a "domain process"?

To me such a refinement seems to be warranted for both conversions. They are at the heart of the application. That's really the core domain of the program.

Here is the exemplary decomposition of "convert from roman": My approach to solve the problem consists of three processing steps.

  • Each roman digit of the roman number is translated into its value, e.g. "XVI" -> [10, 5, 1].
  • The values are summed to calculate the arabic number, e.g. [10, 5, 1] -> 16.
  • In case of a smaller value standing in front of a larger value the smaller value is negated, e.g. [10, 1, 5] -> [10, -1, 5]. I call this "applying the subtraction rule".

Correctly wired together the "domain process" consisting of these three steps looks like this:


If you, like try the same for the conversion of arabic to roman numbers ;-)

Should the new functional units be refined further? I don't think so. I feel confident to just write down the code. It's pretty obvious, isn't it?

So much for the happy day case. But what about a rainy day with invalid input? What to do about it? I think some error output would be nice. I'll extend the data flows from above like this:


It's never too late to improve a design ;-)

Let's step back: What is this I've designed so far? It's data flows; I call it Flow Design. It's a declarative description of what's supposed to happen to solve a problem (which always is some transformation of input into output and side effects).

At this point I don't have a clue how the actual logic in each of the operations will look like. That's the imperative part. It's implementation details.

But my feeling is ;-), if all operations are implemented correctly, then the integration of them in the depicted manner will lead to a correctly working program.

Here's the complete final Flow Design for the program with operations marked with dark color:


Data Design

So much for the I(ntegration) and O(peration) strata of the IODA Architecture. But what about the D as in Data?

Of course it's already there. Data is flowing between the operations. Here's a section of the Flow Design across levels of abstraction with focus on the data:


Due to the simplicity of the scenario the data types are all primitive. It's strings and integers. Maybe, yes, maybe it would be nice to introduce specific types for domain concepts like roman number and arabic number. But right now I don't see much value in that; it would feel kinda artificial/overengineered. Especially since I'm going to use C# as the implementation language. New types are comparatively expensive using that syntax; I'd be more inclined to define types like that if I was using F#.

Anyway, whether the scenario calls for its own data types or not, this does not change the basic architecture: operations depend/work on data. If there were any specific data structures they'd reside beneath the operation stratum.

Class Design

The "bubbles" you've seen so far will be translated to functions in a systematic way. Wait for the implementation...

Functions are modules. And I think each of the bubbles designed so far has a pretty narrow responsibility. It focuses on a single aspect of the requirements. See my article on the Single Responsibility Principle (SRP) for details.

But as important as it is to focus the responsibility of each function (aka bubble), there are more module levels available. I should use them to form higher level responsibilities.

Also, any functions that might be created from the bubbles designed need a class as its home.

Before moving on to the implementation I need some clarity about what classes the program should consist of.

Notice the difference between the usual OOD/P approach. I haven't thought about classes first, but about behavior/functions. Behavior is primary, classes are secondary.

So I did not have to come up with any "candidate classes". My classes are the real ones right away. And I know exactly which functions go where. Because I abstract classes from patterns I see in the Flow Design.

Here's my idea of how to group bubbles into classes:


Some of it might be obvious for you, some not. So let me explain my thinking:

  • Providers: Using an API is a "hard" aspect in any software. It should be isolated into specific modules. Usually I'd say each API should be encapsulated by its own class - but in this scenario there is so little of it, I guess a single class will do.
  • FromRomanConversion and ToRomanConversion: Doing the actual conversion is an aspect of its own. Since there are two directions of conversion a module for each seems to be in order.
  • RomanConversions: This class belongs to the coarse grained domain aspect of the application. It contains "helper functions" not actually concerned with the transformations.
  • Body: This class represents the overall functionality - but without the interaction with the user. It's like an internal API to what the program is supposed to do.1
  • Head: The head is responsible for triggering body behavior. It integrates the body with input from and output to the user.

Interestingly the class diagram follows the IODA Architecture like the bubbly Flow Design:


Only the data stratum is missing. But that's just because the scenario is so simple.

Interfaces and Static Classes

Class design is more than determining which classes there are. It's also about how the services of those classes should be accessible.

Question #1: Can any classes be static? I'd say that's a legitimate question; not all classes need to be instantiated. This only makes sense for classes with data or those which should be replaceable or into which replacements (mocks etc.) should be injected.

Looking at the class design for this application I think, RomanConversions, ToRomanConversion, and FromRomanConversion can be static. There is no state involved, no APIs need to be replaced for testing purposes.

This is different for Providers. It encapsulates APIs which might make it hard to test its users like Head. An interface seems in order or maybe even two: one for the CLI part and one for the console output part of the class. Why not apply the Interface Segregation Principle (ISP)?

From that follows Head cannot be made static. It needs to get instances of the provider interfaces injected, since it statically depends on the interfaces and dynamically on some implementation of them.

Also Body should be described by an interface. That would make it easy to test the head in isolation.

Library Design

Aspects are hierarchical so they should be mapped to a hierarchy of modules. Functional units of the Flow Design have been aggregated into classes. But what about the classes?

The next level of modules after classes is libraries. At least in my thinking.

Also libraries are a means of organizing code in many IDEs under the name "project" (eg. Visual Studio) or "module" (eg. IntelliJ).

Again I don't start with any library candidates, but rather let the classes inspire me. What should be grouped into a library? Here's my answer:


  • RomanConversions contains all classes of the business domain.
  • Providers is the home for all classes dealing with APIs.
  • Head is the top level integrator, the entry point of the application.
  • Body does the integration of the "backend". That's where behavior is created.

Component Design

The next level in the module hierarchy are components. A component consists of one or more libraries and is described by a separate platform specific contract. This enables a more "industrial" style of software production, since work on different components can be done in parallel, even if they depend on each other at runtime.

The current scenario does not really call for components. I'm working on it alone. But still... I'll make Providers and Body components, since there are interfaces and possibly several implementations over time. It's not much effort, so there will be another library for the contracts.


Implementing the design is easy. Almost boring ;-) There are clear rules how functional units of Flow Designs should be translated into functions. It took me some 90 minutes to write down the code including all tests.


Test-first coding wasn't necessary since all operation functions are so small. Also the design was already there so I did not need tests to drive it. The tests I set up are there to avoid regressions.

You can view the sources on GitHub.

When looking at the code there should not be any surprises. The project layout follows the component/library design.


The dependencies between the libraries/classes are as planned.


The code has a IODA architecture on all module levels from functions to components.2

One thing, however, is noteworthy, I guess. How did I translate the bifurcations in the Flow Design, i.e. where the flow splits in two? There are two outputs to "convert", "determine number type", and the validations.

When you look at the following code you might need a minute to understand what's happening:


But actually it's simple. Just read the function calls from top to bottom and left to right. Indentations denote different paths in the flow.

Here's the topmost flow side by side with its implementation:


Each bubble translates to a function call. If there is only one output, the function returns a value. If there is more than one output, the function passes on a value using a continuation (function pointer). Those continuations are what makes the code a bit hard to read. But only at first. It's unusual, not wrong. You just have been trained for so long to read nested function calls from right to left and inside-out; but once you get used to continuations you'll come to appreciate how easy it is again to read code.

Elements of Functional Programming are used to translate the declarative Flow Design into OO language code.

But that's only details. My main point was to show you, that designing a IODA Architecture for an application is not difficult. And implementing the design is not difficult either.

The universal aspects of integration, operation, and data (admittedly not much in this case) have been separated. And orthogonal to that behavioral aspects have been identified and encapsulated into modules on different levels.

  1. This root class could also be called Application or maybe viewed as a use case and named accordingly.

  2. There are slight exceptions. Although the classes FromRomanConversion and ToRomanConversion are operations from the point of view of the class design, they contain both integration functions and operation functions. As long as there is a large imbalance between the two aspects that’s ok. If the number of integration functions would grow in such a hybrid class, though, it would hint at splitting it into dedicated integration and operation classes.

Posted On Saturday, May 2, 2015 2:19 PM | Comments (2) | Filed Under [ Software architecture ]

Wednesday, April 29, 2015

The IODA Architecture

The common architectural patterns are all pretty much the same. Layered architecture, MVC, Hexagonal architecture, Onion Architecture, Clean architecture… they all do two things:

  • Define domains of responsibility
  • Put functional dependencies in order

Look at these depictions of some of the patterns:


What you find are shapes (rectangles, circles) with the responsibility to produce part of the required behavior of a software. All models forsee certain responsibilities like interaction with the user, persisting data, representing and working with domain data etc. Some patterns are more detailed, some are less. That way they are more or less applicable to concrete requirements.

And those shapes are explicitly or implicitly connected in service relationships. One shape uses another; one shape depends on another. Sometimes arrows denote these relationships, sometimes mere location (e.g. a shape on top of another means “top uses bottom”).

These uses-relationships are cleanly directed and there are no cycles.

This is all nice and well. It helped us along quite a while.

But again and again I’ve seen developers scratching their heads. What about stored procedures in a database? They implement business logic, but don’t reside in the Business Layer. What about JavaScript validation logic? It’s part of the Presentation Layer, but implements business logic. Or what if I don’t want to go down the DDD path with Domain Entities and Services? Maybe the application is too small for this.

Most of all, though, what boggles the mind of developers, as far as I can see, are the functional dependencies between all those responsibilities. It’s deep hierarchies which won’t go away by switching to IoC and DI.

Looking at a function on any level of this hierarchy is an exercise in archaeology. Put on your head lamp, get out the brush, be careful while you uncover mysterious code hieroglyphs of the past. There is so much to learn from those 5,000 lines of code… And there, look, another tunnel leading you to yet another level of code.

As useful as the architectural patterns are, they are only coarse grained and they leave us with a very crude idea of the fundamental structure of software. According to them, software is just a deep hierarchy of services with particular responsibilities calling other services calling other services etc. for help.

Thus the DI container has become the developers best friend. And his second best friend has become the mock framework.

This is bothering me.

Certainly every software system is made up of aspects which should be encapsulated in modules. There might be a UI aspect or a persistence aspect or a use case aspect or an encryption aspect or a reporting aspect etc. etc. They are all a matter of the Single Responsibility Principle, which I’ve already put under the microscope.

At this point I don’t want to argue with the established architectural patterns about the responsibilities they perceive as inevitable, be that Business Layer or Repository or Controller etc.

Let’s call them behavioral responsibilities. Because each responsibility is about a functional or non-functional aspect of the behavior the customer wants a software to show.

But what I question is, whether the fundamental notion of software being just a uniform hierarchy of services is useful. And service being some kind of behavioral responsibility to be used by another service.


Essentially there is no distinction. Look at the code in a function in a Presentation Layer or a Use Case or a Controller or a Respository. You won’t see a difference - except in behavioral purpose. All functions on all levels of the dependency tree contain logic and calls to other functions.

“That’s how it is. That’s the nature of software, isn’t it?”, you might say. But I beg to differ. This is just how we structure software today. It’s a mush. And that’s why there are functions with 500 or even 10,000 lines of code. There is no technical limit to how many LOC a function can have. And there is no binding rule about how many LOC it should have (except for some recommendations). It’s all up to us developers. So if push comes to shove a method just grows - one line at a time. Because it can.

This is what has been happening for the past 50 years. And it has been happening despite all well meant design patterns or architecture patterns.

Why’s that? My guess, it’s because all those patterns do not question the basic assumption of all levels of dependency being created equal.

Patterns so far have been concerned with defining certain behavioral responsibilities and nicely ordered dependencies between them. That’s all.

So I suggest, if we don’t like the current state of codebase affairs, then we should try something new. Here’s what:

Formal responsibilities

I’d like to propose to search for the fundamental architecture of software in another dimension. The current architectural patterns are just evolutionary steps in the same dimension. So let’s step out of it. Let’s turn 90° and start moving down another path.

It’s one thing to search for responsibilities in the behavioral domain. That’s necessary, but also it is limited. Like the Flatlanders baffled by the effects of a 3D object passing through their 2D space, we’re again and agin baffled by the temptation to name some class “manager” or “coordinator”. It doesn’t sound right.

Locked in our current view of software design the need for managercoordinatorcontroller classes seems like a symptom of bad object-orientation.

But what if we broaden our view, step out of our dimension or behavioral responsibilities?

Let me introduce three formal responsibilities. I call them formal, because they are not concerned with creating a certain behavior. These formal responsibilities are orthogonal to all behavioral responsibilities.


Operation I call the responsibility of any logic. (With logic being transformations/expressions, control statements, hardware access/API calls.)

Any module containing just logic, is an operation. What an operation does, whether it stores data in a file, calculates a price, parses a string, stuffs data into a view is of no concern.

It’s implicit in that operations only contain logic. However, since this is so important, let me state it also explicitly: Operations may not call any other operation. Operations don’t know of each other. There is no functional dependency between operations. Calling one of your functions is not part of the definition of logic (see above).

Operations just execute their own logic which works on data. They are I/O-processors: given some input they produce output (and possibly side effects).


Giving structure to data is a responsibility separate from operating on data. Operations of course work on data - but they are not data.

This might sound opposed to object-orientation, but it is not. Data structures may provide services to work with them. Those services just should be limited to maintaining structure and consistency of the data.

If something is data, then that’s what its functionality should be limited to. If, on the other hand, something has data, it’s free to operate on it in any way.


Since operations don’t know each other, there needs to be some other party to form a visible behavior from all those small operational behaviors. That’s what I call integration.

Integration is the responsibility to put pieces together. Nothing more, nothing less. That means, integration does not perform any logic. From this also follows, integration is not functionally dependent on operations or other integration.

Formal dependencies

The formal responsibilities (or aspects) are independent of any domain. They neither suggest nor deny there should exist adapters or controllers or use cases or business logic modules in a software. They are also independent of any requirements scenario. Software can (and should) be structured according to the formal aspects if it’s a game, a web application, a batch processor, a desktop application or a mobile application or some service running on a device.

The formal aspects are truly universal. They define fundamental formal responsibilities to be separated into modules of their own. Put operations into other modules than integration or data.

And interestingly what goes away if you do this are functional dependencies between the modules of an application.

If you decide to have some operation module for displaying data, and some for doing business calculations, and some for persisting data… then those modules won’t be calling each other.


They just float around independently, only maybe sharing some data.


As you can see, operations depend on data. Of course that’s necessary. Otherwise there would be no purpose for software. Logic needs to have raw material to process.

But data is not functionality. It’s, well, data. It’s structure.

Finally those operations floating around need to be wired-up to work in cooperation towards a common goal: some larger behavior of which each is just a part.

This is done by integration on possibly many levels.


Integration depends on operations and other integration. But also this dependency is not “bad”, because it’s no functional dependency.

As you can see, dependencies won’t go away. We can’t build software without them. But the functional dependencies of the behavioral architecture patterns are gone. There are no more services calling other services etc.

The dependencies here are just formal. Or empty, if you will. They don’t have to do with any functional or non-functional efficiency requirements.

Separating What from How

I call this universal separation of formal responsibilities IODA Architecture. There are several strata of Integration sitting on top of one stratum of Operations using Data - and APIs from outside the scope of the requirements to at least cause any tangible effects on hardware.


Or if you are more inclined towards “onions” here’s another one ;-)


Or this one, depending on how you want to look at it. You could say, the environment interacts with some integration, so they need to be on the surface. Or you could say, the environment is only accessible through APIs, so they need to be on the surface.


In any case, operations are about how behavior is created. Logic is imperative, it’s nitty gritty details.

Integration on the other hand defines what is supposed to happen. It does not contain logic, it thus is not imperative, but declarative. Like SQL is declarative. Integration assumes the stuff it integrates just to work. If that assumption is valid, then it promises to wire it up into something larger and also correctly functioning.

Also data is about the what. No logic in data except to enforce a certain structure.


I’ve long sought an image to symbolize the IODA architecture. And now I’ve found one which I like quite a bit: a Sierpinski triangle.


I think it fits because it’s a self-similar figure. It’s a triangle made up of triangles made up of triangles… It’s triangles all the way down :-) A fractal.

IODA is also self-similar: An operation on one level of abstraction can in fact be a IODA structure - which you only see when you zoom in.




Operations on a high level of abstraction are black boxes. They are leaves of the the behavioral tree - which works on data and uses APIs.

But if you think, an operation is too coarse grained, you may open it up at any time. Refine it. Zoom in. Then it’s decomposed into another “miniature” IODA hierarchy. The integration formerly integrating the operation with others then integrates the root integration of the operation.[1]


To escape from “dependency hell” it’s not enough to wave the IoC/DI wand. The problem is more fundamental. It’s the very assumption software should be built as deep hierarchies of functionally dependent modules that’s problematic.

Functional dependencies lead to ever growing methods. The SRP is too weak to keep developers from adding more lines to an already excessively long method.

This is different in a IODA architecture.[2] There simply is no way to write an operational method with more a dozen lines of code. Because any developer then will tend to extract some logic into a method to be called. But that would be a violation of the IODA rule of not calling any (non-API) functions from an operation. So if a method is extracted from an operation’s logic the operation has to be turned into an integration.

Integration methods are also short. Without any logic being allowed in an integration method it simply does not feel right to have more than a dozen lines of code in it. And it’s easy to extract an integration method if the number of lines grows.

IODA does not only get rid of functional dependencies - of which a palpable effect is, the need for mock frameworks drastically diminishes -, it also creates a force keeping module sizes small.

Software architecture is not a matter of whether a Presentation Layer may only call a Business Layer or Use Cases may call Entities or the other way around. There should not be any service dependencies between these or other kinds of behavioral responsibilities in the first place.

Which behavioral responsibilities there should be is only a function of the concrete requirements. But what is truly universal and unchanging in my view is the distinction between integration, operation, data - and whatever framework APIs deemed helpful to get the job done.


  1. The same is true for data, by the way. Data structures are black boxes to operations. But since they can contain logic (to enforce their structure and consistency), they might not just consist of operations but a whole IODA hierarchy.

  2. Which is not just a theoretical thought, but experience from several years of designing software based on IODA.

Posted On Wednesday, April 29, 2015 3:15 PM | Comments (10) | Filed Under [ Thinking outside of the box Software architecture ]

Monday, April 27, 2015

Sweet Aspects

When a module has a single responsibility that means it’s focused on producing the behavior of a single aspect. That’s how I described the Single Responsibility Principle (SRP) in my previous article.

Since the SRP is so important and at the same time a bit elusive, I thought, I try to illustrate this “traits and aspects thing” a bit more. Maybe you can even develop a taste for the SRP :-)

Look at this heap of sweets:


Let them represent logic in your code. You know, those 5,000 lines of code in a method, or those 200 methods in a class, or those 1000 classes in a library… It’s all the same. Unstructured stuff. High entropy. Disorder. Not clean. Yet at least ;-)

At some point it might have been a good thing to throw all this together. Maybe that way delivery was faster/easier? You know all those explanation how legacy code just happened over time. Should get into technical debt very consciously; have a plan how to pay it back. But in my experience that’s not how it works for teams. Like many private households (and companies) they stumble and falter and fall; or each time they they borrow it’s an exception and sure will never happen again. Promise! Until the next emergency which requires an exception.

5,000 line methods happen. They happen one line at a time. And each time there is a very good reason why to add the line instead of first refactor the whole heap of logic.

So that’s what there is at the end for 5,000 very good reasons: a huge bag of stuff. A brownfield.


But now is the time to bring order to the chaos ;-) Refactoring time. Let’s apply the SRP. It will make it easier to get an overview of what’s there, to make it easier to pick just the sweet you like most. Or in code: Order will make it easier to understand code, when you want to add new features or fix a bug. Also extending the code will be smoother. Less refactoring, less digging around, less unwanted side effects.

But what kind of order should there be? How to group the parts? Which traits to select to define sets with high cohesion between their elements?

I think, with regard to the depicted sweets people are divided into those who like licorice and those who don’t ;-) So “taste” could be a single trait to split the sweets into sets.

One set has the responsibility to make people happy who like licorice. The other one has the responsibility to make those happy who don’t like it. And then there is a third set with just one sweet which tastes like a gummibear, but with licorice.


Picking just a single trait splits a whole into as many parts as there are values the trait can assume. That might lead to a coarse grained grouping aka modularization as you can see above.

So now let’s pick two traits and see what happens, e.g. “taste” and “form”:


You see, this doubles the groups! There is one for (licorice, cube), (licorice, tube), (gummibear, bear), (gummibear, fruit), (gummibear, frog), (licoricegummibear, bat).

And again. Now with yet another trait: “taste”, “form”, “color”.


Wow! More than 20 groups of sweets. A fine grained modularization, I’d say.

The closer you look, the more traits you learn to distinguish, the smaller the groups/modules covering an aspect become. Yes, that what I’d call the selected traits: an aspect.

The combination of taste+form+color we could call the “gourmet aspect” ;-) It’s relevant for someone who relishes these kinds of sweets.

But someone who needs to pack them up (or is very hungry) might be more interested in the “size aspect” which groups according to the trait volume.


Or someone who wants do use sweets to lay out a picture with them might just look at the “color aspect”:


So as you can see, cohesion sometimes is tangible, but sometimes it’s not. For sweets color, form, size are “hard” traits. You don’t need experience with “using” the sweets in order to group them along these lines. In software it’s easily visible traits like APIs, data, logic that lead to “hard aspects”. You don’t need to understand the domain for modularization - but a scrutinizing view helps.

The trait taste is different though. You need to experience the sweets. Only then you can distinguish values and group the sweets. That’s comparable to parts of logic where just looking at it won’t help to find aspects. You have to really understand it’s parts. What’s the purpose of each line, of groups of statements? Only then you realize their “taste” and can separate them into focused modules.

The result: What formerly was just a heap of stuff - sweets or logic - now is a nicely structured, clean, modularized system of code. Each module having a clear single responsibility on one of several levels of abstraction.


I hope this excursion into an analogy helps you to get am even better grip on the SRP. It’s worth trying. Good modularization is at the heart of any clean coding effort.

Bon appétit! :-)

Posted On Monday, April 27, 2015 8:59 PM | Comments (0) | Filed Under [ Thinking outside of the box Software architecture ]

Saturday, April 25, 2015

The Single Responsibility Principle under the microscope

It’s equally important and mysterious: the Single Responsibility Principle (SRP). And even though its originator Robert C. Martin tried to explain it again, I feel there is something lacking. Some concreteness, some tangibility.

First the definition of the SRP in Martin’s own words:

“The Single Responsibility Principle (SRP) states that each software module should have one and only one reason to change.”

This is more informative than Wikipedia’s tautology:

“In object-oriented programming, the single responsibility principle states that every class should have responsibility over a single part of the functionality provided by the software, and that responsibility should be entirely encapsulated by the class.”

But still… Martin’s definition is not crystal clear. It leaves much room for interpretation during TDD refactoring or code reviews. Even he admits this:

“However it begs the question: What defines a reason to change?”

To improve on this, he developed the notion of a “responsibility owner” as I’d call it:

“When you write a software module, you want to make sure that when changes are requested, those changes can only originate from a single person, or rather, a single tightly coupled group of people representing a single narrowly defined business function.”

That’s a clever idea - but it’s only useful for pretty coarse grained responsibilities. Take an online shop for example. I assume we can agree on such a shop having a couple of distinct parts like catalog, shopping cart, recommendation engine, order processing, billing etc.


For each of those parts there probably is a person (stakeholder) owning it, i.e. having a clear vision of how this part of the shop should behave and look like. According to Robert C. Martin this would mean, I guess, each functionality should be put into a separate module.

Who would disagree?

But that’s not really a difficult decision. Yet.

Now, what about a small company ordering such an online shop. There is only a single person responsible for all the parts. Would that mean everything has to go into just one module? Certainly not. Because what Robert C. Martin probably means it not person, but role.

Even if there is only a single responsible person for so many different parts of the software, this person views the whole from different perspectives. Mentally she switches between different roles like accounting, sales, marketing, support, even customer.

Does that help defining what a “reason to change” is? The SRP could be rephrased like this:

Separate code into modules so that each module is owned by only a single business role.

This is not bad. But still it breaks down once you zoom in on such a module. How to apply the SRP within (!) the catalog module? The whole catalog is owned by the sales role. But since it does not seem realistic to implement the whole catalog in a single function or even class, there is no guidance from the owning role as to how to form smaller modules.

I’m sorry, but even this augmented definition of the SRP does not seem to help much. My guess is, we need to look at the matter a bit more systematically.


First let’s be clear about what responsibilities are: If a module has a single or even multiple responsibilities, then that means it contributes to the fulfillment of requirements. A module can be responsible for some functional requirement and/or some non-functional efficiency requirement (e.g. performance, scalability).

Contribution to the fulfillment of such requirements means, the logic in a module produces a part of the desired behavior. Because that’s what software is all about: behavior. Customers want software to behave in a certain way, e.g. show the products in a certain category when clicking on an image (functional requirement) or display a query result in less than 1 second (efficiency requirement).

Behavior is triggered by some input from a user. It consists of output to the user and/or side effects (e.g. changes to a database or printing a document).

imageBehavior is produced solely by logic, which to me are transformational statements (expressions), control statements (e.g. if, switch, for, repeat), and API calls/hardware access.

That’s it. That’s what customers want. Nothing more, nothing less. Which means, they don’t want modules. Not a single one.


To wrap logic up into modules does not contribute to the behavior of software. So why do it? Why should we wreck our brains about whether a module has already just a single responsibility or not?

We need modules for the non-functional requirements of evolvability and productivity. That’s their sole purpose. Modules don’t contribute to functionality or efficiency; they only make it easier to keep adding features to the code and fixing bugs.

Customers want that, too. Unfortunately they take it for granted. And we developers have a hard time to explain why keeping evolvability high is not that easy. Especially because there is no hard-and-fast metric for evolvability.[1]

So there are now orthogonal responsibilities: the responsibility with regard to functional and efficiency requirements - I call that logic responsibility -, and the responsibility with regard to evolvability.

Following the SRP thus means to identify some logic responsibility and wrap it up into a module to fulfill the evolvability responsibility.



From the above example it should be obvious there is not just a single level of responsibilities. Responsibilities can be coarse grained or fine grained.

A module representing the catalog functionality of an online shop has a pretty broad, but still single responsibility.

But this functionality consists of different sub-functionalities and efficiencies. I imagine there to be maybe a query engine and some price calculation and a cache and several other parts to play together in order to give users a “catalog experience”. And although they are all owned by one role they should not go into the same module, I’d say.

That means a hierarchy of requirements has to be mapped onto a hierarchy of modules. But should those be all of the same kind, e.g. classes? That would dilute the hierarchy. It would not be visible in the code base anymore, at least not that easily. There would be just a bunch of classes (for query engine, calculation etc.), of which maybe one would be conceptually on a higher level (for the overall catalog).

That would work, but I suggest we broaden our view of modules. Let’s think of modules forming a hierarchy of their own.

In his first version of the SRP Robert C. Martin spoke of classes. In the updated version he speaks of modules - but leaves it to the reader to figure out what modules are. I think that doesn’t make it easier to understand what the SRP means.

So here is my take on what modules are:

Modules are design time code containers to enable evolvability and productivity.

This might sound a bit lofty, but in fact it’s pretty simple. To make it more concrete here is my hierarchy of modules from fine grained to coarse grained:

  1. Functions
  2. Classes
  3. Libraries
  4. Components
  5. (µ)Services


A level n module can be said to physically contain level one or more n–1 modules. Of course each level has its own characteristics. It brings something special to the table: Functions abstract logic behind a name, classes encapsulate details about data and functions, libraries are true black boxes, components separate contract from implementation, and (µ)Services sport platform neutral contracts.

Now think of it: The sole purpose of all these containers is to make it easier for your to maintain your code. They are used to encapsulate details, they save time by letting you re-use them, they decouple through contracts…

None of these containers was invented to improve functionality or performance or scalability or security.[2]

Even µServices, the newest kind of module, are not a matter of scalability or security. They are supposed to improve evolvability at design time and runtime and make polyglot development easier.[3]

With such a physical hierarchy of modules it’s easier to map a hierarchy of required behavior to code. For example the catalog could become a (µ)Service. Then the query engine and the calculation could become components within this (µ)Service. And the even smaller behavioral parts could become libraries and classes and functions. All nested within each other like russian dolls.

With a physical hierarchy of modules this can be done. It helps to map the reality of hierarchical requirements. But of course even this comes to an end. There might be more behavioral levels in the requirements than levels of modules. So in the end modules of the same level need to be arranged in some kind of logical hierarchy.

Anyway, each module on each level is supposed to have a single responsibility. But if there are maybe 5 components in a software system with 3 libraries each with 10 classes with 10 functions each… that would amount to 1,500 modules. And I don’t think you can or want to find 1,500 persons (or roles) out there who own these modules.

Such a simple calculation makes it obvious: The notion of “responsibility equals owner” is only helpful for very coarse grained modules. Don’t neglect it, but don’t put all your responsibility eggs in this basket.


The field the SRP targets now has more structure, it has become more detailed. But my feeling is, we’re still close to square one. Robert C. Martin’s question has not been answered yet:

“What defines a reason to change?”

Which logic to put into this function and not into that one? Which functions to split among two classes instead of keeping them in just one? Which classes to group in a library? Why split a libary into three? etc. That’s the questions we have to ask during design/refactoring all the time.

Let me try to move closer to an answer by introducing the notion of an aspect. The term is not new to software development, but I want to use it in a more general way than AOP (Aspect Oriented Programming) does.

Every module has a purpose. In some way it contributes to the overall behavior of a software. The more it focuses on a certain aspect of the behavior, the better. Then it has a clear responsibility.

A module’s responsibility is to implement an aspect of requirements.

An aspect can be functional or non-functional. Aspects form a hierarchy, they can be on different levels of abstraction.

The above mentioned catalog, shopping cart etc. are aspects of the overall solution. They are easy to identify, but below that…? They are just examples. So what’s the definition of aspect? Mine is:

An aspect is a set of traits on a common level of abstraction which are likely to change together.

This might sound complicated, but in fact we deal with aspects every day. Think of a person. What are aspects of a person?

I’d say for example hairdo and clothes are aspects of a person. Each stands for a set of traits, and each trait can take on one of several values, e.g.

  • Hairdo: hair color (black, blond, red…), hair length (long, short, bald…), hair style (curly, lank, pony tail…) and more.
  • Clothes: color (black, brown, purple…), style (casual, festive, rugged…), fabric (denim, silk, linnen…) and more.


What binds traits together to form an aspect, what makes up their high cohesion is how they change. We view traits as cohesive when they are affected by the same causes which means they tend to change at the same time.

It’s more likely to change the hair length at the same time as the hair color and the hair style. This happens when you go to a hair dresser. The synchronicity is high, the frequency every couple of weeks. The cause might be you want to look nicer for a party or because bringing your hair in order every morning has become exhausting ;-)

It’s also more likely to change the color of your clothes at the same time as the style and the fabric. This happens when you dress yourself in the morning or dress up to go to the opera. The synchronicity is high, the frequency a couple of times per day. The cause might be you want to feel more comfortable or more appropriate for the occasion.

On the other hand it’s less likely to change your hair style and at the same time also change the color of your clothes. At least it’s that way for the people I know ;-) Which means: If there are circumstances which lead to changing hair style and clothes color more or less together, if there is a common cause for both, then… well, then it’s probably useful to view them as an aspect.[4]


This kind of abstraction we do every day. There are thousands of already agreed upon common aspects.

But when you are confronted with software requirements… then there are no aspects at first. It’s just a vast land of traits. It’s tons of details without abstractions. That’s why the SRP is at the core of our profession. We’re aspect hunters.

Here’s the analogy again:

First there is just a bunch of traits: hair color, clothes color, hair length, fabric etc.

Then we see those traits change their values: hair color goes from black to blond, hair length goes from long to short - but clothes color stays the same.

Then we realize there are patterns in those changes.

Then we abstract from the traits. We group them together according to the observed (or imagined) change patterns (which represent causes). Aspects are born.

In a myriad of real (or anticipated) changes to traits we need to spot patterns and then abstract. Aspect hunters are masters of abstraction. That’s what the SRP is about.[5]

Sure, ultimately all changes originate from people. That’s why Robert C. Martin says:

“[The SRP] is about people.”

But this is only helpful at the beginning. It’s a shortcut to abstraction. We don’t want to wait for changes to happen before we venture into abstraction. We want the abstractions to be served on a silver platter.

There is nothing wrong with shortcuts. But we should recognize them as such - an be prepared to go beyond them.

Hunting for aspects top-down

Spotting cohesion is no easy feat. It becomes easier with experience, though. Fortunately. But still, it’s tough. So we should look for help.

Robert C. Martin provides help when he recommends to look to stakeholders and their roles to find aspects that are worth to be represented by a module. Such roles as owners of a coarse grained aspect mostly help to structure the domain of a software into coarse grained modules.

Take the online shop for example again. There will be a person most interested in the presentation of the products, and a person responsible for the pricing, and a person responsible for order processing, and a person for billing etc. What should you make of them?

Since that’s pretty coarse grained aspects of the problem domain I find it natural to derive from this high level modules, e.g. (µ)Services or components. A catalog service or pricing component would make sense to me, but probably not a class to represent the whole aspect of price calculation.

This is an example of finding aspects top-down. You start with a business role as the root and drill down into to find requirements aspects.

When you look at patterns like the layered architecture or Ports-and-Adapters or Clean Architecture you find more examples of top-down aspect determination.


These patterns are congealed experience telling us: There are aspects common to all software regardless of the domain. Let’s call them cross-cutting aspects as opposed to domain aspects.

For example, interacting with the user is an aspect to be separated from persisting data or communicating with other software. These should be wrapped into modules. But of what size, on what level of the module hierarchy?

I think, such aspects should be encapsulated by at least a library; to view them as black boxes helps testing and exchanging them to keep up with technological development. These aspects form the interface of a software.

But there is more: Robert C. Martin zooms in on what’s hidden behind this interface. That’s use cases and entities to name just two aspects which are less tangible.

Whereas interface aspects can be identified pretty easily by looking at hardware or APIs, domain aspects are harder to spot. But one thing should be clear: Activities are not the same as data. That’s why use cases should be viewed as aspects of their own separted from the entities they work on.

And to represent those aspects classes seem to be the module level of choice.

How to find such “soft” aspects? Talk to the business roles, understand their domain. Certainly each role will give you ideas for lots of different such aspects. But that does not mean they all belong in just one module. Split them up!

And be open to the possibility that the role won’t be able to lead you to all of them. As your understanding of a domain evolves you will see patterns of change in traits - and will realize there are more and the existing aspects should be cut differently. That’s what Eric Evans calls “refactoring to deeper insight”.

Universal aspects

In addition to the interface and domain aspects I’d like to point out three more aspects which to me are orthogonal to the ones presented so far:

  • Integration
  • Operation
  • Data

I think they are so fundamental, they even come before the other aspects. They are the main shearing layers because they “evolve in different timescales”.[6]


Thinking about user interface or persistence or use cases already is pretty specific. It’s about behavioral aspects.

But behavior needs to be coordinated. And behavior needs to be fed with material to work on. To me this distinction is universal. It comes before any of the above mentioned architectural patterns.

Behavior is produced by logic. Logic is encapsulated by modules I call operations. Operations can do some business calculation, query a database, encrypt data, draw a graph or whatever.

Operation is different from data structures. Sorry, OO-folks ;-) Yes, I think for the past 25 years too much data has been entwined with too much logic. We need to start separating data from logic. Data is a very different aspect compared to behavior. Data is about structure, behavior is about process.

And then all the operations need to be “orchestrated”, put into the right order to produce an overall behavior. I call that integration. And it also is a distinct aspect different from data and operations.

Think about a company: There are documents (data) to be processed by clerks (operation) who are overseen by managers (integration). That’s even the case in software development. A product owner produces user stories (data) to be transformed into code (data) by developers (operation). And the whole process is overseen by a Scrum master (integration).

And it’s an anti-pattern if managers (or leaders) try to do the work of the employees they manage. Remember micro-management? ;-)

In code this happens if an integration module does not just “wire-up” other modules to form a process, but also contains logic to add itself to the behavior.

How to translate these universal aspects? Functions and classes are the starting point for me. Focus them on one of these aspects. Let the distinction between these aspects be the fundamental organizing principle for higher level modules/aspects.

Abstracting aspects bottom-up

So much for finding aspects top-down. But there are only so many you can identify this way. But we need to spot more, many more. Because in the end, each function we write should have just a single responsibility. Because a function is a module, the smalles module there is.

That’s where bottom-up aspect hunting starts.

It’s hard to give a definition of an aspect a priori, but it’s comparatively easy to see it as a pattern once there are traits on the table. It’s like the difference between an artist and a spectator. I find a difficult to create a good painting; whereas “good” just means “I like it” ;-) But it’s easy to recognize one when I see it.

So don’t start with the usual object oriented dance around higher level modules like classes - except for the obvious ones. Don’t play the CRC card game.

Instead start from the bottom up. Start with functions. Which means: Start with genuine behavior. All else will follow from there.

Pick some behavior requested by the customer and decompose it. What are the processing steps necessary to transform input into output plus side effects? Then repeat this for each processing step found - until the leafs of this decomposition tree become so small, their implementation is obvious to you.

imageThe processing steps you identify form flows, data flows. Integrating them into a whole is declarative programming, no logic involved. Only the leafs contain logic and only logic (operations). I call this approach “radical object-orientation” or “OOP as if you meant it”. The result of such behavior decomposition are “objects” as Alan Kay thought about them: logic encapsulated in a way so that it only communicates with the outside through messages.[7]

Anyway, what you get are many functions “without a home”. You are at the bottom of abstraction. Now build up from this higher level modules. Take each function as a trait. Which traits seem to be cohesive? Shared state or usage of the same API are signs of cohesion. Likewise focus on different requirements like functionality vs. efficiency. Or within efficiency performance vs. security vs. scalability. Or within functionality calculation vs. transformation or command vs. query. Or of course: integration vs. operation.

Slice and dice the functions (processing steps) in your decomposition hierarchy like you see fit according to observed patterns. First group them into classes. Then group classes into libraries. If you can nest classes in your language, use it to group classes within classes to form ever more coarse grained aspects. The same holds for functions.

But be careful! Within an aspect the traits should be on roughly the same level of abstraction. Abstracting aspect clothes from color and style is fine. But clothes color and collar form seem to be traits on different levels.

This is were experience is needed. Beyond hard and fast rules like suggested above there is a large area where the cohesion between traits and their level of abstraction need to be explored. Try to group them this way, then that way, and feel what makes more sense. Don’t try to avoid “refactoring to deeper insight” later on. Often you can’t be right the first time.

The one reason to change

So where are we after all of this? What about the “one and only one reason to change” which is the hallmark of a module’s responsibility?

You have to learn to spot traits in your code. Like read, write, command, query, calculate, store, data, behavior, encryption, integration, operation, caching, communication, persistence, catalog, billing, logging etc. etc. First see those traits, coarse grained and fine grained, functional as well as non-functional, domain specific vs. cross-cutting.

And then… compare them. Compare them with regard to what causes them to change. And if they have a common cause compare them with regard to how likely it is they change at the same time.

If two traits have a common cause for change and are likely to change at the same time then encapsulate them in a module on an appropriate level. You have defined an aspect and the module’s single responsibility is to implement it.

Learn to do this bottom-up. But don’t set recommendations for common aspects at naught. You find them in patterns and architectural models. Then you can hunt for aspects top-down.

  1. No, don’t even think of metrics like cyclomatic complexity or afferent coupling etc. They do not unambiguously measure evolvability. Also no customer orders evolvability in terms of such metrics. “I want functionality F, with performance P, and evolvability E.” Customers just silently assume the software they order to be easily changeable for an indefinite time. And if that’s not the case, if legacy code problems creep up and slow down development, they are always surprised. As Dag Sjøberg, a Norwegian professor in software engineering, suggests, the only metric to predict evolvability is the number of lines of code. So what customers could say is: For given functional and efficiency requirements keep the LOC at a minimum. But even that I haven’t heard of from customers. They are simply not really aware of the fundamental problem of keeping code bases evolvable.

  2. Ok, maybe with the exeption of functions. Functions where invented to save memory which might have an impact on performance, because more logic can be kept in RAM.

  3. Servers on the other hand, not (µ)Services, serve efficiency requirements like scalability or security. Servers are not modules. They are runtime containers to enable non-functional requirements and I call them hosts to distinguish them from modules. In software architecture they should not be confused with modules. They require independent design consideration.

  4. Hairdo and clothes are pretty obvious aspects, I guess. But if aspects are not hard facts, but abstractions resulting from observations and as such pretty subjective, couldn’t there be an aspect where hair style and clothes style are highly cohesive? Yes, of course. Just think of when hair style and clothes style pretty much change in sync due to a common cause. When is that the case? Say, a child becomes a teenager. Or an adult changes from hippie to conservative. Or a manager becomes a monk. How could this aspect be called when hair style and clothes style (and probably other traits as well) change in unison? How about “worldview”?

  5. If you think, aspects as abstractions are like classes, you’re right. Classes were introduced to be able to bind together what belongs together, to formally express cohesion. But classes are not the only means to define aspects. Aspects can be smaller than what should be encapsulated by a class, and they can be larger. That’s why we should understand modules to form a hierarchy like the one described above.

  6. Which is also true for many aspects, especially the coarse grained. Interface aspects evolve in a different timescale than domain aspects; the UI interface aspect evolves in a different timescale than the persistence aspect. etc.

  7. But I don’t want to delve into this right now. Let it suffice to say: This is not the functional decomposition from the last century. It’s a different kind of design approach, it’s flow design.

Posted On Saturday, April 25, 2015 5:47 PM | Comments (1) | Filed Under [ Thinking outside of the box Software design ]

Wednesday, December 31, 2014

In need of more abstraction

The ultimate product of software development is this: CPU executable binary code.


Decades ago we used to “write” this more or less directly into memory. But that was very tedious and error prone. Code was hard to reason about, hard to change.

Abstractions in code

So we looked for ways to make coding easier. Enter a higher level of abstraction: Assembler.


By representing machine code instructions as text and throwing in macros productivity increased. It was easier to read programs, easier to think them up in the first place, and they were quicker to write with less errors.

According to Jack W. Reeves Assembler source code was a design of the ultimate code which got built by an automatic transformation step.

Soon, though, software was deemed hard to write even with Assembler. So many details needed to be taken care of again and again, why not hide that gain behind some abstractions?

That was when 3GL languages were invented like Fortran or Algol, later C and Pascal etc. But I want to paint this evolution differently. Because from today’s point of view the next level of abstraction on top of Assembler is not a 3GL like Java or C#, but an intermediate language like Java Byte Code (JBC) or the .NET Intermediate Language (IL).[1]


Solving the problem of overwhelming details of concrete hardware machines was accomplished by putting a software machine on top of it, a virtual machine (VM) with its own machine code and Assembler language.

Where Assembler provided symbolic instructions and names and macros to abstract from bits and bytes, VMs for example provided easier memory management. Not having to deal with CPU registers or memory management anymore made programming a lot easier.

Now IL Assembler source code was a design for IL byte code, which was source code for machine code Assembler, which was source code for the ultimate machine code. Ok, not really, but in principle.

IL made things simpler - but not simple enough. Programming still left much to be desired in terms of readable source code and productivity. Partly the solution to that were libraries. But also another level of abstraction was needed. Enter 3GLs with all there control structures and syntactic sugar for memory management and sub-program access.


That’s where we are today. Source code written in Java or C# is the design for some IL Assembler, which is the design for IL VM byte code, which is the design for machine code Assembler, which is the design for the ultimate machine code. OK, not really, but in principle.[2]

Abstraction beyond code

We like to think of 3GL source code as the design for the executable machine code. As it turned out, though, yesterday’s design, yesterday’s source code became today’s target code.

Yesterday’s abstractions became today’s details. Nobody wants to reason about software on the level of abstraction of any Assembler language. That’s why Flow Charts and Nassi-Shneiderman diagrams were invented.

And what was pseudo-code once, is now a real programming language.

Taking this evolution as a whole into view it begs the question: What’s next?

There is a pattern so far. As many levels of abstractions as have been put onto each other there is one aspect that hasn’t changed. All those languages - Assembler, IL, 3GL - are all about control flow.

Mainstream reasoning about software hasn’t changed. Today as in the 1950s it’s about algorithms. It’s about putting together logic statements to create behavior.

So how can this be extended? What’s our current “pseudo-code” about to be turned into source code of some future IDE?

My impression is: It’s over.

Control flow thinking, the imperative style of programming is at its limit.

There won’t be another level of abstraction in the same vain. I mean language-wise. The number of frameworks to be glued together to form applications will increase. There will be more levels of abstractions.

But to actually design behavior, we will need to switch to another paradigm.

Accessing data has become hugely more productive by the introduction of declarative programming languages like SQL (and modern derivatives like Linq) or Regular Expressions.

So my guess is, we need to go more in that direction. Programming has to become more declarative. We have to stave off imperative details as long as possible.

Functional Programming (FP) seems to be hinting in that direction. Recursion is a declarative solution compared to loops. Also simple data flows as f |> g in F# have declarative power because they leave open whether control flows along with data. f could (in theory) still be active while g already works on some output from f.

Still, though, even with FP there is one question unanswered: How do you think about code?

Is there a way for us to express solutions without encoding them as heaps of texts right away? Is there a way to communicate solutions without and before actually programming them? Can we describe software behavior in a systematic way on a next level of abstraction - and then systematically translate this description into Groovy or Haskell?

Object-orientation (OO) has given us more ways to describe data structures than most developers know. Think of all the relationship types defined in UML.

But software firstly is not about data structures, it’s about functionality, about behavior, about activities. How can that be described, planned, designed above today’s source code, even FP source code?

Because if Assembler, the code design of the 1950s, nowadays is just the output of a compiler translating today’s 3GL source code design… then what kind of design can be translated into today’s source code as a target?

Model-Driven Software Development (MDSD) seems to be trying to answer this question. But despite all efforts it has not been widely adopted. My guess is, that’s because the design of a modelling language is even harder than the design of a decent framework. Not many developers can do that. Also, not many domains lend themselves to this. And it’s not worth the effort in many cases.

But still, MDSD has gotten something right, I guess. Because what I’ve seen of it so far mostly is about declarative languages.

So the question seems to be: What’s a general purpose way to describe software behavior in a declarative manner?

Only by answering this question we’ll be able to enter a next level of abstraction in programming - even if that currently only means to enable more systematic designs before 3GL code and without automatic translation.

We have done that before. That’s how we started with object-orientation or querying data. First there was a model, a way of thinking, the abstraction. Then, later, there was a tool to translate abstract descriptions (designs) into machine code.

The above images all show the same code.[3] The same solution on different levels of abstraction.

However, can you imagine the solution on yet another level of abstraction above the 3GL/C# source code?

That’s what I’m talking about. Programming should not begin with source code. It should begin with thinking. Thinking in terms of models, i.e. even more abstract descriptions of solutions than source code.

As long as we’re lacking a systematic way of designing behavior before 3GL source code - be it OO or FP - we’ll be suffering from limited productivity. Like programmers suffered from limited productivity in the 1950s or 1990s before the invention of Assembler, IL, 3GLs.

And what’s the next level of abstraction?

In my view it’s data flow orientation. We have to say goodbye to control flow and embrace data flows. Control flow will always have its place. But it’s for fleshing out details of behavior. The big picture of software behavior has to be painted in a declarative manner.

Switching from OO languages to FP languages won’t help, though. Both are limited by textual representation. They are great means to encode data flows. But they are cumbersome to think in. Nobody wants to design software in machine code or byte code. Nobody wants to even do it in Assembler. And why stop with 3GLs?

No, think visually. Think in two or three or even more dimensions.

And once we’ve designed a solution in that “space”, we can translate it into lesser textual abstractions - which then will look differently.

This solution


surely wasn’t translated from a design on a higher level of abstraction. How the problem “wrap long lines” is approached conceptually is not readily understandable. Even if there were automatic tests to be seen they would not explain the solution. Tests just check for certain behavior.

So, as an exercise, can you imagine a solution to the problem “Word Wrap Kata” on a higher level of abstraction? Can you depict how the expected behavior could be produced? In a declarative manner?

That’s what I mean. To that level of discussion about software we have to collectively rise.


PS: Ok, even though I did not want to elaborate on how I think designing with data flows can work – you find more information for example in my blog series on “OOP as if you meant it” –, I guess I should at least give you a glimpse of it.

So this is a flow design for the above word wrapping problem:



This shows in a declarative manner, how I envision a process for “producing” the desired behavior. The top level/root of the hierarchical flow represents the function in question. The lower level depicts the “production process” to transform a text:

  • First split the text into words,
  • then split words longer than the max line length up into “syllables” (slices).
  • Slices then are put together to form the new lines of the given max length.
  • Finally all those lines are combined into the new text.

This sounds like control flow – but that’s only due to the simplicity of the problem. With slight changes the flow design could be made async, though. Then control would not flow along with the data anymore.

The data flow tells a story of what needs to be done, not how it exactly should happen. Refinement of a flow design stops when each leaf node seems to be easy enough to be written down in imperative source code.

Here’s a translation of the flow design into C# source code:



You see, the design is retained. The solution idea is clearly visible in code. The purpose of Wrap() is truly single: it just integrates functions into a flow. The solution can be read from top to bottom like the above bullet point list. The code is “visually honest”.

Such a flow design can be easily drawn on a flipchart. With it I can communicate my idea of how to create a certain behavior quickly to my fellow team members. It’s easy to translate into code. And since it does not contain imperative logic, it leads to very clean code, too. Logic is confined to functionally independent leafs in the decomposition tree of the flow design. Read more about this in my blog series “The Incremental Architect’s Napkin”.

  1. I even remember P-Code already used in the 1970s as the target of Pascal compilers.

  2. Of course, this is not what happens with all 3GL languages. Some are interpreted, some are compiled to real machine code. Still, this is the level of abstraction we’ve reached in general.

  3. Well, to be honest, the first image is just some arbitrary binary code. I couldn’t figure out how to get it for the Assembler code in the second image.

Posted On Wednesday, December 31, 2014 12:39 PM | Comments (1) | Filed Under [ Thinking outside of the box Software modelling ]

Powered by: