The Architect´s Napkin

Software Architecture on the Back of a Napkin
posts - 69 , comments - 227 , trackbacks - 0

My Links

News

Archives

The IODA Architecture

The common architectural patterns are all pretty much the same. Layered architecture, MVC, Hexagonal architecture, Onion Architecture, Clean architecture… they all do two things:

  • Define domains of responsibility
  • Put functional dependencies in order

Look at these depictions of some of the patterns:

image

What you find are shapes (rectangles, circles) with the responsibility to produce part of the required behavior of a software. All models forsee certain responsibilities like interaction with the user, persisting data, representing and working with domain data etc. Some patterns are more detailed, some are less. That way they are more or less applicable to concrete requirements.

And those shapes are explicitly or implicitly connected in service relationships. One shape uses another; one shape depends on another. Sometimes arrows denote these relationships, sometimes mere location (e.g. a shape on top of another means “top uses bottom”).

These uses-relationships are cleanly directed and there are no cycles.

This is all nice and well. It helped us along quite a while.

But again and again I’ve seen developers scratching their heads. What about stored procedures in a database? They implement business logic, but don’t reside in the Business Layer. What about JavaScript validation logic? It’s part of the Presentation Layer, but implements business logic. Or what if I don’t want to go down the DDD path with Domain Entities and Services? Maybe the application is too small for this.

Most of all, though, what boggles the mind of developers, as far as I can see, are the functional dependencies between all those responsibilities. It’s deep hierarchies which won’t go away by switching to IoC and DI.

Looking at a function on any level of this hierarchy is an exercise in archaeology. Put on your head lamp, get out the brush, be careful while you uncover mysterious code hieroglyphs of the past. There is so much to learn from those 5,000 lines of code… And there, look, another tunnel leading you to yet another level of code.

As useful as the architectural patterns are, they are only coarse grained and they leave us with a very crude idea of the fundamental structure of software. According to them, software is just a deep hierarchy of services with particular responsibilities calling other services calling other services etc. for help.

Thus the DI container has become the developers best friend. And his second best friend has become the mock framework.

This is bothering me.

Certainly every software system is made up of aspects which should be encapsulated in modules. There might be a UI aspect or a persistence aspect or a use case aspect or an encryption aspect or a reporting aspect etc. etc. They are all a matter of the Single Responsibility Principle, which I’ve already put under the microscope.

At this point I don’t want to argue with the established architectural patterns about the responsibilities they perceive as inevitable, be that Business Layer or Repository or Controller etc.

Let’s call them behavioral responsibilities. Because each responsibility is about a functional or non-functional aspect of the behavior the customer wants a software to show.

But what I question is, whether the fundamental notion of software being just a uniform hierarchy of services is useful. And service being some kind of behavioral responsibility to be used by another service.

image

Essentially there is no distinction. Look at the code in a function in a Presentation Layer or a Use Case or a Controller or a Respository. You won’t see a difference - except in behavioral purpose. All functions on all levels of the dependency tree contain logic and calls to other functions.

“That’s how it is. That’s the nature of software, isn’t it?”, you might say. But I beg to differ. This is just how we structure software today. It’s a mush. And that’s why there are functions with 500 or even 10,000 lines of code. There is no technical limit to how many LOC a function can have. And there is no binding rule about how many LOC it should have (except for some recommendations). It’s all up to us developers. So if push comes to shove a method just grows - one line at a time. Because it can.

This is what has been happening for the past 50 years. And it has been happening despite all well meant design patterns or architecture patterns.

Why’s that? My guess, it’s because all those patterns do not question the basic assumption of all levels of dependency being created equal.

Patterns so far have been concerned with defining certain behavioral responsibilities and nicely ordered dependencies between them. That’s all.

So I suggest, if we don’t like the current state of codebase affairs, then we should try something new. Here’s what:

Formal responsibilities

I’d like to propose to search for the fundamental architecture of software in another dimension. The current architectural patterns are just evolutionary steps in the same dimension. So let’s step out of it. Let’s turn 90° and start moving down another path.

It’s one thing to search for responsibilities in the behavioral domain. That’s necessary, but also it is limited. Like the Flatlanders baffled by the effects of a 3D object passing through their 2D space, we’re again and agin baffled by the temptation to name some class “manager” or “coordinator”. It doesn’t sound right.

Locked in our current view of software design the need for managercoordinatorcontroller classes seems like a symptom of bad object-orientation.

But what if we broaden our view, step out of our dimension or behavioral responsibilities?

Let me introduce three formal responsibilities. I call them formal, because they are not concerned with creating a certain behavior. These formal responsibilities are orthogonal to all behavioral responsibilities.

Operation

Operation I call the responsibility of any logic. (With logic being transformations/expressions, control statements, hardware access/API calls.)

Any module containing just logic, is an operation. What an operation does, whether it stores data in a file, calculates a price, parses a string, stuffs data into a view is of no concern.

It’s implicit in that operations only contain logic. However, since this is so important, let me state it also explicitly: Operations may not call any other operation. Operations don’t know of each other. There is no functional dependency between operations. Calling one of your functions is not part of the definition of logic (see above).

Operations just execute their own logic which works on data. They are I/O-processors: given some input they produce output (and possibly side effects).

Data

Giving structure to data is a responsibility separate from operating on data. Operations of course work on data - but they are not data.

This might sound opposed to object-orientation, but it is not. Data structures may provide services to work with them. Those services just should be limited to maintaining structure and consistency of the data.

If something is data, then that’s what its functionality should be limited to. If, on the other hand, something has data, it’s free to operate on it in any way.

Integration

Since operations don’t know each other, there needs to be some other party to form a visible behavior from all those small operational behaviors. That’s what I call integration.

Integration is the responsibility to put pieces together. Nothing more, nothing less. That means, integration does not perform any logic. From this also follows, integration is not functionally dependent on operations or other integration.

Formal dependencies

The formal responsibilities (or aspects) are independent of any domain. They neither suggest nor deny there should exist adapters or controllers or use cases or business logic modules in a software. They are also independent of any requirements scenario. Software can (and should) be structured according to the formal aspects if it’s a game, a web application, a batch processor, a desktop application or a mobile application or some service running on a device.

The formal aspects are truly universal. They define fundamental formal responsibilities to be separated into modules of their own. Put operations into other modules than integration or data.

And interestingly what goes away if you do this are functional dependencies between the modules of an application.

If you decide to have some operation module for displaying data, and some for doing business calculations, and some for persisting data… then those modules won’t be calling each other.

image

They just float around independently, only maybe sharing some data.

image

As you can see, operations depend on data. Of course that’s necessary. Otherwise there would be no purpose for software. Logic needs to have raw material to process.

But data is not functionality. It’s, well, data. It’s structure.

Finally those operations floating around need to be wired-up to work in cooperation towards a common goal: some larger behavior of which each is just a part.

This is done by integration on possibly many levels.

image

Integration depends on operations and other integration. But also this dependency is not “bad”, because it’s no functional dependency.

As you can see, dependencies won’t go away. We can’t build software without them. But the functional dependencies of the behavioral architecture patterns are gone. There are no more services calling other services etc.

The dependencies here are just formal. Or empty, if you will. They don’t have to do with any functional or non-functional efficiency requirements.

Separating What from How

I call this universal separation of formal responsibilities IODA Architecture. There are several strata of Integration sitting on top of one stratum of Operations using Data - and APIs from outside the scope of the requirements to at least cause any tangible effects on hardware.

image

Or if you are more inclined towards “onions” here’s another one ;-)

image

Or this one, depending on how you want to look at it. You could say, the environment interacts with some integration, so they need to be on the surface. Or you could say, the environment is only accessible through APIs, so they need to be on the surface.

image

In any case, operations are about how behavior is created. Logic is imperative, it’s nitty gritty details.

Integration on the other hand defines what is supposed to happen. It does not contain logic, it thus is not imperative, but declarative. Like SQL is declarative. Integration assumes the stuff it integrates just to work. If that assumption is valid, then it promises to wire it up into something larger and also correctly functioning.

Also data is about the what. No logic in data except to enforce a certain structure.

Self-similarity

I’ve long sought an image to symbolize the IODA architecture. And now I’ve found one which I like quite a bit: a Sierpinski triangle.

image

I think it fits because it’s a self-similar figure. It’s a triangle made up of triangles made up of triangles… It’s triangles all the way down :-) A fractal.

IODA is also self-similar: An operation on one level of abstraction can in fact be a IODA structure - which you only see when you zoom in.

image

 

image

Operations on a high level of abstraction are black boxes. They are leaves of the the behavioral tree - which works on data and uses APIs.

But if you think, an operation is too coarse grained, you may open it up at any time. Refine it. Zoom in. Then it’s decomposed into another “miniature” IODA hierarchy. The integration formerly integrating the operation with others then integrates the root integration of the operation.[1]

Summary

To escape from “dependency hell” it’s not enough to wave the IoC/DI wand. The problem is more fundamental. It’s the very assumption software should be built as deep hierarchies of functionally dependent modules that’s problematic.

Functional dependencies lead to ever growing methods. The SRP is too weak to keep developers from adding more lines to an already excessively long method.

This is different in a IODA architecture.[2] There simply is no way to write an operational method with more a dozen lines of code. Because any developer then will tend to extract some logic into a method to be called. But that would be a violation of the IODA rule of not calling any (non-API) functions from an operation. So if a method is extracted from an operation’s logic the operation has to be turned into an integration.

Integration methods are also short. Without any logic being allowed in an integration method it simply does not feel right to have more than a dozen lines of code in it. And it’s easy to extract an integration method if the number of lines grows.

IODA does not only get rid of functional dependencies - of which a palpable effect is, the need for mock frameworks drastically diminishes -, it also creates a force keeping module sizes small.

Software architecture is not a matter of whether a Presentation Layer may only call a Business Layer or Use Cases may call Entities or the other way around. There should not be any service dependencies between these or other kinds of behavioral responsibilities in the first place.

Which behavioral responsibilities there should be is only a function of the concrete requirements. But what is truly universal and unchanging in my view is the distinction between integration, operation, data - and whatever framework APIs deemed helpful to get the job done.

image


  1. The same is true for data, by the way. Data structures are black boxes to operations. But since they can contain logic (to enforce their structure and consistency), they might not just consist of operations but a whole IODA hierarchy.

  2. Which is not just a theoretical thought, but experience from several years of designing software based on IODA.

Print | posted on Wednesday, April 29, 2015 3:15 PM | Filed Under [ Thinking outside of the box Software architecture ]

Feedback

Gravatar

# re: The IODA Architecture

Thank you for yet another interesting article. (Yeah I kind of discovered your blog, and I see many interesting articles and links.)

I'm however confused about the self-similarity you declared, stating an operation can in itself be a complete new ioda structure itself. This seems contradictory to me.
So in your view, how can an operation contain sub-integrations and sub-operations without violating the constraints of what an operation is?
5/1/2015 4:00 PM | Johan Samyn
Gravatar

# re: The IODA Architecture

I have another question about the ioda architecture:

Currently I'm working in a procedural environment. But I have a strong feeling that the ioda architecture could as well be applied in such an environment as it would in an oop environment. Where one could consider the development env as orthogonal to ioda. It even reminds me somewhat of the 'structured programming' techniques from the nineties; specially the decomposition aspect of ioda. What is your view on this?
5/1/2015 4:02 PM | Johan Samyn
Gravatar

# re: The IODA Architecture

And I also have a humble request about the ioda architecture:

I would really like to see a simple/small example application illustrating all the relevant concepts of this architecture (certainly the one in my first question). (A request I realize may be rather/too time consuming. But perhaps you have some open source illustrating it?)
5/1/2015 4:06 PM | Johan Samyn
Gravatar

# re: The IODA Architecture

I'm working on an example.

As for "structured programming": IODA is about decomposition. But not the decomposition of the 1980s. This will become clear if you read this: https://leanpub.com/messaging_as_a_programming_model. But let me just state the two main reasons here:
* Functional decomposition does not limit what can be done on each level of decomposition, but IODA does. There must not be any logic in an integration (a non-lead node of the decomposition tree).
* Functional decomposition does not care about data. Data is assumed to be global. But IODA cares about data. It flows. And if not it's contained in functional units (i.e. it's pretty local).

As for self-similarity: You have to view a design on different levels of abstraction. Like a map. A 1:1000000 map shows some atomic details like a forest areas; it's just a green patch without any structure. But on a 1:25000 the forest has details, has structure, is not atomic anymore.

The point is: If you decide to refine an operation then you need to apply IODA again.
5/1/2015 5:22 PM | Ralf Westphal
Gravatar

# re: The IODA Architecture

Yeah, I kind of overlooked the abstraction levels when having the self-similarity confusion. Maybe I was applying the 'logic only in operations' all too strictly. I take it the example should help make it all fully clear.

I admit it were the images that made me think of structured programming. You are right, I understand they are really not the same.

Looking forward to the example :) .
Btw what's the best way to be notified when it's ready?
5/2/2015 6:39 PM | Johan Samyn
Gravatar

# re: The IODA Architecture

The sample is online already. See my latest blog post.

To be notified about new articles follow me on twitter: @ralf (de + en) or @ralfen (en only).
5/2/2015 7:39 PM | Ralf Westphal
Gravatar

# re: The IODA Architecture

Very interesting. There is another methodology, Object-Process Methodology, that was also a result of analyzing current architecture, but the result of that was that "Process" was the thing that was incorrectly abandoned by the OO frenzy. I'm wondering how that compares to your identification of "Operation" as a crucial architectural concept. A quick intuitive idea is that Process are themselves composed of Operations. That to me seems more semantically viable.
5/27/2015 1:26 PM | Josef
Gravatar

# re: The IODA Architecture

Operation is more specific than OPM's process. Because it not only processes input, but does that in a certain way: by just employing logic. I don't see that kind of constraint in OPM.

So, yes, maybe you could say, process is a term encompassing operation as well as integration.
5/27/2015 10:41 PM | Ralf Westphal
Gravatar

# re: The IODA Architecture

I like the "dependency-free" thinking that drives IODA, but I'm stuck working through how I would manage complex types (like orders in an enterprise domain). There are types that represent data, types exposed/used by operations, and (possibly) types created by integrations. Minimizing types is a natural thing for app designers to do, but it can introduce unwanted dependencies if "big" types are shared across all layers. At the other end of the spectrum, creating a type for every usage duplicates effort and makes evolution of the solution more difficult.

I was wondering if you had any thoughts along those lines? I'm just thinking out loud, but could there be some rationale for categorizing or managing types that are relavent all the way from the API thought the integration head? Maybe that's not feasible or desirable, so the opposite approach might be to avoid sharing types between operations or integrations altogether and instead rely on late (dynamic) binding at runtime. The latter is what I'm leaning toward. Hope this make sense!

-Erik
5/30/2015 12:23 AM | Erik Johnson
Gravatar

# re: The IODA Architecture

I presume you mean data structures since your example is an order.

IODA is oblivious to how big or small you design your data structures. There can be many or few. IODA only prescribes data structures to be free of dependencies on hardware access and to contain only very focused logic.

Functions on data structures should focus on enforcing structure and consistency.

Data structures are containers which "are" data. In addition there are containers which "have" data.

Integrations might create data structures for convenience' sake. But it's operations who are really working with data. So the question is: Which operations should depend on certain data? So when you say "relevant all the way up to the integration root" that's a misunderstanding. Integrations just wire-up operations. There's not even a hierarchy of integrations necessary. Just one would be required - at least technically to wire-up all operations.

But the question is perfectly valid: If there is an operation reading data from a database and producing some data structure as its result, which other operations should this data structure be shared with? Should a domain logic operation work on it? Should a presentation operation use it, too?

It's a matter of coupling. But IODA does not answer this question. I'm sorry.

I'm struggling with this question, too. To me this means I still haven't uncovered some "principle" or "rule" or heuristic.

So for the moment we need to be conscious about this. And whenever something hurts we need to look closely. Maybe there is a pattern to be uncovered.

But in the end some trial and error will remain. That's because problem domains need to be explored. We are doing software development (!). That's basically doing research. And research requires trial and error/experiments and changes course all the time due to newly won insights.

That's why Eric Evans coined the phrase "refactoring to deeper insight".

You might start out with tight coupling and need to loosen it. Or you might start out with loose coupling and need to tighten it.

It's the same with IODA. Just because IODA defines a certain basic structure for software does not mean it's obvious which operations and integrations there should be.

It's a matter of the design process to play around with alternatives. And then it's a matter of reviews to evaluate how the implemented alternative is doing.

Hope this was of some help to you.
5/30/2015 7:15 AM | Ralf Westphal
Post A Comment
Title:
Name:
Email:
Comment:
Verification:
 

Powered by: