Geeks With Blogs

News


Dylan Smith ALM / Architecture / TFS

The past few weeks I’ve been helping a client come up with an Enterprise Architecture and I realized that I seem to have zero’d in on an EA that I would probably use at most places.

First off what do I mean by Enterprise Architecture?  I know lots of people use this to mean different things, for this post I’m using the term Enterprise Architecture to describe how the various applications and systems in an Enterprise will interconnect and integrate with each other (where necessary). Effective Enterprise Architecture should enable powerful integration scenarios and application re-use, while encouraging loose coupling to minimize the cost of change and impact of change on other systems.

The EA described below relies on Web Services and a SOA approach, while also leveraging the PubSub (Publish/Subscribe) pattern common in Message-Based Architecture.

While this Enterprise Architecture describes the external interfaces each system exposes, and how data flows between them, it does not describe the inner workings of each specific System (that’s the Application Architecture). Having a well-thought out Enterprise Architecture enables flexibility in choosing Application Architectures. Within each system you can choose to use a different Application Architecture, or even change a System’s Application Architecture in the future with minimal impact on other systems.

 

Integration at the Database

Most teams I encounter do integration exclusively by reading/writing directly from each Systems’ databases.  Although the majority of software teams out there probably do integration by database, in my experience the majority of software teams also deeply regret this decision.  Integrating at the DB level tightly couples Applications to the database schema design, making it risky to ever change that design.  It also limits reuse of application logic limiting you to the reuse of data only.

Service Oriented Architecture

Enterprise Architecture consists of breaking down the software ecosystem into independent Systems, and defining a well-known way for those Systems to integrate and/or exchange data. A common approach to manage this integration is to take a SOA approach, and wrap each system in a (Web) Service with a well-defined Service Contract. This moves the inter-system dependencies to the Service Layer rather than the Database layer. At first glance this would seem to simply move the coupling from the DB to the Service Contract, making it risky to ever change the Service Contract. However, it does enable re-use of application logic in addition to data. But more importantly there are well-known techniques to evolve Service Contracts while maintaining compatibility.

The most common approach to Service evolution is to expose separate End-Points for each version of the Service Contract. This way if you wish to modify the Service Contract, you publish a new End-Point with the new Service Contract while leaving the End-Point with the previous Service Contract active. Then implement a compatibility layer that translates service calls from the old Service Contract to the new Service Contract. This way the core System logic needs only support the most recent version of the Service Contract. It also provides a convenient place to introduce logging to understand which Systems are still depending on the old Service Contracts and plan the upgrade work required to move them to the new Service Contract, enabling the eventual retirement of older Service Contracts/End-Points.

PubSub Pattern

The introduction of an SOA approach, would be a marked improvement over integration at the DB level, however there are still challenges with a Service only approach to integration:

  • Availability - Service-level integration introduces availability concerns. Imagine a scenario where System A depends on Systems B, C, and D via their Service Contract. System A either needs to retrieve data from B/C/D to perform some work, or some operation in System A needs to request B/C/D to perform an operation as part of the System A operation (or often both of the above). If any of the B/C/D systems become un-available it will also impact the availability of System A, as now any System A operations that interact with B/C/D will also fail. System A’s availability is now tied to B, C, and D’s availability.

  • Coupling - Let’s imagine that we’re System A developers, and when some important operation in System A occurs, Systems B, C and D need to be notified so they can perform some related action. There are really two options, either B, C and D can poll the System A’s Service constantly querying to see when the relevant data in A has changed. This will have significant performance impacts on System A. The alternative is for System A to explicitly call some Service method in B/C/D when the relevant operation in System A happens. This not only incurs the availability concerns noted above, but what happens when another team develops System E that also wishes to be notified? Does the System E development team now need to ask System A’s development team to make changes to System A in order for System E to work? This is not a good situation to be in.

My approach is that in addition to wrapping every System with a Web Service, we also borrow from Message-Based Architecture and use a PubSub (Publish/Subscribe) pattern. In this pattern each System would publish “Domain Events” when things of interest occur within the System (ideally a System would publish an event every time any data owned by that System changes). Using one of the readily available messaging frameworks, this makes it easy for any System to subscribe to Events from any other System. Whenever anything of note happens in System A it simply publishes an event with the related data, and any other interested systems can subscribe to that Event and react accordingly. This way System A (from the above example) has no knowledge or dependency on the other Systems that may subscribe to its events (System E developers can create their System without having to ask System A developers to make any changes). If System A goes down it will not affect Systems B/C/D, and if System B/C/D go down it will not affect System A.

The other scenario is if System A requires some data from Systems B/C/D to perform an operation in System A. Rather than calling Service methods on B/C/D when that data is required, instead System A can subscribe to the relevant events in System B/C/D when that data changes and System A can maintain its own data cache of data “owned” by B/C/D updating it when the relevant B/C/D events are received. This way the System A operation can complete, even if B/C/D are all unavailable at the time. The important principle to keep in mind with this approach is that a given piece of data can only be “owned” by a single system. System A is free to cache data from B/C/D, but if System A wishes to change any data owned by B/C/D it must “ask” those systems to change it via the B/C/D Web Service. We also need to ensure that the messaging infrastructure we put in place has guaranteed delivery; meaning if System A happens to be down, when it comes back online it will still receive any Events that occurred while it was offline (modern messaging frameworks mostly handle this for us).

Each System under this Enterprise Architecture should look like the following:

Custom System

In this case the “Core Domain” is the actual implementation of that system (we’re assuming in the above diagram that there is some Domain DB contained within it, but that’s not necessary). The actual Application Architecture contained within the Core Domain is irrelevant to the Enterprise Architecture. The “Core Domain” in this case may even be a Commercial Software package such as AX or SAP.

In the case of a Commercial Software system, it often won’t support the Enterprise Architecture proposed here. In that case we need to wrap it with the appropriate integration layer to support the Enterprise Architecture. Consider the below example of a Dynamics AX System:

AX System

In the case of AX I believe it already exposes a Web Service, however, in the case that it didn’t and AX Integration was performed some other way (e.g. DB integration, copying files in a specific format to a specific directory, etc) we would create a custom Web Service that exposed those integration methods over a Service Contract (we don’t want any non-AX Systems talking directly to the AX Database except AX itself, and any Integration Layer we would write). Likewise, AX doesn’t publish Domain Events (and even if it did it wouldn’t do so using the Messaging framework we chose), again we can write some plumbing code to add support for this. In the above diagram we are writing a custom component “AX Event Generator” that would poll the AX Database looking for interesting changes in data and would raise the appropriate Domain Events that other Systems could subscribe to (some COTS Systems may have some notification system or way to “hook” system events eliminating the need to poll the DB). If we wanted AX to respond to Domain Events from other Systems, we would write a simple Event Consumer component that subscribed to Domain Events from other systems and executed the appropriate action in AX.

Using this approach, for other Systems to integrate with AX they no longer need to understand AX Database schema, they don’t need to understand any unusual integration mechanism that AX may use, they only need to understand the Web Service Contract and the Domain Events raised by the AX System (just like every other System).

 

Typically System-To-System Communication is done primarily via Domain Events. Clients (i.e. UI’s) primarily communicate via the Web Service(s).

Posted on Sunday, May 19, 2013 7:40 AM | Back to top


Comments on this post: Enterprise Architecture – SOA with a Dash of PubSub

No comments posted yet.
Your comment:
 (will show your gravatar)


Copyright © Dylan Smith | Powered by: GeeksWithBlogs.net