|
|
Tuesday, December 14, 2010
The SOA consultants invaded the executive suite at your company or agency, preached the true religion, and converted the unbelievers. Now by divine imperative you must convert your legacy applications into a suite of reusable services. But as usual, you lack the time and resources that you need in order to develop the services properly. So you googled or bing’ed, found this blog post, and began crying in gratitude. Yes, as the title implies, I am going to reveal my easy, 3-step, works-every-time process for converting silos of legacy applications into the inventory of services your CIO has been dreaming about. So just close your eyes and count to 3 … now open them … and here it is…. Not. While wishful thinking is too often the coin of the IT realm, even the most naive practitioner knows that converting legacy applications into reusable services requires more than a magic wand. The reason is simple: if your starting point is your legacy applications, then you will simply be bolting a web service technology layer on top of your legacy API. And that legacy API is built in the image of the silo applications. Enter the wide gate of the legacy API, follow the broad path of generating service interfaces from existing code, and you will arrive at the siloed enterprise destruction that you thought you were escaping. The Straight and Narrow Path This past week I had the opportunity to learn how the FBI Criminal Justice Information Systems department has been transitioning from silo applications to a service inventory. Lafe Hutcheson, IT Specialist in the architecture group and fellow attendee at an SOA Architect Certification Workshop, was my guide. Lafe has survived the chaos of an SOA initiative, so it is not surprising that he was able to return from a US Army deployment to Kabul, Afghanistan with nary a scratch. According to Lafe, building their service inventory is a three-phase process: - Model a business process. This requires intense collaboration between the IT and business wings of the organization, of course. The FBI uses IBM Websphere tools to model the process with BPMN.
- Identify candidate services to facilitate the business process.
- Convert the BPMN to an executable BPEL orchestration, model and develop the services, and use a BPEL engine to run the process. The FBI uses ActiveVOS for orchestration services.
The 12 Step Program to End Your Legacy API Addiction Thomas Erl has documented a process for building a web service inventory that is quite similar to the FBI process. 
Erl’s process adds a technology architecture definition phase, which allows for the technology environment to influence the inventory blueprint. For example, if you are using an enterprise service bus, you will probably not need to build your own utility services for logging or intermediate routing. Erl also lists a service-oriented analysis phase that highlights the 12-step process of applying the principles of service orientation to modeling your services. Erl depicts the modeling of a service inventory as an iterative process: model a business process, define the relevant technology architecture, define the service inventory blueprint, analyze the services, then model another business process, rinse and repeat. (Astute readers will note that Erl’s diagram, restricted to analysis and modeling process, does not include the implementation phase that concludes the FBI service development methodology.) The service-oriented analysis phase is where you find the 12 steps that will free you from your legacy API addiction. 
In a nutshell, you - identify the steps in the process that need services;
- identify the different types of services (agnostic entity services, service compositions, and utility services) that are required;
- apply service-orientation principles; and
- normalize the inventory into cohesive service models.
Rather than discuss each of the 12 steps individually, I will close by simply referring my readers to Erl’s explanation.
Sunday, September 12, 2010
In Part 1, I introduced Thomas Erl's notion of adopting the reusability analysis practices of ISVs when modeling reusable services. Today we look at what this entails.
First, let's take a look at what happens to software that's supposed to be reusable if you do *not* perform reusability analysis. When my employer was launching its first enterprise product a few years ago, we allowed our first few customers to dictate the details of many features. This practice had a certain logic to it: we needed customer feedback as to what the product features should look like, and we needed to sell some software to interested customers. So inviting a potential customer to help design the features they wanted (provided they would sign the contract) looked to be a win-win proposition. What we have discovered in the intervening years, however, is that a few of the early customer-designed features don't make sense for the rest of the customers who have since joined our happy family of users. And a couple of features don't even make sense for the customer(s) who originally requested them!
The root problem in these situations was that we did not have enough subject matter expert involvement, and not enough enough attention given to implementing the feature generically (i.e., in a way that could be used by many customers). Sure, we would have had to provide a little extra customization code so that the initial customer could have what they wanted (on top of a generic implementation). Unfortunately, designing the feature for the peculiar needs of a specific customer was tantamount to putting their customization code into the code base for everyone else--not a good idea. And now that the customer-specific code is in there and part of the API, it's hard to extract.
I'm not knocking the overall feature set of our product, by the way. It provides a lot of value for non-profit organizations, and is for the most part implemented generically enough to avoid putting our customers in awkward situations. I'm just saying we made a few mistakes along the way, that's all.
Now let's apply this lesson to enterprise SOA: do not throw the task of modeling and designing a reusable service over the proverbial fence to the first department in the enterprise that needs the service. That department may very well drive the service design towards meeting its peculiar needs and computing environment, much as some of my employer's customers drove our enterprise CRM product in the occasionally odd direction. And then you'll have a service that's hard to reuse. Erl mentions the importance of getting subject matter experts (SMEs) and technology experts involved in modeling services to avoid this danger, and I heartily concur.
However, ISVs are doing a poor job if they restrict their reusability analysis to the involvement of SMEs and technologists. Successful ISVs always have a product management team (or at least a product manager, if the product is small) that is responsible for making sure the product's features and design meet the needs of the target market...not just the first customer, or even the existing customers. Of course, the product management team has subject matter expertise, but they also have the important responsibility of keeping the needs of the entire target market in mind. Without product management, a product can become incoherent, reflecting small and immediate concerns rather than the long-term big picture.
Likewise, an enterprise that is building an inventory of reusable services needs to have an enterprise architecture team that bears the responsibility for modeling the services in the SOA. The enterprise architecture team plays a role similar to the product management team in an ISV. Do not delegate the responsibility for modeling reusable services to departmental implementers--or even a collection of implementers from various departments--if you want them to be truly reusable.
Our product management team doesn't arrive at their conclusions by simply theorizing, or by having many conversations with individual customers. They do those things, yes, but they have also invited key customers to join a product advisory group (PAG). This group meets regularly to discuss which features are most important, and how they can be most effectively implemented. They have to deal with the trade-offs in the iron triangle (time, resources, features) jointly. Because these customers are talking with each other, and not just with our product management team, they can hammer out agreement about which features should ship when, and how they can be generically implemented.
Similarly, if you are trying to work out a service inventory blueprint in your enterprise, you should be gathering decision-makers from all the participating departments and asking them to help determine which services should do what, along with the order in which they should be developed. What shall we call this group? How about "Service Priority Advisory Group Helpfully Elaborating TimeTables of Implementation" (SPAGHETTI)? Whatever you call it, though, form your group and work with 'em.
To conclude the lesson, here are the principles we can apply to modeling reusable services that we have learned from ISVs:
- Do not rely on departmental implementers to model a reusable service for the enterprise--the resulting service will likely reflect the unique concerns of the department, and be hard to reuse elsewhere.
- Involve subject matter experts and technology experts when modeling reusable services.
- Host a regular gathering of those who will be using the services (departmental implementers) so they can help shape the service profiles and priorities.
- However, delegate final responsibility for modeling the service inventory and the capabilities of reusable services to an enterprise architecture team.
Saturday, September 11, 2010
Thomas Erl insists in his magnum opus, SOA Principles of Service Design, that you do not need to goldplate a service's capabilities, or consult with Madame Zelda and her crystal ball, to make the service reusable for future consumers and compositions. Certain types of software that we have been using for decades--operating systems, business productivity software, almost anything an ISV produces--have benefited from a lot of up-front analysis of reusable capabilities. If your software is being used by millions or even mere dozens of organizations, by definition it must possess capabilities that are used in a variety of contexts. In other words, its functionality is being reused. Therefore, if you're modeling the services in your inventory, you just have to adopt the reuse analysis practices that commercial software vendors have been using these past decades.
Of course, to a business that is only familiar with developing siloed applications for internal use, these are unfamiliar practices. The practice you're used to is this: you just look at the immediate requirements, build your application to meet those, deploy, and try to rake in some short-term ROI. For anything bigger than a Mom-n-Pop store, though, this can lead to an uncoordinated mess of overlapping, inconsistent, and uncooperative systems (and business processes). And that's why everyone has been paying attention to SOA recently.
So you've got to learn from the Microsofts and Oracles of the world, unless you happen to work for an ISV already. I've been consulting with ISVs and working at them for most of the past decade, so I'm familiar with the struggles that pertain to making functionality reusable. I'll post more on that shortly....
Sunday, September 05, 2010
Services, designed right, have much more runtime autonomy than components. That autonomy gives you much more ability to manage the security, reliability, and reusability of the encapsulated logic, although there can be a small cost in system performance. And service autonomy makes changes quicker and easier. That ability can save your bacon from time to time, just like it did for me one morning when I got to work....
Sunday, August 15, 2010
Long Island Expressway...painless dentistry...dry wine...educational television. Inventing oxymorons like these is a wonderful party game; what others can we come up with today? How about: airline food...random order...House Ethics Committee...Service-oriented business intelligence....
That last phrase does seem like an oxymoron, at first glance. Service-Oriented Architecture (SOA) and Business Intelligence (BI) appear to be very different animals in enterprise architecture....
As I wa
Monday, July 26, 2010
Collocation of development team members from various disciplines (design, development, test) has many benefits. If you've been working in an agile environment, you know about the enhanced communication. But don't stop there! Get your teammates to review your work before you check it in, and you get crucial feedback and catch some bugs ASAP.
Sunday, July 25, 2010
Just because a feature looks cool in a 5-minute demo doesn't mean it will do the job in real life. It might not scale for large amounts of data, for example. So make sure you're designing, developing, and testing for the real world! I saw this principle in action recently when I was called on to debug a performance problem....
Wednesday, June 23, 2010
We software engineers and architects sometimes feel like the rope in a tug-of-war. Pulling from one side is the short-term goal of delivering functionality, preferably yesterday. Indeed, our customers cannot justify paying for our services unless we deliver a working product, better and faster than our competitors. Pulling from the other side, however, is the long-term goal of quality. If our code becomes too disorganized or hard to understand, we cannot long remain in business, because we w
Wednesday, December 30, 2009
Some combinations go together well: New York, Yankees, and pinstripes; Oscar Peterson, Ray Brown, and Ed Thigpen; a loaf of bread, a jug of wine, and thou. To this list you may now add: unit tests, extension methods, and lambda expressions. Read on if you're interested in writing elegant unit tests.
Sunday, March 29, 2009
If you're writing a reusable framework, you might want to avoid declaring a class or method as public if you don't want to support it as part of the interface. At the same time, you might want to use it from a different internal assembly--especially if you're writing tests. Believe it or not, there is a way to achieve both seemingly contradictory ends in one codebase. Read on to find out how you can have your cake and eat it, too!
Thursday, February 05, 2009
I believe strongly in code clarity, and will reluctantly agree to greater complexity only if there is a clear gain that will clearly benefit the user--e.g., to optimize a bottleneck in a heavily-used code path. I was actually surprised to find that not everyone agrees with my stance. Read further for a friendly but vigorous debate between the two perspectives....
Monday, October 06, 2008
We all know how to perform data lookups based on last name, zip code, and so forth. But what if you need to do a lookup based on encrypted data? This could be an extremely expensive operation, if you have to decrypt the data in every record. But if you use secure hash codes, you can perform the lookup very inexpensively.
Tuesday, August 05, 2008
Evaluating technology options can be very murky--kind of like choosing a college or picking a spouse, only a lot more risky! If you can analyze the features you need, and score the technologies based on their ability to meet those needs, you can actually make some headway.
Friday, June 06, 2008
While I was busy customizing Microsoft's Exception Management Application Block to classify and log all exceptions thrown in our web and Windows apps, and writing instrumentation code that published timing events via System.Diagnostics.Trace, Microsoft was busy writing ASP.NET Health Monitoring. Microsoft has more resources, so their product is a little more advanced and customizable. Here's what it provides:
- An event model: There are event classes for web request failures, for authentication failures, for errors, for SQL errors, for heartbeats, and so forth. You can subclass the WebBaseEvent in order to define your own custom events, as well.
- A provider model: Health monitoring can publish events with 4 built-in providers (email notification, SQL Server, WMI, and event log). The WMI provider is especially interesting, but you will have to write your own application to manage and report WMI events, according to Microsoft.
- A customization model: You can edit web.config in order to map events to providers and to configure providers (for example, to specify a connection string for the SQL Server provider).
Since we have already customized so much infrastructure, especially by writing a bug portal that makes use of the customized Exception Management block, we will probably not be migrating to the ASP.NET Health Monitor any time soon. However, for those of you who have been whistling in the dark with respect to the health of your web applications and services, get busy! I highly recommend Scott Allen's post as a starting place for your efforts.
The .NET Framework provides built-in capabilities for creating components and services as singletons. If you want to create a singleton process, though, you're on your own. But not really; if you read on, you will find out how to use a Mutex to do the job.
|