ESB

Why do SOA projects succeed? Why do some fail?

Brian Loesgen — Mon, 19 May 2008 20:59:51 GMT

I'm part of a curriculum advisory panel for California State University for an upcoming SOA program. As part of that, I spoke with someone there recently, relating real-world experiences, in a rather wide-ranging conversation. I started thinking "gee, I should write some of this down, it could be a good blog post". So, here it is!

I'll start off by saying that I could probably write a book about this, and there's likely a bazillion of my peers that could co-author. Many have already written books about this. However, instead, I'm going to distill selected thoughts down into a soap box-ish ranting blog post. I have boiled down some key points into just a few phrases. Maybe it works, maybe the value gets lost in a haze of over-simplification. Maybe I should do a series of posts (but I have NO plans to do so!).

Think big, do small

I'd say the single most important factor is planning. And by that, I don't mean lock a bunch of smart people in a room for a year and have them emerge with a detailed waterfall Gantt chart. I mean realizing that you're embarking on a path that will transform the way you create, deploy and manage your business-critical applications. Done right, your enterprise becomes more agile, cost and time to deploy new functionality decreases, and you achieve the nirvana of code re-use we as an industry have been seeking for decades. Done wrong, and you'll squander precious people resources and budget on an initiative with little or no return.

Mitigation: "boil the ocean" approaches fail. Think it all through, and then select some small services, then gradually pilot/deploy/build-up from there.

Think across the enterprise

In a large company with multiple IT sub-organizations, you need to have enterprise-level vision. Create a governing body. Identify a services portfolio. Without this, you could be doomed to costly inefficiencies such as duplication of efforts where multiple departments create different variants on a service, inconsistent naming, and eventually, a maintenance nightmare.

Mitigation: If you don't already have an enterprise architecture group chartered with setting enterprise-wide standards and policies, form one.

Take a holistic view

A developer will write 12 lines of super elegant code and say "there, the service is done, you can deploy it now". It's no fault of the developer, in their opinion they are done. The problem of course is that you need a services management strategy, and that goes far beyond the realm of what the rank and file developer sees, or in most cases, beyond what they need to see, and certainly beyond what they usually think about.

Mitigation: Think about the entire lifecycle of a service. How will you deploy it? Is there an approvals process? How will you secure it? How will you manage it? How will you monitor it? How will you retire it?

Recognize the people challenges

SOA thinking moves people into the realm of message-oriented, or contract-first if you like, architectures. This requires a different mindset than traditional object-oriented or procedural architectures. You're in a realm where everything is loosely-coupled, and operations often become asynchronous. I have seen firsthand at challenged client sites just how horribly wrong things can go if you try to apply object-oriented thinking and functional decomposition in a message-oriented environment.

Mitigation: recognize this paradigm shift, and invest in your people (architects and developers) to help them make the transition. Get them training and mentoring.

Recognize the infrastructure challenges

Creating an efficient operations environment for services is something that will be new to many IT folks. Your SLA's will drive requirements such as high availability and response times. Policy-driven SLAs and policy-driven security add a layer of indirection that result in a more change-tolerant and resilient infrastructure.

Mitigation: Plan for the near-term future, but look to the long-term future to ensure the environment you are putting in place will be able to scale to meet future demands. Your tooling may be great when you have 6 services in production, but how will it be when you have 1,000?

Think beyond the technologies and into the processes

Today's fast-paced world and rapidly changing business requirements of course lead to rapidly changing needs from the applications we create. Back when applications were monolithic silos, change came slowly, and evolution and deployments occurred at the monolith level. In a services-oriented world where there are numerous services acting as cogs in the machine, the new unit of deployment is at the service or service-composition level. Change is assured, and solution subset deployments become normal. "cowboy deployments" although almost always bad, in this type of environment become disastrous.

Mitigation: recognize Application Lifecycle Management as a key part of your process, and invest in it. Put procedures in place to assure repeatable, reliable builds. Have a testing strategy. Have a versionning strategy. Code migration should not be something people are afraid of.

Plan for disasters

An interesting side effect in the reuse of code assets by implementing intelligent stratification is that you inevitably build dependencies. For example, if all your services use an exception management service to log and respond to exceptions, what happens if that service is unavailable? The impact of a single service outage could ripple through your entire environment.

Mitigation: Think carefully about the dependencies, and architect and build robust services that incorporate mechanisms to respond to outages.

Benefiting from a successful SOA strategy is possible, but it's something you (as an architect) need to think about. As technologists, we often naturally gravitate towards and get caught up in the technologies. However, there are many touch-points beyond just the technology decisions, what I've listed above are just some of the things I've observed out in the real world, out where people are actually implementing SOA-based initiatives. I'm hoping by posting this that I'll help ease the path for some of you....

Technorati Tags: SOA,BizTalk,Service Oriented,ESB

Neudesic to assist in Microsoft ESB Implementation for Jordanian Government

Brian Loesgen — Wed, 07 May 2008 12:55:41 GMT

Last year I went to Jordan to work with Microsoft on an ESB proof-of-concept for the Jordanian government. The press release below explains how it ended :)

I am really excited about this project. We have a stellar team forming around this project, and some very cool work ahead. .

Keep scrolling down, there are some good photos further down!

Here's a picture from the contract signing:

Here's a picture of Bill Gates and His Excellency the minister at the official contract signing kicking off this project (which happened at an e-government symposium in the UAE in January).

Here's our official press release:

NEUDESIC TO ASSIST IN MICROSOFT ESB IMPLEMENTATION FOR JORDANIAN GOVERNMENT

Enterprise Service Bus To Strengthen Jordanian Government’s e-Government Offerings

IRVINE, CALIF. – APRIL 25, 2008 – Neudesic, a leading Microsoft National Systems Integrator and Gold Certified Partner, recently ranked # 197 on the Inc. 500 List, has announced that it has been selected by Microsoft to assist with the implementation of an Enterprise Service Bus (ESB) for the Jordanian government.

“Microsoft’s selection of Neudesic for this strategically important project is a further testament of Neudesic’s leadership role in the services-oriented architecture and enterprise service bus space,” said Neudesic’s VP of Technology Tim Marshall.

The project will include the design, implementation and rollout of an ESB that will be used across all government ministries, and with external trading partners and agencies, to further extend, automate and streamline the range of government services that are available as e-government offerings to citizens and partners.

“We are looking forward to working with Microsoft and the Jordanian government to rollout an ESB that will serve as foundational infrastructure and meet their e-government communications needs for years to come,” said Brian Loesgen, Neudesic’s BizTalk architect for the project. “The government has recognized that a contemporary services-oriented infrastructure would provide them with the most agile and flexible messaging and integration platform, and that the Microsoft technology stack was perfectly suited to their needs.”

Jordan has historically been at the vanguard of e-government services, offering efficient processes with completion time unheard of in many countries, such as passport issuance within 2 days and drivers’ licenses in 2 hours. Their adoption of ESB messaging will strengthen this position.

“We look forward to working with Neudesic on this project. With their proven track record of success and depth of experience with these technologies, they will ensure that the results of this project will meet, or exceed, the government’s expectations,” said Khaled Chebat, Microsoft’s Services Lead for the Eastern Mediterranean.

And lastly...

I work hard, and I play hard. I'm not the sort to sit in a hotel room wishing everything was just like at home. The Jordanian people are super-friendly, generous and hospitable (well, except for when they get behind a steering wheel :)). I had lots of fun last year exploring the country, and I have more great adventures planned.

Here's a photo of me from a couple of weeks ago, at about 25 meters depth in the Red Sea, in front of the Cedar Pride wreck in Aqaba Jordan. It was deliberately sunk approx 25 years ago, and was far more interesting than other wrecks I've done. I took a bus down (about 4 hours) from Amman, spent the weekend in Aqaba, then took a taxi back following the Dead Sea route. I wanted to go that route primarily because the desert route would have been a steep altitude gain right at the start, not a good idea after a morning of diving, and as an added bonus, got the spectacular Dead Sea coast.

Much better way to spend my off-hours than hanging around the hotel watching CNN!

Technorati Tags: ESB Guidance,ESB,BizTalk,SOA

ESB Guidance Webcast next Friday (May 9th 2008)

Brian Loesgen — Sat, 03 May 2008 19:17:36 GMT

I'll be doing a webcast next Friday on Microsoft's ESB Guidance. It will be recorded if you can't make it live. This will be the first in a series, but I am not going to commit to future dates yet as I have a ton of things going on and am looking at 80%+ travel for an extended period of time. I will however be doing more webcasts. Future topics will be more advanced, include themes such as a deeper drilldown into SOA governance, extending the ESB Guidance, etc.

Details for this first webcast are:

Title

Introduction to Microsoft’s ESB Guidance

Description

The Microsoft ESB Guidance uses Microsoft BizTalk Server 2006 R2 to support a loosely-coupled messaging architecture, and extends the functionality of BizTalk Server to provide a range of new capabilities focused on building robust, connected, service-oriented applications that incorporate itinerary-based service invocation for lightweight service composition, dynamic resolution of endpoints and maps, Web service and WS-* integration, fault management and reporting, and integration with third-party SOA governance solutions.

This webcast will be an introduction to Microsoft’s ESB Guidance. The goal of this session is to explore the architecture of Microsoft’s ESB Guidance 1.0, and explain how it can be applied to create ESB-based business solutions. We will examine the core components of the ESB Guidance, as well as the built-in extensibility points.

Agenda

· Business drivers behind an ESB (Why do I care? Why do I want to do this?)

· Architectural overview (What’s in the box? How does it work?)

· Demos

Meeting Details

Friday May 9, 2008, 1:00-2:15pm Pacific GMT -8 (75 minutes)

Call for audio: 866-500-6738 or +1-203-480-8000, passcode: 221223#

Live Meeting: https://www.livemeeting.com/cc/microsoft/join?id=BTSBAG&role=attend&pw=35DKTQ

Technorati Tags: BizTalk,ESB Guidance,ESB,SOA

Upcoming San Diego Events

Brian Loesgen — Tue, 12 Feb 2008 14:42:11 GMT

There are a few things coming up over the next couple of weeks in San Diego that may be of interest:

Tonight, Feb 12, I will be doing a presentation on Microsoft's ESB Guidance at the San Diego .NET User Group meeting of the Connected Systems SIG. We will meet at the Microsoft office in La Jolla, 6:00 for pizza, meeting starts at 6:30. I'll probably go for a couple of hours, more if you want, because that's what a SIG is all about!
Tuesday, February 26, Mickey Williams (MVP, author, Neudesic Technical Director) will be doing a session on .NET 3.0 language extensions at the main meeting of the San Diego .NET User Group. Meeting will be at the Scripps Ranch library, 6:00 for pizza, meeting starts at 6:30.
Wednesday, February 27, Jason Noble (Neudesic Portal and Collaboration Practice Manager) will be participating in a panel on mashups ("Mashups are in your future!") for the SOA group I run for the San Diego Software Industry Council. Meeting will be at Websense in Sorrento Valley, 5:30 for snacks/networking, meeting starts at 6:00

Some cool stuff in there folks, hope you can make it!

Technorati Tags: ESB,SOA,Mashups

SOA Design Patterns book: public review now in progress, your input welcome

Brian Loesgen — Fri, 01 Feb 2008 17:17:27 GMT

For the past year or so I have had the pleasure and honor of working with Thomas Erl on a couple of forthcoming SOA/ESB books (I'll post more details when we get closer -- soon I hope!), that are part of his Prentice Hall Service-Oriented Computing series of books.

Another one of the books in the series that he's currently working on is SOA Design Patterns (to be release in May 2008). I've already contributed a bit, and am working on further contributions. You can help too! There is a public review in progress right now through the end of the Feb, your comments are welcome.

If you're an architect or patterns type (regardless of your technology preferences) check out http://www.soapatterns.org. We want as much community input as possible from you guys and gals in the trenches that are eating/sleeping/breathing SOA so that this becomes the definitive work on SOA Design Patterns.

Technorati Tags: SOA,ESB,Patterns,Software Architecture Patterns,Design Patterns

New blog aggregated feed: BizTalkBlogs.com

Brian Loesgen — Sat, 19 Jan 2008 19:47:21 GMT

There's a cool new resource out there that acts as a central registry for bloggers that are blogging about BizTalk, WCF, WF, SOA, ESB (which pretty much covers everything I think about all day long :)) and related technologies and topics. If you look at the names of the bloggers, you'll see the top names in this space, so this will surely become a great resource for Oslo content too as we enter that exciting era.

You can now go to a single point and get an aggregated view across a wide breadth of blogs, without needing to maintain the bloglist yourself. Pretty cool, and it has an RSS feed you can subscribe to. If you're doing anything in this space, then this should be a "must see" resource for you. I know I'll be checking the feed regularly, and thanks Microsoft for making this happen!

Check it out at http://www.BizTalkBlogs.com

Technorati Tags: BizTalk,ESB,SOA

New issue of BizTalk Hotrod now available....

Brian Loesgen — Fri, 11 Jan 2008 13:26:38 GMT

To help ensure everyone knows, the new edition (#3) of BizTalk Hotrod is now available. There's lots of good content, keep up the great work guys!

http://biztalkhotrod.com/Issues.aspx

Neuron team is growing, looking for people to add.

Brian Loesgen — Mon, 31 Dec 2007 08:42:21 GMT

I normally don’t post job listings here (we’re growing too fast, I wouldn’t be able to keep up!), however, if you read my blog, then it follows that you are likely interested in SOA, BizTalk, Enterprise Service Bus, so, you fall into the category of people that could potentially be a fit.

Neudesic’s Neuron team continues to grow, and it’s super-exciting to see the growth and all the good work coming from them. We’re looking for a couple of talented engineers to come on board and not only join a highly dynamic company with stellar growth prospects, but also to play a role in the development of a world class ESB product, built from the ground up on top of WCF.

I believe you’d be reporting to Marty Wasznicky (yes, *THE* MartyWaz), who joined Neudesic recently as VP Product Development for Neuron. The positions would be located in SoCal (Irvine).

If you’re interested in either of these positions, pls drop me a note or email me through this blog.

And, Happy New Year all!!

Senior Software Design Engineer (SDE)
Are you a senior SDE looking for your next technical leadership role? Do you like the excitement of shipping cutting edge server products? Do you want to work with talented people? Then come join the Neuron Development team to help ship a world class Enterprise Service Bus product for building Event Driven Business Solutions on the Microsoft Platform. Come work in a diverse and fun environment that gives you the opportunities to work with exciting technology such as Windows Workflow Foundation (WF), Windows Communication Foundation (WCF), Visual Studio 2008 in an AGILE/SCRUM environment.

The Neuron Development team has an exciting and challenging opportunity for a Senior SDE Technical Lead with strong problem solving skills, experienced design and coding capabilities, a passion for technical innovation, and a drive to build the next generation of distributed enterprise service bus software. You will have the opportunity to contribute to architecture discussion and/or product-wide working groups.

Primary responsibilities include participating in planning for multiple feature areas through interacting with and understanding customer scenarios and experience, participating in, and writing product specifications and architecture design documents, proficient in project planning across multiple milestones, implementing/coding features and developing unit tests. The candidate will work with their development, test & Program Management counterparts to determine the vision, scope, design, testability & requirements of the Neuron product.

The ideal candidate must be proficient in C#, WCF, .Net Framework, software design with a strong aptitude in messaging middleware and distributed applications. The ideal candidate will also have 6+ years professional experience in a SDE role, and at least one full software product cycle experience. BS in Computer Science or equivalent experience required. Familiarity with Microsoft Connected System technology such as Biztalk Server, SQL Server including T-SQL, WF, WCF, competitive offerings (Java, J2EE, WebLogic or WebSphere), and a security background are a strong plus.

Software Design Engineer in Test (SDET)
Tests and critiques software components and interfaces in more technical depth, writes test programs to assure quality, and develops test tools to increase effectiveness.

Are you a senior SDET looking for your next technical leadership role? Do you like the excitement of shipping cutting edge server products? Do you want to work with talented people? Then come join the Neuron Development team to help ship a world class Enterprise Service Bus product for building Event Driven Business Solutions on the Microsoft Platform. Come work in a diverse and fun environment that gives you the opportunities to work with exciting technology such as Windows Workflow Foundation (WF), Windows Communication Foundation (WCF), Visual Studio 2008 in an AGILE/SCRUM environment.

The Neuron Development team has an exciting and challenging opportunity for a Senior SDET Technical Lead with strong problem solving skills, experienced design and coding capabilities, a passion to lead technical innovation, and a drive to improve our test methodology. You will have the opportunity to lead architecture discussion and/or product-wide working groups.

Primary responsibilities include participating in planning for multiple feature areas through interacting with and understanding customer scenarios and experience, participating in, and writing effective test plans, proficient in project planning across multiple milestones, implementing improvements to test methods and practices, and measuring, analyzing & reporting test status. Other responsibilities include designing and implementing tools/framework to reach optimal automation, drive root cause analysis of defects, and provide technical leadership. The candidate will work with their development & Program Management counterparts to determine the vision, scope, design, testability & requirements of the Neuron product.

The ideal candidate must be proficient in C#, .Net Framework, software design and have a strong test aptitude and methodology background. The ideal candidate will also have 6+ years professional experience in a SDE or SDET role, and at least one full software product cycle experience. BS in Computer Science or equivalent experience required. Familiarity with Microsoft Connected System technology such as Biztalk Server, SQL Server including T-SQL, WF, WCF, competitive offerings (Java, J2EE, WebLogic or WebSphere) are a strong plus.

Events coming up this week....

Brian Loesgen — Tue, 11 Dec 2007 11:19:36 GMT

OK, so it’s short notice, but….. I have a couple of events I’d like to mention here:

1) Tomorrow night (Dec 12 2007) will be the second meeting of the San Diego .NET User Group Connected Systems SIG. Chris Romp, Microsoft Technology Specialist for BizTalk, SOA and Process Platform in Southern California, will be presenting “What’s new in BizTalk Server 2006R2". Meeting will be at the Microsoft office in La Jolla at 6:00PM. Details (and mailing list sign-up) at http://SanDiegoDotNet.com

2) On Wednesday (Dec 13 2007), I’ll be doing an MSDN GeekSpeak with Lynn Langit and Glen Gordon. Powerpoints are forbidden. It’ll be a free-ranging casual conversation, moving around between BAM, SOA and ESB and wherever else we happen to go. Details at http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032359319&Culture=en-US

Microsoft Patterns and Practices ESB Guidance 1.0 shown, and will be available in a few days

Brian Loesgen — Tue, 06 Nov 2007 23:24:10 GMT

The second huge news from the SOA Conference is that the ESB Guidance 1.0, from the Microsoft Patterns and Practices group, is done. A lot has been done since the first early versions. The size of the team has grown substantially, as has the scope of the project. The end result will be available on MSDN within a week or so. Marty Wasznicky did a couple of sessions on it at the SOA conference last week in Redmond, both of them were in a huge room, and both were at full capacity with people getting turned away at the doors.

In a nutshell, the ESB Guidance is a set of prebuilt components, guidance, and best practices that allow BizTalk developers to quickly produce ESB-based solutions. If you’ve been following my blog, you’ve seen the evolution of this over the past couple of years. The core concepts from the very early versions remain, but significant new functionality has been added (now up to about 60 projects, with tons-o-samples). The end result is a polished set of tools that are a huge benefit to BizTalk developers looking to deploy ESB-based solutions.

There’s far too much richness there to capture in a blog posting, but I’ll highlight a couple of the biggest new features.

Governance

Governance means different things to different people. The ESB Guidance now has hooks in it to both AmberPoint and SOA Software. The capabilities provided depend on which of those products you’re working with, but you can get goodness such as policy-driven security for services, policy-driven SLAs, end-to-end monitoring, and much more. This is a HUGE benefit.

Management Portal

There’s a new management portal that provides a highly functional out-of-the-box experience to both view system metrics and interact with the exception management and other subsystems.

WCF Integration

It’s all WCF now. The ESB Guidance is based on BizTalk Server 2006 R2. Services, on-ramps, etc are now exposed as WCF services.

Smart Caching

As an optimization, endpoint information is now cached using a “smart caching” scheme that precludes the need to do resolution for every call.

[updated] It's now live at MSDN:

http://msdn2.microsoft.com/en-us/library/bb931189.aspx