Rant

Drinking from a firehose

Kent Brown — Wed, 14 Nov 2007 01:12:55 GMT

Is it just me getting old, or is it getting harder and harder to keep up these days?

From an industry point of view, it seems to me that the waves of new technology keep coming faster than ever, but the adoption rate isn't quite what it used to be. Maybe it never was what I thought it was. I guess I always enjoyed trying to stay on the cutting edge and wrote off people that didn't as lazy or no fun. It just seems that post the dotcom bust, companies are a lot more cautious about taking on new technologies.

But it's not just the companies, who have a legitimate business need to do the cost/benefit analysis before rushing in, who are dragging their feet. It seems I see more developers these days writing off the latest cool thing they don't have time to learn yet as unimportant. Or too risky or just too hard. How many projects have you seen on WF, or WPF, or WCF, which have been out for a while now? How many developers do you know who are all over VS2008, which is supposed to RTM by the end of this year?

I've seen brand new development efforts, where developers have a lot of leeway to choose the technology, but they are hesitant to try WCF, instead opting for ASMX because it's tried and true. Either they are content with what they already know, are suspicious of the value of the latest and greatest, are scared of the risks, or are just too tired to learn something new. In a way it's good that developers have "matured" and consider the risks and business value instead of rushing starry-eyed into the next wave of technology. But it's sort of strange to me, and a little sad, because the pedal-to-the-medal optimistic mentality of most developers has always been what drew me to software development in the first place. Let the managers reign you in. But developers should chomp at the bit to push forward!

On a personal level, I had noticed myself slowing down just a bit as well (I've been in software for about 19 years). I need to specialize in something (for me it's BizTalk/WF/WCF, basically middle-tier technologies). So I have to choose wisely how I spend my limited cycles for learning (and how I direct the team of developers I lead). How can I possibly be an expert in BizTalk, WCF, WF, CardSpace, SharePoint, InfoPath, LINQ, Silverlight, SQL Server Analysis Services, SSIS, etc.? Not to mention the myriad of LOB and legacy systems an integration expert should know. These are the full range of technologies I've seen opportunities in over the past year. So obviously there is a place for choosing wisely or you'll be spread too thin.

But if you don't watch out, an insidious overly-cautious, lazy mindset can creep in. If you lose that wild-eyed youthful attitude that "I can learn anything with a good book, a running environment, a free weekend, and a few pots of coffee" it's not too long before you are sneaking around afraid a new technology is going to bite you. Pretty soon you've become that crusty old COBOL developer you always said you'd never be like.

Lately I've been inspired to jump back in the fray and start drinking from the firehose again. Partly it’s from necessity. Partly being inspired by some energetic young guns on my team.

I relate it to keeping in physical shape. If I get out of shape, the idea of running a mile or so is a nasty thought, quickly expelled by the click of the remote to a new channel. But if I dig in and start training for a few weeks, I get back to a point where my body craves the workout. Rigorous exercise seems to inject more energy into my body and mind than it demands. In the end I find I attack life more aggressively and positively when I am in good physical shape.

The same effect occurs when I shed the FUD in my mind of learning new technology and jump in with both feet. Lately I've dug deeper into WCF and WF. I've done some programming with LINQ. I've been watching videos on VS2008 during my commute, even going into the areas that are not my core focus. And I've been forced by different client-driven scenarios to learn and build POCs on several new technologies. There is still a large stack of StuffIReallyNeedToLearnSoon, but the juices are flowing. I'm back in the optimistic aggressive mindset that drew me to software development and helped me succeed in the first place.

I am interested in comments about ways people stay sharp out there. As an architect or manager of developers, do you commit a certain amount of time each day to doing hands-on work? As a developer, do you spend a certain amount of off-hours time keeping pace?

Is BizTalk Dead?

Kent Brown — Tue, 06 Nov 2007 19:20:36 GMT

Ok, so I have a theme going and thought I might as well run with it. I promise this is the last "Is x Dead?" post.

Of course BizTalk isn't dead. But it is going to change in the next couple of years. What I am talking about here is "Oslo", the recently announced, next-generation distributed computing vision from Microsoft.

Oslo takes SOA to the Internet

I was out at the SOA conference last week where Microsoft first publicly shared the vision that they are code-naming Oslo. There is a great story here and I can tell you from meeting with people on the product team that this is real - there are large numbers of people in the Connected Systems division at Microsoft already working on various products to support this vision. However, it will take some time, roughly two years, before significant portions of this will be ready for release. In the meantime, it is important as always to understand where things are going in order to make good decisions with the technology available today. The following is my current impression of the vision, the components, and the impact of Oslo on BizTalk Server.

The Vision of Oslo
Oslo is first of all a vision about the future of distributed application development and hosting. If Windows DNA was about realizing the vision of distributed computing through Microsoft technology, and .NET Connected Systems was about extending this vision to the entire enterprise by embracing interoperability standards, Oslo is about extending the vision one more time out to the breadth of the internet.

This vision had been previously announced at the MIX conference and termed "Software + Services". Software + Services, or S+S, is a subtle but meaningful variation on the currently popular Service-Oriented Architecture (SOA) and Software as a Service (SaaS) themes. The best way to explain it is to compare and contrast it with the current buzzwords.

Whereas SOA is largely used to describe the architecture within the enterprise, S+S creates a world in which services inside and outside the enterprise are tied together seamlessly to build a new breed of applications - "Composite Applications". While the Enterprise Service Bus (ESB) is currently all the rage, Microsoft wants to take it up a step and build the Internet Service Bus (ISB).

And while SOA is primarily targeted toward Corporate IT as an enabler for controlling quality and consistency within an organization, S+S actually encourages and facilitates the building of "opportunistic application development" by power users. These users will be more tech-savvy than ever before (having grown up with video games and blogs and iPods) and will not want to wait on the Corporate IT group to create the applications they need to be more productive. Think of the wild and crazy days of the PC revolution. But the advantage is that it will be easier for Corporate IT to adopt these renegade applications and officially support them when they gain critical mass because they will be built in a service-oriented fashion.

Software as a Service (SaaS) is best exemplified by SalesForce.com, which is an immensely successful CRM system that is completely hosted externally and consumed by subscribing companies. However, SaaS so far has only been successful when an entire application is hosted externally. The vision of S+S is that services will be hosted externally and applications within the enterprise will be built consuming services from within and without the enterprise security boundaries. The crux of this an availability of services built on interoperable web service standards, tools for building composite applications, and a federated security model that allows enterprise applications to trust externally hosted services.

The Components of Oslo

1. Continued emphasis on and evolution of core Service-Oriented programming technologies (WCF, WF). This will become ".NET Framework 4.0".

2. A first class Repository for storing, discovering, and governing services.

3. Simplified developer experience for building distributed applications through better modeling tools. This will of course be realized in a new version of Visual Studio.

4. Federated security model (InfoCard).

5. Leveraging of Groove techniques for getting around firewall barriers to collaboration between services.

6. Evolution of BizTalk -

o Adapters migrating out from BizTalk-specific realm to general .NET availability. This has already begun through the WCF Adapter framework and the WCF Adapter Pack.

o Other BizTalk tools, such as the schema editor and mapper become more mainstream as part of the .NET Framework.

o Orchestration will be done using WF as the XLANG/S engine is deprecated (this has been public knowledge for over a year)

o A new in-memory pub/sub engine will make persistence/durability of messages optional, making BizTalk more suitable for low-latency applications.

o BizTalk as a server product may be primarily about the hosting model and look something like the old AppCenter.

7. Potential for Microsoft (and others) to host useful services "in the cloud" that will be subscribed to by enterprise clients to reuse in their composite applications. There is a CTP version of some utility services already hosted at http://biztalk.net/Default.aspx.

What it means for the future of BizTalk

Eventually BizTalk as we know it will be dead in the sense that it will be deprecated and replaced by a shiny new technology. But I see it just like the change from COM to .NET. COM as a technology was dead, while the core concepts of component-based programming and interoperability between components lived on bigger and better than ever in .NET. And in hindsight, COM was a quirky technology and most of us are glad to be freed from GUIDs and the registry and vTables, etc.

In the same way, I expect that the architectural principals that BizTalk made real (interoperability, pub/sub, legacy integration, web service orchestration, long running transactions, massive scalability, model-driven development, etc.) will live on, in fact be magnified, in Oslo. But some of the specific implementation details that we may have grown sentimentally attached to (the MessageBox, XLANG/S, etc), will go away. And while we'll moan and fret as this transition happens, we'll eventually embrace the new and be happy we did.

In fact, Oslo represents an expansion of the role BizTalk plays in the entire Microsoft platform. While today BizTalk development is a niche are that a relatively small number of developers have braved, in Oslo, the tools will become more mainstream.

In the meantime, while we wait for the next version of BizTalk and the other tools that will make up Oslo, we have to make decisions about the applications we need to build today. This will be the topic of many blogs and white papers to come (in fact I have a white paper in the works regarding choosing between BizTalk and WF). For now I will summarize my view like this:

Oslo confirms Microsoft's commitment to Service-Oriented Architecture and development. So keep moving in that direction. Embrace WCF. Use WF where appropriate. And use BizTalk where appropriate.
You have to build your applications today on the available technology. Oslo is really too far away to significantly impact your development plans today. If you need what BizTalk Server provides, you should use it. Maybe when a Beta 2 or RC version is out with firm RTM dates, then you might want to delay a project to use the new technology.
Microsoft cannot make promises about upgrade paths at this point in the product cycle, but I feel there is no way they will fail to provide a way to upgrade orchestrations built on BizTalk 2006 R2 to run in the next WF-based orchestration engine. First of all, WF's roots are in XLANG/S so it isn't that hard to translate most features/constructs. Secondly, BizTalk is just too important and has too many mission-critical enterprise applications running on it for Microsoft to leave it hanging.
Finally, although the current BizTalk engine will be deprecated, it will still be available and supported for a long time to come.

Is BizTalk Dead?

Kent Brown — Tue, 06 Nov 2007 19:12:56 GMT

Ok, so I have a theme going and thought I might as well run with it. I promise this is the last "Is x Dead?" post.

Oslo takes SOA to the Internet

The Components of Oslo

1. Continued emphasis on and evolution of core Service-Oriented programming technologies (WCF, WF). This will become ".NET Framework 4.0".

2. A first class Repository for storing, discovering, and governing services.

3. Simplified developer experience for building distributed applications through better modeling tools. This will of course be realized in a new version of Visual Studio.

4. Federated security model (InfoCard).

5. Leveraging of Groove techniques for getting around firewall barriers to collaboration between services.

6. Evolution of BizTalk -

o Adapters migrating out from BizTalk-specific realm to general .NET availability. This has already begun through the WCF Adapter framework and the WCF Adapter Pack.

o Other BizTalk tools, such as the schema editor and mapper become more mainstream as part of the .NET Framework.

o Orchestration will be done using WF as the XLANG/S engine is deprecated (this has been public knowledge for over a year)

o A new in-memory pub/sub engine will make persistence/durability of messages optional, making BizTalk more suitable for low-latency applications.

o BizTalk as a server product may be primarily about the hosting model and look something like the old AppCenter.

What it means for the future of BizTalk

Oslo confirms Microsoft's commitment to Service-Oriented Architecture and development. So keep moving in that direction. Embrace WCF. Use WF was appropriate. And use BizTalk where appropriate.
You have to build your applications today on the available technology. Oslo is really too far away to significantly impact your development plans today. If you need what BizTalk Server provides, you should use it. Maybe when a Beta 2 or RC version is out with firm RTM dates, then you might want to delay a project to use the new technology.
Microsoft cannot make promises about upgrade paths at this point in the product cycle, but I feel there is no way they will fail to provide a way to upgrade orchestrations built on BizTalk 2006 R2 to run in the next WF-based orchestration engine. First of all, WF's roots are in XLANG/S so it isn't that hard to translate most features/constructs. Secondly, BizTalk is just too important and has too many mission-critical enterprise applications running on it for Microsoft to leave it hanging.
Finally, although the current BizTalk engine will be deprecated, it will still be available and supported for a long time to come.

Is Kent's blog dead?

Kent Brown — Tue, 06 Nov 2007 15:48:17 GMT

I've been busy, but obviously not blogging. My last blog entry was May 2006! Since then I've:

Continued to lead the Enterprise Integration Practice at twentysix New York (http://www.26ny.com/index.html) - we've built up a team of BizTalk talent that is second to none. We have 7 certified BizTalk consultants: Jeff Bolton and Joe Tsai were from the farm team; Seong-moh Yang, Alex Star, Shashi Raina, and Juan Suero were brought in as free agents. And I am the player/coach. Seriously, it's great working with such a bunch of motivated guys. BizTalk developers tend to be very passionate about middle-tier server-oriented programming and it's a blast to work with these guys.
Continued to lead the NYC Connected Systems User Group (www.nyccsug.org) - Thanks to Ian Murphy for being my Microsoft sponsor, and to several guys on my BizTalk team listed above who offer lots of logistics support. We've got around 250 registered members. We average 30-40 attendees, have had upwards of 60 at some meetings. If you are interested in connected systems programming and live in NYC area you should be part of this group.
Been part of the Microsoft BizTalk Virtual Technical Specialist (VTS) Team - This is an elite group of Microsoft consulting partners who are committed to sustaining a strong practice in BizTalk Server. The VTS role is to support Microsoft field sales team in presenting BizTalk Server to customers and performing POCs to show its value. Then we have the expertise to help customers implement BizTalk solutions successfully. I am the VTS in our organization and it's been a real pleasure to be part of this group. Marty Wasznicky makes sure the program is not just talk - he ensures we get the best training available, the best access to the BizTalk product team, and helps us work closely with the local Microsoft sales team.
Recently named Connected Systems Developer MVP! - This is a huge honor. I am grateful to the people who nominated me and to those who somehow saw fit to choose me. I also feel compelled to explain what it really means as I think there is somewhat of a misunderstanding out there. While it would be flattering to allow the impression that it means "super-smart BizTalk guy" to persist, it actually has to do with community involvement such as user groups, blogs, conferences, etc. I've been running user groups for years, which does take a lot of work and is somewhat "thankless", so this recognition from Microsoft is very rewarding. In a practical sense it gets me more connected to the Microsoft product teams and gives me input into the upcoming releases (such as "Oslo") and access to the most recent information available on beta releases. I look forward to leveraging this connection to benefit the developer community.
Continued to work on my golf game. I know golf isn't considered cool by developers. That is something that dumb, lazy "sales guys" get to do. Actually, I don't have much choice in the matter because I've been pretty much addicted to it since I started about 5 years ago. And, while I am a developer at heart, I do also sometimes play a "sales guy" on TV, so it's only fair that I get to golf every now and then in exchange for selling out to "the man". Anyway, I've struggled for 2 years to break 90, always coming close but falling short. Then boom, a couple of weeks ago I shot a 79! Unless you golf there is no way to explain what a quantum leap that is. I haven't played since. I am sort of savoring it and hoping somehow it wasn't a fluke. If you know golf, you know that the next time out I'll probably stink it up miserably, lose tons of balls, throw clubs, and swear off golf for life. And that'll last about a week.

Is Remoting Dead?

Kent Brown — Mon, 22 May 2006 23:05:00 GMT

There have been a lot of threads lately on various discussion lists and blogs, such as this one, grappling with the future of Remoting with WCF coming down the pike.

To explain my position, I first have to tell you my rule of software projects - “Every project sucks, except the one you are about to start.” We usually start projects with enthusiasm, eager to learn new technologies, convinced that this time we'll get the methodology and project management right and and the project will be written up as a case study and many awards and bonuses will be handed out. Somewhere after a few weeks or months the honeymoon phase ends and we get bogged down in the reality that software development is real hard work and not always fun. But that next gig always shines brightly on the horizon.

When it comes to distributed computing technologies, the rule is “Every distributed computing technology sucks, except the one Microsoft is shipping next year.” COM/DCOM was the greatest thing known to man. That is until its replacement, .NET Remoting, was in the works. Since .NET Remoting was going to solve all our problems it was then OK to admit the warts on the old platform.

Now, just a few years later, we are being told that Remoting sucks and WCF will solve all the problems. Only a dufus would deploy something on .NET Remoting.

Well, I am usually an early adopter and I generally agree that progress is being made with each wave, so I'll be eating and breathing WCF when it releases and singing its praises too. But we just have to remember that in designing something as complex as a platform for distributed computing, lots of compromises have to be made. Depending on the system you are building, and to an extent on your personal style and value judgments, there will be things about WCF that we will all hate in a few years and I dare say some of us will long for the days of .NET Remoting. And in some future stack that Microsoft releases, there will be a mix of features from all preceding platforms mixed in with a new paradigm and a new set of compromises. And then it will be OK to admit that WCF sucks too.

Winning developers over to the BizTalk way

Kent Brown — Wed, 15 Mar 2006 09:58:00 GMT

A few of us who evangelize BizTalk have noticed that it is sometimes tough to get developers excited about BizTalk. In general, the BizTalk class at a developer conference or event, e.g. VSLive, will not be the biggest draw. I think there are several reasons why this is:

General lack of knowledge and understanding of exactly what BizTalk does.
The name probably doesn't convey what it does very well. Microsoft Enterprise Integration Server might have been a better name (but its too close to Host Integration Server I guess).
The earlier versions of BizTalk were tough to work with.
A lot of developers simply aren't doing integration type work - BizTalk won't help you build web pages or Windows apps.
If people are trying to do SOA, ASP.NET makes web services so easy that a lot of people don't see the need to look further.
Developers like code. They may be threatened by the “drag and drop“ impression that you get from BizTalk at first. Where's the code?
Related to #6, a lot of people take pride in the gory multi-threaded, low level plumbing code they write and aren't eager to hand that over to BizTalk.
If you do try to get into BizTalk it can be overwhelming at first how much you have to learn.
The impression is that there aren't that many BizTalk projects out there so the career opportunities aren't so great.
Somehow the impression is that BizTalk is not as “cool“ as say Indigo or something like that.

So besides helping customers understand the value of BizTalk, one of my missions it to help developers get it. I've recently given talks at the NYC Code Camp, and at a local NJ user group, that I think went over very well. My angle is to help developers see that BizTalk a) is very much .NET, b) adds value in SOA implementations, c) is currently in very high demand.

Here are some of my reasons why you should look at BizTalk as a technology to get into if you are a developer:

BizTalk forces you to think in “Contract First“ terms, something most Web Services developers are only recently coming around to.
BizTalk is probably the most “SOA“ technology that Microsoft ships. Even after WCF ships, it still might be a dead heat.
While BizTalk is highly “SOA“, meaning it supports the latest XML and Web Service standards, it is also very pragmatic in that it hooks in nicely with legacy technologies that aren't yet up to the latest standards.
BizTalk 2004, and now BizTalk 2006 are vastly improved over the 2000/2002 versions.
BizTalk 2004/2006 development is done right in Visual Studio. There is actually quite a bit of .NET code in the typical BizTalk project.
Letting BizTalk's well-tested code handle the gory plumbing frees you up to focus on design and architecture and ship faster.
Interest from customers to do BizTalk projects is actually rising dramatically in the last few months. Right now there is more work than qualified people. The career opportunities are excellent.
It takes a fairly senior .NET developer to make a good BizTalk developer. I personally see it as a bar that the junior/intermediate developer should strive for to demonstrate they are ready to be called “senior“, “architect“, etc.
Because BizTalk is tough to learn at first, there is a high barrier to entry - it's one of the ways to differentiate yourself at a time when development skills are being commoditized by off-shoring.
The suite of individual features BizTalk provides (scalability, fault tolerance, Orchestration, Rules Engine, Business Activity Monitoring, Enterprise Single Sign-on, etc.) combine to make an overwhelming whole that is very compelling for using BizTalk to host true “Services“.

Looks like we got ourselves a convoy

Kent Brown — Fri, 24 Feb 2006 19:29:00 GMT

That's right a BizTalk convoy. I don't mean like a bunch of messages being handled by a single orchestration. I mean like a bunch of people who realize they are going in the same direction and are a force to be dealt with. The inaugural meeting of the NYC Connected Systems User Group (www.nyccsug.org) was a roaring success. Only in NYC can you get 50 people out to the first meeting of a user group with only a month of publicizing. It was exciting to see the clear interest in Enterprise Integration and BizTalk Server. We've got a ton of cool topics to cover, some great people helping organize it, and clearly a ton of support from the developer community. This is going to be a fun group. If you didn't make it to the first meeting, plan to come on out on April 17. Marty Wasznicky, http://blogs.msdn.com/martywaz/, will be presenting. You don't want to miss it.

(Non) Trivial Pursuit - Taking the 74-135 BizTalk 2004 Partner Competency Exam

Kent Brown — Wed, 01 Feb 2006 13:01:00 GMT

I just took (and passed!) the 74-135 Developing E-Business Solutions Using Microsoft BizTalk Server 2004 Partner Competency Exam.

This exam has a bit of a reputation of being hard, probably because BizTalk has such a broad range of technologies packaged together and because there are no Transcender-type practice exams to be had. A search on the internet for how to prepare for this exam will leave you frustrated. The answer is basically: “Know the product inside and out and have lots of real-life hands-on experience with it“.

I'm afraid I can't add much, because you have to sign a non-disclosure agreement before you take the test that certainly makes it sound like they can hang you by your toenails, pull out your nose hairs, and threaten to do the same to your kids, if you reveal anything about the questions on the exam. However, that doesn't stop me from making some general observations and describing what I did to prepare.

How I prepared:

Spent about a month preparing, mostly a couple hours a day during my train ride to and from work. Then I crunched the final weekend.
Attended the Deep Dive Training about a year ago. It would be better of course to take it closer to exam time.
Read the BizTalk Server 2004 Unleashed book cover to cover. Like most books written by a collection of authors, there are sections that are not as readable as you would like, but this book is actually very good. The downloadable samples are very helpful.
Worked through the tutorials and online labs.
Read pretty much all of the BizTalk 2004 MSDN White Papers .
Watched several BizTalk-related MSDN web casts.
Immersed myself in the matrix of excellent BizTalk Blogs. (As Bilbo Baggins says: “It’s a dangerous business, Frodo, going out your door. Your feet hit the road and if you’re not careful there’s no telling where you might be swept off to.")

How it went:

When I went through the the questions the first time, I was a little worried, as they weren't all slam dunks. But with the solid prep and some good test-taking, process of elimination, skills, I managed to whittle it down to a handful of questions that were 50/50. The final score was something like 864, 700 being required to pass, so in the end I solidly passed.

I have mixed feelings about this test. Like all certification exams I have taken, you could pass the test and not be a very good BizTalk developer, or conversely you could be a great BizTalk developer and not pass the test. Because BizTalk is so broad, there are only a couple of questions on each topic. You could know the topic pretty well (for example Human Workflow Services) and not know the specific question asked. The test questions are very evenly spread over all possible topics - BizTalk functional areas (Messaging, Orchestration, BAM, BAS, HWS, SSO, Adapters, etc.) as well as disciplines (development, configuration, management, debugging).

The ultimate value of preparing for and taking this exam is that you end up knowing all of the features of the product inside and out. Not having a practice exam, forces you to be very thorough and therefore to be familiar with the non-mainstream areas of BizTalk. This is a good thing, because you can make better design decisions when you are fully aware of the options. I wouldn't make it a pre-requisite for a BizTalk Developer, but perhaps would for a BizTalk Architect.

A season for every trick under heaven

Kent Brown — Wed, 28 Dec 2005 13:54:00 GMT

When pontificating on subjects like SOA, one has usually has to decide which side of the fence to stand on:

Unqualified, wild-eyed, bandwagon-riding enthusiasm. This usually involves pronouncements that the new paradigm will make computing infinitely easier and more fun, that it should universally and immediately be adopted for all new development and most if not all legacy systems should immediately be re-written because the new paradigm renders them hopelessly obsolete. In addition any developer who cares about his career had better start learning and using the new paradigm, and any tools that do not facilitate the new paradigm should immediately be discarded, while any tools that do support it are worth it, no matter how much they cost because of the ensuing productivity, efficiency, 'robustness“, flexibility, and general bliss.
Cool sophisticated skepticism. I love David Ing's post on this: “Unlike the word 'Objects', which found it's mindshare reality place in language constructs and the long winding road of design concepts through modeling languages ... 'Services' is a bit of an unbounded chimera at the moment. I can't think of a single architectural concept that is left behind. Reuse, Integration, Business Process Management, Macro-componentisation, Distributed Systems, Asynchronous Messaging all join the clowns and elephants of the SOA Circus Parade as it rumbles into town. ...This is a fantastic industry to work in as we get to watch a lot of these parades; they come every couple of years. The downside is that we often end up with elephant sh** on the street that needs to be cleared up by someone (certainly never the clowns in the parade, they're long gone...). “ LOL.

The problem is the tendency for people to take a specific paradigm for solving a specific set of problems and to generalize and extrapolate it out to the full universe of computing problems. I try to take a different approach:

There is a time for every paradigm,
       and a season for every trick under heaven:
a time to in-line and a time to abstract ,
       a time to inherit and a time to aggregate,
a time to build and a time to buy,
       a time to refactor and a time to patch,
a time to design and a time to ship,
       a place for Remoting and a place for SOAP,
a time for indirection and a time to hard code,
       a time to use a pattern and a time to refrain,
a time to accommodate feature requests and a time to push back,
       a time to push hard to met a deadline and a time to give up,
a time to train and a time to fire,
       a time to scale and a time to simplify,
a time to architect and a time to hack,
       a time to document and a time to run to the next project.

(sung to the tune of Turn, Turn, Turn, by the Byrds).

To me, architecting a system is finding the right solution for the given set of constraints. Those constraints involve the technical and functional requirements of the system being built, but also include a whole host of other business and cultural realities of the organization for which the system is being designed. This is so obvious that I don't want to expound on it, but surprisingly I hear and read so much along the line of “best practices” that does not make this clear. For example, in a web app, should you use session variables or should you keep things completely “stateless“? It depends. Are you are building Amazon.com, or an intranet application? Will you ever scale beyond a single web server? How much simpler or faster will your app be if you use some session state? But I often hear people state categorically that they always make web apps completely stateless because “you never know when it might need to scale“. There is of course some truth to that, but sometimes you do know it will never need to scale.

While a big proponent of SOA (I actually invented SOA, but that is for a future post ;-) ), I recently architected and built a large system that used .NET Remoting extensively - very little XML and not a web service in sight at a time when the conventional wisdom is “don't use Remoting, its not the future of the platform“. Why? Because while it was an extremely large and complex system, there were no requirements to integrate with other platforms, and the performance and simplicity of .NET Remoting outweighed any advantage of being ready for interoperability in case the need arises in the future. Shortsighted? Maybe, but in my opinion the right choice given the requirements, time line, budget, team expertise, etc. Besides, SOA is Just a Facade (another post to come soon), so we can always create an SOA interface to this system when needed.

On another project, the salesman at my previous consulting company had sold a project as using BizTalk Server 2004, a technology we were very eager to use. However, after some initial proof of concept work in BizTalk, I was asking some questions about the development staff at the client company. It turned out they had mostly SQL Server experience, which a couple of developers who had dabbled in VB6. No .NET experience. No XML experience. No BizTalk experience. What the heck were they going to do with the system after we left? It turned out the need was easily handled by some slightly complex DTS packages. Not quite as “elegant” as using BizTalk, but a much better fit for the situation.

It's tougher to take this approach to architecture and design because you have to be more aware of the various ways of solving a given problem, and the strengths and weaknesses of each solution. You have to really dig into all the requirements, including business and cultural, and you have to make value judgments. You can't just repeat the current industry mantra. You have to be willing to prescribe what fits the situation, not just what is cool and exciting for you as a technologist. It's more work, but its what we owe to those we serve. I don't want a mechanic who always changes the break pads at a certain mileage whether they need it or not. I don't want an architect who has designed skyscrapers to blindly use the same techniques on the new addition to my house.

Hopefully I don't sound like I am on the skeptical side of the fence when it comes to SOA. My job is to manage and grow a consulting practice around Enterprise Integration. I see SOA as an important paradigm with very real benefits that should be embraced, and I think it is actually underutilized in the industry. I just think we have to be careful not to oversell it because then we lose credibility.