BizTalk

Microsoft ESB Guidance for BizTalk Server 2006 R2 is live!

Kent Brown — Fri, 09 Nov 2007 17:06:04 GMT

One of the best attended sessions at the SOA Conference in Redmond last week was actually the last session on the last day when usually most people would be headed towards the airport. It was on the ESB Guidance that has been in the works for several months. Marty Wasznicky demo'd the final release bits and discussed the most recent refinements and changes. He was basically done at that point, just waiting for the final i's to be dotted and t's to be crossed on the deployment package and for it all to be put up live on MSDN. I've been hitting refresh on my browser all week trying to get the latest for a demo we are working on. Finally it's here: http://msdn2.microsoft.com/en-us/library/bb931189.aspx!

I won't go into detail here on what the ESB Guidance is since its all in the documentation, but in a nutshell it is a set of free components that work with BizTalk and enable Enterprise Service Bus (ESB) scenarios. This is huge because ESB is currently all the rage in SOA-land. Time and again we see BizTalk in head-to-head competition with the likes of IBM/Tibco/BEA to be the ESB of a large SOA initiative at a company. BizTalk was SOA-before-SOA-was-cool, but it does lack out of the box a handful of the features currently expected in an ESB. This ESB Guidance provides the needed components (mostly implemented as pipeline components) to supplement BizTalk so it can legitamately claim to be an ESB. The architecture and components here were harvested from large customer engagements where BizTalk is successfully being used as the ESB.

Congrats to Marty Wasznicky for his vision and relentless hard work to make this a reality.

Is BizTalk Dead?

Kent Brown — Tue, 06 Nov 2007 20:20:36 GMT

Ok, so I have a theme going and thought I might as well run with it. I promise this is the last "Is x Dead?" post.

Of course BizTalk isn't dead. But it is going to change in the next couple of years. What I am talking about here is "Oslo", the recently announced, next-generation distributed computing vision from Microsoft.

Oslo takes SOA to the Internet

I was out at the SOA conference last week where Microsoft first publicly shared the vision that they are code-naming Oslo. There is a great story here and I can tell you from meeting with people on the product team that this is real - there are large numbers of people in the Connected Systems division at Microsoft already working on various products to support this vision. However, it will take some time, roughly two years, before significant portions of this will be ready for release. In the meantime, it is important as always to understand where things are going in order to make good decisions with the technology available today. The following is my current impression of the vision, the components, and the impact of Oslo on BizTalk Server.

The Vision of Oslo
Oslo is first of all a vision about the future of distributed application development and hosting. If Windows DNA was about realizing the vision of distributed computing through Microsoft technology, and .NET Connected Systems was about extending this vision to the entire enterprise by embracing interoperability standards, Oslo is about extending the vision one more time out to the breadth of the internet.

This vision had been previously announced at the MIX conference and termed "Software + Services". Software + Services, or S+S, is a subtle but meaningful variation on the currently popular Service-Oriented Architecture (SOA) and Software as a Service (SaaS) themes. The best way to explain it is to compare and contrast it with the current buzzwords.

Whereas SOA is largely used to describe the architecture within the enterprise, S+S creates a world in which services inside and outside the enterprise are tied together seamlessly to build a new breed of applications - "Composite Applications". While the Enterprise Service Bus (ESB) is currently all the rage, Microsoft wants to take it up a step and build the Internet Service Bus (ISB).

And while SOA is primarily targeted toward Corporate IT as an enabler for controlling quality and consistency within an organization, S+S actually encourages and facilitates the building of "opportunistic application development" by power users. These users will be more tech-savvy than ever before (having grown up with video games and blogs and iPods) and will not want to wait on the Corporate IT group to create the applications they need to be more productive. Think of the wild and crazy days of the PC revolution. But the advantage is that it will be easier for Corporate IT to adopt these renegade applications and officially support them when they gain critical mass because they will be built in a service-oriented fashion.

Software as a Service (SaaS) is best exemplified by SalesForce.com, which is an immensely successful CRM system that is completely hosted externally and consumed by subscribing companies. However, SaaS so far has only been successful when an entire application is hosted externally. The vision of S+S is that services will be hosted externally and applications within the enterprise will be built consuming services from within and without the enterprise security boundaries. The crux of this an availability of services built on interoperable web service standards, tools for building composite applications, and a federated security model that allows enterprise applications to trust externally hosted services.

The Components of Oslo

1. Continued emphasis on and evolution of core Service-Oriented programming technologies (WCF, WF). This will become ".NET Framework 4.0".

2. A first class Repository for storing, discovering, and governing services.

3. Simplified developer experience for building distributed applications through better modeling tools. This will of course be realized in a new version of Visual Studio.

4. Federated security model (InfoCard).

5. Leveraging of Groove techniques for getting around firewall barriers to collaboration between services.

6. Evolution of BizTalk -

o Adapters migrating out from BizTalk-specific realm to general .NET availability. This has already begun through the WCF Adapter framework and the WCF Adapter Pack.

o Other BizTalk tools, such as the schema editor and mapper become more mainstream as part of the .NET Framework.

o Orchestration will be done using WF as the XLANG/S engine is deprecated (this has been public knowledge for over a year)

o A new in-memory pub/sub engine will make persistence/durability of messages optional, making BizTalk more suitable for low-latency applications.

o BizTalk as a server product may be primarily about the hosting model and look something like the old AppCenter.

7. Potential for Microsoft (and others) to host useful services "in the cloud" that will be subscribed to by enterprise clients to reuse in their composite applications. There is a CTP version of some utility services already hosted at http://biztalk.net/Default.aspx.

What it means for the future of BizTalk

Eventually BizTalk as we know it will be dead in the sense that it will be deprecated and replaced by a shiny new technology. But I see it just like the change from COM to .NET. COM as a technology was dead, while the core concepts of component-based programming and interoperability between components lived on bigger and better than ever in .NET. And in hindsight, COM was a quirky technology and most of us are glad to be freed from GUIDs and the registry and vTables, etc.

In the same way, I expect that the architectural principals that BizTalk made real (interoperability, pub/sub, legacy integration, web service orchestration, long running transactions, massive scalability, model-driven development, etc.) will live on, in fact be magnified, in Oslo. But some of the specific implementation details that we may have grown sentimentally attached to (the MessageBox, XLANG/S, etc), will go away. And while we'll moan and fret as this transition happens, we'll eventually embrace the new and be happy we did.

In fact, Oslo represents an expansion of the role BizTalk plays in the entire Microsoft platform. While today BizTalk development is a niche are that a relatively small number of developers have braved, in Oslo, the tools will become more mainstream.

In the meantime, while we wait for the next version of BizTalk and the other tools that will make up Oslo, we have to make decisions about the applications we need to build today. This will be the topic of many blogs and white papers to come (in fact I have a white paper in the works regarding choosing between BizTalk and WF). For now I will summarize my view like this:

Oslo confirms Microsoft's commitment to Service-Oriented Architecture and development. So keep moving in that direction. Embrace WCF. Use WF where appropriate. And use BizTalk where appropriate.
You have to build your applications today on the available technology. Oslo is really too far away to significantly impact your development plans today. If you need what BizTalk Server provides, you should use it. Maybe when a Beta 2 or RC version is out with firm RTM dates, then you might want to delay a project to use the new technology.
Microsoft cannot make promises about upgrade paths at this point in the product cycle, but I feel there is no way they will fail to provide a way to upgrade orchestrations built on BizTalk 2006 R2 to run in the next WF-based orchestration engine. First of all, WF's roots are in XLANG/S so it isn't that hard to translate most features/constructs. Secondly, BizTalk is just too important and has too many mission-critical enterprise applications running on it for Microsoft to leave it hanging.
Finally, although the current BizTalk engine will be deprecated, it will still be available and supported for a long time to come.

Is BizTalk Dead?

Kent Brown — Tue, 06 Nov 2007 20:12:56 GMT

Ok, so I have a theme going and thought I might as well run with it. I promise this is the last "Is x Dead?" post.

Oslo takes SOA to the Internet

The Components of Oslo

1. Continued emphasis on and evolution of core Service-Oriented programming technologies (WCF, WF). This will become ".NET Framework 4.0".

2. A first class Repository for storing, discovering, and governing services.

3. Simplified developer experience for building distributed applications through better modeling tools. This will of course be realized in a new version of Visual Studio.

4. Federated security model (InfoCard).

5. Leveraging of Groove techniques for getting around firewall barriers to collaboration between services.

6. Evolution of BizTalk -

o Adapters migrating out from BizTalk-specific realm to general .NET availability. This has already begun through the WCF Adapter framework and the WCF Adapter Pack.

o Other BizTalk tools, such as the schema editor and mapper become more mainstream as part of the .NET Framework.

o Orchestration will be done using WF as the XLANG/S engine is deprecated (this has been public knowledge for over a year)

o A new in-memory pub/sub engine will make persistence/durability of messages optional, making BizTalk more suitable for low-latency applications.

o BizTalk as a server product may be primarily about the hosting model and look something like the old AppCenter.

What it means for the future of BizTalk

Oslo confirms Microsoft's commitment to Service-Oriented Architecture and development. So keep moving in that direction. Embrace WCF. Use WF was appropriate. And use BizTalk where appropriate.
You have to build your applications today on the available technology. Oslo is really too far away to significantly impact your development plans today. If you need what BizTalk Server provides, you should use it. Maybe when a Beta 2 or RC version is out with firm RTM dates, then you might want to delay a project to use the new technology.
Microsoft cannot make promises about upgrade paths at this point in the product cycle, but I feel there is no way they will fail to provide a way to upgrade orchestrations built on BizTalk 2006 R2 to run in the next WF-based orchestration engine. First of all, WF's roots are in XLANG/S so it isn't that hard to translate most features/constructs. Secondly, BizTalk is just too important and has too many mission-critical enterprise applications running on it for Microsoft to leave it hanging.
Finally, although the current BizTalk engine will be deprecated, it will still be available and supported for a long time to come.

Is Kent's blog dead?

Kent Brown — Tue, 06 Nov 2007 16:48:17 GMT

I've been busy, but obviously not blogging. My last blog entry was May 2006! Since then I've:

Continued to lead the Enterprise Integration Practice at twentysix New York (http://www.26ny.com/index.html) - we've built up a team of BizTalk talent that is second to none. We have 7 certified BizTalk consultants: Jeff Bolton and Joe Tsai were from the farm team; Seong-moh Yang, Alex Star, Shashi Raina, and Juan Suero were brought in as free agents. And I am the player/coach. Seriously, it's great working with such a bunch of motivated guys. BizTalk developers tend to be very passionate about middle-tier server-oriented programming and it's a blast to work with these guys.
Continued to lead the NYC Connected Systems User Group (www.nyccsug.org) - Thanks to Ian Murphy for being my Microsoft sponsor, and to several guys on my BizTalk team listed above who offer lots of logistics support. We've got around 250 registered members. We average 30-40 attendees, have had upwards of 60 at some meetings. If you are interested in connected systems programming and live in NYC area you should be part of this group.
Been part of the Microsoft BizTalk Virtual Technical Specialist (VTS) Team - This is an elite group of Microsoft consulting partners who are committed to sustaining a strong practice in BizTalk Server. The VTS role is to support Microsoft field sales team in presenting BizTalk Server to customers and performing POCs to show its value. Then we have the expertise to help customers implement BizTalk solutions successfully. I am the VTS in our organization and it's been a real pleasure to be part of this group. Marty Wasznicky makes sure the program is not just talk - he ensures we get the best training available, the best access to the BizTalk product team, and helps us work closely with the local Microsoft sales team.
Recently named Connected Systems Developer MVP! - This is a huge honor. I am grateful to the people who nominated me and to those who somehow saw fit to choose me. I also feel compelled to explain what it really means as I think there is somewhat of a misunderstanding out there. While it would be flattering to allow the impression that it means "super-smart BizTalk guy" to persist, it actually has to do with community involvement such as user groups, blogs, conferences, etc. I've been running user groups for years, which does take a lot of work and is somewhat "thankless", so this recognition from Microsoft is very rewarding. In a practical sense it gets me more connected to the Microsoft product teams and gives me input into the upcoming releases (such as "Oslo") and access to the most recent information available on beta releases. I look forward to leveraging this connection to benefit the developer community.
Continued to work on my golf game. I know golf isn't considered cool by developers. That is something that dumb, lazy "sales guys" get to do. Actually, I don't have much choice in the matter because I've been pretty much addicted to it since I started about 5 years ago. And, while I am a developer at heart, I do also sometimes play a "sales guy" on TV, so it's only fair that I get to golf every now and then in exchange for selling out to "the man". Anyway, I've struggled for 2 years to break 90, always coming close but falling short. Then boom, a couple of weeks ago I shot a 79! Unless you golf there is no way to explain what a quantum leap that is. I haven't played since. I am sort of savoring it and hoping somehow it wasn't a fluke. If you know golf, you know that the next time out I'll probably stink it up miserably, lose tons of balls, throw clubs, and swear off golf for life. And that'll last about a week.

BAM Portal Pivot Table Error

Kent Brown — Fri, 31 Mar 2006 09:23:00 GMT

Getting ready for my talk on Business Activity Monitoring at the NYC BizTalk 2006 First Look Clinic this week, I ran into an apparent bug on the BAM Portal. It wasn't showing the Pivot tables but was giving a message: “The query cannot be processed: Safety settings on this machine prohibit accessing a data source on another domain.“.

Time was getting close to my presentation. Of course I could ask the audience to “trust me, it really works I'm just having some unexplainable bug right now“, but that's not fun.

I was urgently searching through the newsgroups and found someone else with the same error but he hadn't figured it out yet. I even had some Microsoft people trying to help but they couldn't reproduce the problem.

Turns out its just what the error says. My setting in IE for Trusted Sites (which the BAM portal needs to be) had the security setting “Allow data access across domains” set to “Disable“. This setting is in the Miscellaneous category. Setting this to “Enable” does the trick.

Contract First and BizTalk

Kent Brown — Wed, 15 Mar 2006 12:43:00 GMT

Not only is BizTalk sort of “Contract First” out of the box, there is now a tool for BizTalk to actually give you the same experience the thinktecture WSCF tool gives you(http://www.thinktecture.com/Resources/Software/WSContractFirst/default.html).

The new tool is written by Molnar Tibor and is called BiztalkWscfClient: http://geekswithblogs.net/mtex/. It currently supports BizTalk 2004 and VS.NET 2004, but he is working to upgrade it for BizTalk 2006 and VS.NET 2005 very soon.

This is very cool! I can't wait to get the latest version.

Winning developers over to the BizTalk way

Kent Brown — Wed, 15 Mar 2006 10:58:00 GMT

A few of us who evangelize BizTalk have noticed that it is sometimes tough to get developers excited about BizTalk. In general, the BizTalk class at a developer conference or event, e.g. VSLive, will not be the biggest draw. I think there are several reasons why this is:

General lack of knowledge and understanding of exactly what BizTalk does.
The name probably doesn't convey what it does very well. Microsoft Enterprise Integration Server might have been a better name (but its too close to Host Integration Server I guess).
The earlier versions of BizTalk were tough to work with.
A lot of developers simply aren't doing integration type work - BizTalk won't help you build web pages or Windows apps.
If people are trying to do SOA, ASP.NET makes web services so easy that a lot of people don't see the need to look further.
Developers like code. They may be threatened by the “drag and drop“ impression that you get from BizTalk at first. Where's the code?
Related to #6, a lot of people take pride in the gory multi-threaded, low level plumbing code they write and aren't eager to hand that over to BizTalk.
If you do try to get into BizTalk it can be overwhelming at first how much you have to learn.
The impression is that there aren't that many BizTalk projects out there so the career opportunities aren't so great.
Somehow the impression is that BizTalk is not as “cool“ as say Indigo or something like that.

So besides helping customers understand the value of BizTalk, one of my missions it to help developers get it. I've recently given talks at the NYC Code Camp, and at a local NJ user group, that I think went over very well. My angle is to help developers see that BizTalk a) is very much .NET, b) adds value in SOA implementations, c) is currently in very high demand.

Here are some of my reasons why you should look at BizTalk as a technology to get into if you are a developer:

BizTalk forces you to think in “Contract First“ terms, something most Web Services developers are only recently coming around to.
BizTalk is probably the most “SOA“ technology that Microsoft ships. Even after WCF ships, it still might be a dead heat.
While BizTalk is highly “SOA“, meaning it supports the latest XML and Web Service standards, it is also very pragmatic in that it hooks in nicely with legacy technologies that aren't yet up to the latest standards.
BizTalk 2004, and now BizTalk 2006 are vastly improved over the 2000/2002 versions.
BizTalk 2004/2006 development is done right in Visual Studio. There is actually quite a bit of .NET code in the typical BizTalk project.
Letting BizTalk's well-tested code handle the gory plumbing frees you up to focus on design and architecture and ship faster.
Interest from customers to do BizTalk projects is actually rising dramatically in the last few months. Right now there is more work than qualified people. The career opportunities are excellent.
It takes a fairly senior .NET developer to make a good BizTalk developer. I personally see it as a bar that the junior/intermediate developer should strive for to demonstrate they are ready to be called “senior“, “architect“, etc.
Because BizTalk is tough to learn at first, there is a high barrier to entry - it's one of the ways to differentiate yourself at a time when development skills are being commoditized by off-shoring.
The suite of individual features BizTalk provides (scalability, fault tolerance, Orchestration, Rules Engine, Business Activity Monitoring, Enterprise Single Sign-on, etc.) combine to make an overwhelming whole that is very compelling for using BizTalk to host true “Services“.

NYC Code Camp

Kent Brown — Thu, 23 Feb 2006 09:36:00 GMT

I am presenting at my first Code Camp this Saturday in NYC. The matrix of sessions is here: http://nyc.codecamp.us/Matrix.htm

The registration is full, but you can register for the wait list here: http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032289984&Culture=en-US

My talk is on Orchestrating Web Services with BizTalk Server 2006. I've found from my .NET user group in NJ and other venues that its often tough to get the average developer enthused about BizTalk. I'm not sure why it is. Some may be overwhelmed by the fact that it is so different. Some may feel threatened that it de-emphasizes custom code. I personally see it as the culmination of a lot of architectural principles I've come to hold over the years. My angle at Code Camp is to leverage developers' interest in Web Services and SOA by showing the value BizTalk adds in “choreographing” web services and how that helps realize the SOA dream.

BizTalk Server Compiled Help files

Kent Brown — Tue, 14 Feb 2006 19:12:00 GMT

I needed a copy of the help files I could print out from my laptop which does not have BizTalk Server 2006 installed (I have it on a VPC but had trouble printing topic and all subtopics from there for some reason).

This CHM version just came out last week and is very good. The content has been greatly improved since the Beta 2 release and overall seems a massive improvement over the BTS 2004 documentation: http://blogs.msdn.com/luke/archive/2006/02/03/524534.aspx.

NYC Connected Systems User Group - Inaugural Meeting

Kent Brown — Wed, 01 Feb 2006 14:34:00 GMT

Things are coming together for the new NYC Connected Systems user group (NYCCSUG).

We now have:

A web site: www.nyccsug.org.
A meeting location: The Microsoft NYC office.
An Inaugural Meeting planned:

6-8:30 PM on Thursday Feb 23rd
Speakers:

I will open up the meeting to introduce the group, discuss the purpose and logistics.
Ian Murphy, a Business Process and Integration (BPI) Technical Specialist with Microsoft to present a BizTalk 2006 Overview.

We are going to be an INETA group which hooks us into their great Speakers Bureau.
We have a Microsoft sponsor in Redmond who is going to help arrange for speakers from the product teams.
I have started contacting other big-name presenters and it seems being in NYC makes it a little easier to attract them.

Please go register and tell your friends about the group.