BizTalk

BizTalk Light and Easy Sample Code

Michael Stephenson — Sat, 07 Nov 2009 19:47:29 GMT

Ive had a few people ask recently about getting the source code for the BizTalk Light and Easy videos I did about testing and build.

The code is all on Codeplex where everyone should be able to easily get a copy.

http://biztalklaealm.codeplex.com/

Enjoy

Mike

SBUG Webcast - BPM & Intro to Global 360

Michael Stephenson — Tue, 03 Nov 2009 21:39:48 GMT

Hi All,

Following this evenings webcast the recording of the event is available from the following location:

https://www311.livemeeting.com/cc/mvp/view?id=Z9M8RH

We were interested on feedback of what we would like to see at a future SBUG session, the suggestions were as follows:

Getting more value from Sharepoint in a BPM solution

Integrating BPM and Biztalk

An industry sector case study

More about the persona based approach

Claims and case management

Nationwide case study?

Hands on where we can play around with Global 360 product and make some processes

If you would like to see any of the above or an alternative please drop me an email or use the forum post for feedback

http://sbug.org.uk/forums/p/178/266.aspx#266

Cheers

Mike

Microsoft UK Business Integration Roadshow

Michael Stephenson — Fri, 30 Oct 2009 22:32:41 GMT

Theres a really cool looking event coming up in the UK in early december.

There are also some great speakers

Check it out

http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032430518&Culture=en-GB

Bizmonade Followup

Michael Stephenson — Fri, 30 Oct 2009 13:35:36 GMT

Ive been having a few discussions with Christos Karras and Jason Hyland about Bizmonade over the last few days as a followup to some thoughts and questions i posted about Bizmonade.

Christos has posted a response to some of my questions on the below link, and i must say i think his plan for Bizmonade looks really good and it should become an important tool in the biztalk development cycle.

Check out his response here : http://www.nootaikok.com/2009/10/more-about-bizmonade.html

SBUG Meeting on Tuesday

Michael Stephenson — Fri, 30 Oct 2009 12:52:13 GMT

Just a reminder about the SBUG webcast on tuesday

Global 360 will be introducing themselves and looking for feedback on what topic members would like to see them present at the next full SBUG meeting

All the details are here: http://sbugminihelloglobal360.eventbrite.com/

SOA is just outsourcing for applications – Discuss?

Michael Stephenson — Thu, 29 Oct 2009 15:06:01 GMT

There's been a few discussions going on recently with various colleagues and community members on the back of the SOA Manifesto announcement. In this discussions it made me think back to a year or so ago when I was watching some presentations about various SOA things. At the time there were discussions about why SOA was good, what it offered, why companies struggled with it and all of the usual stuff.

At the time I was reflecting on things on the way home and was thinking you know SOA is really just outsourcing. So if business has been doing outsourcing for so long and become fairly effective at it why is SOA something which is still very unclear?

At the time I didn't really take this any further but this week I thought id note down some of my thoughts in a bit of a brain dump fashion to see where I end up.

Before we get into more detail to set some context I am thinking about outsourcing in terms of how throughout history organisations have evolved to identify specialisations where a unit of work can be abstracted from the mainstream tasks and performed by a group specifically for that task. Overtime this has obviously evolved to the point where organisations developed departments to deal with specific types of work. This again evolved to the use of external business partners who would do these units of work for the organisation.

In the systems context I'm thinking about SOA in terms of how we want applications to work together doing work for each other. This can be either internal applications or applications outside of the core organisation. I guess this could also extend beyond applications to infrastructure services. I don't really want to get into a discussion of SOA vs. SAAS in this post but I'm sure I will mention things where some readers may think ah..but that's SAAS not SOA. To state my view upfront I believe that SAAS is what you are delivering in terms of applications as services, and SOA is one way you might do this. I guess I'm saying that to me SOA is an architectural approach to help you build effective SAAS applications and services across your organisation, and to consume services provided by yourself and others.

I thought it was best to discuss some things that outsourcing and SOA have in common, some of their differences and then a summary of my thoughts.

What have SOA and Outsourcing got in Common

Some of the things SOA and Outsourcing have in common include:

Lower Cost

Usually the vision for any project which talks about SOA or outsourcing includes some statement about how the project will save the organisation money.

Reuse

On of the best ways to get ROI on both services and outsourcing is through reuse opportunities. A business example of this would be when an organisation creates a specialist customer retentions team. Nomally all customer service people may deal with customers who become unhappy and want to cancel their account.

It's not really what we do

Business and applications are the same in that they have a core function and then a set of things that they need to do to support that function. To a business these support functions are usual candidates for outsourcing. Take the example of Acme Widgets. The thing they do is to manufacture and sell widgets. Acme really want to invest their resources into building the most profitable widgets and the best sales processes. Acme aren't really interested in things like human resources or finance departments and want to do these things to a good enough standard and to a low cost. These are obvious outsourcing candidates.

In the systems world Acme would be interested in investing heavily in functionality to support their sales, R&D and manufacturing departments but they don't really want to have to deal with application support things like user management. This is where services can come in. By using services from other systems such as Active Directory you can remove most of the user management requirements from your application.

Its really about focusing on the high value stuff and getting the low value stuff don't as efficiently as possible.

Someone else does it better

A common reason for using outsourcing or a service provided by someone else is that they may be a domain expert in that area and be really good at what they do. Rather than try to do it myself if it's not too expensive shouldn't I just get this expert to do it.

Contract

Having a well understood contract is key part of both successful services and outsourcing. A contract usually defines the following

what messaging/communications needs to take place
What levels of service are required
How much will be paid and how often
How exceptions will be managed
The way things are expected to work together
What everyone is expected to do

I'm sure I could come up with others, but if you think about it these things are equally applicable to an application service provided internally or externally as they are to an outsourcing project.

Enhancements from partner

Both application based services and outsourcing would offer opportunities for the consumer to benefit from enhancements the partner made to their offering. It is the partner (external company or internal team) whose core function depends on the outsourcing or service your organisation wants. You are their customer, but you are unlikely to be their only customer.

To improve their business the partner will probably try to add value to their offering by enhancing it or offering complimentary offerings which may benefit your organisation. A systems example of this might be where enhancements made to an application for one consumer can be offered to another. If I am the provider of a service which allows companies to get quotes for travel bookings. If I enhance my service to integrate with more travel companies then my customers would benefit from better prices and more choice.

Governance

Governance is important in both SOA and Outsourcing worlds to ensure you have made the right decisions and things are put in place to support your initiative and the ensure that these things are followed through.

An example of governance might be the policy which is created by a company to help them evaluate and choose outsourcing partners or application service providers. This can be done without this governance, but the importance of the policy is that there is a standard and consistent approach to doing this which can be evolved over time, but who's aim is to protect the interests of the business.

Monitoring

Without monitoring for service usage in IT or outsourcing usage and organisation would not be clear on how much it should pay, if it is getting value for money and a whole lot of other things.

In an SOA approach organisations would often be looking for a monitoring platform which they can reuse across their services for a consistent approach and to lower the cost of monitoring. In outsourcing projects monitoring is probably less standard because the nature of the projects could be very different. Obviously some outsourcing projects could involve IT and applications which could take advantage of SOA approaches.

The benefits of a Middleman

My initial thoughts were that one of the key differences between SOA and Outsourcing is that when using an SOA approach organisations often look to platform technologies such as a service bus to help them effectively connect their applications and services. This is quite a well understood concept in IT. However after a little bit of reading about advice for successful outsourcing apparently one of the common things for successful outsourcing projects is to have a middleman who can act as a broker between stakeholders. I didn't really know much about the detail of this but I guess this person is the equivalent of the service bus.

Barriers to entry

If you are doing some new work then both outsourcing and using services can help reduce those barriers to entry to start your new business initiative. An example from outsourcing might be an SME who want to begin doing online sales when they previously had very little IT capability. The cost of setting up a sufficient capability might mean the initiative would never start. Outsourcing this might be one way that the company could partner with someone who can do this for them could massively reduce the costs of the initiative.

Likewise in organisations who have an SOA approach, the more mature their SOA capabilities then the lower their barriers to entry for any new project could be because they can reuse or buy services which could be the difference between a project being viable or not.

Security

When considering outsourcing or SOA, security is an important factor in both. In initiatives involving both there is probably going to be information which is sensitive to the business moving around the organisation and potentially leaving its boundaries. Security considerations are important in both areas to ensure the business is not at risk.

Economy of scale

Because both services and outsourcing are normally dealing with more than one customer each consuming organisation can often benefit from the economies of scale offered by the service or outsourcing partner. For example in outsourcing a company who handles calls for many companies can probably negotiate much better deals with their vendors because of the number of customers they have where as each individual organisation if they didn't outsource would have a smaller volume of work which would be less attractive when negotiating deals with vendors.

An effective SOA strategy can also offer the benefits of economies of scale in a number of ways, one of which could be by using external service providers in a similar way to described about for outsourcing.

Where do SOA and Outsourcing Differ

I have listed below some thoughts on how SOA and Outsourcing are clearly different.

Competitive Advantage

One of the aims of SOA in an organisation is to make the organisation more competitive and to get an edge by being more dynamic and able to react to change. In outsourcing work is usually outsourced to save money. You do get some cases where work which is important to the income of an organisation is outsourced but this is probably unusual. Consider the case of a company who sells insurance, they may outsource something like random sales lead generation by SMS to a company who specialises in text messaging but they are unlikely to outsource the really important bit of the sales process where they get to speak to their customers and really show what they are about. Any company who is looking to get a competitive edge by outsourcing work which is where their differentiation lies has probably lost some of its focus.

SOA is more organic then outsourcing

Services tend to evolve and be enhanced over time, adapting them for more consumers and to do new things. Outsourcing units of work tends to be very fixed and with a clearly defined time where you are and are not outsourcing the work.

How are cost savings achieved

In an outsourcing initiative it is likely that the cost savings from the delivery of the project will begin immediately or very soon after delivery. In a project involving SOA the cost savings from a service may not be seen in the short term. SOA projects usually begin to see real cost savings when services begin to be reused and this is usually a longer term cost saving.

Return on investment

Because of the above comment about cost savings the ROI is likely to be seen quicker in outsourcing projects than projects which use SOA. However I would assume most outsourcing projects get to a certain point and then the ROI levels off to a fairly static level, where as with an SOA initiative the more times you reuse the service the higher the ROI. As you become more mature in your services then your opportunity to reuse them goes up closely followed by the return on that initial investment.

To discuss an example here, a few years ago I worked with a company in the banking sector. This company was a financial institution but they had this great and quite innovative idea at the time to offer outsourced online banking solutions to the rest of their sector. Initially they had been taking their own online banking application and creating copies of it to be setup for any business partner they teamed up with. Although this did the job, they did a project which I was involved with where they would reuse the things that their mainframe did and develop a services layer on top of this and a white labelled banking application which consumed these services. One of the original requirements was an upgrade of the old application to lower maintenance costs, but by also including requirements which allowed this to be properly treat as a service they made a larger initial investment. When they begun using the application themselves and with their main business partner the success they had meant they were able to win a number of other good business partners who had significantly reduce costs and time to get that business onboard. This was a good example of where the greater initial investment to include service based requirements payed off in the long term with an ROI that gets steeper over time.

One project helps another

In projects with SOA practices and services being delivered one of the core aims of the project is to deliver something which will help you in other projects. In a traditional business outsourcing project the requirements of the project are unlikely to care about the next project.

Outsourcing doesn't have to involve IT

Outsourcing does not have to involve IT, but I've never heard of a project or initiative which talked about SOA which did not have a heavy IT element.

Ownership

In my observations when parts of a process are outsourced the business side of an organisation always has a significant amount of control and ownership over the process however in lots of organisations when services are used or created it is a much bigger challenge to get the business to own those services and champion them. If you asked the manager of a business area within an organisation what services he/she had and how they played into the strategy for his/her business unit I think you would get very mixed responses.

Payment/Cost Model

In outsourcing you have probably negotiated a contract based on an expected amount of work. The outsourcing partner may expect to require 100 people to support the work. If the real volume of work is actually much lower than that they will probably still have to change you the same amount, your organisation is taking on the risk of how accurate the work estimate is. In a services based costing model (particularly for cloud platforms) the cost model tends to be much more usage based. You tend to pay per click or per request or with some kind of band width algorithm. Pay as you go models really support good scalability.

Centre for Excellence

Creating a Centre for Excellence is a term which I've heard many times in discussions about both SOA and Outsourcing. What I think is interesting is that in the two contexts however they often are realized in different ways. In SOA the centre for excellence usually refers to a group of people who actually get SOA and can apply the principles to real things. In Outsourcing it tends to be the different because it's the group doing the outsourced unit of work is generally what is considered to be the centre for excellence.

I guess this one could go in either similarities or differences?

Conclusion

Ok so I've wrote down a whole load of things so far, have I proven that SOA is just outsourcing for applications, well of course not and that wasn't really the point. However what I was trying to show is that there are a hell of a lot of similarities between SOA and Outsourcing as an approach and although business usually struggles to get to grips with SOA and has done for quite a while, they do understand Outsourcing very well indeed.

Maybe one way forward could be to talk about SOA using an outsourcing metaphor, this might help with those business/IT interaction challenges and remove some of the fear factor. One of the difficulties in communicating SOA is the lack of a common language for describing and explaining it. There aren't many well defined patterns or models. Maybe this is where we could look to outsourcing to see if any are transferable.

When you look at the differences you probably notice that actually a lot of the differences tend to come in the detail and implementation which often falls within the IT area anyway for projects based on SOA principles.

It would be good to know if anyone agrees with any of the above or if I'm just talking a load of rubbish

Bizmonade & Orchestration Testing

Michael Stephenson — Tue, 27 Oct 2009 00:43:33 GMT

I've just been looking over Bizmonade which is a testing tool for testing BizTalk orchestrations in isolation from the BizTalk execution environment. It's similar in concept to BizMock however the implementation is different. The simple way to describe how BizMock is implemented is that it uses a custom adapter which allows you to manipulate the messaging around an orchestration which is executing within BizTalk. Bizmonade is different in that it creates a custom version of the XLANG Engine which it then allows you to load an orchestration completely outside of BizTalk for testing.

Both projects allow you to test the orchestration from NUnit or MsTest with a syntax similar to most mocking frameworks.

As I've wrote a bunch of stuff in the past about testing, I'll be keeping an eye on these projects. What I'm really interested in is their positioning within the testing you would normally do. Both projects position themselves in that gap between grey-box testing with BizUnit and the component testing of schemas/maps/etc. There is clearly a gap there but how big of a deal it is I'm not sure.

My current questions and thoughts about where I would use the above two components are as follows:

Would I really want to extensively test all orchestration paths with Bizmonade and probably have an amount of duplication in my BizUnit testing
Would I be looking to use Bizmonade for edge or obscure test cases which would be a pain to test with BizUnit
Would I be looking to trust Bizmonade enough to reduce the amount of BizUnit testing I do
Could a key place for Bizmonade be where I usually have a chain of orchestrations which could be difficult to test with BizUnit, and I could test the orchestrations individually with Bizmonade
What risk does running my orchestration outside of BizTalk pose. Might I end up dealing with issues which wouldn't happen in XLANG or missing conditions that would happen in XLANG
Would Bizmonade offer a better way to ensure good test coverage for orchestrations
Would I lose the ability to debug orchestrations
Am I going to have to pay for Bizmonade (longer term)

Hopefully the answers to a lot of these questions will become clear as I get more experience with the component and with future releases (remember this is only a preview release).

As mentioned earlier there is a gap in the testing capabilities between BizUnit and the ability to unit test orchestrations, however this hasn't really caused me too much pain in the past. I think to be successful Bizmonade needs to be able to save me time in my development cycle and improve the quality of any deliverable maybe one of the best things Bizmonade could look to do would be to interpret an orchestration in the same way that Visual Studio Unit Tests can for C# and then generate some code which has tests for most of the common conditions and branches within my orchestration. Now that id really like.

A nice BizTalk day

Michael Stephenson — Sun, 25 Oct 2009 19:26:06 GMT

We had a situation last week where BizTalk really showed some of its value and it was nice to get some good feedback from one of the application service managers. I thought id share to brighten all of your days.

So the situation was that we had a production incident where the volume of messages the BizTalk environment had been processing was well in excess of what had been performance tested. The performance tests had been done against assumptions for one of our processes which had been incorrect and we processing a lot more messages on a day to day basis. This wasn't really too big a deal as BizTalk was handling it fine, however one particular morning there had been a backlog in the batch processing in one of the systems which seemed to have been flushed out and caused a flood of messages to come into BizTalk within a very short period of time.

The result of this flood of messages was that we ended up with a lot of orchestration instances running at the same time and we ended up with a memory based throttling situation for the host which was running these orchestrations. This normally wouldn't have been an issue as the 3 main orchestrations which were running are all asynchronous processes and would just work their way through and no external systems would be affected by a delay in these getting processed.

What was a problem however was that we had some other orchestrations where a delay in their processing would be an issue. These other orchestrations share the same host and for a long time had never experienced any issues. However since the latest release of one of the applications this unexpected increase in load was now a problem and this was the first incident.

The good news starts with the fact that because BizTalk could be monitored easily we had a good heads up on this issue starting to happen and we were able to begin doing something about it before it became a big problem. It's always good to be seen as being proactive and being able to deal with things without the business feeling any pain. So we were getting alerts from our HP Openview setup about this happening. We were able to quickly get on top of the issue by creating a new host and moving a couple of orchestrations to this new host so they wouldn't be affected in large floods of messages from other processes. The second thing we did was to increase the throttling settings for the first BizTalk host so that we could clear the backlog of messages a little quicker. It took about 10 mins to modify the BizTalk settings and to get things running nicely again. Although it took a couple of hours for BizTalk to work through the backlog of messages.

When we went over this issue with some of the Application Service Managers it was very pleasing to see their reaction to how we dealt with this. From other applications they had been used to expecting to require code changes, lots of analysis and testing and probably a number of weeks before they could implement changes to correct problems properly. They were also used to a significant cost in terms of having to implement workarounds to deal with the fall out of these type of situations.

In our case however BizTalk really showed its strength as an execution platform. Its flexibility and manageability are really powerful features.

SBUG Webcast BizTalk Messagebox Viewer

Michael Stephenson — Tue, 13 Oct 2009 22:32:41 GMT

Thanks to Jean-Pierre for a good SBUG event today, as discussed in the meeting below is some followup information.

Video

The video from this session is available here:

https://www311.livemeeting.com/cc/mvp/view?id=M56GT6

Forum/Q&A

If anyone has a question about the video please add it to the below forum.

http://sbug.org.uk/forums/p/169/253.aspx#253

Resources

A number of useful links related to todays meeting:

Terminator Tool: http://blogs.msdn.com/biztalkcpr/pages/biztalk-terminator-download-install-info.aspx

Jean-Pierres Blog: http://blogs.technet.com/jpierauc/default.aspx

SBUG Mini Meeting - Hello Global 360

Michael Stephenson — Fri, 02 Oct 2009 20:21:48 GMT

Global 360 are a new partner who have joined the user group and have contributed a few interesting ideas and suggestions for the user group.

The plan is at this short online meeting they will do a small introduction to who they are and what they do. From here we can suggest things that user group members would be interested in knowing more about and one of the Technical Architects from Global 360 will do a deep dive BPM session at a future SBUG event.

Signup for the meeting at: http://sbugminihelloglobal360.eventbrite.com/